← All AI/ML open problems

Learning dynamics & scaling / Open question

Training Versus Thinking

The useful split between training and inference depends on transfer and how often the resulting model will be deployed.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

For a fixed finite set of model/training recipes and QQ deployment queries, model each recipe's expected task loss as an unknown LL-Lipschitz function of four bounded log-compute coordinates: pretraining, SFT, RL and inference. Charged pilot runs give conditionally sub-Gaussian observations; shared-checkpoint dependence must be included. Under C=Cpre+CSFT+CRL+QCtestC=C_{\mathrm{pre}}+C_{\mathrm{SFT}}+C_{\mathrm{RL}}+Q C_{\mathrm{test}}, derive a finite-budget allocation regret bound against an oracle knowing these response surfaces.

Assumptions and scope

  • Fix architecture options, training distributions, inference procedure and deployment workload; state any smoothness/shape assumptions on response surfaces.
  • Use true downstream performance, not pretraining loss alone, as the final objective. Include verifier/tool costs where used.
  • Learning-curve noise, checkpoints shared across stages and possible nonmonotone model-size optima must be handled or explicitly excluded.

What would settle it?

A declared lifecycle model, an adaptive pilot/allocation procedure and a finite-budget oracle comparison, or lower bounds demonstrating which transfer assumptions are needed.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Controlled pretraining-to-RL prediction and stage-specific scaling fits provide partial evidence. A universal allocation across arbitrary transformers is not asserted; the model and cost assumptions must be specified first. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Shen et al. — Understanding Reasoning from Pretraining to Post-Training ↗empirical motivation · source checked 2026-11-19

    Version-three abstract checked: controlled chess pretraining/RL and a smaller math-language replication. Do not treat a fitted dependence on pretraining as a universal compute optimum.

  2. Khatri et al. — The Art of Scaling Reinforcement Learning Compute for LLMs ↗empirical motivation · source checked 2026-11-19

    Abstract checked: empirical sigmoidal fits, recipe-dependent efficiency/ceilings and held-out-scale prediction; not a universal asymptotic theorem.

  3. Tan et al. — Scaling Behaviors of LLM Reinforcement Learning Post-Training ↗empirical motivation · source checked 2026-11-19

    ACL 2026 publisher abstract checked: Qwen2.5 scaling, model-size efficiency saturation and data reuse in the investigated regimes.

  4. Zheng et al. — An Empirical Study of Reinforcement Learning Scaling Laws ↗empirical motivation · source checked 2026-11-19

    Abstract checked: actor-critic experiments in Starpilot/Craftax, Elo-like metric and nonmonotone compute-optimal model size. This is not an LLM post-training theorem.