RL & post-training · Transfer & distribution shift / Open question
The Optimal Curriculum
Capability-adaptive generators maintain reward variation, but initial success rates need not predict future learning or cross-task benefit.
Seed catalog · Showing the bundled record.
Problem statement
Use N finite-horizon, binary-reward tasks with a shared bounded-feature softmax policy and a fixed evaluation mixture. The curriculum chooses the next task; task-to-task gradient influence is assumed to have rank at most r and bounded variation over training, with probes charged to the budget. Determine the excess cost of learning difficulty and transfer online relative to an equally resourced oracle that knows this influence, including negative transfer and failures of static difficulty ranking.
Assumptions and scope
- Evaluate on a fixed target distribution; selecting only easy training tasks must not count as progress.
- Specify the task generator and transfer structure, such as a bounded-rank influence model, and the feedback available to the curriculum.
- Include the cost of difficulty probes and any coverage floor or importance correction. State how negative transfer enters the guarantee.
What would settle it?
An implementable curriculum with explicit finite-budget regret/error bounds under stated transfer assumptions, plus a counterexample or lower bound identifying when static difficulty ordering cannot suffice.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
RLVE and sGPO already provide adaptive or initially profiled curricula. The source-motivated extension is a transfer-aware online guarantee, not the existence of curriculum learning. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Zeng et al. — RLVE: Adaptive Verifiable Environments ↗empirical motivation · source checked 2026-11-19
ICML 2026 publisher abstract checked: 400 procedurally generating environments and capability-adaptive difficulty; empirical environment-scaling evidence.
- Sudalairaj et al. — sGPO: Trading Inference FLOPs for Training Efficiency in RLVR ↗related algorithm · source checked 2026-11-19
Abstract checked: initial-policy profiling already motivates inverse-success group sizes, filtering and curricula. The simple inverse-success rule is not listed here as new.
- Cheng et al. — How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws ↗partial theory · source checked 2026-11-19
Full-text model and theorem statements checked: two-stage linear softmax policies with frozen features, realizability/orthogonality assumptions and gradient-flow limits. Coverage can change without new strategy support.