Statistical learning theory · Transfer & distribution shift / Open question
How Many Worlds Does an AI Need?
A theorem explaining the value of diversity needs to connect experience structure to statistical complexity, not just count tokens or environment names.
Seed catalog · Showing the bundled record.
Problem statement
Study horizon-H finite MDPs on common state/action sets whose transition kernels are mixtures of r unknown shared kernels. A world draws a mixture vector; each structure perturbs it within total-variation radius eta; tasks draw bounded linear rewards from a k-dimensional shared basis. Under fixed world/structure/task sampling laws and coverage constants, derive matching new-world policy-risk bounds in the numbers of worlds, structures, tasks and trajectories, and optimize their allocation at explicit acquisition costs.
Assumptions and scope
- Define latent dimension/rank, world and task sampling laws, horizon and coverage. Distinguish training generalization to a new world from a new task in an observed world.
- Do not impose empirical exponent values or their ordering as axioms. Include identifiable classes where the allocation ordering differs.
- Account for dependent trajectories and the sparse pattern of observed task-environment pairs; a full Cartesian product is a separate assumption.
What would settle it?
A fully specified hierarchical family with matching risk-rate bounds and a derived cost-optimal allocation, including a classification of the structural conditions behind different orderings.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
He et al. fit per-axis laws and separate world/structure diversity. Multi-task RL already has restricted guarantees, including finite-MDP-family transfer (Brunskill and Li) and pooled shared-representation offline rates (Manda and Diddigi). A matching multi-axis lower bound and a general fixed exponent ordering are neither established nor asserted here. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- He et al. — How Data Scales in Agentic Reinforcement Learning ↗empirical motivation · source checked 2026-11-19
Full-text Sections 3–6 checked: distinguish worlds, environment structures, tasks, trajectories and reward density. Exponents and synthesis discounts are empirical, regime-specific fits; allocation assumes a stated cost model.
- Schnitzer et al. — Probabilistic Performance Guarantees for Multi-Task RL ↗partial theory · source checked 2026-11-19
Abstract checked: composition of per-task finite-rollout confidence bounds with task-level generalization is existing theory.
- Brunskill and Li — Sample Complexity of Multi-task Reinforcement Learning ↗partial theory · source checked 2026-11-19
Abstract checked: tasks drawn from an unknown distribution over a finite MDP set; reduced per-task exploration complexity without negative transfer under stated assumptions.
- Manda and Diddigi — Generalisation in Multitask Fitted Q-Iteration and Offline Q-learning ↗partial theory · source checked 2026-11-19
Abstract checked: finite-sample guarantees with shared low-rank action-value structure and pooled 1/sqrt(nT) dependence under realizability and coverage. Separate world/structure/task axes and matching lower bounds are not claimed.
- Zeng et al. — RLVE: Adaptive Verifiable Environments ↗empirical motivation · source checked 2026-11-19
ICML 2026 publisher abstract checked: 400 procedurally generating environments and capability-adaptive difficulty; empirical environment-scaling evidence.