← All AI/ML open problems

Statistical learning theory · Transfer & distribution shift / Open question

How Many Worlds Does an AI Need?

A theorem explaining the value of diversity needs to connect experience structure to statistical complexity, not just count tokens or environment names.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

Study horizon-H finite MDPs on common state/action sets whose transition kernels are mixtures of r unknown shared kernels. A world draws a mixture vector; each structure perturbs it within total-variation radius eta; tasks draw bounded linear rewards from a k-dimensional shared basis. Under fixed world/structure/task sampling laws and coverage constants, derive matching new-world policy-risk bounds in the numbers of worlds, structures, tasks and trajectories, and optimize their allocation at explicit acquisition costs.

Assumptions and scope

  • Define latent dimension/rank, world and task sampling laws, horizon and coverage. Distinguish training generalization to a new world from a new task in an observed world.
  • Do not impose empirical exponent values or their ordering as axioms. Include identifiable classes where the allocation ordering differs.
  • Account for dependent trajectories and the sparse pattern of observed task-environment pairs; a full Cartesian product is a separate assumption.

What would settle it?

A fully specified hierarchical family with matching risk-rate bounds and a derived cost-optimal allocation, including a classification of the structural conditions behind different orderings.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

He et al. fit per-axis laws and separate world/structure diversity. Multi-task RL already has restricted guarantees, including finite-MDP-family transfer (Brunskill and Li) and pooled shared-representation offline rates (Manda and Diddigi). A matching multi-axis lower bound and a general fixed exponent ordering are neither established nor asserted here. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. He et al. — How Data Scales in Agentic Reinforcement Learning ↗empirical motivation · source checked 2026-11-19

    Full-text Sections 3–6 checked: distinguish worlds, environment structures, tasks, trajectories and reward density. Exponents and synthesis discounts are empirical, regime-specific fits; allocation assumes a stated cost model.

  2. Schnitzer et al. — Probabilistic Performance Guarantees for Multi-Task RL ↗partial theory · source checked 2026-11-19

    Abstract checked: composition of per-task finite-rollout confidence bounds with task-level generalization is existing theory.

  3. Brunskill and Li — Sample Complexity of Multi-task Reinforcement Learning ↗partial theory · source checked 2026-11-19

    Abstract checked: tasks drawn from an unknown distribution over a finite MDP set; reduced per-task exploration complexity without negative transfer under stated assumptions.

  4. Manda and Diddigi — Generalisation in Multitask Fitted Q-Iteration and Offline Q-learning ↗partial theory · source checked 2026-11-19

    Abstract checked: finite-sample guarantees with shared low-rank action-value structure and pooled 1/sqrt(nT) dependence under realizability and coverage. Separate world/structure/task axes and matching lower bounds are not claimed.

  5. Zeng et al. — RLVE: Adaptive Verifiable Environments ↗empirical motivation · source checked 2026-11-19

    ICML 2026 publisher abstract checked: 400 procedurally generating environments and capability-adaptive difficulty; empirical environment-scaling evidence.