AI safety & alignment · Statistical learning theory / Open question
When Can Debate Reveal the Truth?
Oversight should remain informative even when the system being judged can solve problems its supervisor cannot solve directly.
Seed catalog · Showing the bundled record.
Problem statement
Use claims that a bounded-fan-in Boolean circuit of size n and depth D outputs a specified bit on a given input. Debaters submit and challenge gate-value traces; the judge can inspect a bounded number of local gate checks, each with independent error nu below one half. In this finite extensive-form game, characterize when learning debate strategies yields correct root decisions with probability at least 1-epsilon, charging strategy training, repeated checks and trusted adjudication.
Assumptions and scope
- Fix claim decomposition, evidence access, debater computational classes and the protocol's equilibrium or learning criterion.
- Specify judge noise and dependence, including repeated-query behavior. Arbitrary systematic errors on decisive claims need not be correctable through debate.
- Keep access to an honest strategy distinct from a guarantee that optimization discovers it; define capability asymmetry and any communication or collusion restrictions.
What would settle it?
A robust learnability theorem with explicit judge-error and resource thresholds, paired with impossibility cases or lower bounds outside those thresholds.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Doubly-efficient debate already gives computational protocols; Li et al. claim instance-optimal honest dominant-strategy guarantees for stable decompositions. Lechner et al. formulate debate as online learning and show judge-error amplification. The residual target combines noisy judging with strategy learning and charged supervision. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Brown-Cohen, Irving and Piliouras — Scalable AI Safety via Doubly-Efficient Debate ↗partial theory · source checked 2026-11-19
Primary search abstract checked: polynomial-cost honest strategies can verify stochastic computation against much stronger dishonest strategies in a specified debate protocol.
- Li et al. — How to Have a Sensitive Debate: An Instance-Optimal Protocol for AI Debate ↗partial theory · source checked 2026-11-19
Primary search abstract checked: stable decompositions, worst-case correctness, honest dominant strategies and instance-wise black-box optimality are already claimed. Fallible-judge training must be a separate extension.
- Lechner et al. — Towards a Theory of Learnability via Debate ↗partial theory · source checked 2026-11-19
Primary search abstract checked: online learning formulation, a cooperative-protocol combinatorial characterization, and amplification of occasional judge errors. Debate-based learnability is not wholly unstudied.