← All AI/ML open problems

AI safety & alignment · Statistical learning theory / Open question

When Can Debate Reveal the Truth?

Oversight should remain informative even when the system being judged can solve problems its supervisor cannot solve directly.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

Use claims that a bounded-fan-in Boolean circuit of size n and depth D outputs a specified bit on a given input. Debaters submit and challenge gate-value traces; the judge can inspect a bounded number of local gate checks, each with independent error nu below one half. In this finite extensive-form game, characterize when learning debate strategies yields correct root decisions with probability at least 1-epsilon, charging strategy training, repeated checks and trusted adjudication.

Assumptions and scope

  • Fix claim decomposition, evidence access, debater computational classes and the protocol's equilibrium or learning criterion.
  • Specify judge noise and dependence, including repeated-query behavior. Arbitrary systematic errors on decisive claims need not be correctable through debate.
  • Keep access to an honest strategy distinct from a guarantee that optimization discovers it; define capability asymmetry and any communication or collusion restrictions.

What would settle it?

A robust learnability theorem with explicit judge-error and resource thresholds, paired with impossibility cases or lower bounds outside those thresholds.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Doubly-efficient debate already gives computational protocols; Li et al. claim instance-optimal honest dominant-strategy guarantees for stable decompositions. Lechner et al. formulate debate as online learning and show judge-error amplification. The residual target combines noisy judging with strategy learning and charged supervision. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Brown-Cohen, Irving and Piliouras — Scalable AI Safety via Doubly-Efficient Debate ↗partial theory · source checked 2026-11-19

    Primary search abstract checked: polynomial-cost honest strategies can verify stochastic computation against much stronger dishonest strategies in a specified debate protocol.

  2. Li et al. — How to Have a Sensitive Debate: An Instance-Optimal Protocol for AI Debate ↗partial theory · source checked 2026-11-19

    Primary search abstract checked: stable decompositions, worst-case correctness, honest dominant strategies and instance-wise black-box optimality are already claimed. Fallible-judge training must be a separate extension.

  3. Lechner et al. — Towards a Theory of Learnability via Debate ↗partial theory · source checked 2026-11-19

    Primary search abstract checked: online learning formulation, a cooperative-protocol combinatorial characterization, and amplification of occasional judge errors. Debate-based learnability is not wholly unstudied.