← All AI/ML open problems

Verification & proof search · Statistical learning theory · AI safety & alignment / Open question

Who Verifies the Verifier?

Optimization concentrates on a verifier's mistakes, so a small static validation error may not bound error after post-training.

OpenEditorially scopedSpecified modelCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

In a finite contextual bandit with a learned verifier and a separately accessible trusted correctness audit, determine the minimum audit budget required to control accepted errors throughout discrete stochastic policy updates. Permit the verifier to be updated from prior audits, and derive a true-reward versus accepted-error guarantee under explicit policy-shift assumptions.

Assumptions and scope

  • Distinguish true correctness from verifier acceptance and specify false-positive/false-negative behavior on the changing policy distribution.
  • Audits supply additional information; verifier scores alone are not assumed to identify false acceptances.
  • Define the auditor error model, adaptive audit selection and the quantity controlled uniformly over training time; state overlap or exploration requirements.

What would settle it?

An adaptive auditing algorithm with high-probability accepted-error/true-reward bounds and matching audit-information lower bounds, or an explicit impossibility regime for insufficient coverage or feedback.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Moya et al. already establish identifiability limitations and audit-based selective control for fixed-verifier gradient flow. The residual target is finite-sample, discrete, jointly updated policy/verifier dynamics. AB-MCTS separately illustrates why search coverage is not selected-answer correctness: Appendix D.3 reports 19.2% selected Pass@2 versus roughly 30% coverage at 250 generations. That motivates auditing but is not a result in this training model. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Moya, Thornley and Lin — Verifier Errors in RLVR ↗partial theory · source checked 2026-11-19

    Primary abstract checked: fixed-verifier gradient flow, feedback-identifiability limitations and audit-based selective control are already analyzed.

  2. Rahman et al. — ScalePRM: Scaling Verification Compute Without Ground Truth ↗empirical motivation · source checked 2026-11-19

    EMNLP publisher abstract checked: synthetic step labels from self-consistency/meta-critique and observed reward exploitation. Learned process scores are not kernel certificates.

  3. Bhamidipaty et al. — Imperfect World Models are Exploitable ↗partial theory · source checked 2026-11-19

    Abstract checked: model exploitation, limits of reward-hacking analogies and a relaxed safe planning horizon are already studied.

  4. Inoue et al. — Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search ↗related algorithm · source checked 2026-11-20

    Full HTML Sections 3–5 and Appendices A, C.10 and D checked: GEN/CONT selection, mixed-model and aggregation variants, empirical multi-model search. Finer real-world costs and final-answer selection are explicit directions. No finite-budget optimality theorem is supplied; candidate coverage is not selected-answer accuracy.