AI safety & alignment · Verification & proof search / Open question
Can Hidden Backdoors Be Ruled Out?
A model can pass extensive evaluations while retaining a rare hidden failure mode. Understanding what evidence can rule one out is a foundational problem in AI assurance.
Seed catalog · Showing the bundled record.
Problem statement
For depth-L ReLU classifiers with rational weights of at most b bits and bounded layer norms on [0,1]^d, allow triggers that replace at most k input coordinates by fixed values. Given a trusted clean-data sampler and labels, determine the sample and runtime needed to certify that no allowed trigger causes a wrong target-class prediction on more than an epsilon fraction of clean inputs, or to produce a violating trigger. Compare weight access with query-only auditing and expose dependence on k, L and b.
Assumptions and scope
- Fix a finite-precision network class, an explicit unsafe-output predicate, a deployment distribution and a trigger class; distinguish worst-case absence from small distributional failure probability.
- Specify the auditor's access and computational budget. Positive results must exclude the known indistinguishable constructions through a substantive assumption, not by assuming detectable backdoors.
- Charge both inspection and testing; any minimum trigger mass, reference-model guarantee or provenance trust must be explicit.
What would settle it?
An efficient sound audit for a nontrivial structured class with sharp detection/sample bounds, or a hardness reduction locating the limits of the proposed access model. Ordinary rare-event sampling bounds alone are a baseline.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Goldwasser et al. already construct computationally undetectable backdoors, including white-box cases; Sleeper Agents shows empirical persistence through safety training. The target is an achievable structural frontier for certification, not universal backdoor detection or a claim that observed models are deceptive. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Goldwasser et al. — Planting Undetectable Backdoors in Machine Learning Models ↗partial theory · source checked 2026-11-19
Primary abstract checked: cryptographic constructions give black-box indistinguishability and, in random-feature/ReLU models, white-box undetectability. Generic efficient detection is obstructed; a positive certificate needs additional structural or access assumptions.
- Hubinger et al. — Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training ↗empirical motivation · source checked 2026-11-19
Primary abstract checked: constructed backdoor behaviors persist through tested safety-training methods. This is empirical evidence in deliberately trained examples, not evidence that arbitrary deployed models have hidden objectives.