← All AI/ML open problems

AI safety & alignment · Verification & proof search / Open question

Can Hidden Backdoors Be Ruled Out?

A model can pass extensive evaluations while retaining a rare hidden failure mode. Understanding what evidence can rule one out is a foundational problem in AI assurance.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

For depth-L ReLU classifiers with rational weights of at most b bits and bounded layer norms on [0,1]^d, allow triggers that replace at most k input coordinates by fixed values. Given a trusted clean-data sampler and labels, determine the sample and runtime needed to certify that no allowed trigger causes a wrong target-class prediction on more than an epsilon fraction of clean inputs, or to produce a violating trigger. Compare weight access with query-only auditing and expose dependence on k, L and b.

Assumptions and scope

  • Fix a finite-precision network class, an explicit unsafe-output predicate, a deployment distribution and a trigger class; distinguish worst-case absence from small distributional failure probability.
  • Specify the auditor's access and computational budget. Positive results must exclude the known indistinguishable constructions through a substantive assumption, not by assuming detectable backdoors.
  • Charge both inspection and testing; any minimum trigger mass, reference-model guarantee or provenance trust must be explicit.

What would settle it?

An efficient sound audit for a nontrivial structured class with sharp detection/sample bounds, or a hardness reduction locating the limits of the proposed access model. Ordinary rare-event sampling bounds alone are a baseline.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Goldwasser et al. already construct computationally undetectable backdoors, including white-box cases; Sleeper Agents shows empirical persistence through safety training. The target is an achievable structural frontier for certification, not universal backdoor detection or a claim that observed models are deceptive. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Goldwasser et al. — Planting Undetectable Backdoors in Machine Learning Models ↗partial theory · source checked 2026-11-19

    Primary abstract checked: cryptographic constructions give black-box indistinguishability and, in random-feature/ReLU models, white-box undetectability. Generic efficient detection is obstructed; a positive certificate needs additional structural or access assumptions.

  2. Hubinger et al. — Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training ↗empirical motivation · source checked 2026-11-19

    Primary abstract checked: constructed backdoor behaviors persist through tested safety-training methods. This is empirical evidence in deliberately trained examples, not evidence that arbitrary deployed models have hidden objectives.