← All AI/ML open problems

AI safety & alignment · Verification & proof search / Open question

Certifying Jailbreak Resistance Without Blanket Refusal

A defense must protect against an entire specified perturbation set while remaining useful. Stronger detection can otherwise buy safety by rejecting most requests. Efficiency payoff: Reduce the number of extra generations or exhaustive edit checks needed for a meaningful robustness/utility certificate.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-20

Seed catalog · Showing the bundled record.

Problem statement

Fix a finite-precision depth-LL autoregressive transformer, vocabulary VV, maximum prompt length nn and generation horizon HH. Let Ek(x)\mathcal E_k(x) contain all payloads obtained by at most kk token insertions, deletions or substitutions, with trusted instructions immutable. For an explicit finite-state predicate Bad⁡(x,y)\operatorname{Bad}(x,y) and a randomized defense DD, certify sup⁡x′∈Ek(x)Pr⁡[Bad⁡(x′,D(x′))=1]≤α\sup_{x'\in\mathcal E_k(x)}\Pr[\operatorname{Bad}(x',D(x'))=1]\leq\alpha, while preserving benign-task utility within β\beta of the undefended model. Determine the certificate's inference/sample cost.

Assumptions and scope

  • Fix tokenizer, numeric/rounding semantics, parameter bit bounds, sampling rule and the known finite-state output predicate before certification. The predicate defines a restricted testable safety property; it is not an oracle for all harmful meanings. Inputs after editing have length at most n+k.
  • Use a supplied benign prompt distribution and bounded task-loss function, with paid independent utility samples. Refusal is scored as a task failure when the task requires a substantive answer, preventing an always-refuse certificate from meeting the utility target.
  • State white-box or query access, confidence 1-delta of randomized certification, and whether the guarantee is pointwise or distributional over base prompts. Count all model calls and preprocessing. Robust classification by a safety head does not by itself bound harmful generated responses.

What would settle it?

A sound nonvacuous defense/certificate with explicit n,k,H,|V| and confidence complexity on a stated useful model class, or matching computational/query lower bounds. Exhaustive finite enumeration supplies a baseline; attack-suite failure rates are not a certificate.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Erase-and-check already certifies harmful-prompt detection relative to a base filter; SmoothLLM, CluCERT and output-oracle smoothing give further methods and conditional guarantees. The added target combines an explicit generated-output predicate, edit set and utility/cost constraint. Its architecture/defense design remains an editorial research framework. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Kumar et al. — Certifying LLM Safety against Adversarial Prompting ↗partial theory · source checked 2026-11-20

    HTML Sections 3–7 and Theorem 4.1 checked: erase-and-check inherits harmful-prompt detection guarantees from a base filter; general infusion enumerates erased subsets. Benign utility is evaluated empirically and the guarantee is not unrestricted semantic output safety.

  2. Robey et al. — SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks ↗related algorithm · source checked 2026-11-20

    Primary abstract checked: perturb-and-aggregate defense with empirical robustness/utility trade-offs. Threat-model and instability assumptions matter; arbitrary semantic jailbreak immunity is not established.

  3. Wang et al. — CluCERT: Certifying LLM Robustness via Clustering-Guided Denoising Smoothing ↗partial theory · source checked 2026-11-20

    Primary search abstract checked: semantic clustering and denoising seek tighter certificates and lower sampling cost. This is an existing efficiency baseline, not a certificate against arbitrary meanings or input transformations.

  4. Seferis et al. — Randomized Smoothing Meets Vision-Language Models ↗partial theory · source checked 2026-11-20

    Primary search abstract checked: generated responses mapped to discrete classes by a bounded-error oracle, with sample/radius laws. Output-level smoothing already has theory conditional on the classifier assumptions.