AI safety & alignment · Statistical learning theory / Open question
Can a Model Truly Forget?
A model may stop expressing information while still retaining its influence. Meaningful forgetting needs a guarantee tied to the training process and future releases.
Seed catalog · Showing the bundled record.
Problem statement
Fix a two-layer ReLU learner trained by T clipped, Gaussian-noised SGD steps on bounded labelled examples. After each release, an adversary chooses deletions using the releases seen so far. Determine the retained state and update work needed so that the joint transcript of subsequent model releases is (epsilon,delta)-indistinguishable from rerunning that same randomized learner on retained data, while preserving a specified prediction-risk tolerance. Count deletion number, dataset size, width and stored precision explicitly.
Assumptions and scope
- Fix the randomized training algorithm, retained-data access, deletion budget and allowed auxiliary storage; central storage and user-held deletion tickets are distinct models.
- Define the post-deletion indistinguishability experiment and adaptive adversary, including allowed public history. Previously released outputs cannot be erased retrospectively.
- Distinguish data deletion from permanent inability to relearn a concept from new data; state the utility objective and composition of guarantees across repeated deletions.
What would settle it?
A sharp resource trade-off with a certified procedure for a nontrivial nonlinear class and lower bounds under adaptive deletion, specifying how guarantees compose with continuing training.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Realizability-testing memory lower bounds, ticketed-memory schemes, retain-sensitivity certificates and continual-learning unlearning analyses already exist. The target is a joint adaptive-release/resource characterization. Failure of a particular extraction test is not proof of deletion. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Cherapanamjeri et al. — The Space Complexity of Learning-Unlearning Algorithms ↗partial theory · source checked 2026-11-19
Publisher abstract checked: realizability-testing deletion can require linear central storage even for constant VC dimension; ticketed and bounded-deletion models have different bounds. This is not a blanket lower bound for every approximate deletion task.
- Heinzler, Malihi and Sanyal — Less Noise, Same Certificate: Retain Sensitivity for Unlearning ↗partial theory · source checked 2026-11-19
Primary search abstract checked: fixed-retain-set sensitivity suffices for certified unlearning and can reduce noise compared with DP-style global sensitivity; PCA and ERM examples are existing results.
- Hu, Duan and Zhang — The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning ↗partial theory · source checked 2026-11-19
Primary search abstract checked: nonconvex continual-learning excess-risk decomposition and gradient/Hessian-based certified unlearning. The residual target concerns adaptive-release guarantees and sharp memory/time costs.