THEORIA / RESEARCH CATALOG

Open problems
in foundations of AI/ML.

Find your next research direction. Explore conjectures and open questions through their models, mathematical targets and connections.

Theory & computationMathematical models & guarantees
119open problems

Explore with /next-steps ↗ or /attack slash commands

A source-linked collection of theoretical open problems. Explore challenges in AI safety, learning theory, transformers, world models, optimization, memory and inference. Each entry gives a precise research target, assumptions and known progress. Open status is provisional; broader programs explicitly need a framework. Sources & methodology ↗

Seed catalog · Browse the bundled collection. Search matches words; Supabase mode uses ranked full-text search.

START EXPLORING
SubjectAll subjects
TypeAll types
ReviewAll entries

Choose multiple options. Results match any selected value within each filter.

119 problems

AI safety & alignment · RL & post-trainingOpen question

Alignment Without Reward Hacking

Optimizing a learned reward can exploit its blind spots. Alignment needs guarantees that survive the distribution shift created by optimization itself.

Editorially scoped

4 sources · Framework needs definitionRead
AI safety & alignment · Statistical learning theoryOpen question

Can a Model Truly Forget?

A model may stop expressing information while still retaining its influence. Meaningful forgetting needs a guarantee tied to the training process and future releases.

Editorially scoped

3 sources · Framework needs definitionRead
AI safety & alignment · Verification & proof searchOpen question

Can Agents Resist Prompt Injection?

Agents must read untrusted material without granting it the authority to redirect their actions or disclose protected information.

Editorially scoped

2 sources · Framework needs definitionRead
Inverse RL & imitation · Statistical inference · AI safety & alignmentOpen question

Can Behavior Reveal the Reward?

Matching an expert's actions need not reveal why they act that way. Reliable reward learning must describe what behavior can and cannot tell us.

Editorially scoped

3 sources · Framework needs definitionRead
Statistical learning theoryOpen question

Can Every Learnable Rule Be Compressed?

If a rule can be learned from limited evidence, must every consistent dataset have a comparably small explanation? This is a precise form of the connection between simplicity and generalization.

Editorially scoped

2 sources · Specified modelRead
AI safety & alignment · Verification & proof search · Statistical inferenceOpen question

Can Finite Audits Certify an Adaptive Conversation?

Multi-turn certificates rely on worst-case transition and persistence properties, while practical audits observe only a finite selection of conversations. Efficiency payoff: Spend audits on safety-relevant states and transitions instead of repeatedly testing whole conversations without a coverage argument.

Editorially scoped

3 sources · Framework needs definitionRead
AI safety & alignment · Verification & proof searchOpen question

Can Hidden Backdoors Be Ruled Out?

A model can pass extensive evaluations while retaining a rare hidden failure mode. Understanding what evidence can rule one out is a foundational problem in AI assurance.

Editorially scoped

2 sources · Framework needs definitionRead
Neural architecture & complexity · Tabular learning · Statistical learning theoryOpen question

Can KANs Learn Their Best Decomposition?

KANs can approximate known smooth decompositions accurately; their architectural promise depends on discovering such decompositions from data. Efficiency payoff: Find a small graph of reusable univariate functions rather than paying for a wide generic model or a supplied symbolic structure.

Editorially scoped

5 sources · Framework needs definitionRead
Statistical inference · Embeddings & representations · Optimization & optimizersOpen question

Can Neural Networks Recover Interaction Graphs?

Neural networks can represent complex interactions parsimoniously. Turning their learned features into a trustworthy scientific interaction graph requires more than a good fit.

Editorially scoped

3 sources · Framework needs definitionRead
Neural architecture & complexity · Learning dynamics & scalingOpen question

Can Sparse Experts Learn Without Starving?

A large pool of experts is efficient only if the router discovers their roles and sends enough learning signal to each useful expert. Efficiency payoff: Train a high-capacity model with low active computation without losing rare features through capacity overflow or early routing collapse.

Editorially scoped

3 sources · Framework needs definitionRead
Transformer theoryOpen question

Can Transformers Learn to Search?

Searching a graph is a simple test of whether a transformer can discover a scalable reasoning procedure.

Editorially scoped

4 sources · Framework needs definitionRead
RMT & weight spectra · Statistical learning theoryOpen question

Can Weight Spectra Predict Generalization?

Weight matrices are available even when training data are not. Turning their spectral signatures into reliable diagnostics could make model selection and auditing far more informative.

Editorially scoped

3 sources · Framework needs definitionRead