Tabular learning · Bayesian learning / Open question
Learning the Right Prior for a Table
A synthetic-data prior can make a tabular model fast, but the wrong prior can waste its inference budget or systematically distort predictions. Efficiency payoff: Adapt to table structure while evaluating only a small subset of expensive prior-specific inference paths.
Seed catalog · Showing the bundled record.
Problem statement
Let prior , , select coordinates of and a degree- additive Fourier regression function with independent centered Gaussian coefficients of known variances. A task draws , then one function and noisy labeled rows. From meta-training tables, learn a router that evaluates at most prior-specific predictors on a new table. Determine squared-risk overhead relative to exact Bayes prediction under the unknown mixture , charging pretraining and routing.
Assumptions and scope
- The K component priors, support-size/degree ranges and coefficient/noise variances are known and bounded; mixture weights are unknown. Train/test tables are independent draws from the same mixture. A contamination or covariate-shift extension needs a separate quantitative assumption.
- Each component predictor's approximation error and evaluation cost must be given or learned with a bound. Exact posterior integration over all supports is not a free primitive. The router is a specified permutation-invariant neural network trained on tables, without access to the latent component index at test time.
- Use expected squared prediction risk on fresh rows. Charge all K scores if the router computes them before selecting k. Distinguish lack of component identifiability from predictive indistinguishability; the optimal mixture predictor need not identify a unique component.
What would settle it?
A finite-meta-sample, finite-training router guarantee with explicit dependence on K,k,n,m and component overlap, plus an unavoidable risk/cost trade-off. Exact Bayesian model averaging alone is a baseline.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
TabPFN, algebraic prior mixtures and a linear test-time-training improvement theorem already exist. The proposed finite Fourier-prior mixture and costed neural gate make a distinct editorial benchmark; optimization and gating choices still need to be frozen. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Hollmann et al. — TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second ↗related algorithm · source checked 2026-11-20
Primary abstract checked: synthetic-prior pretraining and in-context tabular classification. These experiments do not give a uniform risk/resource guarantee for arbitrary prior mismatch.
- Bartz-Beielstein — Structured Priors: Towards an Algebra of Component Priors for Prior-Data Fitted Networks ↗partial theory · source checked 2026-11-20
Primary abstract checked: prior operations and exact-Bayes mixture identities; empirical component identification can fail. A reliable learned finite-cost prior gate is not implied by exact averaging.
- Gozeten et al. — Test-Time Training Provably Improves Transformers as In-Context Learners ↗partial theory · source checked 2026-11-20
Primary abstract checked: a linear-transformer test-time update theory and empirical TabPFN gains. Nonlinear prior adaptation with charged training/stopping costs is beyond that stated result.