Tabular learning · Data-efficient learning · Neural architecture & complexity / Open question
Learning Tables with Only the Features That Matter
A small subset of columns often drives tabular predictions. Neural models should be able to discover that subset without paying a high-dimensional inference cost forever. Efficiency payoff: Match an established statistical target with a compact learned predictor and a quantified cost of discovering useful columns.
Seed catalog · Showing the bundled record.
Problem statement
For and , let , each have zero mean and unit Sobolev norm, and . Can a gradient-trained additive ReLU network with learned feature gates attain risk from rows, while retaining only active coordinate subnetworks at deployment? Give sample, width, optimization-step and inference bounds without a known-support oracle.
Assumptions and scope
- Work in the regime where the displayed rate tends to zero. The Sobolev bound is per univariate component, not a fixed global bound on their sum; changing that condition can change the high-dimensional minimax rate.
- Use independently initialized coordinate ReLU subnetworks and a specified differentiable gate/regularizer with a final thresholding rule. State initialization scale, subnetwork width, step sizes and any proximal operation. Inserting a fully fitted kernel estimator into a network does not establish the requested gradient-training result.
- Charge all d-coordinate screening and training work, model/optimizer storage and numerical precision. The O(s) active-subnetwork condition permits reporting a width-dependent O(sm) serving cost; it is not a claim of O(s) arithmetic for arbitrary accuracy.
What would settle it?
An explicit finite-step neural learning theorem at the displayed rate, up to named logarithmic factors, with a sparse-serving certificate; or a training obstruction for a precisely specified update/initialization class.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Minimax sparse-additive estimation is established by Raskutti et al.; it is not the open target. Sparse neural mixers also exist. The remaining claim concerns actual random-initialized gated-network training and resource accounting; a dedicated optimization-literature audit is still needed for this editorial specialization. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Raskutti, Wainwright and Yu — Minimax-Optimal Rates for Sparse Additive Models over Kernel Classes via Convex Programming ↗partial theory · source checked 2026-11-20
Primary abstract checked: matching L2 estimation bounds for sparse sums of bounded-kernel univariate RKHS functions, including Sobolev classes. These statistical rates are established; the new target concerns a finite gradient-trained neural implementation.
- Eslamian and Cheng — TabNSM: Neural Sparse Mixer for Tabular Regression ↗related algorithm · source checked 2026-11-20
Primary search abstract checked in the tabular pass: sparse interactions, near-linear computation at fixed sparsity and empirical regression gains. Finite-step feature discovery at that cost needs separate analysis.