← All AI/ML open problems

Tabular learning · Data-efficient learning · Neural architecture & complexity / Open question

Learning Tables with Only the Features That Matter

A small subset of columns often drives tabular predictions. Neural models should be able to discover that subset without paying a high-dimensional inference cost forever. Efficiency payoff: Match an established statistical target with a compact learned predictor and a quantified cost of discovering useful columns.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-20

Seed catalog · Showing the bundled record.

Problem statement

For X∼Unif⁡([0,1]d)X\sim\operatorname{Unif}([0,1]^d) and Y=∑j∈Sfj(Xj)+ξY=\sum_{j\in S}f_j(X_j)+\xi, let ∣S∣=s|S|=s, each fjf_j have zero mean and unit W2,2W^{2,2} Sobolev norm, and ξ∼N(0,1)\xi\sim N(0,1). Can a gradient-trained additive ReLU network with learned feature gates attain L2L^2 risk O(sn−4/5+slog⁡(ed/s)/n)O(s n^{-4/5}+s\log(ed/s)/n) from nn rows, while retaining only O(s)O(s) active coordinate subnetworks at deployment? Give sample, width, optimization-step and inference bounds without a known-support oracle.

Assumptions and scope

  • Work in the regime where the displayed rate tends to zero. The Sobolev bound is per univariate component, not a fixed global bound on their sum; changing that condition can change the high-dimensional minimax rate.
  • Use independently initialized coordinate ReLU subnetworks and a specified differentiable gate/regularizer with a final thresholding rule. State initialization scale, subnetwork width, step sizes and any proximal operation. Inserting a fully fitted kernel estimator into a network does not establish the requested gradient-training result.
  • Charge all d-coordinate screening and training work, model/optimizer storage and numerical precision. The O(s) active-subnetwork condition permits reporting a width-dependent O(sm) serving cost; it is not a claim of O(s) arithmetic for arbitrary accuracy.

What would settle it?

An explicit finite-step neural learning theorem at the displayed rate, up to named logarithmic factors, with a sparse-serving certificate; or a training obstruction for a precisely specified update/initialization class.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Minimax sparse-additive estimation is established by Raskutti et al.; it is not the open target. Sparse neural mixers also exist. The remaining claim concerns actual random-initialized gated-network training and resource accounting; a dedicated optimization-literature audit is still needed for this editorial specialization. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Raskutti, Wainwright and Yu — Minimax-Optimal Rates for Sparse Additive Models over Kernel Classes via Convex Programming ↗partial theory · source checked 2026-11-20

    Primary abstract checked: matching L2 estimation bounds for sparse sums of bounded-kernel univariate RKHS functions, including Sobolev classes. These statistical rates are established; the new target concerns a finite gradient-trained neural implementation.

  2. Eslamian and Cheng — TabNSM: Neural Sparse Mixer for Tabular Regression ↗related algorithm · source checked 2026-11-20

    Primary search abstract checked in the tabular pass: sparse interactions, near-linear computation at fixed sparsity and empirical regression gains. Finite-step feature discovery at that cost needs separate analysis.