Neural architecture & complexity · Tabular learning · Statistical learning theory / Open question
Can KANs Learn Their Best Decomposition?
KANs can approximate known smooth decompositions accurately; their architectural promise depends on discovering such decompositions from data. Efficiency payoff: Find a small graph of reusable univariate functions rather than paying for a wide generic model or a supplied symbolic structure.
Seed catalog · Showing the bundled record.
Problem statement
Let and , where is a depth- KAN of maximum hidden width and at most nonzero edges: . Each edge has norm at most , , and all hidden values remain in . From noisy samples, can gradient training plus pruning learn a spline KAN whose prediction error tracks the best sparse decomposition's approximation/statistical trade-off, with polynomial training cost?
Assumptions and scope
- Noise is independent centered Gaussian with known variance bound. The layered graph and edge functions are unknown; d,L,w,E,s,B,R are supplied complexity bounds. The target is prediction, not recovery of a unique hidden representation.
- Use splines of degree at least s−1, explicit grids, initialized coefficients, regularization, optimizer and pruning rule. Charge architecture search and failed restarts. A global penalized-risk minimizer or the teacher graph cannot be provided as a free oracle.
- A rate must expose dependence on composition Lipschitz constants, width and depth, as well as spline coefficients and precision. Compare to composition-aware MLPs at matched data and arithmetic budgets; arbitrary Sobolev functions do not satisfy the sparse smooth-edge promise.
What would settle it?
An explicit finite-step learning/pruning theorem with an oracle-style population-risk and resource bound for the full stated class, or a concrete optimization/computational barrier and a sharply defined tractable subfamily.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
The original KAN theorem is an approximation result conditional on smooth edge functions. Related RKHS-superposition minimax theory and recent tight piecewise-linear region counts already exist; worst-case dimensionality barriers remain. This editorial target adds unknown decomposition discovery and actual gradient-training cost, not a universal KAN advantage. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Liu et al. — KAN: Kolmogorov-Arnold Networks ↗partial theory · source checked 2026-11-20
HTML Sections 2.1–2.5 and Theorem 2.1 checked: smooth-edge spline approximation with representation-dependent constants, grid extension, pruning and empirical training. The representation theorem does not provide smooth low-cost decompositions for every target or a gradient-training guarantee.
- Chen, Zhong and Ma — Wahkon: A Statistically Principled Deep RKHS Superposition Network ↗partial theory · source checked 2026-11-20
Primary search abstract checked: deep representer theorem, regularized estimator and minimax rates for a related RKHS-superposition model. Statistical estimator guarantees are existing baselines, not automatically finite-step spline-KAN optimization guarantees.
- Ni et al. — Kolmogorov-Arnold Networks: Curse of Dimensionality and Structural Bias ↗partial theory · source checked 2026-11-20
Primary abstract checked: worst-case exponential-dimensional spline-coefficient lower bound on Sobolev balls and transfer of selected spline constructions to cubic-ReLU MLPs. Favorable smooth-composition scaling is not a universal architecture-specific escape from dimension.
- Su and Liu — Tight Asymptotic Bounds on the Maximum Number of Linear Regions of Piecewise-Linear Kolmogorov-Arnold Networks ↗partial theory · source checked 2026-11-20
Primary abstract checked: matching region-count growth at fixed architecture and a stated parameter-matched comparison with ReLU networks. The old missing lower region-count bound is not retained as open; expressivity does not establish trained risk.
- Yu, Yu and Wang — KAN or MLP: A Fairer Comparison ↗empirical motivation · source checked 2026-11-20
Primary abstract checked: matched parameter/FLOP comparisons generally favor MLPs outside symbolic representation; spline activations can explain part of the advantage. The reported continual-learning comparison differs from the original KAN setting.