Neural architecture & complexity · Learning dynamics & scaling / Open question
Refining Splines Without Destabilizing Learning
Grid extension adds expressive capacity without starting over, but new weakly observed spline directions can create ill-conditioning and overfitting. Efficiency payoff: Spend fine-resolution parameters and optimization only where needed while retaining the work done at coarse resolution.
Seed catalog · Showing the bundled record.
Problem statement
For on , assume norms bounded by , hidden sums in and there. Train a cubic-spline KAN from uniform-input samples with unit Gaussian noise, doubling uniform knot resolution during training. Can a data-driven refinement/stopping schedule achieve up to logarithms, with controlled gradient-step and coefficient-bit costs?
Assumptions and scope
- Fix centering of phi and psi and a scale normalization to remove redundant hidden-coordinate shifts/scales. All teacher functions and their decomposition are unknown. Constants may depend on a,B,R but not n or G.
- Use exact nested-knot insertion to initialize a refined spline, so preserving the current function at refinement is a known baseline. Specify gradient optimizer, parameter/range constraints, regularization and the independent validation or corrected data-reuse rule selecting G.
- The displayed oracle trade-off is a target, not an imported convergence theorem. Charge all coarse and fine stages, conditioning-dependent steps and storage. Establish population risk rather than only faster descent of the training loss; moving-grid variants require separate stability control.
What would settle it?
A nonasymptotic refinement-and-optimization theorem at the stated risk trade-off, with a cost comparison to finest-grid training, or a quantified conditioning/optimization obstruction under explicit initialization and updates.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
KAN grid extension and classical knot insertion already exist; the source reports stalled optimization and U-shaped test loss at fine grids. Smooth approximation and related statistical estimator rates do not analyze the full changing-parameter training trajectory. This fixed three-edge benchmark is an editorial specialization. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Liu et al. — KAN: Kolmogorov-Arnold Networks ↗partial theory · source checked 2026-11-20
HTML Sections 2.1–2.5 and Theorem 2.1 checked: smooth-edge spline approximation with representation-dependent constants, grid extension, pruning and empirical training. The representation theorem does not provide smooth low-cost decompositions for every target or a gradient-training guarantee.
- Chen, Zhong and Ma — Wahkon: A Statistically Principled Deep RKHS Superposition Network ↗partial theory · source checked 2026-11-20
Primary search abstract checked: deep representer theorem, regularized estimator and minimax rates for a related RKHS-superposition model. Statistical estimator guarantees are existing baselines, not automatically finite-step spline-KAN optimization guarantees.
- Yu, Yu and Wang — KAN or MLP: A Fairer Comparison ↗empirical motivation · source checked 2026-11-20
Primary abstract checked: matched parameter/FLOP comparisons generally favor MLPs outside symbolic representation; spline activations can explain part of the advantage. The reported continual-learning comparison differs from the original KAN setting.