← All AI/ML open problems

Neural architecture & complexity · Learning dynamics & scaling / Open question

Refining Splines Without Destabilizing Learning

Grid extension adds expressive capacity without starting over, but new weakly observed spline directions can create ill-conditioning and overfitting. Efficiency payoff: Spend fine-resolution parameters and optimization only where needed while retaining the work done at coarse resolution.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-20

Seed catalog · Showing the bundled record.

Problem statement

For f∗(x,y)=ρ(ϕ(x)+ψ(y))f_*(x,y)=\rho(\phi(x)+\psi(y)) on [0,1]2[0,1]^2, assume C4C^4 norms bounded by BB, hidden sums in [−R,R][-R,R] and ρ′≥a>0\rho'\geq a>0 there. Train a cubic-spline [2,1,1][2,1,1] KAN from nn uniform-input samples with unit Gaussian noise, doubling uniform knot resolution GG during training. Can a data-driven refinement/stopping schedule achieve E∥f^−f∗∥L22≤Cinf⁡G{G−8+G/n}\mathbb E\|\hat f-f_*\|_{L^2}^2\leq C\inf_G\{G^{-8}+G/n\} up to logarithms, with controlled gradient-step and coefficient-bit costs?

Assumptions and scope

  • Fix centering of phi and psi and a scale normalization to remove redundant hidden-coordinate shifts/scales. All teacher functions and their decomposition are unknown. Constants may depend on a,B,R but not n or G.
  • Use exact nested-knot insertion to initialize a refined spline, so preserving the current function at refinement is a known baseline. Specify gradient optimizer, parameter/range constraints, regularization and the independent validation or corrected data-reuse rule selecting G.
  • The displayed oracle trade-off is a target, not an imported convergence theorem. Charge all coarse and fine stages, conditioning-dependent steps and storage. Establish population risk rather than only faster descent of the training loss; moving-grid variants require separate stability control.

What would settle it?

A nonasymptotic refinement-and-optimization theorem at the stated risk trade-off, with a cost comparison to finest-grid training, or a quantified conditioning/optimization obstruction under explicit initialization and updates.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

KAN grid extension and classical knot insertion already exist; the source reports stalled optimization and U-shaped test loss at fine grids. Smooth approximation and related statistical estimator rates do not analyze the full changing-parameter training trajectory. This fixed three-edge benchmark is an editorial specialization. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Liu et al. — KAN: Kolmogorov-Arnold Networks ↗partial theory · source checked 2026-11-20

    HTML Sections 2.1–2.5 and Theorem 2.1 checked: smooth-edge spline approximation with representation-dependent constants, grid extension, pruning and empirical training. The representation theorem does not provide smooth low-cost decompositions for every target or a gradient-training guarantee.

  2. Chen, Zhong and Ma — Wahkon: A Statistically Principled Deep RKHS Superposition Network ↗partial theory · source checked 2026-11-20

    Primary search abstract checked: deep representer theorem, regularized estimator and minimax rates for a related RKHS-superposition model. Statistical estimator guarantees are existing baselines, not automatically finite-step spline-KAN optimization guarantees.

  3. Yu, Yu and Wang — KAN or MLP: A Fairer Comparison ↗empirical motivation · source checked 2026-11-20

    Primary abstract checked: matched parameter/FLOP comparisons generally favor MLPs outside symbolic representation; spline activations can explain part of the advantage. The reported continual-learning comparison differs from the original KAN setting.