Kernel learning · Embeddings & representations · Statistical learning theory / Open question
Learning the Right Kernel
Kernel methods need not keep their representation fixed. Understanding how they learn a useful similarity geometry could bridge classical statistics and neural feature learning.
Seed catalog · Showing the bundled record.
Problem statement
Let and , where , is a degree-at-most- polynomial with bounded coefficients and nondegenerate gradient covariance, and is Gaussian noise. Start with an isotropic Gaussian kernel, alternate ridge fitting with damped, trace-normalized empirical gradient-outer-product metric updates, and reuse the same sample. Prove multi-round feature-subspace and prediction-risk bounds relative to the initial kernel, or identify a failure regime caused by symmetry, noise or unstable reuse.
Assumptions and scope
- Fix a positive-definite kernel family, ridge strength, average-gradient-outer-product update, normalization and damping; the learned kernel must be well defined on fresh points.
- Specify input/target structure, noise and the sample-reuse protocol. An accurate population gradient outer product cannot be assumed for free.
- Measure both representation recovery and population prediction risk, and charge kernel solves or their approximations. One successful feature update does not guarantee the full recursive trajectory.
What would settle it?
A controlled nonlinear multi-round learning theorem with sample/iteration complexity and a strict matched fixed-kernel advantage, plus failure regimes for symmetry, noise or unstable reuse.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Linear RFM trajectories already have finite-sample recovery theory; AGOP already recovers multi-index subspaces before accurate prediction; algebraic-task symmetry effects have been studied empirically. The target is a joint nonlinear recursive optimization-and-generalization result. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Cheng et al. — Exact Dynamics and Finite-Sample Trajectory Recovery of Linear Recursive Feature Machines ↗partial theory · source checked 2026-11-19
Primary abstract checked: regularized noisy multi-output linear RFM trajectories remain close to an infinite-data reference at every iteration under stated sub-Gaussian assumptions.
- Zhu et al. — Average Gradient Outer Product in Kernel Regression Provably Recovers the Central Subspace for Multi-Index Models ↗partial theory · source checked 2026-11-19
Primary abstract checked: AGOP subspace recovery can require fewer samples than accurate fixed-kernel prediction when low-degree components expose all relevant directions. This is an established representation/prediction separation.
- Bernal, Mallinar and Belkin — Breaking Data Symmetry Is Needed for Generalization in Feature Learning Kernels ↗empirical motivation · source checked 2026-11-19
Primary abstract checked: empirical RFM grokking and symmetry-breaking mechanisms on algebraic tasks. The title's necessity claim is not imported as a universal theorem.