← All AI/ML open problems

Learning dynamics & scaling · Neural architecture & complexity · Optimization & optimizers / Open question

How Fast Can Local Learning Propagate Credit?

Local primal-dual feedback turns credit assignment into a coupled dynamical system. Nonlinear transients determine whether this can be a practical alternative to global backpropagation. Efficiency payoff: Establish when local feedback can train deep, narrow networks with a controlled synchronization and memory cost.

OpenEditorially scopedSpecified modelCompiled 2026-11-20

Seed catalog · Showing the bundled record.

Problem statement

For width-mm, depth-LL residual tanh networks hi=hi−1+L−1tanh⁡(Wihi−1)h_i=h_{i-1}+L^{-1}\tanh(W_i h_{i-1}), with ∥Wi∥op≤M\|W_i\|_{\mathrm{op}}\leq M, use squared output loss and lifted residuals ri=hi−hi−1−L−1tanh⁡(Wihi−1)r_i=h_i-h_{i-1}-L^{-1}\tanh(W_i h_{i-1}). Run PC-ALM on Lρ=ℓ+∑iλi⊤ri+ρ2∑i∥ri∥2\mathcal L_\rho=\ell+\sum_i\lambda_i^\top r_i+\frac\rho2\sum_i\|r_i\|^2. Determine the local-step budget T(L,m,M,ε)T(L,m,M,\varepsilon) making the resulting weight gradient gTg_T satisfy ∥gT−∇ℓBP∥≤ε\|g_T-\nabla\ell_{\mathrm{BP}}\|\leq\varepsilon, uniformly over unit-norm input/target pairs and bounded readouts.

Assumptions and scope

  • Initialize activities by the forward pass and multipliers at zero. Use T−1 simultaneous primal steps h<-h-eta_h grad_h L followed by dual steps lambda<-lambda+alpha r(h_new), then one final primal step; weights are fixed during inference. This fixes the source's update order.
  • Choose positive rho, eta_h and alpha as functions of L,m,M,epsilon, not of an unknown exact gradient. The gradient discrepancy is the Euclidean norm over all weight entries, evaluated at the original weights. Any locally valid basin condition must be stated and verified for forward initialization.
  • A training extension must keep weights inside the stated norm class by explicit projection and account for accumulated gradient error. Charge O(Lm) activity/dual storage and O(TLm^2) arithmetic per dense sample, as well as parallel round complexity; local updates are not automatically fewer FLOPs than BP.

What would settle it?

A nonlinear finite-time gradient-error bound with explicit stable stepsizes and depth dependence, or a bounded-weight counterexample obstructing a proposed uniform rate. Distinguish linear wavefront arrival from settling to an epsilon-accurate gradient.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

The paper proves linear equilibrium convergence and analyzes linear stability/wavefronts; its nonlinear comparisons use finite inference empirically. Its resource comparison is retained as a baseline. This residual-tanh uniform-error benchmark is an editorial specialization, not a claim that all predictive-coding convergence is open. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Seely and Gould — Augmented Lagrangian Predictive Coding ↗partial theory · source checked 2026-11-20

    HTML Sections 3–5 and Appendix structure checked: Algorithm 1 uses T−1 primal-dual steps and a final primal step; linear equilibrium convergence, modewise stability and linear wavefront calculations already exist. Nonlinear finite-inference training is empirical; Appendix F includes a resource comparison.

  2. Sakana AI — Augmented Lagrangian Predictive Coding technical report ↗research agenda · source checked 2026-11-20

    Primary technical overview checked in the lab review: local PI-control interpretation and deep-network credit propagation experiments. The experimental wavefront is not a uniform nonlinear convergence theorem.