JEPA & world models · Learning dynamics & scaling / Open question
Prediction Without Representation Collapse
Predicting another learned representation admits trivial solutions. A training theory should explain how useful structure emerges and survives stochastic updates.
Seed catalog · Showing the bundled record.
Problem statement
Use paired views of a rank- Gaussian latent signal, two-layer tanh encoders and a linear predictor. Train the online encoder by minibatch squared embedding-prediction loss with stop-gradient target and update target weights by . Determine when persists and the learned embedding retains predictive information about , as noise, masking, batch size and vary. State any added variance regularizer explicitly.
Assumptions and scope
- The base algorithm is squared prediction loss with the EMA/stop-gradient update in the statement. LeJEPA is a separate comparison, not an interchangeable update; regularized variants must give the coefficient and empirical statistic.
- Fix layer widths, initialization, learning rates, loading-matrix singular-value bounds and the masking probabilities; report their effects rather than choosing an easy data family after training.
- Use both the covariance eigenvalue criterion and excess latent-prediction risk against the same-view oracle; nonzero variance alone is insufficient. Analyze beyond a local linearization around zero.
What would settle it?
A finite-time nonlinear stability/convergence guarantee with a nontrivial representation criterion and explicit failure boundaries, or a lower bound showing which data or regularization condition is needed.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Drive vs. Decay already analyzes early linearized JEPA stability and mode growth; LeJEPA supplies a regularized, theory-motivated objective without teacher–student heuristics. The additional target is finite-batch nonlinear trajectory control and useful-rank persistence. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- De Melo Costa et al. — Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures ↗partial theory · source checked 2026-11-19
Primary abstract checked: early gradient-flow linearization near collapse, approximate spectral decoupling, rank-emergence predictions and a residual predictor. This is not a global finite-batch nonlinear guarantee.
- Balestriero and LeCun — LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics ↗partial theory · source checked 2026-11-19
Primary abstract checked: isotropic-Gaussian target embeddings under the stated theory, SIGReg and a predictive objective without stop-gradient or teacher–student heuristics. Collapse-avoidance objectives already exist.
- Cui et al. — A Generalization Theory for JEPA-Based World Models ↗partial theory · source checked 2026-11-19
Primary abstract checked: action-conditioned co-occurrence factorization, finite-sample prediction/planning-regret bounds and a latent-dimension trade-off. A prediction-to-planning theory is already available under the paper's framework.