← All AI/ML open problems

Embeddings & representations · JEPA & world models / Open question

Learning a Shared Geometry

Images, language and other observations describe overlapping aspects of the world. Shared embeddings should preserve that common information without pretending every modality reveals the same facts. Efficiency payoff: Share useful modality features and reduce embedding dimensions with a downstream-risk guarantee.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

Let views satisfy xv=Avz+Bvsv+ξvx_v=A_vz+B_vs_v+\xi_v, with shared z∼N(0,Ik)z\sim N(0,I_k), independent Gaussian nuisances/noise and bounded, identifiable loading matrices. Train finite-width nonlinear encoders from partly missing paired views, then align their whitened outputs using mm paid pairs. Bound inf⁡O⊤O=IE∥e1(x1)−Oe2(x2)∥2\inf_{O^\top O=I}\mathbb E\|e_1(x_1)-Oe_2(x_2)\|^2 together with excess risk for bounded linear functions of zz, relative to the noise-limited oracle. Expose finite-training error, view overlap and alignment-sample costs.

Assumptions and scope

  • Fix the latent model, observation maps, paired/unpaired data access and precisely which latent functions both modalities can identify.
  • State encoder objectives, dimension, normalization and optimization procedure; distinguish population optima from the encoders actually learned.
  • Define agreement up to an explicit transformation group and charge any alignment pairs. A similarity score alone need not certify semantic equivalence or causal structure.

What would settle it?

A finite-sample learning-and-alignment theorem with quantitative transfer error, plus necessity or ambiguity results under incomplete views and unequal information.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Balestriero and LeCun already claim a JEPA alignment theorem under good-view assumptions and bounds otherwise. Content/style identifiability and approximate sufficient representations also have theory. The residual target combines imperfect observation access, finite-data optimization and a charged alignment procedure. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Balestriero and LeCun — Proving and Improving the Platonic Representation Hypothesis with JEPAs ↗partial theory · source checked 2026-11-19

    Primary abstract checked: an October 2026 alignment theorem under good-view assumptions, with a bound otherwise and empirical alignment scaling. The full optimization and observation assumptions were not independently audited.

  2. von Kügelgen et al. — Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style ↗partial theory · source checked 2026-11-19

    Abstract checked: invariant content is identifiable up to an invertible map under the latent augmentation model; this does not identify every content coordinate.

  3. Lin and Mei — A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics ↗partial theory · source checked 2026-11-19

    Abstract checked: contrastive losses yield approximately sufficient encoders and downstream risk bounds, including augmentation error.

  4. Schechter Vera et al. — EmbeddingGemma 2: Unified and Efficient Multimodal Representations ↗related algorithm · source checked 2026-11-20

    November revision abstract and Google DeepMind page checked: selective encoder loading, token budgets, truncatable dimensions and quantization. These motivate a task-aware resource/risk guarantee.