← All AI/ML open problems

Multi-agent communication & games · Embeddings & representations · Neural architecture & complexity / Open question

Learning to Communicate in Hidden States

Passing hidden states can save decoding work, but successful communication requires the receiver to interpret the sender's representation and preserve information relevant to its own task. Efficiency payoff: Save decode/prefill work while paying for alignment, adapter training and message bandwidth.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-19

Seed catalog · Showing the bundled record.

Problem statement

Let two agents observe oi=Aiz+ξio_i=A_i z+\xi_i for shared z∼N(0,Ik)z\sim N(0,I_k), independent Gaussian noise and bounded, well-conditioned AiA_i, encoded as frozen neural states hi=fi(oi)h_i=f_i(o_i). The receiver predicts u⊤zu^\top z for a known task direction ∥u∥2=1\|u\|_2=1, using its own state and an rr-coordinate, pp-bit message from the sender. From mm paired training episodes, derive sample, message-bit and adapter-compute bounds for squared risk within ε\varepsilon of the Bayes predictor observing both states, including unknown cross-model alignment.

Assumptions and scope

  • Fix the frozen encoders, state dimensions, accessible layer, noise covariance bounds and which pairs/directions are observed during training; require a quantitative information-preservation condition on the states. Encoders and alignment data are not free oracle knowledge.
  • Use an explicitly bounded neural adapter/decoder class and name its initialization, optimizer and quantizer before a training theorem. The receiver has no access to the sender's original observation at deployment.
  • Charge r times p payload bits, quantizer scales, adapter parameters and local computation. Compare text and latent protocols under equal resources: an unrestricted discrete code can serialize the same finite-bit vector, so latent form alone gives no information-theoretic separation.

What would settle it?

A finite-data risk/communication/computation bound for the stated two-agent observation model and a concrete learned protocol, with an alignment or bit-budget lower bound. Feature reconstruction or a probe score alone does not certify task-risk preservation.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Heterogeneous KV transfer, compressed cache exchange and latent-identification frameworks already exist. The new target is a costed finite-training guarantee for a concrete observation/decision problem; it remains an editorial benchmark specialization rather than a published closed-form conjecture. Black-box APIs without hidden-state access are a different interface. This is an editorial, source-motivated research target. The preliminary review through 2026-11-19 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Liu — Beyond Tokens: A Unified Framework for Latent Communication in LLM-Based Multi-Agent Systems ↗review · source checked 2026-11-19

    HTML framework and research agenda checked: distinguish hidden states, embeddings and KV caches, cross-model alignment and quantized transport. A vector's raw bit capacity is not its task information or a same-budget advantage over text.

  2. Chen et al. — See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents ↗related algorithm · source checked 2026-11-19

    Primary October abstract checked: trained cross-model KV alignment and experiments with shared and separate observations. Existing heterogeneous communication is a baseline; no checked finite-sample minimax risk guarantee is claimed here.

  3. Zheng et al. — Thought Communication in Multiagent Collaboration ↗partial theory · source checked 2026-11-19

    Abstract checked: latent-variable identification and sharing-structure guarantees under the paper's assumptions, plus empirical collaboration. Those assumptions and proofs require fuller audit before importing an identifiability theorem; the catalogue target concerns finite training and charged communication.

  4. Rossi, Raghunath and Wu — Latent Cache Flow: Model-to-Model Communication Without Text ↗related algorithm · source checked 2026-11-19

    Abstract checked: compressed jointly translated KV caches, including different-context communication. Adapter size, transport and task performance must be charged separately in a theoretical comparison.