Back

↓

Bridging models’ internal states

Models compute far more than they output. We built a low-loss channel to allow models to cooperate by directly sharing internal states.

To produce one token, roughly one word, a language model builds more than a hundred hidden vectors. About a million numbers, describing everything it currently holds about the problem in front of it. Around two megabytes of internal state. Then it picks one entry out of a vocabulary of around 150,000 and throws the rest away.

Seventeen bits leave the model. The two megabytes are never used again.

Every system that connects models together runs on those seventeen bits. Coding subagents, model councils, routing providers: text is the only channel any of them has[1] . Systems built to think in thousands of dimensions are effectively talking to each other through a keyhole.

Today we are coming out of stealth to say what we have been doing about that.

I think it's important to emphasize these have been on the rise lately; dynamic workflows are very buzzy, routing rises in prominence, claude and pplx use advising; the oai HF incident is all about model coordination, if you wish :)

maybe the emphasis goes to the "why this matters now" part, but then we need tp figure out the split without looking overly repetitive

What gets discarded is not spare capacity

The assumption is that the hidden state is scaffolding, and that the token is the finished product. Interpretability research from the past two years says otherwise.

Transformers plan ahead. Consider how a model chooses between "a" and "an." To get the article right it has to already know the noun, and it does: Hanna and Ameisen (ICLR 2026, arXiv:2604.12493) show that Qwen-3 carries a representation of "accountant" several tokens before writing it, and that this representation is what produces the "an." They also find the effect grows with model size, so larger models hold more that the text never shows.

Anthropic's interpretability team found the same mechanism running over a full line of verse in Claude 3.5 Haiku: the model settles on its rhyme before writing toward it (Lindsey et al., 2025).

A second result from the same group goes further. Their Jacobian lens identifies the concepts a model is poised to say at any moment, a set they call J-space: a small, shifting collection of unspoken words naming what the model is working with right now, neither an echo of the input nor a guess at the next token. A model can be asked to hold a concept there and will silently keep it while its visible output carries on unchanged. Their own summary is that these measurements uncover internal reasoning and reactions that never appear in the output (Gurnee et al., 2026).

So when one model writes a sentence for another model to read, what is lost is not formatting. It is the deep computation–the considerations behind the choice.

Why this matters

So far, AI has scaled along two axes.

Pre-training, scaling data and parameters, brought world knowledge. Scaling test-time compute through single-agent reinforcement learning brought reasoning and planning. Both have been flattening for a while.

The next axis for scaling is models coordinating with each other, and it has to be reached through multi-agent RL, which is a qualitatively harder problem than single-agent training. Efficient coordination requires efficient communication. Right now models coordinate by writing sentences to each other, losing almost everything they computed to produce those sentences.

That is the axis we are working on, and communication is the part of it we started with.

This doesn't seem right. Knowledge of the world? Still doesn't seem right.

definition from google: World knowledge encompasses the general information, facts, concepts, and cultural context shared by humans across geography, history, science, and society

I think it's meant to sound similar to "world models" which is a widely used term in AI

What we built

One model hands its hidden states to another through a small trained bridge, and the receiving model works with them directly. No output-type text passes between them, and neither model's weights are touched. The bridge is the only new part of the system.

It is also the company's name: Mostik means “little bridge”.

There are 16 of us: 12 PhDs and a Fields Medalist. We started four months ago.

We started with the geometry

How easily two models can be bridged is a question about the geometry of the spaces they compute in. So that is where we began: what those spaces look like, and how much structure any two models already share.

Both answers turned out to matter.

Alignment between models does not increase on its own as models get more capable, so a bridge has to be built rather than expected to emerge as models scale. Some alignment is there, however, and it varies in ways we can predict. The question is never whether a translation exists between two models, but rather how much work it takes to find.

The first result

The first thing we built is a bridge from a large model to a small one. We chose that pairing because deployments already want it: a large model that knows things, and a cheap model that does the writing. Our sender and receiver are GLM-5.2 (753B) and Qwen-3.5 (4B), a leading open-source model and a popular edge-device-class one.

The economics come from an asymmetry in how models run. Writing an answer happens one token at a time and costs a lot. Reading a prompt is one parallel pass and costs significantly less.

So, in these experiments the large model never writes. It reads the problem, its hidden states cross the bridge, and it stops. The small model does all the generation.

On its own, the small model scores well below the large one, and the distance between those two scores is the gap to close. With the bridge attached, the small model closes 50% of that distance, which lifts its own accuracy by 25%. On harder subsets, where the gap between sender and receiver is more pronounced, that uplift reaches 2x.

The same result reads a second way. Somewhere between our small model and our large one sits a mid-sized model that would score exactly what the bridged pair scores–and you could deploy that instead. Running the bridged pair costs 2.5x less compute than running such a model.

We also compared the setup against other standard ways of combining one or several models. Across those setups, the bridged system shows a clear Pareto improvement.

[FIGURE: Pareto comparison against standard multi-model baselines]

We compared our method, which allows models to communicate by sharing their internal states, with existing approaches for handing work from one model to another. In each case, the larger model worked through part of the problem before passing what it had produced to a smaller model. In the case of the traditional text hand-off, it passed the text it had written so far; with our latent hand-off, it passed the internal state it had built up while reasoning about the problem.

The latent hand-off performed better at every level of large-model compute we tested, improving performance by as much as 10 percentage points. The advantage was greatest when the hand-off happened very early, before the larger model had written anything. At that point, a text-based approach has nothing to pass along, while the latent state already contains information the model has developed from reading and processing the problem. As more compute is given to the larger model, the two approaches eventually converge toward large-model performance, but the latent approach provides a better performance–compute trade-off throughout.

Both models stay frozen

Our approach doesn’t alter either model. Both remain completely frozen; the bridge is trained separately and is the only part of the system that learns anything.

That matters for two reasons.

First, it tells us something about model representations. Two models built by different teams and trained on different data develop internal representations similar enough that we were able to translate between them without fine-tuning either model. We froze both models deliberately to make this a strict test: if useful information can pass between them without either model being adapted to the channel, then the bridge is exploiting a structure that was already present.

Second, it makes the approach practical to deploy. Many systems already use a powerful model as an advisor to a smaller, cheaper model. At key decision points, the advisor reads the conversation and writes out guidance for the smaller model. That requires paying for the expensive model to process the full context and then paying again for it to turn what it knows into words.

With our approach, the expensive reading still happens. But the expensive writing doesn’t have to. The latent channel can take the place of the tokens that previously carried the advice from one model to the other.

Observability

The best tool the field has for checking what a model is doing is reading its chain of thought. The paper that raised the alarm about losing that tool says as much itself: Korbak et al. (2025, arXiv:2507.11473) call CoT monitoring imperfect and fragile, and note that some misbehaviour passes straight through it.

The failures are already documented. A model can produce the same sentence several times while provably holding different things in mind, with the text showing no sign of the difference (Gurnee et al., 2026). Behaviours transmit through strings of generated numbers that look random and carry no visible trace of what they encode. And a model penalised for misbehaviour visible in its reasoning has a cheaper move available than becoming well behaved, which is to make it less visible.

Text is a record of what a model chose to say. It has never been a record of what the model did.

Getting past that means reading the computation directly, and reading the computation means understanding the geometry of representation spaces. That is the same research that makes a bridge between two models work at all. We are doing it either way, and we think it is where real observability has to come from.

There is a practical consequence specific to the channel. The representations the bridge learns decode: what crosses can be read back out, sampled during a run or inspected on demand, without interrupting either model. So a latent hand-off can be watched at the level where the computation actually happens.

Where this goes

The bridge is trained, which means gradients run through it, so it does not have to arrive after both models are finished. It can be present while one of them is still being trained. That is what we are working on now, using distillation and specialisation..

Distillation. A teacher's supervision normally arrives at the student's output layer: at best the student sees what the teacher was about to say at each position, not the computation behind it. A channel moves that supervision inside the generation, so the teacher guides the student while the answer is being produced rather than grading it once it exists. Early signals suggest a student trained that way stays closer to its teacher for the same budget.

Specialisation. Small models appear to pick up narrow skills more efficiently when they train with the channel open, and to keep the general ability they came in with. That points at a way of extending a large model by attaching new modules onto it instead of retraining the whole thing.

Both are early and in progress.

Lastly, nothing in the mechanism is particular to a pair, and the same channel that lets one model supervise another lets several train together, combining what each of them knows into a single interwoven system.

Connecting finished models is worth doing on its own, and it is what our results show works. A channel that is present during training means models can be built to work together from inception, which is a different and much larger question than whether two of them can be joined afterwards.

Both models stay frozen

Our approach doesn’t alter either model. Both remain completely frozen; the bridge is trained separately and is the only part of the system that learns anything.

That matters for two reasons.

Our approach doesn’t alter either model. Both remain completely frozen; the bridge is trained separately and is the only part of the system that learns anything.

That matters for two reasons.

Working with us

We are backed by General Catalyst.

If you serve models, the bridge fits the prefill and decode split you already run: decoding stays on a small model, part of the prefill comes from a larger one, and it arrives as latent state rather than as tokens. A second version lets the latents cross the wire, so the two models do not have to sit in the same facility.

If your product already has models handing work to each other, through councils, advisors, escalation or subagent workflows, that hand-off is what the bridge replaces, and it replaces it in place.

If you have a small model fine-tuned on your own data and have been choosing between it and whatever stronger model shipped last month, that is the choice we are trying to remove.

If you work on representation geometry, cross-model transfer, or interpretability for latent channels, we would like to hear from you either way.

[info@mostik.ai] · mostik.ai

WIRED has the first outside account of the company and the work: [WIRED LINK]

We are backed by General Catalyst.

If your product already has models handing work to each other, through councils, advisors, escalation or subagent workflows, that hand-off is what the bridge replaces, and it replaces it in place.

If you have a small model fine-tuned on your own data and have been choosing between it and whatever stronger model shipped last month, that is the choice we are trying to remove.

If you work on representation geometry, cross-model transfer, or interpretability for latent channels, we would like to hear from you either way.

[info@mostik.ai] · mostik.ai

WIRED has the first outside account of the company and the work: [WIRED LINK]

Talk to us.

info@mostik.ai

2026