There is a failure mode in retrieval-augmented systems that nobody puts in the demo. You ask a question, the system embeds it, pulls the top ten chunks, stuffs them into a prompt, and the model writes a fluent answer built on whichever ten chunks happened to score highest.
If the answer needed an eleventh chunk, you do not find out. You get the same confident paragraph either way.
The fix is not a better embedder. It is letting something look again — and that is what the RLM does.
Two components, clearly separated
The Sidecar is a fleet agent. It runs where the compute is, reports what it has, and serves inference requests. Fantom MCP Server registers as one of the masters a sidecar reports to.
The RLM is a retrieval language model — a dedicated model whose job is not to write prose but to gather. It iteratively calls code-search tools until it has the exact code needed to answer, then hands back the evidence it collected plus a brief draft. A separate synthesis step turns that into the final cited answer.
The split matters. The gathering model is optimised for tool use and can be small and fast. The synthesis model is optimised for writing. Asking one model to do both is how you get an answer that is either badly researched or badly written.
How the fleet reports in
The sidecar initiates the connection, not the server — which is the right way round when the compute lives on workstations that come and go.
The wire protocol is small on purpose. A sidecar opens a WebSocket and sends register with its address, hostname and containers. After that it sends heartbeat frames carrying containers, active request counts and a status blob. The master replies registered, and can push command and config frames back. If the WebSocket is unavailable, the same exchange works over HTTP endpoints for heartbeat, poll and result.
Fantom MCP does not currently issue work commands over this channel — its actual inference calls go over HTTP. What it does do is mine every heartbeat for capability data, so the routing layer always knows which roles are actually being served right now.
One detail worth naming because it was learned the hard way: that registry is rewritten on every heartbeat, read-modify-write, into a config file that also holds search settings. An earlier version returned an empty object on a torn read, so a single corrupt read could persist only the sidecar list and wipe everything else — including the backup. Reads now recover from a sibling backup and writes are atomic. Fleet code touches config far more often than anyone expects.
The gathering loop
Here is what actually happens when a question arrives.
The model builds a plan, issues tool calls, and the results are gathered with bounded concurrency. Evidence blocks are split, merged and budgeted against the context window — which is read dynamically from the serving endpoint rather than hardcoded, because the same loop runs against different models with different limits.
If what came back is not sufficient, it goes again. That is the entire difference between this and one-shot retrieval: the system is allowed to notice it does not have enough.
A recent optimisation is a good illustration of where the real cost sits. Rather than re-embedding for every widening step, the query is embedded once and the search widens to sibling chunks in parallel, skipping the reranker on that widening pass. The expensive part of "look again" was never the looking — it was redundantly re-encoding the same question.
Failure is a first-class path
Every endpoint in this chain belongs to a machine that might be asleep, mid-reboot, or busy. So the rule throughout the RLM path is blunt: any endpoint error returns null and the caller falls back. It never throws.
That sounds unambitious. In practice it is what makes the feature usable on a real fleet, where "the RLM host is offline" should degrade the answer, not produce a stack trace in a controls engineer's editor. The default fallback mode is local-only, and the gathering step is an enhancement to retrieval rather than a dependency of it.
Capacity without hardware
The last piece connects this to the cloud. A sidecar can route a role to OpenRouter instead of its local model, and Fantom MCP treats each such pairing as a virtual container — a separate logical provider, named on the <PCName>-OR-<Role> pattern, that represents real serving capacity with no GPU and no VRAM behind it.
This is how embedding fans out across local GPUs and cloud at the same time, and how reranking gets a backend at all on a fleet whose machines are busy doing other work.
Two properties of that arrangement are worth stating plainly:
- Discovery goes over HTTP, not the heartbeat. The heartbeat carries a trimmed snapshot that does not include virtual container data. A system that mined the heartbeat for it would silently find nothing, forever, and look like it was working.
- The API key never touches Fantom MCP. The sidecar holds it. Fantom only ever names a provider and a model. Nothing in the discovery path reads, stores, logs or transports a key.
How those providers are then chosen, gated and budgeted is the subject of the next post.
Fantom MCP Server is source-available at github.com/Project-SandStar/FantomMcpServer.