Many tiny specialists.One shared model.All answering at once.
Train a small specialist for every AI job in your app, then run them together on one base model, on one GPU or right in the browser. A specialist stops being a whole model and becomes a few megabytes. You get one base loaded once, dozens of experts riding it, and the same answers you would get running each one alone.
Three ways to mix specialists
The same word points at three different mechanisms. What gets routed, and where the specialists actually live.
MoE
Mixture of Experts
Routes tokens among experts baked inside one network. The experts are trained together with the base, so no single expert is a thing you can name, test, or ship. Adding one means retraining.
MoA
Mixture of Agents
Layers of whole models take turns: each reads the last layer's text and proposes, then a combiner merges them. Composable after the fact, but every member is a full model, so the memory and the runs both add up.
MoTA
Mixture of Tuned Adapters
Routes task lanes among small tuned adapters over one frozen base. Every member is a named, testable file you can add or replace for megabytes, and they all run in a single batched pass.
One GPU. One pass. Every specialist answers at once.
Requests fan into decode lanes that move in lockstep. Each lane wears its own specialist, and all of them ride one shared base on one GPU.
Every lane is verified: its output is exactly what the same prompt and specialist would produce running alone. The fan-out finishes in about the time of the slowest lane, not the sum of the lanes.
Weight memory barely moves as you add specialists
One base plus N small adapters against N full models, on a log scale. The base is about 450 MB; a specialist is a few MB.
Data table
| N | Base + adapters | N full models | Ratio |
|---|---|---|---|
| 1 | 454 MB | 450 MB | 1.0x |
| 2 | 458 MB | 900 MB | 2.0x |
| 4 | 466 MB | 1.8 GB | 3.9x |
| 8 | 482 MB | 3.5 GB | 7.5x |
| 16 | 514 MB | 7 GB | 14.0x |
| 24 | 546 MB | 10.5 GB | 19.8x |
| 32 | 578 MB | 14.1 GB | 24.9x |
Each new specialist costs about 4 MB
The same numbers, felt as a race. Watch what every added specialist puts on each side.
Ship a fine-tune as a full model and it pays the whole base again. Ship it as an adapter and it costs a few megabytes on the base that is already loaded. Thirty-two different specialists fit in 578 MB of weights.
Running them together multiplies throughput
Aggregate tokens per second as concurrent streams grow. Every point is verified against single-stream decode, measured and logged per run.
A faster tier that relaxes the exactness check reaches even higher, but it can change outputs, so it stays opt-in and never the default. The desktop GPU easing past 16 streams is real and reported as measured.
Declared like a model, run like a function
Two calls. registerAdapter loads a specialist once by name; generateBatch takes your prompts plus the specialist for each one, and mixes specialists and the plain base in the same pass.
01import { WebGPUEngine } from "@tryhamster/gerbil/gpu";02
03const engine = await WebGPUEngine.create({04 repo: "mlx-community/Qwen3.5-0.8B-4bit", // the shared base, loads ONCE05});06
07// Register each specialist once, by name. The small factors are packed08// into shared GPU buffers, ready for any lane to use.09await engine.registerAdapter("classifier", "hf:acme/ticket-classifier-lora");10await engine.registerAdapter("redactor", "hf:acme/pii-redactor-lora");11await engine.registerAdapter("router", "hf:acme/handoff-router-lora");12
13// One batched pass. Each prompt names its specialist; null runs the14// bare base. Every lane returns exactly what it would return alone.15const results = await engine.generateBatch(16 [classifyPrompt, redactPrompt, routePrompt, draftPrompt],17 {18 adapters: ["classifier", "redactor", "router", null],19 sampling: { temperature: 0 },20 },21);What one pass does
plan · route · reduce
One request fans out to a lane per specialist, they decode together over the one shared base, and their outputs join into a single record. Same answers, zero quality tradeoff. Available in the Node SDK today, with the browser next.
Batched equals running alone. Bit for bit.
A greedy lane in a 32-lane batch produces the same token ids as the same prompt decoded on its own. Not close. Identical.
Every batched kernel is built from the single-sequence one it came from, so only the addressing changes and the math never does. A lane in a busy batch returns exactly the tokens it would return running alone. Verified token for token, with mixed adapters and plain-base lanes in the same pass.
Switching specialists moves megabytes, not the model
What has to move to change what the engine is good at. Log scale.
A swap rebuilds only the small specialist factors; the base weights never leave the GPU. Swapping one specialist for another and back restores the base output exactly. And with per-lane batching you often skip swapping entirely: different lanes just wear different specialists in the same pass.
The shrinking cost of a token
Each cell is one GPU dispatch in a single decode step. Merging kernels removed 93 of them, output unchanged.
On some GPUs the per-dispatch overhead, not the math, is what bounds decode speed. Every merged cell is host and driver work that stops happening on every single token, and every merge keeps the output byte-for-byte the same.
Time to first token, cut in half
A 512-token prompt on a laptop GPU. Reading the whole prompt in one wide pass, instead of a token-shaped loop, gets the first word out sooner.
Prompt throughput on the same input rose by well over half, and under mixed load the same change lifted requests per second while cutting the median wait to first token by roughly a third to a half. Same output as the previous path.
A specialist is a file you own
Tune once, get an artifact measured in megabytes, keep it forever, and compose it with the ones you tune next.
The weights are the value. A tuned behavior stops being a subscription to someone else's model and becomes a file in your repo: diff it, roll it back, test it on its own, ship it to a browser, and stitch it into a swarm next to the specialists you tune later.
On the user's device, the next AI moment costs nothing
Rough cost of 1,000 product AI moments, about 300 generated tokens each. Order of magnitude, log scale.
On-device costs the user a sliver of electricity and a one-time cached download. What it changes is product design: classification, extraction, and drafting can run on every keystroke and every row, because nobody is metering the calls.
What each approach routes, trains, and costs
MoTA members are pre-tuned adapters, not co-trained experts and not full models. Running them together is just batched decode.
| MoE | MoA | MoTA | |
|---|---|---|---|
| Unit of specialization | One expert inside a single network | A whole language model | A tiny adapter on a shared base |
| Add one more | Retrain the whole network | One more full model, one more full run | A few megabytes, registered once |
| Memory for N specialists | One fixed model | N full models | One base plus a few MB each |
| Can you name a member | No, the experts are opaque | Yes, each is a whole model | Yes, each is a named, testable file |
| How they run together | One model, nothing to coordinate | One after another, results combined | One batched pass, all lanes at once |
| Where it runs | Wherever the one model fits | Wherever N models fit | One consumer GPU, or the browser |
Best for jobs you can split up
Sorting and routing, pulling many fields out of one document, working through a pile of records, asking several specialists and keeping the best answer, agents that explore a few approaches at once. The win comes from splitting the work well, which is your call to make.
Not a frontier model
Thirty-two small specialists running at once do not add up to one big brain. Open-ended reasoning, long creative work, and broad general knowledge stay a job for a frontier model. For the many production jobs you can split up, a swarm of tuned specialists on your own hardware is faster, cheaper, private, and works offline.
Specialists are tuned ahead of time. Nothing trains while your app runs.
Every specialist you tune is a swarm member
Gerbil Tune trains specialists on a shared base, so every one you train today already fits in a swarm. Start with one, then run the whole desk together.