GerbilGerbil

Getting Started

Gerbil gets you running a local LLM in just a few minutes. Install the package, load a model, and generate your first response, all on your own machine.

Installation

One-off usage, try without installing:

Terminal
npx @tryhamster/gerbil "Write a haiku about coding"

Global install, use gerbil command directly:

Terminal
npm install -g @tryhamster/gerbil

Local install, for programmatic use in your project:

Terminal
npm install @tryhamster/gerbil
Note: CLI examples on this site use gerbil and assume global install. Substitute npx @tryhamster/gerbil if using without installing.

Quick Start

One-liner API

The simplest way to use Gerbil. No setup required:

index.ts
import gerbil from "@tryhamster/gerbil";


const text = await gerbil("Explain recursion in one sentence");
console.log(text);
// → "Recursion is when a function calls itself to solve smaller instances of the same problem."

Engine API

For more control, create a WebGPUEngine directly. It downloads safetensors from the HuggingFace Hub and caches them in IndexedDB on first run:

standalone.ts
01import { WebGPUEngine } from "@tryhamster/gerbil/gpu";
02

03// dtype "auto" picks int4 on mobile, the repo's native precision on desktop.
04const engine = await WebGPUEngine.create({
05  repo: "mlx-community/Qwen3.5-0.8B-4bit",
06  dtype: "auto",
07});
08

09// Generate text
10const result = await engine.generate("Write a haiku about coding", {
11  maxTokens: 100,
12  sampling: { temperature: 0.8 },
13});
14

15console.log(result.text);
16console.log(`Speed: ${result.tokensPerSecond.toFixed(1)} tok/s`);
17

18// Clean up when done
19engine.destroy();

In the browser

In a React app, the one-liner is the useEngine hook from @tryhamster/gerbil/hooks. It lazy-loads the model, streams into completion, and needs no model argument:

Chat.tsx
import { useEngine } from "@tryhamster/gerbil/hooks";


function Chat() {
  const { complete, completion, isGenerating } = useEngine();
  return (
    <button onClick={() => complete("Write a haiku about coding")} disabled={isGenerating}>
      {completion || "Generate"}
    </button>
  );
}

Outside React (vanilla JS / Node), drive the native WebGPUEngine directly, a lean bundle that runs the same code across desktop and mobile. The repo is optional (omit it for a device-aware default):

browser.ts
import { WebGPUEngine } from "@tryhamster/gerbil/gpu";


// Zero-config: no repo → a device-aware default. Or pass any repo.
// Downloads + caches the model in IndexedDB on first run.
const engine = await WebGPUEngine.create();


const { text } = await engine.generate("Write a haiku about coding");
console.log(text);

See the Browser documentation for chat, vision, and embeddings.

Streaming

Stream responses token by token:

streaming.ts
import { WebGPUEngine } from "@tryhamster/gerbil/gpu";


const engine = await WebGPUEngine.create({
  repo: "mlx-community/Qwen3.5-0.8B-4bit",
});


for await (const token of engine.stream("Tell me a story")) {
  process.stdout.write(token);
}


engine.destroy();

Thinking Mode

Qwen3 models reason before answering. The chain-of-thought is separated out for you on result.thinking, so you can show or hide it:

thinking.ts
const result = await engine.generate("What is 127 × 43?");


console.log("Thinking:", result.thinking);
// → "127 × 43 = 127 × 40 + 127 × 3 = 5080 + 381 = 5461"


console.log("Answer:", result.text);
// → "5461"

Structured JSON Output

Get structured data with Zod schema validation:

json.ts
import { json } from "@tryhamster/gerbil";
import { z } from "zod";


const person = await json("Extract: John is 32 and lives in NYC", {
  schema: z.object({
    name: z.string(),
    age: z.number(),
    city: z.string(),
  }),
});


console.log(person);
// → { name: "John", age: 32, city: "NYC" }

Embeddings

Generate embeddings for semantic search and RAG with the native EmbeddingGemma-300M model (dim 768, ~173 MB). Load it with embedding: true; embed() returns a unit-L2-normalized vector:

embeddings.ts
import { WebGPUEngine } from "@tryhamster/gerbil/gpu";


const engine = await WebGPUEngine.create({
  repo: "mlx-community/embeddinggemma-300m-4bit",
  embedding: true,
});


// EmbeddingGemma is asymmetric — tag queries vs. documents.
const query = await engine.embed("how do I cache a model?", { taskType: "query" });
const doc = await engine.embed("Models are cached in IndexedDB.", { taskType: "document" });


// Cosine similarity is just a dot product (vectors are L2-normalized).
const score = query.reduce((sum, q, i) => sum + q * doc[i], 0);

Vision AI

Vision runs on Qwen3.5-0.8B's own built-in ViT tower (no separate vision model). Build the engine with enableVision: true and call describeImage():

vision.ts
01import { WebGPUEngine } from "@tryhamster/gerbil/gpu";
02

03// The ViT tower (~192MB) ships inside the lean Qwen3.5 4-bit repo.
04const engine = await WebGPUEngine.create({
05  repo: "mlx-community/Qwen3.5-0.8B-4bit",
06  enableVision: true,
07});
08

09// Pass a data/http URL or pre-decoded RGB pixels.
10const { text } = await engine.describeImage(
11  { pixels, width, height },
12  "What's in this image?"
13);
14

15console.log(text);
16// → "A golden retriever playing fetch in a sunny park..."

See the Vision documentation for more details.

Text-to-Speech

Synthesize speech on-device with native Kani-TTS-2. Call engine.speak() and it returns raw 22.05 kHz mono PCM:

tts.ts
01import { WebGPUEngine } from "@tryhamster/gerbil/gpu";
02

03const engine = await WebGPUEngine.create({
04  repo: "nineninesix/kani-tts-450m-0.2-ft",
05});
06

07const { pcm, sampleRate } = await engine.speak(
08  "Hello from Gerbil! This runs entirely on-device.",
09);
10

11// pcm → Float32Array mono, sampleRate → 22050

See the Text-to-Speech documentation for voices and controls.

Speech-to-Text

Transcribe audio on-device with native Moonshine. It takes raw 16 kHz mono PCM (Float32Array), runs an encoder-decoder ASR pass, and returns the full transcript:

stt.ts
01import { MoonshineSTT } from "@tryhamster/gerbil/gpu";
02

03const stt = await MoonshineSTT.create(); // UsefulSensors/moonshine-base
04

05// pcm must be 16 kHz mono Float32 (resample mic/file audio first).
06const { text, audioSeconds } = await stt.transcribe(pcm);
07

08console.log(text); // "Hello world"

See the Speech-to-Text documentation for details.

Using Any HuggingFace Model

Load any compatible model from HuggingFace:

custom-model.ts
import { WebGPUEngine } from "@tryhamster/gerbil/gpu";


// HF repo id (the lean 4-bit Qwen3.5 checkpoint)
const engine = await WebGPUEngine.create({
  repo: "mlx-community/Qwen3.5-0.8B-4bit",
});


// Faster/smaller alternative: LFM2.5-350M (~199MB)
const fast = await WebGPUEngine.create({
  repo: "LiquidAI/LFM2.5-350M-MLX-4bit",
});

💡 Pro Tip: Preload Models

Create the engine during app initialization so weights are downloaded, compiled, and cached in IndexedDB before the user's first prompt. Reuse the same instance afterwards:

preload.ts
import { WebGPUEngine } from "@tryhamster/gerbil/gpu";


// Warm up at startup — downloads + caches the model.
const engine = await WebGPUEngine.create({
  repo: "mlx-community/Qwen3.5-0.8B-4bit",
  onProgress: (loaded, total, message) =>
    console.log(message, total ? loaded / total : 0),
});


// Subsequent prompts reuse the same warm engine — instant.
await engine.generate("Hello!");

See the Browser documentation for caching details.

Make It Yours

Once you're generating, two features turn Gerbil into your own specialist. Capture turns the AI calls your app already makes into a task-tagged dataset, recorded locally, at the source, so every call you paid for becomes a labeled example. Each tag becomes a clean dataset you can train a small specialist from, and nothing leaves your machine until you choose to. Read the Capture docs.

Adapters (flavors) let you cache one heavy base model once, then hot-swap small LoRA flavors on top, megabytes, not gigabytes. One base powers many skills: a support agent, a refunds classifier, a code reviewer, each a tiny delta that ships over-the-air and runs offline on-device. Read the Adapters docs.

Next Steps