Autocomplete & Rewrite

Gerbil can add inline autocomplete — the gray ghost text that completes what you're typing — and one-call rewrites to any input in your app, with the model running on the user's own device. The useAutocomplete hook owns the debounce, in-flight, and stale-response bookkeeping so your component only has to render the suggestion and handle accept and dismiss.

Ghost text with useAutocomplete

The useAutocomplete hook is the easy path. Feed it each keystroke through onInput, and it schedules a debounced request, cancels stale ones, and exposes the current suggestion to render. Wire Tab to accept() and Esc to dismiss(), and render the suggestion as gray text overlaid behind the textarea:

01"use client";
02
03import { useState } from "react";
04import { useAutocomplete } from "@tryhamster/gerbil/hooks";
05
06export function GhostTextArea() {
07  const [text, setText] = useState("");
08  const { suggestion, isFetching, onInput, accept, dismiss } = useAutocomplete({
09    model: "mlx-community/Qwen3.5-0.8B-4bit",
10  });
11
12  function handleChange(e: React.ChangeEvent<HTMLTextAreaElement>) {
13    setText(e.target.value);
14    onInput(e.target.value); // debounced request + clears any stale suggestion
15  }
16
17  function handleKeyDown(e: React.KeyboardEvent<HTMLTextAreaElement>) {
18    if (e.key === "Tab" && suggestion) {
19      e.preventDefault();
20      setText((t) => t + accept()); // commit the ghost text
21    } else if (e.key === "Escape") {
22      dismiss();
23    }
24  }
25
26  return (
27    <div style={{ position: "relative" }}>
28      {/* Ghost layer: the typed text (invisible) + the gray suggestion */}
29      <div
30        aria-hidden
31        style={{
32          position: "absolute",
33          inset: 0,
34          whiteSpace: "pre-wrap",
35          pointerEvents: "none",
36          color: "transparent",
37        }}
38      >
39        {text}
40        <span style={{ color: "#9ca3af" }}>{suggestion}</span>
41      </div>
42
43      <textarea
44        value={text}
45        onChange={handleChange}
46        onKeyDown={handleKeyDown}
47        placeholder="Start typing…"
48        style={{ position: "relative", background: "transparent" }}
49      />
50      {isFetching && <span>…</span>}
51    </div>
52  );
53}

The hook shares the same reference-counted engine as the other hooks, so it never uploads the model to the GPU a second time. Tune its behavior with a few options: debounceMs (default 550), minChars (default 8 — no request fires below this trimmed length), maxTokens (default 16), temperature (default 0.3), and autoLoad (default false — the model loads on the first request). The return also carries isReady, load, and error for explicit loading and error UI.

The lower-level autocomplete call

When you want to drive autocomplete yourself — debouncing in your own state, or calling it outside React — use engine.autocomplete(prefix, opts) directly, or useEngine().autocomplete for the same method on the shared engine. It returns the cleaned continuation string:

01"use client";
02
03import { useEngine } from "@tryhamster/gerbil/hooks";
04
05function Editor() {
06  const { autocomplete, isReady, load } = useEngine({
07    model: "mlx-community/Qwen3.5-0.8B-4bit",
08    autoLoad: true,
09  });
10
11  async function suggest(prefix: string) {
12    if (!isReady) await load();
13    // Defaults: maxTokens 16, temperature 0.3, stop ["\n"], singleLine true
14    const continuation = await autocomplete(prefix);
15    return continuation; // e.g. " jumps over the lazy dog."
16  }
17
18  // …
19}

The options are maxTokens (default 16), temperature (default 0.3), stop (default ["\n"]), and singleLine (default true, which trims the suggestion at the first newline). The same method exists on a standalone engine via await engine.autocomplete(prefix).

Autocomplete runs in continuation mode: it continues the text you pass instead of answering it like a chatbot. Give it "The quick brown fox" and it returns " jumps over the lazy dog.", not a reply about foxes. The output is also cleaned up for inline use — wrapping quotes are stripped, an echoed copy of your typed text is dropped, and a single leading space is added unless the continuation hugs punctuation — so you can render it straight as ghost text.

Rewrite & tone

engine.rewrite(text, opts) rephrases a block of text and returns only the rewritten text — no preamble, no quotes, no commentary. Use it for "Improve writing" buttons, tone switchers, and tighten-this-up actions. Pass a tone such as professional, friendly, concise, or playful:

01"use client";
02
03import { useEngine } from "@tryhamster/gerbil/hooks";
04
05function ToneButtons() {
06  const { rewrite, isReady, load } = useEngine({
07    model: "mlx-community/Qwen3.5-0.8B-4bit",
08    autoLoad: true,
09  });
10
11  async function makeProfessional(draft: string) {
12    if (!isReady) await load();
13    return rewrite(draft, { tone: "professional" });
14    // "hey can u send the file" → "Could you please send me the file?"
15  }
16
17  async function makeFriendly(draft: string) {
18    return rewrite(draft, { tone: "friendly" });
19  }
20
21  // …
22}

For anything that isn't a single tone word, pass free-form instructions instead — they replace the tone instruction entirely, so you control exactly how the text is transformed:

01const tightened = await rewrite(paragraph, {
02  instructions:
03    "Rewrite this as two short bullet points. Output only the bullets, no preamble.",
04});

The remaining options are maxTokens (default 256) and temperature (default 0.7). The method is available both on the React hook via useEngine().rewrite and on a standalone engine via engine.rewrite. Either way you get a plain string back, ready to drop into the editor.