Dictation clean-up that hears “no wait”

Problem

People don't dictate the way they type. They say “um”, they repeat a word, and they change their mind halfway through: “book the 7pm flight, no wait, the 9pm one.” A speech model writes all of that down faithfully. So a dictation app needs a second, small language model that turns the transcript into the sentence you meant before it lands at your cursor.

Clean-up models are good at the easy part: dropping fillers, fixing capitals and punctuation. The hard part is the self-correction. Most tidy the text and keep the item the speaker retracted, so you get something like “Book the 7pm flight, no wait, the 9pm one.” with nicer commas. For MacWispr, my on-device dictation app for macOS, the fix also had to be small and fast enough to run on every utterance without leaving the Mac.

How it works

1. Speech becomes a raw transcript.MacWispr transcribes on device with an open speech model (Qwen3-ASR or Parakeet). The result is lower case, mostly unpunctuated, and keeps every “um” and every change of mind.
2. The transcript goes into a plain prompt.The model was trained on a raw completion format, not a chat template: the transcript after ### Input:, then ### Output: for the model to continue.
3. A 350M model rewrites it.Decoding is greedy (temperature 0), so the same input always gives the same output. The model removes fillers and stutters, writes numbers as digits, fixes light grammar, and, when you correct yourself, drops the retracted item and keeps the replacement and the rest of the sentence.
4. The clean text is pasted.For in-app testing I ran the model as a small local server that loads it once, warms it up and answers each request with the cleaned text and its latency. Inputs under three words pass through untouched.

### Input:
set the oven to three fifty no wait three seventy five for the lasagna

### Output:
Set the oven to 375 for the lasagna.

That example, prompt and output, is from the model card. The base is a 5-bit MLX build of Liquid AI's LFM2.5-350M that a community author had already trained for dictation clean-up (“Sotto”). It was strong at light clean-up and weaker at self-corrections, so I didn't train from scratch. I taught the one skill it was missing.

The fine-tune: a small adapter, then fused

LoRA (low-rank adaptation) leaves the model's weights frozen. Beside each chosen weight matrix it trains two thin matrices: one squeezes the layer's input down to 8 numbers, the other expands them back out. Their product is a correction to the frozen weight. Because the squeeze is so narrow, there is very little to train, which matters when the base is only 350M parameters and the skill is narrow.

LFM2.5-350M has 16 layers: ten short-convolution layers and six attention layers. I put adapters on the last eight, four of each kind. In attention layers they sit on the query, key, value and output projections; in convolution layers on the input and output projections; and in every adapted layer on the three feed-forward weights. After training, I fused the adapter into the weights so the model loads directly in MLX in Python or mlx-swift-lm in Swift, with no adapter step. The Hugging Face repo has both: the fused model and the standalone adapter (6 MB) in lora-adapter/.

Every value in the table comes from the published adapter_config.json and the adapter file itself. I kept the learning rate low: the base was already good at most of the job, and a small nudge is less likely to make it forget light clean-up.

Training data, and keeping the test honest

I didn't have a corpus of people correcting themselves, so I generated one. An LLM wrote messy-transcript and clean-text pairs on fresh topics, and an LLM quality check filtered out the bad ones. The pairs fall into three groups, and each group teaches something different.

The first pair is the model card's. The other two are illustrative; the training data itself isn't published.

The third group matters most for trust. A model trained only on corrections can learn a shortcut: whenever it hears “no” or “wait”, cut something. Preserve examples make that shortcut expensive.

Then the leakage gate. Synthetic data makes it easy to fool yourself: if the test sentences share topics or phrasing with the training sentences, the model can score well by recognising them. So the held-out set of 94 items was generated on topics kept apart from training, and I checked it against the training data before scoring. Overlap: 0 of 94.

What was measured

An LLM judge graded the base and the fine-tune on the held-out items. The model card reports three categories, 36 items in all. Each square below is one item.

• 13/16held-out course-corrections, up from 10/16
• 7/8preserve items left intact, up from 6/8
• 50–100 msper utterance on Apple Silicon
• 0.4%of the weights trained: 1.5M of 350M

Limits

• The test is small. Three more passes out of 16 is a real direction but not a precise size; a different sample could move it by a couple of items either way. No error bars or repeated evaluations are reported.
• An LLM judged the outputs, not people, and the card doesn't name the judge model.
• The training pairs and the held-out set are not published. The public bench/polish_finetune/ folder has the in-app test server and a follow-up formatting script, not the course-correction data builder, so you can run the model but can't yet re-run the scores.
• The 5-bit base occasionally corrupts a rare word: “simmer” came out as “smear”. A higher-precision base would likely help.
• Long corrections are sometimes over-shortened, with a trailing clause dropped.
• At 350M parameters it only cleans text. It does not answer questions found in the transcript, and it isn't a general assistant.
• A follow-up script in the repo continues training from this checkpoint to teach bullet and numbered lists. I haven't published results for it. MacWispr's 1.2.4 beta bundles a separate Qwen3.5-0.8B clean-up model; that is a different experiment and is not reported here.

Credits and links

The base is the community Sotto 5-bit MLX clean-up build (juanquivilla/sotto-cleanup-lfm25-350m-mlx-5bit) of Liquid AI's LFM2.5-350M, used under the LFM Open License. Training and fusing use Apple's open-source MLX and mlx-lm. I have no affiliation with Liquid AI or Apple. The data pipeline, fine-tune and evaluation are mine.

Run it

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("vasanth009/LC-lfm2.5-350m")
raw = "set the oven to three fifty no wait three seventy five for the lasagna"
prompt = f"### Input:\n{raw}\n\n### Output:\n"
out = generate(model, tok, prompt=prompt, max_tokens=64, sampler=make_sampler(temp=0.0))
print(out.split("###")[0].strip())   # Set the oven to 375 for the lasagna.

Read more

• Model on Hugging Face: fused weights, the standalone LoRA adapter, usage and the evaluation table.
• Training scripts: the local test server MacWispr calls, and the follow-up formatting fine-tune.
• MacWispr on GitHub: the on-device dictation app this was built for, with its speech-model benchmarks.
• The base model: the Sotto clean-up build this fine-tune starts from.