Transcription

On-device speech recognition using WhisperKit and the Apple Neural Engine. No audio ever leaves your Mac.

How Transcription Works

Harvestry transcribes video audio using WhisperKit, an optimized implementation of OpenAI's Whisper automatic speech recognition (ASR) model built for Apple Silicon. The model runs directly on the Apple Neural Engine (ANE) — the dedicated machine-learning accelerator present in every M-series chip.

Key properties of the transcription engine:

Fully on-device. No audio or text is sent anywhere. The ANE processes everything locally.
Word-level timestamps. WhisperKit returns timestamps for every individual word, not just sentence or segment boundaries. These timestamps power the per-word highlight sync in the exported audio player.
Multilingual. Whisper detects the spoken language automatically, or you can name it yourself. Coverage is wide but accuracy varies by language — see Spoken Language & Translation.
Transcribe or translate. Output stays in the spoken language by default; Harvestry Pro can instead translate it into English in the same pass.
Background processing. Transcription runs asynchronously and does not block the UI. You can continue using Harvestry while transcription runs.

ℹ️

Apple Silicon required. The Apple Neural Engine is only available on M-series chips (M1 and later). Intel Macs are not supported.

Whisper Models

Five model sizes are available. Choose based on how you want to balance accuracy against speed and disk space.

Model	Disk Size	Speed	Accuracy	Best For
Tiny	~75 MB	Fastest	Basic	Testing; very short clips; clearly-spoken English where accuracy is less critical
Base	~150 MB	Very fast	Good	Good balance of size and quality for simple content; podcasts; meeting recordings
Small ★	~250 MB	Fast	Better	Recommended starting point; handles most lectures, accented speakers, and technical terms well
Medium	~770 MB	Moderate	High	Dense academic content; multiple speakers; stronger accents; non-English material
Large Turbo	~800 MB	Moderate	Best	Maximum accuracy; complex or highly technical lectures; situations where every word matters

All five models produce word-level timestamps. The accuracy difference is most noticeable with technical jargon, accented speech, and non-English languages. ★ Small is selected by default.

Downloading a Model

Models are not bundled with the app — they are downloaded on demand from Hugging Face and cached in ~/Library/Caches/. You must download at least one model before processing your first lecture.

Open Settings

Click the gear icon in the top-right toolbar, or use the menu bar: Harvestry → Settings.

Go to the Transcription tab

The first tab in Settings is Transcription. It shows the five available models and their download status.

Select a model and click Download

Click the model row to select it, then click Download. A progress bar shows download progress. The download requires roughly 75 MB–800 MB depending on the model.

Wait for completion

The model status changes to Ready with a green checkmark. You can close Settings and begin processing.

💡

Start with Small. At ~250 MB it downloads quickly, loads fast, and handles the vast majority of lectures well. If you need more accuracy — dense technical content, heavy accents, or non-English material — try Medium or Large Turbo and retranscribe. Tiny and Base are useful when you just need a quick result and do not require high accuracy.

Selecting a Model Per Lecture

Each lecture can be transcribed with any downloaded model, regardless of the global default set in Settings. On the lecture detail view, the Transcription step row shows the currently selected model in its subtitle. Click the model name to open a menu picker and select a different model before starting processing.

The model choice is stored per lecture and shown in the step row even after processing completes, so you can always see which model was used.

Spoken Language & Translation

Whisper understands 100 languages. By default Harvestry transcribes each one in the language it was spoken — a French lecture produces a French transcript. Both options below live in the same menu on the Transcription step row, beside the model picker.

Choosing the spoken language

The Spoken language section offers Auto-detect (the default) or any of the 100 supported languages by name.

Auto-detection inspects only the opening of the audio. That is usually enough, but it can be misled by a video that begins with a music bed, a silent title card, applause, or an introduction in a different language from the body of the talk. Because the detected language is applied to the whole file, one bad guess at the start affects the entire transcript. If you already know what is being spoken, naming it explicitly is more reliable and costs one click.

Transcribe vs. Translate Pro

The Output section of the same menu chooses what Whisper does with the audio:

Transcribe (original language) — speech becomes text in the language it was spoken. French in, French out. Included in both Standard and Pro.
Translate to English — foreign-language speech becomes an English transcript in a single pass. The model goes from audio directly to English text; this is not a second pass over a finished transcript. Pro only.

Translation runs entirely on-device, like transcription — no audio or text is sent anywhere. Everything downstream is unaffected: screenshots, timestamps, word-level audio sync, annotations, consolidation, and both export formats all operate on the English result.

⚠️

Translation targets English only. Whisper was trained to translate into English and no other language, so pairs like French → German are not possible at this stage. That is a property of the speech model, not a Harvestry restriction. If you need another language, see Getting notes in another language below.

Getting notes in another language Pro

If what you actually want is French → German, Japanese → Spanish, or any pair that doesn't end in English, don't use Translate. Instead:

Leave this menu on Transcribe (original language), so you get a faithful transcript in the language spoken.
Set Settings → Consolidation → Notes language to your target language.
Choose Reprocess → Run consolidation on the lecture.

The consolidation model writes the notes in your target language, working from the text. Your export then holds the original-language transcript alongside notes in the language you chose. Full details and a worked example are in LLM Consolidation → Notes Language.

💡

Use Medium for translation, not Large Turbo. Large Turbo is a pruned-decoder version of Large v3 that OpenAI fine-tuned for transcription only — its translation quality is noticeably worse despite being the larger model. Medium is the strongest translation-capable model Harvestry ships. When you select Translate while Large Turbo is active, the menu offers to switch for you.

📝

Quality varies a great deal by language. French, Spanish, German, Italian, Portuguese, and Dutch translate very well. Low-resource languages — Maltese, Amharic, Shona, Lao, Yoruba and similar — are supported but far weaker, and the failure mode is fluent-sounding English that does not match what was said. Check results against a source you trust before relying on them.

Model Mismatch Warning

If you change the global default model in Settings after a lecture has already been transcribed, the step row for that lecture will appear in orange. This indicates a mismatch between the model that was used and the current global default — it does not mean anything is wrong, just that the transcript was generated with a different model than your current preference.

To clear the mismatch warning, either:

Change the per-lecture model selection back to match the transcript, or
Retranscribe the lecture with the new model (see below).

The same applies to the language settings. If you change the spoken language or switch between Transcribe and Translate on a lecture that has already been processed, the status line reports what the existing transcript was made with and what your current settings would produce, and a Re-process button appears beside it. Existing consolidated notes and screenshots are carried through the re-export.

Retranscribing

You can re-run transcription on a completed lecture at any time, for example to use a more accurate model or to pick up improvements after updating WhisperKit.

Select the lecture in the sidebar.
Set the model and the language/Translate options on the Transcription step row the way you want them.
Click Reprocess in the action row and choose Re-transcribe. If your settings differ from the ones the current transcript was made with, the item shows what it will use, e.g. "Re-transcribe as French → English".
A confirmation dialog warns that the existing transcript will be replaced. Confirm to proceed.

The menu item is always available on a completed lecture, even if nothing has changed. It's only disabled when the source video file is missing or the selected model isn't downloaded.

Retranscription runs only steps 1 and 4 of the pipeline. Your existing screenshots and consolidated notes are kept, and the HTML export is regenerated once the new transcript is ready. Notes written from the old transcript aren't updated automatically; choose Reprocess → Run consolidation afterwards if you want them to match.

⚠️

Annotations are replaced on retranscription. Highlights and margin notes anchored to specific transcript passages may not survive retranscription if the passage text changes substantially. Export your current HTML first as a backup before retranscribing important lectures.

Updating WhisperKit Models

Harvestry checks for model updates from Hugging Face every 24 hours. When an update is available, the model row in Settings shows an update badge. Click Update to download the newer weights. The update check compares local snapshot SHAs against the Hugging Face Hub HEAD.

Previous The Processing Pipeline

Next Screenshot Capture