LLM Consolidation

Send your transcript to Claude, ChatGPT, or a local Ollama model to generate polished, structured study notes alongside the verbatim transcript — in the transcript's language or another of your choosing.

What Consolidation Does

The raw transcript produced by WhisperKit is accurate but verbatim — it captures exactly what was said, including filler words, repeated phrases, and the natural looseness of spoken language. LLM Consolidation sends this transcript to a language model and asks it to produce structured Markdown study notes: headings organized by topic, bullet-point summaries, and key takeaways.

The consolidated notes appear as a separate "Consolidated Notes" tab in the transcript panel and in the exported HTML page. They sit alongside the original verbatim transcript — the raw transcript is never replaced.

You can annotate consolidated notes independently of the transcript (highlights, margin notes, inline notes all work in both views).

Choosing a Mode

The mode picker on the Pipeline Step 3 row is a segmented control with four options:

Off — No LLM is called. The pipeline skips Step 3 entirely and proceeds directly to export. This is the default.
Claude — Uses the Anthropic API. Your transcript is sent to Anthropic's servers over HTTPS. Requires an API key.
ChatGPT — Uses the OpenAI API. Your transcript is sent to OpenAI's servers over HTTPS. Requires an API key.
Ollama — Uses a locally running Ollama instance at localhost:11434. Fully on-device. Requires Ollama to be installed and a model pulled.

Turning consolidation on or off is stored per lecture; the provider you pick is shared across all lectures (see Per-Lecture Control). Changing the mode on a completed lecture never re-runs consolidation by itself — choose Reprocess → Run consolidation when you want new notes.

Notes Language Pro

The consolidated notes are written in the transcript's own language by default. You can have them written in a different one instead — and this is the only place in Harvestry that produces an arbitrary language pair.

Whisper's own translation is one-directional into English, so French → German is impossible at the transcription stage. A consolidation model has no such limit: it can write notes in any language it knows, working from the text rather than the audio.

💡

Where to find it: Harvestry → Settings (⌘,) → Consolidation → Notes language. It sits below the provider picker and applies to whichever provider you have selected — Claude, ChatGPT, or Ollama.

A worked example: French video, German notes

On the lecture, set the Transcription step's language menu to Transcribe (original language) — you want a faithful French transcript, not an English translation.
Open Settings → Consolidation, choose a provider, and set Notes language to German.
Back on the lecture, choose Reprocess → Run consolidation.
The status line confirms the target — "writing notes in German…" — and the export ends up with a French transcript alongside German consolidated notes, both in the same document.

Changing the notes language marks existing notes as stale, so the Reprocess menu labels the consolidation item "(settings changed)". Nothing is regenerated until you ask for it.

📝

Output quality depends on the model, not on Harvestry. A frontier model writing German or Spanish is excellent. A small local Ollama model asked for a low-resource target language may quietly answer in the transcript's language instead, or produce awkward output. If the result comes back in the wrong language, try a larger model before assuming the setting failed.

For low-resource languages this route is also simply better than Whisper's speech translation, which was trained on far less data than the text models were.

Claude Setup

To use Claude consolidation, you need an Anthropic API key. Keys are available at console.anthropic.com.

Open Settings → Consolidation

Go to Harvestry → Settings and click the Consolidation tab.

Enter your API key

Paste your Anthropic API key into the API Key field. The key is stored securely in the macOS Keychain — it is never stored in plain text.

Select a model

After the API key is saved, Harvestry fetches your available models from the Anthropic API and shows them in a dropdown. Choose a model. Click the refresh button next to the picker to update the model list if your account gains access to new models.

Claude Model Recommendations

The model list is fetched live from the Anthropic API, so it reflects whatever your account has access to. General guidance:

Model Family	Best For	Cost
Claude Sonnet	Recommended for most use. Excellent balance of quality and speed. Produces well-structured notes with good comprehension of technical content.	Moderate
Claude Haiku	Fastest and cheapest. Good for straightforward lectures with clear structure. May miss nuance in dense academic content.	Low
Claude Opus	Highest quality. Best for highly technical, multi-topic, or non-English content where maximum comprehension matters.	High

⚠️

API usage is billed by Anthropic. Consolidating a long lecture can consume a significant number of tokens (input + output). A 90-minute lecture transcript is typically 8,000–15,000 input tokens. Check your Anthropic dashboard to monitor usage.

ChatGPT Setup

To use ChatGPT consolidation you need an OpenAI API key. Keys are available at platform.openai.com. Note that an OpenAI API key is billed separately from a ChatGPT Plus subscription — a Plus subscription on its own does not provide API access.

Open Settings → Consolidation

Go to Harvestry → Settings, click the Consolidation tab, and choose ChatGPT API as the provider.

Enter your API key

Paste your OpenAI API key into the API Key field. As with the Claude key, it is stored in the macOS Keychain and never written in plain text.

Select a model

Once the key is saved, Harvestry fetches the model list from your OpenAI account and filters it to the chat-capable models. Pick one. Click refresh beside the picker to pull the list again when your account gains access to new models.

📝

Claude or ChatGPT? Both produce good study notes and both cost a fraction of a cent per lecture at current pricing. Claude tends toward longer, more structured academic prose; ChatGPT toward tighter summaries. Switching providers on a finished lecture offers to regenerate the notes, so the practical answer is to run the same lecture through both and keep the one you prefer.

Ollama Setup Pro

Ollama runs language models entirely on your Mac using Apple Silicon's unified memory. No data leaves the device at any point.

Install Ollama

Download Ollama from ollama.ai and install it. Ollama runs as a background service on localhost:11434. You can verify it is running by opening http://localhost:11434 in a browser — it should show "Ollama is running".

Pull a model

Open Terminal and pull a model suited to your Mac (see recommendations below). For example:

ollama pull qwen3:8b

The first pull downloads the model weights and may take a few minutes depending on size. Subsequent loads are instant.

Harvestry auto-detects your models

In the Pipeline Step 3 picker, select Ollama. Harvestry queries localhost:11434/api/tags and shows all installed models in a dropdown. Click the refresh button to re-query if you pull new models while Harvestry is already open.

ℹ️

Ollama endpoint. If you run Ollama on a non-default port or a remote host, you can change the endpoint URL in Settings → Consolidation → Ollama Endpoint. Default: http://localhost:11434.

Choosing an Ollama Model Pro

Not all models are equally suited to lecture summarization. Two things matter most for this use case:

Context window. A 60-minute lecture transcript is typically 12,000–15,000 tokens. A 90-minute lecture can reach 20,000+. Models with short context windows (under 32K) will silently truncate the transcript, producing notes that only cover the first portion of the lecture. Always prefer models with 128K+ context.
Instruction following. Harvestry's consolidation prompt asks for structured Markdown with specific headings and sections. Models that follow formatting instructions reliably produce cleaner, more usable notes.

Apple Silicon's unified memory architecture means your Mac's RAM is the key constraint — the model weights load directly into the memory pool shared by CPU and GPU. As a rule of thumb, a model requires roughly 1.2× its on-disk size in RAM to run comfortably. Any model that would push your system past ~80% total RAM usage will swap to SSD and run unusably slowly.

8 GB RAM (M1 / M2 MacBook Air, Mac mini)

With 8 GB unified memory, you can comfortably fit models up to about 4–5 GB on disk. Larger models will swap to SSD and run too slowly to be practical.

Model	Pull Command	Disk	Context	Notes
Qwen3 4B ★	`ollama pull qwen3:4b`	2.5 GB	256K	Best structured-output quality at this tier. Handles long transcripts without truncation.
Phi-4 Mini	`ollama pull phi4-mini`	2.5 GB	128K	Very fast generation. Slightly less consistent formatting than Qwen3 4B.
Gemma4 E2B	`ollama pull gemma4:e2b`	7.2 GB	128K	Technically fits on 8 GB but leaves little headroom — may swap. Not recommended for this tier.

💡

8 GB honest assessment. Summarization quality on 4B models is noticeably limited compared to larger models — expect good structure but potentially shallow summaries on dense academic content. If quality matters, consider using Claude API instead, which has no local RAM constraint.

16 GB RAM (MacBook Pro M2 / M3, Mac mini M4)

16 GB unlocks 8–14B models and is the sweet spot for most users. You get genuinely useful note quality at comfortable speeds.

Model	Pull Command	Disk	Context	Notes
Qwen3 14B ★	`ollama pull qwen3:14b`	9.3 GB	256K	Top recommendation at this tier. Excellent instruction following. Fits with ~5 GB to spare.
Qwen3 8B	`ollama pull qwen3:8b`	5.2 GB	256K	Faster than 14B with slightly lower quality. Good starting point.
Qwen3 30B (MoE)	`ollama pull qwen3:30b`	19 GB	256K	MoE model (30B-A3B — 3B active parameters). Reportedly outperforms QwQ-32B at a fraction of the inference cost. Excellent if you want to push quality at this tier.
Gemma4 E4B	`ollama pull gemma4:e4b`	9.6 GB	128K	Google's Mixture-of-Experts model. Strong summarization quality; comparable to Qwen3 8B.

32 GB RAM (MacBook Pro M3 Pro / Max, Mac Studio M2)

32 GB is where on-device models become genuinely excellent. Both Gemma4 26B and Qwen3 32B produce output that holds up well against Claude API quality.

Model	Pull Command	Disk	Context	Notes
Gemma4 26B ★	`ollama pull gemma4:26b`	18 GB	256K	Google's flagship MoE model. Runs efficiently — only ~3.8B parameters are active per token, so it's faster than a dense 18B model while drawing on 26B of total knowledge. Excellent prose and structure.
Qwen3 32B	`ollama pull qwen3:32b`	20 GB	256K	Dense 32B model. Slightly slower than Gemma4 26B but outstanding at following complex formatting instructions.
Qwen3.6 35B	`ollama pull qwen3.6:35b`	24 GB	256K	Newest Qwen generation (post-3.5). Efficient architecture — 35B total parameters, fast inference. Excellent for long lectures.
Mistral Small	`ollama pull mistral-small:22b`	13 GB	128K	22B model with a compact 13 GB footprint. Fast generation, clean prose. Good if you want to leave more RAM free for other apps.

64 GB+ RAM (Mac Studio M2 Ultra / M4 Max, Mac Pro)

On high-memory machines, local models match or exceed what you'd get from most commercial APIs.

Model	Pull Command	Disk	Context	Notes
Qwen3.6 35B ★	`ollama pull qwen3.6:35b`	24 GB	256K	Newest Qwen generation (post-3.5). 256K context handles the longest lectures comfortably. At 24 GB it leaves abundant headroom on a 64 GB machine. Top recommendation for this tier.
Llama 3.3 70B	`ollama pull llama3.3`	43 GB	128K	Meta's dense 70B. Highest raw parameter count that fits at this tier. Strong quality but limited to 128K context — may truncate very long lectures.
Gemma4 31B	`ollama pull gemma4:31b`	20 GB	256K	Google's dense 31B. Fast inference, excellent quality, and 256K context. Good alternative if you prefer a Google model.

💡

About Mixture-of-Experts (MoE) models. Models labelled as MoE — including Gemma4 26B and Qwen3 30B-A3B — are larger than they appear. They route each token through only a fraction of their total parameters, so they run with the speed and memory footprint of a smaller model while retaining the knowledge of a larger one. gemma4:26b is an 18 GB download but runs like a sharp 4B model with 26B of knowledge behind it.

The Consolidation Prompt

The system prompt sent to the LLM instructs it how to format the output. The default prompt asks the model to produce structured Markdown with:

A brief summary paragraph at the top
Headings for each major topic covered
Bullet-point summaries under each heading
A "Key Takeaways" section at the end

You can customize the prompt in Settings → Consolidation → System Prompt. The text editor in Settings accepts any Markdown-aware instructions. A Reset to Default button restores the original prompt.

Per-Lecture Control

Whether consolidation runs is set per lecture: the Step 3 mode picker can turn it on for one lecture and off for another, so you can skip it for a quick reference video and use it for a dense seminar.

The provider and model, along with the notes language and system prompt, are global. Picking ChatGPT on one lecture's Step 3 row switches the provider for every lecture that has consolidation turned on. To compare two providers on the same lecture, run consolidation, switch provider, and run it again.

Changing any of these settings never regenerates notes on its own. On a pending lecture the change applies when you next click Begin Processing. On a completed lecture, choose Reprocess → Run consolidation when you're ready. If the existing notes came from different settings, that menu item says "(settings changed)".

Disabling After the Fact

To remove consolidated notes from a completed lecture:

On the Step 3 row, switch the mode picker from Claude, ChatGPT, or Ollama to Off.
If the lecture already has notes, an alert titled "Remove Consolidated Notes?" explains that the existing notes will be removed from the HTML output.
Click Remove from HTML to delete the notes and re-export without them, or Keep Notes to leave everything as it was. Removed notes can be regenerated later with Reprocess → Run consolidation once consolidation is turned back on.

Re-running Consolidation

To generate or regenerate notes on a completed lecture, choose Reprocess → Run consolidation. This runs only Step 3 with your current provider, model, and notes language, then re-exports the HTML. The transcript and screenshots stay as they are.

The item is available whenever a transcript exists and consolidation is turned on. It doesn't wait for Harvestry to detect that something changed. If the existing notes came from different settings, it's labelled "(settings changed)", but you can re-run it at any time, for example to get a fresh take with the same model.

Privacy

The privacy implications differ significantly between modes:

Off — No LLM call. Nothing leaves the device.
Claude — Your transcript text is sent to Anthropic's API servers over HTTPS. This is one of the two intentionally off-device steps in Harvestry's pipeline. Anthropic's data handling is governed by their Privacy Policy.
ChatGPT — Your transcript text is sent to OpenAI's API servers over HTTPS. OpenAI's data handling is governed by their Privacy Policy; note that their API and consumer-ChatGPT terms differ.
Ollama — Fully on-device. The transcript is sent over localhost to the Ollama daemon running on your Mac. No data leaves the device.

Consider using Ollama mode for lectures containing sensitive, confidential, or personally identifiable content.

Previous Screenshot Capture

Next HTML Export