I spent a weekend in my home lab fine tuning an open source Qwen 3.5 model on every YouTube transcript I have ever recorded. The goal was simple. I wanted a model that sounds like me without me having to write a 4,000 word system prompt every single time. After running tests across a 9 billion parameter model, a medium Mistral 3 model, and a 27 billion parameter Qwen, I learned something most tutorials skip over. The choice between LoRA and full fine tuning for personal writing style is not actually a close call. For voice work, LoRA wins almost every time, and the people telling you otherwise are usually trying to sell you compute.

This is the question I keep getting from engineers who watched my fine tuning series. They want to know whether they should retrain billions of parameters from scratch or whether a small adapter is enough to capture how they actually sound. The honest answer depends on a handful of variables: dataset size, how stylistically distinct your writing is, how much voice drift you can tolerate, and whether you have evaluation in place to catch problems before they ship.

What Does LoRA Actually Change Inside the Model?

LoRA stands for low rank adaptation. Instead of fine tuning all the parameters of a language model, you train somewhere between 0.5 and 1.5 percent of them. You are not retraining billions of weights. You are creating a tiny trainable adapter that gets injected into the model alongside the base weights. The base model stays frozen. Your adapter learns the patterns that make your writing yours.

For personal writing style, this is exactly what you want. Your voice is not encoded across every parameter in a 27 billion parameter network. Your voice lives in a relatively small set of patterns: sentence length, transition words, how you open paragraphs, what you refuse to say, the cadence of your explanations. A LoRA adapter has more than enough capacity to capture those patterns from a few thousand high quality training pairs. If you want to understand why this works at the hardware level, my model quantization guide explains how parameter efficient techniques compound to make local AI viable.

Full fine tuning, by contrast, updates every parameter in the model. It is slower, it requires far more VRAM, and for voice work it is mostly overkill. The base model already knows English. It already knows how to reason. You do not need to teach it those things again. You just need to nudge its output distribution toward your style.

When Does LoRA Capture Voice Well Enough?

In my experience, LoRA captures voice well enough in three specific conditions.

First, when your dataset is in the range of one to five million tokens of cleaned, paired data. My YouTube transcripts produced enough material to train a 27 billion parameter Qwen with a LoRA adapter that genuinely sounded like me. When I asked the fine tuned model how I stay up to date with AI tools, it gave a direct, brief answer about using AI agents to scan saved resources. That is exactly how I would actually answer the question in person. The base Qwen model, by comparison, returned a poetic meditation about flow and change that had nothing to do with how I think.

Second, when your style is already represented somewhere in the base model’s training distribution. If you write like a normal English speaker with some quirks, LoRA can shift the model’s behavior toward those quirks without breaking anything else. The base model has seen plenty of conversational, direct prose. LoRA just tells it to prefer that mode for your prompts.

Third, when you can tolerate small stylistic drift. My LoRA adapter occasionally produces em dashes I personally would never use. That is a legitimate stylistic miss, but it is the kind of thing I can clean up in post processing or partially solve with better data engineering. If your tolerance for drift is high, LoRA is fine.

When Do You Actually Need Full Fine Tuning?

Full fine tuning becomes worth considering in narrow cases. If your writing style is wildly different from the base model’s training distribution, for example if you write in a constructed language, a heavily domain specific jargon, or a format that almost never appears on the public web, a LoRA adapter may not have enough capacity to shift the model far enough.

You might also need full fine tuning if you are baking in a large body of factual knowledge alongside the style. Voice plus a hundred million tokens of proprietary technical documentation is a different problem than voice alone. At that scale, the adapter starts to feel cramped, and updating more parameters gives the model room to actually internalize the content.

Honestly though, most people asking about full fine tuning for voice do not actually need it. They need better data. The bottleneck in personal style fine tuning is almost never the number of trainable parameters. It is the quality and structure of the prompt and response pairs you feed in. If you want a deeper look at the local infrastructure side, my VRAM requirements guide for local AI walks through what hardware you actually need for parameter efficient training.

What Is the Dataset Size Threshold?

Here is the rough rule of thumb I use after fine tuning three different model sizes. For an 8 billion parameter base model, you want at least one to two million tokens of clean, paired training data. For a 27 billion parameter model, you want closer to two to five million tokens, though I got reasonable results on the lower end.

The word “clean” is doing a lot of work in that sentence. My YouTube transcripts came from Google’s auto captioning, which produced plenty of misspellings and weird artifacts. If I had fed those raw transcripts into the training pipeline, the model would have learned to reproduce those errors. So before any LoRA training even started, I spent significant time on dataset engineering: cleaning the text, removing artifacts, and transforming the raw transcripts into prompt and response pairs that match the chat format the model expects at inference time.

That last point catches a lot of people. A YouTube transcript is not a training example. You cannot just dump unstructured monologue into a fine tuning pipeline and expect a chat model to come out the other side. You need pairs. I used a local language model to generate plausible questions for each transcript snippet, then paired those generated questions with the actual transcript segments as responses. If you have ever set up a local AI development workflow with Ollama, you already have most of the tooling you need to run this kind of synthetic question generation locally.

Below the threshold, you start seeing voice drift. The model picks up some surface patterns but reverts to base model behavior on anything novel. Above the threshold, returns diminish quickly. More data past a certain point mostly just makes training take longer.

Want to skip the infrastructure setup entirely and start with working RAG and local model projects? Get the Local AI Starter Projects and you will have a foundation to build the rest of this pipeline on top of.

How Do You Detect and Prevent Voice Drift?

Voice drift is the thing nobody warns you about. You fine tune the model, the first ten outputs sound great, you ship it, and three weeks later you notice the model has slowly slid back toward generic LLM behavior on edge cases. This happens because your training data did not cover the full distribution of questions users actually ask.

The fix is evaluation. Before you celebrate a fine tuned model, you need a structured evaluation pipeline that compares your fine tuned outputs against the base model on a fixed set of probe questions. Some of those questions should be in distribution, similar to your training data. Some should be deliberately out of distribution, designed to probe whether the model maintains your voice when asked something it was not trained on.

I caught multiple painful problems this way. In one run, I realized I had transformed my transcripts incorrectly during dataset engineering, and the LoRA adapter had baked in a subtle formatting quirk that only appeared on certain question types. Without a evaluation pipeline, I would have shipped that model and only noticed weeks later. Building a structured probe set is similar in spirit to building an AI knowledge base for retrieval, except the goal is regression testing rather than retrieval quality.

For voice models specifically, I evaluate on three axes. Tone, which I check by reading outputs and asking whether they sound like me. Brevity, which I check by measuring response length distributions, since one of my style markers is being concise. And content correctness, which matters because a model that sounds like me but says things I would never say is worse than a generic model.

How Should You Decide Between LoRA and Full Fine Tuning?

Here is the flowchart I actually use. Try a better prompt first. If that fails, add retrieval augmented generation. If that fails, consider an agentic loop. If all of that still fails, then fine tuning is on the table. And when fine tuning is on the table, default to LoRA.

Reach for full fine tuning only when you have already trained a LoRA adapter, evaluated it rigorously, and confirmed that the adapter capacity is genuinely the bottleneck rather than your data quality or your evaluation rigor. In practice, almost nobody who asks about full fine tuning has done that work first. They jump straight to the most expensive solution and then wonder why their results are not better than what a well tuned LoRA would have produced for one tenth the compute and one fifth the time.

If you want to see the rest of this fine tuning pipeline, including the data engineering, the LoRA training itself, the evaluation harness, and the GGUF export for running the model locally, the next videos in my series walk through every step. Watch the original walkthrough on YouTube here: https://www.youtube.com/watch?v=v7qMjy_RxOs. And if you want to learn this kind of work alongside other engineers building real fine tuned models, join the AI Engineer community at https://aiengineer.community/join.