I have spent the last few years building AI systems for teams who cannot, under any circumstances, leak a single sentence of their source material. Not by accident. Not in a logged prompt. Not buried inside a vendor’s training pipeline. The most demanding of those teams are lawyers reviewing privileged contracts, and the engineering pattern they need is one I want to walk through here.

This is not legal advice. I am a software engineer. What I can tell you is how I would architect a local AI stack for a legal team that wants the productivity gains of large language models without surrendering attorney-client privilege to a third-party API.

Why does sending a privileged contract to a hosted API break the model legal teams operate under?

The standard SaaS AI workflow is simple. A user pastes a contract. The text travels over TLS to a vendor’s server. The vendor runs inference. A response comes back. Somewhere in the middle the document was decrypted, processed, and possibly logged.

For most use cases that is fine. For privileged material it is a structural problem. Once a document leaves the client’s controlled environment and enters a vendor’s infrastructure, the analysis becomes whether that disclosure was necessary, whether the vendor qualifies as an agent, whether retention policies hold, and whether the privilege survives. None of those questions exist if the document never leaves the building.

The engineering answer is to make the document never leave the building. That is what local AI does. The model weights live on a machine the firm controls. Inference happens on that machine. No outbound API call carries contract text anywhere. The transcript I worked from for this post demonstrates exactly that pattern in a different domain (a self-hosted search engine), and the same architecture applies cleanly to contract review.

What does a local AI stack for contract review actually look like?

Picture three layers running on a workstation, a server in the office, or a private cloud instance the firm administers.

The first layer is the model runtime. This is the piece that loads weights and serves completions. Ollama is the most approachable option. You pull a model, you run it, and you have an API on localhost that behaves enough like the OpenAI interface that most tooling just works. In the video I built this post from, I demonstrate exactly this pattern by running a local model and pointing an application at the local endpoint instead of a hosted one. The exact same swap is what makes a legal AI assistant private by construction.

The second layer is the retrieval system. Contract review is rarely about a single document. It is about a clause in this NDA compared to the standard the firm uses, or a representation in this purchase agreement compared to the same representation across forty deals last year. That requires a vector store and a chunking strategy that respects clause boundaries. I cover the engineering depth of this in my guide on building production RAG systems, and the same principles hold whether you are indexing public documentation or a partner’s playbook.

The third layer is the application surface. This is what the associate actually uses. A web interface, a Word add-in, a chat panel inside the document management system. The design choice that matters here is making sure every prompt, every retrieval, every response stays inside the firm’s network. No telemetry. No analytics pixel. No “helpful” cloud sync.

Which models are realistic for legal review on local hardware?

This is the question I get asked the most, and the honest answer is that the landscape changed in 2025 and is still changing.

A modern workstation with a single high-end GPU can comfortably run a fourteen to thirty billion parameter model. That is enough capability to summarize a hundred page agreement, extract defined terms, flag deviations from a template, and answer questions about a clause with citations back to the source paragraph. It is not enough capability to replace a senior partner’s judgment, and nobody serious is claiming it should.

For firms with more budget, a small server with two or four GPUs can host a seventy billion parameter model and serve the entire team. At that size the quality gap between local and frontier hosted models narrows considerably for the specific task of structured contract analysis, which is mostly about following instructions carefully over long context rather than open-ended reasoning.

The decision of where to draw the line is something I worked through in detail in my local versus cloud LLM decision guide. The short version for legal teams: if the document is privileged, the model runs locally, full stop. The cost of a GPU is rounding error compared to the cost of a privilege waiver argument.

How do you build prompts that actually capture legal nuance?

Prompt engineering for contract review is its own craft. The model does not know what your firm cares about. It does not know that this client always strikes the mutual indemnification, or that this jurisdiction requires specific language for limitation of liability to be enforceable. You have to teach it, in the prompt, every time.

The pattern I use is a layered system prompt. The outer layer establishes the role and the constraints. You are reviewing a draft for a specific client. You will not summarize. You will not editorialize. You will identify deviations from the provided playbook and quote the exact contract language for each. The middle layer injects the playbook itself, retrieved from the firm’s repository of standard positions. The inner layer is the document under review, chunked and tagged so the model can cite section numbers accurately.

The discipline that matters most is forcing the model to quote rather than paraphrase. A paraphrase loses the precision that legal language depends on. When the model says “the indemnity is broad,” that is useless. When the model says “Section 9.2 indemnifies the Buyer for any Loss arising out of or related to a Breach, with no materiality qualifier and no cap,” that is a starting point a lawyer can actually use.

This kind of structured prompting connects to a broader pattern I have written about in AI system design patterns for 2026. The prompts are software. They get versioned, tested against a regression set of contracts, and updated as the firm’s positions evolve.

Where does local AI fit into due diligence and eDiscovery?

Due diligence is where the volume problem becomes acute. A mid-size acquisition produces a data room with thousands of documents. The traditional approach is associates reading until their eyes bleed. The cloud AI approach is uploading the data room to a vendor and trusting their security review. The local AI approach is running the entire pipeline inside a controlled environment.

The architecture is the same retrieval-augmented pattern as single-document review, scaled up. Documents get OCR’d if needed, chunked, embedded, and indexed. A reviewer asks questions. The system retrieves relevant passages and the model synthesizes answers with citations back to the original document and page. Nothing leaves the environment.

eDiscovery has the same shape with stricter chain-of-custody requirements. The advantage of local AI here is that the audit trail is yours. You log every prompt, every retrieval, every response, in a system you control. When opposing counsel or a regulator asks how a document was reviewed, you have a complete answer. When the same question is asked of a SaaS vendor, the answer is whatever the vendor’s logs happen to contain.

I keep a working set of these architectures and reference implementations on my open source page. If you want to see the actual moving parts rather than read about them in the abstract, that is the place to start.

What about the search layer that the legal team uses every day?

One of the underrated benefits of going local is that you stop being limited to whatever search experience your document management vendor ships. The same self-hosted AI search pattern I demonstrated in the video this post is based on, where a local model sits in front of a meta-search engine and produces cited answers, applies directly to a firm’s internal knowledge base.

Imagine an associate asking, in plain English, how the firm has handled a specific kind of earn-out dispute across the last fifty deals. A traditional search returns a list of file names. A local AI search returns a synthesized answer with citations to the actual memos and agreements, generated by a model that never sent a single token outside the firm. I have written more about this specific shift in self-hosted search advantages, and the legal vertical is one of the most natural fits for it.

How do you keep this system trustworthy over time?

Three habits matter more than the rest.

The first is evaluation. You build a regression set of contracts with known issues and you run every model update, every prompt change, every retrieval tweak against it. If the system used to catch a missing materiality qualifier and now misses it, you find out before the associate does.

The second is scope discipline. Local AI is for drafting assistance, issue spotting, and summarization. It is not for final legal judgment. The output is reviewed by a lawyer every time. The system is a power tool, not an autopilot. Treating it that way protects both the client and the lawyer.

The third is data hygiene. The whole reason to go local is to keep privileged material inside a controlled boundary. That boundary only holds if you enforce it. No copying outputs into a hosted note-taking app. No screenshots into a cloud chat. The same care that goes into the model deployment has to extend to the workflow around it. I cover the broader picture in my piece on data privacy in AI, and the discipline scales from a solo practitioner to a global firm.

What is the realistic next step for a legal team that wants to start?

Start small. Pick a single workflow. NDA review is a great first target because the documents are short, the playbook is well understood, and the volume is high enough that even modest time savings compound quickly.

Stand up a local model on a single workstation. Index your firm’s NDA playbook. Write a system prompt that compares an incoming draft to the playbook and produces a redline-style report with quoted clauses. Run it against fifty real NDAs from the last year and have a partner grade the output. Iterate the prompt until the grade is consistent.

Once that workflow is solid, extend to the next one. Service agreements. Then employment contracts. Then due diligence. The architecture stays the same. The prompts and the playbooks evolve. The privacy posture is preserved at every step because the model never left the building in the first place.

That is the real promise of local AI for legal teams. Not magic. Not autonomous lawyering. A reliable, private, auditable productivity layer that respects the duty of confidentiality the profession is built on.

If you want to go deeper on the engineering side, I publish video walkthroughs of these architectures on my YouTube channel, and I run a community of engineers building exactly this kind of system at aiengineer.community/join. Come build with us.