How Does an AI Attorney Work? The Legal AI and LLM Behind an AI Lawyer
An AI attorney is legal software built on a large language model — the same class of AI that powers ChatGPT — wrapped in a retrieval layer that feeds it real statutes and case law. It works by turning your question into a prompt, searching a legal knowledge base for relevant sources, and generating a plain-language answer grounded in those sources, a process rooted in the architecture of the large language model itself.
This guide opens the black box: how the LLM is trained, how retrieval-augmented generation keeps it current, what it can and cannot do, and why a human lawyer still signs off on the result.
What Is an AI Attorney — and What Sits Inside It?
An AI attorney is not a person. The term splits into two different things, and mixing them up causes most of the confusion around the technology.
Two meanings of “AI lawyer” — we mean the software
The phrase “AI lawyer” can mean one of two things. The first is a human attorney who advises clients on AI-related law — data protection, algorithmic liability, AI regulation. The second is a software assistant that answers legal questions or helps with legal work using artificial intelligence. Legal AI platforms like Eve and Bloomberg Law draw this distinction explicitly, because the two are easy to conflate in casual conversation. This article is about the second meaning: an AI attorney as a legal AI assistant, a piece of software you interact with through a chat interface rather than a person you retain.
The core is a large language model
Strip away the interface and the legal branding, and what sits inside an AI attorney is a large language model — a neural network trained to predict the next word in a sequence of text. ChatGPT, released publicly by OpenAI in late 2022, is the model that made the category widely known, but it is far from the only one. Anthropic’s Claude, Google’s Gemini, and Microsoft Copilot are all examples of foundation models that can play the same role. A dedicated legal AI wraps one of these base models in a legal-specific layer: a database of statutes and case law, prompt templates tuned for legal tasks, and verification checks designed to catch errors before they reach the user. The foundation models most often used as that core include:
- ChatGPT, built by OpenAI
- Claude, built by Anthropic
- Gemini, built by Google
- Copilot, built by Microsoft
Vendors sometimes switch between these models, or combine more than one, depending on the task.
How the LLM Is Trained: From Text to Predictions
Before an AI attorney can answer a single question, its underlying model has already gone through a multi-stage training process that shapes what it can and cannot reliably do.
Pre-training on huge text corpora
An LLM starts life by training on enormous corpora of text — books, websites, articles, and, for legal-focused models, court opinions and statutes. During this phase the model learns statistical relationships between words, or more precisely between tokens, the small chunks of text the model actually processes. Machine learning research generally recognizes three broad training approaches, and modern LLMs combine elements of all three across their training pipeline:
- Supervised learning — the model is trained on labeled examples
- Unsupervised learning — the model finds patterns in unlabeled text
- Reinforcement learning — the model is rewarded or penalized based on the quality of its output
What matters for legal use is this: the model does not “know” the law in the way a human lawyer does. It predicts statistically likely sequences of tokens based on patterns it absorbed during training.
Fine-tuning and RLHF
After pre-training, most LLMs go through a further round of fine-tuning, often using reinforcement learning from human feedback (RLHF). Human reviewers rate the model’s responses, and the model is adjusted to produce more helpful, accurate, and safe outputs over successive rounds. This step improves how the model behaves, but it does not solve a separate and more important problem: the model’s knowledge is frozen at whatever point its training data ends. It has no built-in way to know about a statute passed last month or a ruling issued last week — which is exactly the gap that retrieval-augmented generation exists to close.
Why raw LLM knowledge is not enough for law
Law changes constantly, and it changes differently in every jurisdiction. A model’s parameters, by contrast, are static once training finishes. That means the “memory” baked into an LLM can be outdated, jurisdiction-mismatched, or simply wrong by the time you ask your question. Serious legal AI tools do not rely on a model’s raw internal knowledge for anything that matters — they pair the model with an external system that retrieves current, verifiable legal text at the moment you ask.
| Training phase | What happens | Key limitation for legal use |
|---|---|---|
| Pre-training | Model learns language patterns from huge general text corpora | No built-in verification of legal accuracy |
| Fine-tuning / RLHF | Human feedback shapes tone, helpfulness, and safety | Knowledge is frozen at the training cutoff date |
Retrieval-Augmented Generation: How an AI Attorney Reads the Law
That external system is called retrieval-augmented generation, or RAG, and it is the piece of the architecture that separates a general-purpose chatbot from a legal AI tool built to cite real sources.
The two-step RAG pipeline
Researchers describe retrieval-augmented generation as a two-step pipeline, and legal-AI analysis from Harvard’s Journal of Law and Technology frames it the same way. First comes retrieval: your query is converted into a numerical representation through natural language processing, and a retrieval system searches a database — often full of statutes, regulations, and prior court decisions — for the passages most relevant to your question. Second comes generation: those retrieved passages are fed to the LLM along with your original question, and the model writes an answer that draws on the retrieved text rather than purely on what it memorized during training.
Grounding answers in real sources
This process is often called grounding, because the answer is anchored to specific, checkable documents instead of floating free on the model’s internal probability estimates. Grounding is what allows a legal AI tool to show you a citation or a link back to the statute or case it relied on, and it measurably reduces — though does not eliminate — the risk of the model inventing facts. The catch is that either half of the pipeline can introduce error: retrieval can pull the wrong document, or generation can misstate what a correctly retrieved document actually says.
The Step-by-Step Workflow, From Prompt to Answer
It helps to walk through what actually happens, end to end, the moment you type a question into a legal AI tool.
- You type a plain-language legal question into the chat interface.
- The AI attorney online converts your question into a prompt and generates a numerical embedding of its meaning.
- The retrieval system searches its legal knowledge base and pulls the statutes, regulations, or cases most relevant to that embedding.
- The LLM synthesizes those retrieved passages into a natural-language answer.
- A well-built tool displays the underlying sources alongside the answer, so you — or your lawyer — can verify the citations before relying on them.
Each step depends on the one before it. A vague prompt produces a vague retrieval, and a weak retrieval produces a shaky answer no matter how capable the underlying model is.
What an AI Attorney Can Actually Do
Once the architecture is in place, the practical uses fall into a handful of categories, ranging from research to drafting to increasingly autonomous task execution.
| Task | What the AI attorney does | Typical time saved |
|---|---|---|
| Legal research | Searches and summarizes relevant statutes and precedent | Hours per query |
| eDiscovery / document review | Scans large document sets for relevant material | Weeks reduced to minutes |
| Contract drafting and review | Generates or flags clauses in agreements and letters | Minutes per document |
| Document summarization | Condenses long filings into short briefs | Minutes per document |
| Client intake | Structures initial client information before human review | Ongoing, per intake |
Legal research and document review
Legal research means finding and summarizing the statutes, regulations, or prior decisions relevant to a question — the core task RAG was built to support. Document review and eDiscovery apply the same underlying technology at a larger scale: complex litigation can involve document sets running into the hundreds of thousands or millions of records, and AI-assisted review is routinely used to compress work that would otherwise take weeks into minutes.
Drafting, summarizing, and client intake
Drafting is one of the most mature use cases. Legal AI tools can produce first drafts of contracts, demand letters, and client correspondence, and legal AI vendor Harvey has described summarizing documents running 150 pages or more down to a working brief in minutes rather than hours. Bar-exam performance became the industry’s headline benchmark. GPT-4, an upgraded version of the model behind ChatGPT, passed a simulated bar exam and outperformed most human test-takers, according to a study by researchers Daniel Katz and Michael Bommarito published in the Royal Society’s Philosophical Transactions A — a result that fueled early optimism about legal AI capability, even though passing a written exam is a different skill from practicing law. Lawyer adoption still lags lawyer belief. A Thomson Reuters survey found that 82% of lawyers see a role for generative AI in legal work, but only 51% believe it should actually be used for the tasks they had in mind, a gap that reflects lingering caution about reliability.
The shift to agentic AI
The newest trend in legal AI is agentic AI — systems that carry out multi-step tasks with less step-by-step prompting, rather than waiting for a human to direct each individual action. Instead of a lawyer editing every paragraph an AI drafts, the workflow shifts toward the lawyer auditing a more complete output the AI produced largely on its own, which changes the nature of the human review from line-editing to verification.
Accuracy, Hallucinations, and Limits
The single biggest limitation of any AI attorney is not a missing feature — it is a structural risk built into how LLMs generate text in the first place.
What a hallucination is
A hallucination is a response the model delivers with full confidence that turns out to be fabricated — a case that does not exist, a statute that says something different from what was quoted, a citation with no real source behind it. In most contexts a hallucination is an annoyance. In law, a fabricated citation submitted to a court is a professional and ethical liability, not just an inconvenience.
The hard numbers on legal AI accuracy
Researchers at Stanford’s RegLab studied this problem directly rather than taking vendor claims at face value. Their study, led by Magesh and colleagues and published on arXiv, tested commercial legal AI products that already use retrieval-augmented generation — not raw general-purpose chatbots — and still found meaningful hallucination rates: roughly 17% of queries to Lexis+ AI and roughly 33% of queries to Westlaw’s AI-Assisted Research produced hallucinated content.
Our findings highlight the need for rigorous, transparent benchmarking and public evaluations of AI tools in law, as well as continued efforts to develop AI systems that are grounded in the corpus of legal knowledge.
Magesh et al., Stanford RegLab / Stanford HAI
The finding that matters most for anyone using these tools: retrieval reduces hallucination, but it does not eliminate it, and vendor marketing that claims a product is “hallucination-free” should be treated with skepticism. Verification of citations remains a human responsibility, not something you can hand off entirely to the tool.
The Limits: Ethics, Confidentiality, and Who Is Responsible
Accuracy risk is only part of the picture. A legal AI tool also operates inside a professional ethics framework that assigns responsibility squarely to the human lawyer, not the software.
The lawyer stays responsible
Under the American Bar Association‘s Model Rules, a supervising lawyer remains responsible for work product generated with AI assistance:
- Rule 1.1 — competence, including understanding the tools a lawyer relies on
- Rule 5.1 — a supervising lawyer’s responsibility for the conduct of the firm
- Rule 5.3 — responsibility for the work of non-lawyer assistance, which extends to AI tools
The ABA’s Formal Opinion 512, issued in July 2024, was the organization’s first formal ethics guidance addressing generative AI in legal practice specifically. By 2025, well over a dozen state bar associations had issued their own guidance on AI ethics, reflecting how quickly regulators moved once the underlying technology became mainstream in legal practice.
Confidentiality and data privacy
Feeding confidential client information into a public, general-purpose chatbot without safeguards can create a real confidentiality problem, since many consumer AI products may use submitted text to further train their models. Before entering any client detail into a legal AI tool, it is worth checking a few things:
- Does the vendor state that inputs are excluded from model training?
- Is data encrypted both in transit and at rest?
- Does the tool have a published data retention and deletion policy?
- Is the vendor built specifically for legal use, rather than a general consumer chatbot?
A dedicated legal AI tool built for professional use typically offers more control over how client data is stored, processed, and retained — but that control varies by vendor, and it is worth confirming before submitting anything sensitive.
YMYL disclaimer
A free AI attorney is an informational tool, not a substitute for a licensed attorney. Using it does not create an attorney-client relationship, and nothing it generates should be treated as formal legal advice for your specific situation. For decisions that carry real legal consequences, consult a qualified attorney licensed in your jurisdiction — your state or local bar association’s lawyer referral service, or a legal aid organization if cost is a barrier, is the right place to start.