I never thought I’d find myself wondering if I had the right legal AI platform while watching the Red Sox try to battle back against the Yankees in the bottom of the eighth. But legal tech has reached that point. You can see it in the million-dollar booths at industry conferences, the robust discussions on online message boards, even in the ads plastered on the walls of professional sports stadiums.

Garbage In, Garbage Out

What some of these providers can do today is incredible, and it’s already having an impact across the profession, ranging from how firms recruit and train talent to how they structure their billings. But as with many new innovations, the old rule has never been more true: “garbage in, garbage out.”

If you’re like me, you probably first heard this phrase when your fourth-grade teacher warned you that a calculator is only as smart as the person entering the numbers. But it fits this moment in legal tech perfectly. Whatever legal AI platform you choose produces an output, and like any output, the fidelity is driven by the quality of the inputs.

I come at this as someone who’s operated within the tech world for the last decade plus, and for all its nuances, capturing and analyzing testimony is a process with inputs, outputs, and specific steps. One key input often used by legal LLMs is prior case law. While we’ve all heard horror stories about models hallucinating cases that never occurred, a real published opinion is a fixed text. Courts can overrule or distinguish it, and other jurisdictions can disagree, but the words on the page are the words the court wrote. Whether it’s still good law is a separate question from whether the text is accurate. Witness testimony is a whole ‘nother ballgame.

Why Testimony Is the Hard Input

Deponents, witnesses, and attorneys don’t typically speak like robots, enunciating clearly in a flat monotone with defined pauses between speakers. Generally, this is a good thing for humanity (and those industry conferences might be a tough slog otherwise), but for the purposes of record capture (which itself is the primary source for an AI output) it creates challenges.

Unique speech patterns, accents, long technical terms, cross-talk, low-talk (thanks, Seinfeld), arguments, and other conversational patterns that differentiate us from the bots all require court reporters to constantly make dozens of micro-decisions.

We’ve all played the age-old game of telephone and watched the classic comedy Multiplicity (some of Michael Keaton’s finest post-Batman work). They both teach us that you lose fidelity with each translation layer. In our case, the accuracy and fidelity of the testimony is the primary source, and any issues or errors in that base record will only be amplified and exaggerated. If it has flaws, no amount of AI wizardry, regardless of how many millions of tokens are burned, will be able to correct it.

The reason for this is based on how these AI models work. They treat whatever record they’re handed as absolute truth. Sure, an LLM can evaluate one record against another to identify inconsistencies or logical fallacies, but it has no way to assess how accurate that baseline record is against the real human interaction in the room or on the Zoom. What’s worse, LLMs are trained to provide polished, confident output, so these errors become incredibly hard to detect, if not outright invisible.

With no downstream correction mechanism available, the fidelity of the record is non-negotiable. Those countless in-the-moment decisions that a court reporter must make, judgments honed from their years of schooling, training, shadowing, and field experience, aren't just nice-to-haves, they’re as critical to any downstream analysis as the foundation of a new home. Cut corners on the foundation and you might still get a home with some nice curb appeal, but watch out when the winds change.

Cheaper Inputs Cost More Downstream

As AI takes on more jobs downstream, upstream inputs matter more, because errors cascade through every step that follows. While there are certainly innovations and technology that can support court reporters, trying to automate their work or pursue “cheaper” alternatives is penny-wise and pound-foolish. You might save a few dollars today, but you’ll suffer at the settlement table or in front of the judge when your AI-supported analysis is fundamentally flawed.

A qualified, certified, experienced court reporter is the ultimate de-risk. When you can have confidence in your “raw material,” you’ve removed a key risk vector from your final output.

It turns out my fourth-grade teacher was right about that calculator. A machine, even one as sophisticated as legal AI, can only work with what you give it. Legal AI will keep getting smarter, but the record it reads will only ever be as good as the person who captured it.