The Price of Unverified AI: Courts Sanction Hallucinated Citations — and Warn of What Comes Next

AI ethics laws and regulations, artificial intelligence legal standards and policy

Generative artificial intelligence has moved from novelty to a daily tool in law offices across the country. It has also produced a fast-growing category of judicial sanctions. In the span of two weeks in mid-2026, courts in Illinois and Connecticut confronted lawyers and litigants who let AI corrupt their filings — one through fabricated “hallucinated” citations, the other through hidden instructions planted for AI to read. Together, Scott v. Illinois Human Rights Commission and Elliott v. New York Bariatric Group, LLC map both the current enforcement landscape and where it is heading.

The Core Problem: Fabricated Citations

In Scott, an Illinois appellate court affirmed the dismissal of a discrimination charge but devoted much of its opinion to sanctioning the petitioner’s attorney for briefs riddled with AI-generated fiction. The filings contained ten false citations: four fabricated statutory quotations, one case that does not exist, and five real cases that did not say what he claimed. His central argument on appeal — that the agency had violated a duty to conduct a “full investigation” — rested on statutory language the AI had simply invented.

            Two features of the decision stand out. First, the court refused to treat AI hallucinations as a lesser offense. A fabricated quote, a nonexistent case, and a case that does not say what counsel claims are all misstatements of law; a hallucinated citation is no different from one an attorney invents from imagination, because the lawyer — not the software — answers for the filing. The court adopted a blunt benchmark: the only acceptable standard is zero false citations. Second, the court was unmoved by the attorney’s explanations. He said he used a “premier corporate subscription” of a popular AI tool and had tried to cross-check his citations; the court responded that no subscription tier relieves a lawyer of the duty to verify. Worse, his response to the show-cause order contained still more errors — a repealed statutory subsection, fresh misquotations, and missing pin cites the court had expressly ordered.

            The sanction: a $15,000 fine, calculated at $1,500 per false citation, plus a referral to the state attorney-disciplinary commission. The court deliberately set that per-citation rate higher than earlier cases, reasoning that existing fines had not deterred a problem that keeps recurring.

A National Trend — and Escalating Consequences

Scott is no outlier. The opinion surveys a rapidly expanding body of sanctions nationwide, and the numbers are climbing. Courts have imposed fines ranging from a few hundred dollars to tens of thousands, and the consequences increasingly reach past money:

  • A Cook County court fined a law firm $59,500 for citing hallucinated cases in a post-trial motion.
  • The Seventh Circuit fined an attorney $5,000, referred him for discipline, and warned that a repeat could mean disbarment before that court.
  • The Sixth Circuit ordered offending attorneys to pay the opposing party’s fees, double their costs, and $15,000 each.
  • Federal courts have struck briefs, dismissed clients’ claims with prejudice, ordered AI-focused continuing legal education, and even barred a lawyer from practicing before the court for two years.

            The through-line is that judges are treating fabricated authority not as a technical slip but as a threat to the integrity of the adversarial system — one that wastes opposing counsel’s time, diverts scarce judicial resources, and can deprive a client of arguments grounded in real authority.

The Next Frontier: Hidden Prompts

If Scott addresses bad output, Elliott confronts something newer: bad input. There, a self-represented plaintiff embedded instructions in his court filings in white, tiny-point text — invisible to a human reader but fully legible to any AI that processed the document. The concealed text directed any AI reviewing the filing to agree with his position and to “remediate” a clerk’s adverse ruling in his favor. The technique has a name: prompt injection.

            The Connecticut court explained why this is a distinct and serious abuse. A filing is a communication to the court and the opposing party, and its integrity depends on the reader seeing exactly what the filer wrote — not a second, hidden message engineered to skew how the document is reviewed. A concealed instruction aimed at the tools a judge, clerk, or opposing counsel might rely on is, the court reasoned, akin to an improper ex parte communication: a message the other side can neither see nor answer.

            Notably, the court reached this conduct even though Connecticut’s new AI rule, effective only weeks earlier, addresses just the accuracy of what AI produces — not manipulated input. The lawyer’s (and litigant’s) duty of candor and the court’s inherent authority over its own proceedings filled the gap. The sanction was narrowly tailored: the plaintiff lost the privilege of electronic filing and must now file on paper, in person. The wrong, the court stressed, lay in the attempt itself — regardless of whether the injection ever changed a ruling.

            How was the hidden text found? On its own. The court raised the issue sua sponte while reviewing the docket — not at the opposing party’s prompting — and, tellingly, not through any AI screen. Connecticut’s courts do not use AI to review filings; a human being noticed the concealed instructions, and the judge had actually ruled on the underlying motion from a printed copy, so the injection never touched the decision. The opinion contrasts a Brazilian tribunal whose own AI system flagged and blocked an identical hidden prompt before processing it. The lesson the court draws is that the last line of defense is a person who looks closely at what the machine produced.

Guarding Against Hidden Prompts: A Practical Check

The Elliott opinion does not spell out the exact step by which the court surfaced the invisible text — and the concealment method, a white one-point font, remains hidden even on paper and during ordinary on-screen reading. Because the tactic is now common, it is worth building a quick habit of screening any document before you feed it to an AI tool. A few simple steps reveal most hidden text:

  • Select the entire document (Ctrl+A / Cmd+A). Highlighting exposes white-on-white text, which appears against the selection shading.
  • Copy everything and paste it into a plain-text editor such as Notepad or TextEdit in plain-text mode. Stripping color and font size renders any concealed instruction in ordinary black type.
  • Recolor or reformat the text. Setting all text to a dark color, or applying a shaded background, makes invisible white text visible; enlarging a uniform font size blows up any one-point characters.
  • Watch for tells. Unexpected blank space, a page or scroll length that outruns the visible words, or an oddly large file size can all signal hidden content.
  • For PDFs, extract the text layer. Copying the document’s selectable text into a plain editor — or running a text-extraction tool — surfaces anything machine-readable but not visually apparent.

            None of this requires special software, and it takes only seconds. The discipline is the same one the court urged: look at what is actually in the document before any tool — or any reader — relies on it.

A Related Caution: The Agreeableness Trap

Elliott offers one more warning worth heeding. The judge observed that generative AI tends toward agreeableness — prompt it only to build the case for a desired outcome, and it will generally oblige, in fluent, confident prose arranged to look like law. A litigant who never asks the tool to test a position, only to advance it, may come to believe a losing argument is a winning one, and to mistake a correct adverse ruling for bias. The discipline the court urged is simple: ask the tool to challenge your position as readily as to support it.

Practical Takeaways

  • Verify every citation and quotation before filing. Courts now apply a zero-tolerance standard, and “the AI generated it” is not a defense.
  • No subscription tier or vendor substitutes for a lawyer’s own cite-checking. Responsibility for the filing stays with the human who signs it.
  • Sanctions are escalating and cumulative — fines, fee-shifting, disciplinary referrals, CLE requirements, and even practice bars are all on the table.
  • Treat incoming documents as potential vectors, too. An opponent’s production, an exhibit, or a client file could carry hidden instructions that skew an AI summary or translation; read AI output with your own judgment.
  • Use AI to stress-test arguments, not just to bolster them. A tool prompted only to agree can quietly lead a case astray.
  • The professional-conduct rules already reach this conduct. Candor to the tribunal, competence (including technological competence), and the bar on frivolous filings all apply — with or without an AI-specific rule.

See Scott v. Illinois Human Rights Commission, 2026 IL App (1st) 251462 (July 28, 2026); Elliott v. New York Bariatric Group, LLC, No. AAN-CV-25-6066141-S, 2026 WL 2323029 (Conn. Super. Ct. Aug. 6, 2026).

This article is provided for general informational purposes only and does not constitute legal advice or create an attorney-client relationship. Outcomes depend on the specific facts of each matter; readers should consult qualified counsel about their particular circumstances. Murray | Lobb Attorneys PLLC has represented businesses, creditors, individuals, and Texas municipalities across Galveston County and coastal Texas since 1991.

Scroll to Top