Der Fall
Rivera & Goldstein LLP was a 30-attorney employment law firm known for meticulous work product. When the firm adopted an enterprise AI tool six months ago, managing partner Diana Rivera championed it as a way to handle the firm's growing caseload without sacrificing quality. Training sessions were held. Guidelines were distributed. Everyone nodded. Most forgot.
Marcus Chen, a senior paralegal with eight years of experience, was juggling assignments for three attorneys when he received an urgent request from associate attorney Sarah Park: prepare a preliminary research memo on wrongful termination claims under state law for a new client intake. The client, a mid-level manager at a tech company, had been fired after reporting safety violations to HR. Sarah needed the memo by the next morning for a meeting with the prospective client.
Marcus opened the AI tool, typed 'Summarize the law on wrongful termination and retaliation claims, including relevant cases and statutory protections,' and pressed enter. The AI produced a polished, four-page memo in under two minutes. It cited three state statutes, four federal cases, and two state appellate decisions. It concluded that the client had 'strong grounds for a wrongful termination claim under both statutory and common law theories.' Marcus formatted it, added the firm's letterhead template, and emailed it to Sarah at 11:47 PM.
Wichtige Meilensteine
6 months ago — AI Tool Adopted
Rivera & Goldstein licenses an enterprise AI assistant. Training sessions are held for all staff. Written guidelines are distributed but not incorporated into existing workflows or supervision protocols.
Tuesday, 4:30 PM — The Assignment
Associate Sarah Park assigns Marcus Chen a preliminary research memo on wrongful termination for a prospective client meeting the next morning. Marcus is already working on two other urgent matters.
Tuesday, 11:47 PM — The Memo Is Sent
Marcus uses a single, unstructured AI prompt to generate the memo. He formats it on firm letterhead and emails it to Sarah without independently verifying any citations or legal conclusions.
Wednesday, 7:15 AM — The Near-Miss
Sarah reviews the memo over coffee and notices that one of the cited state appellate cases does not match any decision she has seen in this area. She checks Westlaw. The case does not exist. She checks the other citations. Two more are fabricated. The statutory analysis conflates protections from two different states. The meeting is in 90 minutes.
Warum das wichtig ist
No sanctions were imposed. No client was harmed. No court was misled. By every visible metric, nothing happened. But the Rivera & Goldstein incident reveals something more insidious than a single catastrophic failure: it exposes the everyday risk that AI-generated work product will pass through busy, understaffed workflows unchallenged. Marcus was not careless — he was experienced, trusted, and overwhelmed. The AI output looked exactly like the memos he had seen attorneys produce for years. The failure was not in the person; it was in the prompt, the process, and the assumption that a tool that produces polished output must be producing accurate output.
Kontextanalyse
Die systemischen Faktoren, die einen Routineauftrag in einen Beinahe-Vorfall verwandelten.
Fehler im Prompt-Design
- The prompt specified no jurisdiction, allowing the AI to conflate laws from multiple states
- No analytical framework was requested, producing a narrative rather than a structured legal analysis
- The prompt asked for a conclusion ('summarize the law') rather than a research survey, encouraging the AI to assert rather than report
- No instruction to flag uncertainty or distinguish between well-established and contested legal positions
Workflow-Lücken
- No verification step was built into the firm's AI-assisted research workflow
- The assignment was delegated without specifying which jurisdiction's law to research
- Time pressure incentivized speed over accuracy — the memo was due in hours, not days
- No second reviewer was available or assigned for AI-generated research output
Aufsichtsversagen
- The assigning attorney did not specify the expected format, depth, or sources for the memo
- No protocol required AI-generated work product to be flagged as such for reviewer awareness
- The firm's AI guidelines existed on paper but were not integrated into day-to-day supervision
- Paralegal workload was not monitored — Marcus was handling three concurrent urgent matters
Institutionelle Faktoren
- AI training was a one-time event, not an ongoing competency requirement
- The firm measured AI adoption rates but not AI output quality
- No incident reporting mechanism existed for AI-related errors or near-misses
- The culture celebrated AI efficiency gains without equal emphasis on AI risk management
Beteiligte & Rollen
Each participant assumes one role with distinct objectives, constraints, and private information. Roles are designed to create productive tension during discussion.
Marcus Chen — Senior Paralegal
Profil
Eight years of experience, consistently rated as a top performer. First time using the AI tool for a substantive research assignment rather than simple summarization. Was handling three urgent matters simultaneously when the assignment came in.
Ziele
- Demonstrate that the error was a process failure, not a competence failure
- Protect his professional reputation and position at the firm
- Advocate for realistic workload management and clearer AI usage protocols
Einschränkungen
Marcus knows that two other paralegals have been using the AI tool the same way — with single prompts and no verification — for months without incident. He has not told anyone this yet.
Sarah Park — Associate Attorney
Profil
Third-year associate who caught the errors during her morning review. She assigned the memo late in the day without specifying jurisdiction or expected sources. She is relieved she caught the errors but aware that she almost forwarded the memo to the partner for the client meeting.
Ziele
- Acknowledge her role in the delegation failure without accepting disproportionate blame
- Push for mandatory verification protocols for all AI-generated work product
- Ensure the firm's response addresses the systemic issues, not just the individual incident
Einschränkungen
Sarah knows the partner she reports to has been pressuring associates to use AI to increase billable efficiency. She has felt unable to push back on unrealistic turnaround times.
Diana Rivera — Managing Partner
Profil
Championed the firm's AI adoption initiative. Genuinely believes AI is essential for the firm's competitiveness but is now confronting the gap between her vision and the firm's implementation. Responsible for firm-wide risk management.
Ziele
- Contain the reputational risk and prevent external disclosure of the incident
- Implement meaningful safeguards without killing the firm's AI momentum
- Determine appropriate accountability without scapegoating individuals for systemic failures
Einschränkungen
Diana has a board meeting with the firm's malpractice insurer next week. She knows the insurer has been asking about the firm's AI practices. If this incident surfaces, premiums could increase significantly.
Dr. James Whitfield — Quality Assurance Lead
Profil
Hired six months ago to oversee the AI integration. Former legal technology consultant with experience at three Am Law 100 firms. Has been advocating for stricter protocols since his first week but has been told to 'let the team get comfortable with the tools first.'
Ziele
- Use this incident to implement the verification protocols he has been proposing since day one
- Establish a formal incident review process for AI-related errors
- Secure budget and authority for ongoing AI competency training
Einschränkungen
James has a draft AI governance policy that he submitted to Diana two months ago. It was never reviewed. He also knows that the enterprise AI vendor's terms of service have a clause about data retention that the firm has not fully evaluated.
Lernaktivitäten
Sechs progressive Aufgabentypen nach der Smoother-Methodik, die von sachlichem Verständnis bis zur beruflichen Selbstreflexion reichen.
- Read the full case narrative. Identify every decision point where the outcome could have been different.
- Reconstruct Marcus's original prompt. Then write the prompt he should have used, identifying each specific improvement.
- Map the chain of events from assignment to near-miss. Identify all actors, their roles, and the points where the chain could have been broken.
- Research the AI tool's capabilities and limitations. What does the vendor's documentation say about legal research accuracy?
- Rewrite the narrative from Marcus's perspective: What was he thinking at 11:47 PM? What did the AI output look like to someone under pressure?
- Explain why polished formatting makes AI errors harder to detect. How does professional-looking output create a false sense of reliability?
- Compare this incident to traditional paralegal research errors. What is fundamentally different about AI-generated errors versus human research mistakes?
- Create a stakeholder impact map: Who would have been affected if the memo had reached the client? Trace the potential consequences.
- Evaluate Marcus's prompt against prompt engineering best practices. Identify every deficiency and explain why each one matters.
- Assess whether the firm's AI guidelines were adequate on paper. If yes, why did they fail in practice? If no, what was missing?
- Analyze the role of time pressure in this incident. Is it possible to maintain AI output quality under real-world deadline conditions?
- Question the assumption that a verification step would have caught all errors. What kinds of AI errors are hardest to detect even with verification?
- Design a three-step AI-assisted research workflow that includes prompt structuring, output verification, and reviewer sign-off.
- Draft a revised AI usage policy for Rivera & Goldstein that addresses the specific failures exposed in this case.
- Create an 'AI Prompt Template' for common legal research tasks that paralegals can use as a starting point.
- Role-play the post-incident review meeting as your assigned character. Prepare a 3-minute opening statement.
- Compare the AI prompt templates created by different teams. Which would be most effective in preventing this specific type of error?
- Assess each team's proposed workflow against real-world constraints: time pressure, staffing limitations, and cost.
- Evaluate the revised AI policies: Do they address root causes or just symptoms? Would they survive contact with a busy Monday morning?
- Review the firm's original AI guidelines. Grade them on a rubric of clarity, specificity, enforceability, and integration with existing workflows.
- Have you ever submitted work product without fully verifying it because it 'looked right'? What made you trust it?
- How does this case change your understanding of what 'using AI responsibly' actually means in daily practice?
- Reflect on the gap between knowing best practices and following them under pressure. What systems would help you bridge that gap?
- Identify one specific change you will make to your own AI usage workflow based on this case study.
Übung: Prompt Engineering in der Praxis
Take the original failed prompt ('Summarize the law on wrongful termination and retaliation claims, including relevant cases and statutory protections') and rewrite it five times, each time improving one specific dimension: jurisdiction specificity, analytical structure, source reliability instructions, uncertainty flagging, and output format requirements. Compare your five iterations to see how each improvement changes the AI's output quality.
Referenzen & Quellen
Berufsrechtliche Standards
- ABA Model Rule 1.1 — Duty of Competence, Comment 8 on technology competence
- ABA Model Rule 5.3 — Responsibilities Regarding Nonlawyer Assistants (applies to AI tool supervision)
- ABA Formal Opinion 512 (2024) — Generative AI Tools and the obligations of competence, confidentiality, and supervision
Ressourcen zum Prompt Engineering
- Legal Prompt Engineering: Principles for Reliable AI-Assisted Research — Stanford CodeX Working Paper (2024)
- AALL Guidelines for the Use of AI in Legal Research — American Association of Law Libraries (2024)
- Thomson Reuters Practical Law — Best Practices for AI-Assisted Legal Drafting (2024)
Bereit, diesen Fall zu üben?
This case study is designed for guided facilitation as part of the Lawra Learning Program. Request a personalized session that includes prompt engineering exercises, role simulation, and expert debriefing.
Kommentare
Kommentare werden geladen...