Failure Mode Mapping for AI Features
AI features fail differently than traditional software. Mapping failure modes before you build is one of the highest-leverage AI PM activities.
Why AI fails differently
Traditional software fails loudly: a null pointer exception crashes the app. You know immediately something went wrong.
AI fails quietly: the model returns a plausible-looking wrong answer. The user may not notice. The error propagates silently through their workflow.
In legal documents, this asymmetry is dangerous. A software crash in a document editor is annoying. An AI-suggested clause that subtly shifts liability is catastrophic — and it looks exactly like a correct suggestion.
In Go, 'dame' are neutral points — but running out of dame (shortage of liberties) can suddenly make a living group vulnerable. AI failure modes are like dame: individually harmless-looking, but they can combine into a serious vulnerability.
A failure mode taxonomy for AI features
Use this taxonomy to systematically map failure modes before writing the PRD:
False positives (wrong alert): The AI flags something that isn't actually a problem. Result: alert fatigue, user stops trusting the AI.
False negatives (missed problem): The AI misses a real problem. Result: user over-relies on AI and misses errors it should have caught.
Distribution shift: The AI was trained on contract type A but is now used on contract type B. Result: degraded performance in ways that aren't obvious.
Adversarial inputs: Unusual formatting, mixed languages, scanned PDFs, or non-standard clause structures confuse the parser. Result: incorrect or absent suggestions.
Model drift: The model's accuracy degrades over time as the distribution of real documents drifts away from training data. Result: gradual quality decline.
Edge case cliffs: The model works well on 95% of inputs but fails dramatically on a specific pattern. Result: a specific user segment has a terrible experience.
The pre-mortem technique
Before launching any AI feature, run a pre-mortem: assume it's 6 months post-launch and the feature has failed badly. Work backwards: what happened?
For a BoostDraft proofreading feature, a pre-mortem might surface: - 'A major law firm disabled the feature because it flagged intentional non-standard formatting as errors 40% of the time' → need to train on more firm-specific styles - 'A Japanese in-house counsel team found the Japanese proofreading missed an entire class of errors common in Japanese corporate contracts' → need Japanese legal domain experts in the annotation process - 'A user sued claiming they relied on BoostDraft's AI and missed a critical clause' → need to review liability language in terms of service
Premortem outputs feed directly into the PRD's risk section and acceptance criteria.