Find Recurring Failure Patterns in AI Outputs

Worked example

Task context

A user notices that AI-generated drafts keep sounding polished but vague. Instead of editing every draft manually, collect failures and detect the repeated pattern.

Open the related tool: AI Recurring Failure Pattern Detector

Input and output

Sample 1: sounds confident but no evidence.
Sample 2: starts with 'Here is an improved version' and repeats the user's goal.
Sample 3: gives a plan but no verification step.
Sample 4: says 'this will increase traffic' without data.

Recurring pattern: polished wrapper + unsupported outcome claims + missing verification.
Fix list: remove wrapper phrases, require evidence labels, add proof-of-done, add stop condition for outcome claims.
Regression test: feed a new draft and check whether it still contains wrapper, unsupported promise, or no verification step.

Checks before copying

• Use at least three samples before calling something a pattern.
• Separate style problems from trust problems.
• Turn patterns into prompt rules or review checks.
• Keep one regression test so the fix can be checked later.

Lesson: Repeated AI mistakes should become a small review system, not a pile of manual edits.

Keep working