title: "CiteWarden \u2014 Forensic Citation Checking, Because AI Fabricates References With Total Confidence" published: true description: "Paste any text; CiteWarden live-verifies DOIs, arXiv IDs, PMIDs, CFR sections and case law against real registries \u2014 and flags the fabricated ones. Built for LexHack 2026." tags: ai, security, showdev, opensource
The $5,000 hallucination
In Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023), a federal judge sanctioned lawyers $5,000 after they filed a brief citing six court decisions that ChatGPT had invented. That was 2023 — the failure mode hasn't gone away, it's just everywhere now: student essays, grant proposals, court filings, and yes, dev blog posts.
CiteWarden is the checking layer. It reads text the way a paranoid reference librarian would. Built for LexHack 2026 (AI × Law × Civic Tech).
Try it: rudra496.github.io/citewarden
What it checks — against live registries
| Citation family | Verification path |
|---|---|
| DOIs | Live Crossref lookup → OpenAlex fallback. 404 on both = flagged as non-existent |
| arXiv IDs | Via their registered DOI mirror (10.48550/arxiv.*) in OpenAlex |
| PubMed IDs | Live NLM E-utilities summary |
| CFR sections | Official eCFR structure API |
| U.S. reporter cites (U.S., F., F.2d, F.3d, S. Ct., F. Supp.) | 464-case verified landmark database + forensic bounds checks |
| Case names | Landmark DB match + documented-fabrication watchlist |
| UK acts / EU regulations | Verified statute list + live Wikipedia/ELI record checks |
The offline forensic layer
Beyond "does this exist," CiteWarden runs fully offline forensics: reporter volume-range bounds, reporters cited before they existed (any F.3d cite dated before 1992 is impossible), decision years contradicting the volume era, malformed titles, placeholder artifacts, and same-cite/conflicting-year collisions.
The integrity score is transparent
- Green = matched a live registry or the verified database
- Amber = well-formed but unverifiable — honest uncertainty, never guessed
- Red = impossible values, watchlist hit, or the registry confirms non-existence
Score = 100 − 100·(red/total) − 30·(amber/total), floored at 0. You can compute it yourself from the report.
Zero-invention data policy
Every shipped row was machine-checked at build time: the 464 landmark cases were parsed from Wikipedia's curated index with volume-bounds, volume-era consistency, and year sanity gates (1791–2026); build logs live in research/. The Mata v. Avianca watchlist contains the six fabricated citations transcribed verbatim from the sanction order.
Links
- Live: rudra496.github.io/citewarden
- Source: github.com/rudra496/citewarden
If you review manuscripts, grade papers, or file briefs — paste your next document in before someone else finds the problem.
Comments(0)
Join the discussion securely with your ZyVOP account.
Continue on ZyVOP to CommentNo comments yet. Be the first to comment!