Research question
How often do real tagged PDFs carry structures that pass casual inspection but mislead assistive technology and conversion tools, and which pathologies dominate?
Overview
A tagged PDF is not an accessible PDF. In the wild, tag trees carry pathologies that no simple checker flags: every element tagged as a paragraph, heading levels that skip or repeat, role maps that rename standard roles into decorative ones, artifacts leaking into the reading order, and replacement text that contradicts the visible characters.
This study measures pathology prevalence across thousands of public government PDFs using the same structure-tree analysis that powers our conversion pipeline, and publishes per-pathology rates with anonymized structural examples.
The study exists because we keep hitting these pathologies in production. The registered starting case: federal fact sheets whose per-character spans carry two-character ActualText values that, honored literally as the PDF specification requires, delete entire list bodies from the extracted text.
Why this matters
Tag-checking tools assume tags mean what they say
Validators verify structural rules, not truthfulness. A document can pass machine checks while its tags actively misdescribe the content. Prevalence data tells tool builders which defenses matter and tells authors which mistakes their tools are silently making.
Conversion pipelines must decide when to trust tags
Every converter faces the same fork: trust the tag tree or re-derive structure from layout. That decision should rest on field data about how often tags lie, not on anecdotes.
What we measure
- Structural pathologies
- Flat trees, everything-as-P, heading level skips, empty structure elements, orphaned marked content
- Semantic pathologies
- ActualText contradicting rendered text, alt text on decorative furniture, tables tagged as paragraphs
- Role map abuse
- Custom roles mapped to standard roles they do not behave like
- Coverage gaps
- Share of visible characters outside the structure tree entirely
- Generator fingerprints
- Pathology rates by producing application where metadata identifies one
Method
Acquire public documents from official government URLs under the same provenance rules as the conversion pilot: recorded hashes, malware scans, no redistribution.
Run structure-tree analysis on every document: role histograms, coverage ratios, ActualText plausibility checks, heading ladder analysis.
Manually verify a stratified sample of each flagged pathology to publish a precision estimate alongside every rate.
Publish rates, methodology, and reproducible pathology descriptions. Individual documents are described structurally, not named, except where the publisher is a government body publishing public records.
Evidence so far
Only recorded findings appear here. Anything not listed has not been measured yet.
The registered starting case is documented
The August 2026 pilot A/B caught per-character spans whose ActualText deleted full text runs when honored spec-literally. The defense, an ActualText plausibility guard, is now in production and described in the public A/B report.
Independent context
Published industry studies report roughly 95 percent of public-facing PDFs failing accessibility checks and PDF/UA validators disagreeing on half of real files. Pathology prevalence is the missing layer under both numbers.
Questions we expect
Is this just a list of validator failures?
No. Most of these pathologies pass validators. The study measures cases where tags are structurally valid and semantically wrong, which is precisely the class of failure that machine checking misses.