A versioned collection of real-world, standards, malformed, and deliberately stressful PDF inputs. It turns PDF reliability into something that can be measured, repeated, and compared.
KillerPDF v1.8.6 results
- 40,563 PDFs saved across the three main collections.
- Exact 1.8.4 outcome match: every file kept the same status and diagnostic detail.
- Repeatable results: all five measured runs matched for every file.
- No crashes or timeouts across 80 damaged inputs.
Tested September 22, 2026. Each main collection had one warmup and five measured runs.
Downloadable regression set
- PDFs tested
- 16,696
- Saved
- 15,171
- Skipped
- 1,233
- Save failed
- 292
- Median full set time
- 2m 49s
Standards and color set
- PDFs tested
- 649
- Saved
- 377
- Skipped
- 235
- Save failed
- 37
- Median full set time
- 5.5s
Private stress set
- PDFs tested
- 29,599
- Saved
- 25,015
- Skipped
- 2,957
- Save failed
- 1,627
- Median full set time
- 8m 58s
Damaged-file safety test
- PDFs tested
- 80
- Rejected or skipped
- 78
- Save failed
- 2
- Crashes
- 0
- Timeouts
- 0
- Timing versus v1.8.4: regression was 18.2% faster, stress 3.4% faster, and standards/color 1.9% faster.
- Coverage: batch opening and saving. Rendering and PDF conformance require separate checks.
Read the full benchmark report, measured runs, and test method.
View all benchmark resultsWhere the PDFs came from
The full run includes 17,328 downloadable inputs and 29,696 locally held PDFs. The local files cannot be redistributed; their sources and counts are listed below.
Downloadable regression set
16,696 downloadable files from real test suites, generated examples, and openly licensed documents.
- iText 76,695
- arXiv Creative Commons batches3,409
- veraPDF corpus2,907
- Mozilla PDF.js874
- PDFium test corpus823
- qpdf684
- pdfcpu453
- xberg test documents326
- Apache PDFBox162
- Apache FOP PDF Images55
- py-pdf sample files34
- pikepdf33
- Asymptote gallery33
- OCRmyPDF31
- Apache Tika30
- Apache FOP26
- gendx25
- libHaru20
- PDF Tools Kit17
- HummusJS14
- pdf-email-optimizer12
- img2pdf9
- OpenPrinting7
- OpenPDF6
- pdf-lib5
- PDF Association SafeDocs2
- PDF Differences2
- PDFKit2
Standards and color set
649 files tested: 552 downloadable standards PDFs and 97 locally held color-suite PDFs.
- OpenPreserve Format Corpus282
- Ghent PDF Output Suite 5.093
- Synthetic PDF Testset88
- Techniques for Accessible PDF82
- PDF Differences34
- BFO PDF/A Test Suite33
- PDF Association SafeDocs23
- PDF 2.0 examples7
- Altona Test Suite 1.2 and 2.04
- PDF COS Syntax fixtures2
- Apache Tika1
Private stress set
29,599 locally held files from the PDF Association Stressful PDF Corpus. They are included in the full run but not in the public download.
- PDF Association Stressful PDF Corpus29,599
Damaged-file safety test
80 downloadable files kept separate from normal documents so damaged inputs can be tested behind a timeout.
- qpdf fuzz corpus77
- iText 71
- PDF Tools Kit1
- pikepdf1
Every public file has its source path, pinned revision, byte count, and SHA-256 fingerprint in the release manifests. The complete source record links each upstream project and explains its license, accepted files, duplicates, and exclusions.
Developer guide
The guided runner turns the corpus into a repeatable open-and-save benchmark for KillerPDF, qpdf, or another command-line PDF tool.
1. Start the runner
Use the Copy run command button above and paste the command into PowerShell. It downloads the current runner to your temporary folder and starts it. Windows PowerShell 5.1 and PowerShell 7 are supported.
Prefer to inspect or save it first? View the PowerShell script.
2. Choose the PDF tool
KillerPDF is checked in its standard Windows install locations. qpdf is checked on PATH and in the runner's own tool cache. If qpdf is somewhere else, choose it from the menu and then select Locate an existing qpdf.exe. The runner can also download and verify either built-in tool when needed.
Another command-line PDF tool can be connected through a reusable adapter. The next section shows the complete setup.
3. Choose the collections and run count
Select the general regression set, standards and conformance files, malformed security files, or every public collection. Five measured runs are the standard comparison run. Each normal collection also receives one warmup pass.
4. Choose where the corpus lives
The default corpus folder is %LOCALAPPDATA%\KillerPDF-Corpus\corpus. Developers who want the collection on another drive can launch a saved copy of the runner with:
5. Read and compare the results
Results are written under %LOCALAPPDATA%\KillerPDF-Corpus\benchmarks. The detailed CSV files contain each measured run. The summary CSV files contain median timing, throughput, outcome counts, tool version, and executable SHA-256. Keep the same corpus version, adapter, run count, and workstation conditions when comparing releases.
.partial files, which the runner removes when cancellation completes.The corpus files themselves are not tied to the runner. Any PDF application can open them manually. GUI-only applications still require manual testing when they do not expose a reliable command-line or automation interface.
Adapt another PDF tool
A tool can be automated when it accepts an input PDF, writes a separate output PDF, and finishes without interactive prompts.
Create the adapter
- Choose Custom command-line tool from the first runner menu.
- Choose Create a new adapter.
- Enter the full path to the tool's executable and a name for the result files.
- Enter each command-line argument separately and in the order required by the tool.
- Enter the exit codes that mean saved or saved with a warning, then choose a per-file timeout.
Map the input and output files
Use {input} where the application expects the source PDF and {output} where it expects the saved PDF. If the tool normally runs like this:
enter these four arguments when the wizard asks for them:
What the runner does with it
The runner launches the executable directly once per PDF. It does not evaluate a PowerShell or command prompt string. For each file it substitutes full temporary paths, applies the timeout, records the exit code, and checks that the output PDF exists before counting the operation as saved.
The adapter is tested with one corpus file before the full benchmark begins. The runner reports the validation result and asks whether to continue. Created adapters are stored under %LOCALAPPDATA%\KillerPDF-Corpus\adapters and appear in the menu on later runs.
Run a saved adapter without the menus
The adapter result files record saved outputs, warnings, failures, timeouts, timing, the executable version, and its SHA-256 hash. Open the complete adapter file reference and example.
What the results mean
Each adapter preserves the outcomes its tool can report instead of pretending every PDF application behaves the same way.
KillerPDF: OK, skipped, and failed
OK means KillerPDF opened and saved the input through the tested batch path. Skipped means it intentionally refused the input or could not enter that path, such as an encrypted, unsupported, or unreadable file. Failed means the save began but did not complete.
qpdf: saved, warning, failed, and timed out
Saved means qpdf produced an output without warnings. Warning means it produced an output while reporting a recoverable problem. Failed means it produced no usable output. Timed out means the per-file safety limit stopped the process.
These results measure safe automated open and save behavior. They do not claim rendering, text extraction, layout reconstruction, or semantic table recovery.