The receipts. Named runs of the eval suite, with their numbers.

The runs

Each recall number is the result of that named run on that corpus. It is not a product accuracy rate, and results on your code depend on your code. New runs of the customer setup get appended to this table as they happen, worse results included.

Fabricated counts a finding published on a benign plant. On the founding runs of 25 September 2026, all 18 benign look-alikes stayed silent. The internal gate behind a configuration change sets a recall bar of 83.3% on the general corpus and a bar of 10 of 12 on the frozen authorization slice, with zero fabricated; a published finding that does not match a planted line fails a run too.

Download this table: JSON or CSV. Both files are generated from the same data as this page.

Honest limits

The corpus is small: single-flaw planted pull requests of a few thousand tokens each, so these numbers rank configurations against each other; they are not a forecast of results on your pull requests. The founding runs of 25 September 2026 are two runs of this configuration; the wider record shows run-to-run swings of up to four plants on other lanes, so read each number as that run's result. The thinnest class is path traversal: the founding runs found one of two traversal plants each, and the second is a consistent miss. The judge is strict by design: a finding that names the right bug but does not quote the planted line counts as a miss, and one plant in the corpus carried a second real bug that produced a true extra finding in a replication run; the record keeps both facts. Zero fabricated over 18 benign plants is the precision record of the founding runs; it says nothing about precision on unseen code. The runs of 1 October 2026 used the second engine and a new corpus: multi-line flaw plants written into real repository files, adversarial clean controls (authorization in an upstream wrapper, a check moved into a helper, row-level-security-backed reads, and pre-existing bugs staged beside a touched line), and unproven-sink plants whose wanted outcome is an unverified listing rather than a published finding; the engine listed all 3 unproven-sink plants in every repeat and published none. The release gate for this corpus sets a bar of 20 of 24 published on the free-tier lane. The free-tier run of 1 October published a median of 16 of 24, below the bar, and same-day runs on successive builds swung from 16 to 20; the paid-plan lane published a median of 21 of 24 with zero published false positives across 72 clean-control runs. Both rows stay in the table.