[Transparency]

The evaluation record. Scope, method, the full matrix, and how to check it.

Aevral is not open source. The scanner engine, the planted-vulnerability corpus, and the judge stay private. This page is what we publish instead: what Aevral scans, how a scan works, the complete frozen evaluation matrix including the configurations that failed, an integrity digest over the private eval assets, and a protocol for anyone who wants to reproduce the numbers.

Version

Report version

2026-09-01.2

Corpus integrity digest

63b1afc64791d6b3d976fd34950282475b4f21e445623fc69d761cacd721cd9c

sha256 over 18 private eval assets (plants, judges, runners, tests). Same digest, same frozen assets.

Not open source

The engine, the planted-flaw corpus, and the judge stay private. This page is what we publish instead.

Version 2026-09-01.2 (1 September 2026): internal operational cost figures, which are not part of the evaluation methodology, have been removed from this report; all evaluation figures, recall and fabrication data, wall-clock results, the reproduction protocol, and the asset digest are unchanged.

Scope and limits

What we scan

Authorization: who can reach what, across files and across modules.
IDOR: object references a caller is not entitled to.
Business-logic access control: role propagation, tenant headers, fail-open guards, negative-quantity and similar logic flaws.

What we do not scan

Memory corruption and binary-level flaws.
Injection in its classic forms and general SAST.
Anything outside the repository: infrastructure, runtime, and deployment are out of scope.

Honest limits

Findings are findings, never verdicts of certainty. A single-pass scan can miss, and re-scans of the same code have produced different finding counts.
A scan that could not finish says so. We print scan_incomplete with the named gaps (failed batches, unresolved hints, files not analyzed) instead of hiding them. An incomplete scan is not a clean bill.
Broader finding classes stay coming soon until an evaluation gate for them is green and published here.

How a scan works

1Probe and snapshot: the repository is resolved to a default-branch commit and snapshotted before analysis.
2File classification: source files are classified by machine into six classes. The bypasses are declared, not silent: tests and eval fixtures are excluded, and secret, vendor, and build paths are skipped by name.
3Trust boundaries: an inventory pass marks trust boundaries over a compact skeleton. This pass is model-assisted and its output is recorded as input to analysis, never as a finding. If inventory fails, the scan is marked incomplete.
4Clustering with deterministic edges: files are grouped by location plus imports, route registrations, and server entry points. The edges are mechanical, not model decisions.
5Whole-file batching: clusters are sent as whole-file batches, attack-surface files first, under a hard character budget.
6Cross-batch candidates: sibling summaries make flows that cross batch boundaries visible as candidates, never as findings.
7Boundary stitching: cross-batch hints are resolved mechanically into stitched reads; unresolved hints force an incomplete verdict, not a guessed finding.
8Deterministic correlation: merge and dedup are mechanical. No model adjudicates a finding into or out of the report.
9Grounding: every published finding is grounded verbatim in a single context the model actually saw. If it cannot be quoted from the snapshot, it is not published.
10Run limits: every evaluation run operates under hard time and compute cutoffs. The applicable cutoffs and any incomplete coverage are recorded with that run's results.

Evidence contract

Every finding in a report is written as evidence you can re-verify, not a sentence to trust:

Finding class and title: what broke, in the class we scan (authorization, IDOR, business-logic access control).
Location: file and line range in the scanned snapshot.
Verbatim evidence: the snippet, quoted from one context the model saw. If it cannot be quoted, it is not published.
The path: entry, the missing check, and the sink, when the flow crosses files.
A fix prompt: paste-ready text for the coding agent you already use.
Fileable: the report names the class and evidence in the shape a security team needs to re-verify it in its own review process. We do not attach control-mapping labels to findings today.

Frozen evaluation matrix

The complete frozen matrix, including the configurations that failed. Publishing only the green row would make this page a brochure.

Corpus	Effort	Gate	Recall	Fabricated	Wall	Coverage
small	none	PASS 12/12	-	0	~2 min total	complete small-corpus run
monorepo	none	PASS	10/10 (100%)	0	44s	38/38 files, 37/37 batches, 8 stitches, 1 unresolved hint, honest scan_incomplete
monorepo	low	FAIL	8/10 (80%)	1	106s	11 unresolved hints
monorepo	medium	FAIL	6/10 (60%)	0	~150s	14 unresolved hints

The corpus is private: plants of authorization, IDOR, and business-logic flaws, plus benign files, in a 38-file monorepo layout where the flaw entry and its sink are provably in different batches.
The judge is mechanical, not the model under test and not a second model's opinion.
Effort none is the production configuration. More reasoning made recall worse at every level (10/10 to 8/10 to 6/10) and did not solve fabrication: effort low fabricated one finding, effort medium dropped recall further. The rows that fail stay on this page.
Every frozen configuration remains in the matrix, including the failed rows. Publishing only the production configuration would turn an evaluation record into a brochure.

Reproduction protocol

Any independent evaluator may supply or escrow an unseen corpus of repositories with planted authorization, IDOR, and business-logic flaws.
The corpus runs against Aevral as a normal hosted scan. We do not pick the corpus and we do not see the plants in advance.
The evaluator may publish aggregate recall and fabrication numbers, without our approval.
Reach us through the contact form to set it up.

Versioning policy

This report is versioned. A new immutable version is published whenever the engine, the ruleset, the model, or any eval asset changes; the version and digest above always identify the exact state. Previous versions stay archived. We do not silently edit a published report.