Confidence & Proof Tiers
Sigilix does not rank findings on a numeric 1–5 confidence scale. A number is just the model’s opinion about its own opinion. Instead, every posted finding carries a proof-tier receipt that records how strongly it was witnessed — and a finding that cannot be witnessed at all does not post.
The principle: the model’s job is interpretation; the deterministic layer is the witness. A finding is only as believable as the evidence the deterministic layer can attach to it.
Proof-tier receipts
Every finding that survives to the PR carries one of four tier pills:
| Tier | What it means | How it’s earned |
|---|---|---|
| VERIFIED | Corroborated by an independent receipt | A scanner in your own CI confirmed the same finding — a signed runner receipt (e.g., a SARIF result) joined to the finding. The strongest witness: two independent systems agree |
| REPRODUCED | Demonstrated by execution | Sigilix’s own sealed execution sandbox re-ran the cited code and reproduced the claimed behavior — the bug was shown, not argued. See Verification & the Trust Ledger |
| GROUNDED | Anchored to cited code / evidence in the diff | The finding quotes specific code that exists at the cited location, or a deterministic check (secret scan, AST rule, dependency advisory) backs it; the grounding gate matched the claim to the source |
| MODEL | Model judgment, no external witness | A specialist’s reasoning the deterministic layer could not independently confirm — still useful, but explicitly labeled as opinion |
The tier is a receipt, not a score. It tells the reader why they should believe a finding, not how confident the model claims to be. A REPRODUCED logic bug and a MODEL-tier naming suggestion are both legitimate; the pill lets you triage at a glance.
Two properties of the tiers are architectural, not stylistic:
- A receipt always outranks self-attestation. A finding corroborated by your own CI (VERIFIED) outranks one Sigilix reproduced itself (REPRODUCED), which outranks one that is merely well-anchored (GROUNDED). The hierarchy is about independence of the witness.
- Tiers only move up. Once a finding has earned a tier, a later, weaker signal can never demote it. Signals are folded together by keeping the strongest witness.
The grounding gate
Between a specialist producing a finding and that finding posting, the synthesizer runs the grounding gate: the finding must be anchored to the code it claims to be about.
| Check | Outcome |
|---|---|
Finding’s path:line exists in the diff | Eligible to anchor |
Finding’s path:line is hallucinated | Dropped |
| Cited code quote matches the source (whitespace-normalized) | Earns GROUNDED |
| Cited code quote paraphrases / doesn’t match | Demoted to MODEL or dropped |
| Claimed unsafe pattern actually present | Confirmed; eligible for REPRODUCED or VERIFIED if an execution or receipt check runs |
| Claimed pattern absent | Suppressed before it reaches you |
The gate is what makes Sigilix’s reviews trustworthy: a finding that survives it is one whose subject genuinely exists in your code. Findings the model invented — an “unused” function that is called, a “missing” guard that is present — are caught here and never posted.
The gate score is deterministic, not self-reported
Whether a finding clears a gate is driven by a deterministic score computed by the pipeline, not by the model’s own confidence number. The score blends four signals:
- The specialist’s self-reported confidence — rescaled conservatively, so a model claiming maximum confidence still doesn’t saturate the score
- Whether the finding carries its required evidence (cited code, a deterministic-fact reference)
- Agreement — a bounded bonus when additional specialists flag the same code
- Severity — higher-stakes categories shift the threshold
The model’s opinion of itself is one input among four, and the weakest one. This is the practical meaning of “the model’s job is interpretation; the deterministic layer is the witness.”
Dismissals feed a false-positive ruler
What happens after a finding posts also feeds back. When your team dismisses a finding, the dismissal is recorded with a bounded reason — not a bug, bad anchor, already covered, too minor, wrong context — and those reasons drive an internal false-positive rate metric, computed per specialist with statistical confidence intervals and treated as a lower bound on the true rate. Repeated dismissals of a category on your repo also soften future findings of that category — see Review Memory. The pipeline is tuned against this ruler: precision regressions show up as a moving number, not anecdotes.
Grounding is content-matched, not line-counted. Models routinely miscount line numbers but quote code verbatim, so the gate matches the quoted code against the source to anchor the finding precisely — including to deleted lines (side: LEFT) when the finding’s subject is removed code.
Agreement still escalates severity
Multiple specialists flagging the same code is a strong signal, and it raises severity (independent of proof tier):
| Inputs | Severity |
|---|---|
| 1 specialist, low confidence | Info |
| 1 specialist, high confidence | Warning |
| 2+ specialists agree | Warning or Critical (by category) |
| Specialist + deterministic check confirms | Critical |
Severity (Info / Warning / Critical) is the categorical label the reader sees. The proof tier is the separate witness pill. A finding can be Critical-and-MODEL (high stakes, model-only evidence) or Info-and-VERIFIED (a confirmed nit) — the two axes are orthogonal.
The verdict
The verdict follows from severity, with the grounding gate having already removed unwitnessed Critical claims:
- Any Critical that cleared the grounding gate → Request changes
- Otherwise → Approve
A Critical finding the gate could not ground is demoted or dropped before it can block a merge — so the verdict is never decided by a hallucination.
Advisory cap
To prevent comment dilution, the synthesizer surfaces the top few low-severity (Info / nit) findings inline and aggregates the rest into a single “advisory nits” line in the summary. The remainder are recorded in telemetry, not posted. Switch profile to assertive if you want low-severity findings surfaced more aggressively.
Tuning what posts
The proof-tier and grounding logic is fixed — it’s the believability guarantee, not a knob. What you can tune via sigilix.json:
profile—"chill"(default, only must-fix) vs"assertive"(include nits + style). Shifts what each specialist flags in the first place.rules.<role>— Per-specialist prompt addendum. Influences what each specialist chooses to flag.deterministicChecks— Regex rules whose severity you set explicitly. These post at the severity you declare and carry a deterministic receipt (GROUNDED, backed by the check itself) — they bypass model judgment entirely.
Telemetry
Every finding’s lifecycle is recorded: which proof tier it earned, whether the grounding gate matched / demoted / dropped it, any agreement escalation, final severity, and whether it was posted, suppressed, or aggregated. Telemetry is internal — used for monitoring and prompt-engineering.