Known limitations (honest boundaries)

ReviewGate is a static LLM quality gate — it does not execute your code. The boundaries below come from real evaluations (see docs/evals/). Stating them plainly is a trust feature: we would rather be clear about what doesn't work than oversell.

1. Subtle algorithmic off-by-one / carry bugs that only surface at runtime

• Several previously missed cases (DST handling, parseISO 24:00, negative-zero diffs) now BLOCK after "execution-simulation" prompt improvements; the reverted addBusinessDays weekend fix is caught 2/2 stably.
• The --exec-verify run_check tool had a real bug (child stdout never captured or returned to the model) that was found and fixed on 2026-07-01 — execution signals now actually reach the model.
• Still true: the most subtle single-step off-by-one and digit-carry bugs can be mis-simulated. The bottleneck is whether the model proactively doubts and verifies. On large repos, individual dimensions can hit the wall-clock timeout (marked incomplete, never silently passed).
• The reliable defense remains unit tests. Do not rely on static review alone for critical algorithmic logic.

1b. Cross-file state-representation drift in large diffs

• Frontier specimen: syncthing #10170 — a bool field and a bitmask acting as dual sources of truth, drifting across 19 files where every hunk looks locally consistent. Both the baseline and a targeted extra check scored 0/2. The case stays in the recall dataset as a known-red frontier.
• Qualitatively, this class of bug needs intent context ("what was this change supposed to do") — it belongs to --intent review territory, not diff-local static analysis.

2. "Bare" dangerous calls with no context signal

• A three-line requests.get(url) with no hint that url is user-controlled will not reliably be flagged as SSRF.
• Mitigation: real code usually carries handler/parameter context (recall is good with context); --samples N stabilizes high-risk categories.

3. Run-to-run variance

• Inherent to LLMs: the set of findings fluctuates between runs; borderline confidences can flip between WARN and BLOCK.
• Mitigation: dedup + counter-evidence judging converge results; --samples N takes the union across samples and keeps the highest confidence.

4. Degraded precision tooling for unsupported languages

• tree-sitter grammars cover rust / cpp / python / go / js / ts / java. Other languages still get full LLM review, but find_definition / callers / references fall back to lexical grep, find_duplicate_functions is unavailable, and per-language rule files don't route. Adding a language = adding a grammar + extension mapping.

5. Reviews the diff, not the repository

• By default only the change under review is examined; pre-existing code is not audited. "Negative diffs" that delete security guards are specifically covered (prompt rule + verified on a real deletion-only CVE in cJSON).

6. Intent review (--intent) checklist semantics

• Intent is parsed into numbered acceptance criteria; any criterion the review didn't explicitly verdict is marked ? not assessed and degrades the result to WARN — the checklist always covers every criterion, and an unchecked one is never silently dropped.
• Model-level limitation remains: the model may spend its budget deep on one criterion, leaving others honestly marked "not assessed" — the fix is a larger --timeout or splitting the intent, not pretending it was checked.
• Intent quality bounds the output: a one-line commit message yields a few coarse criteria; written acceptance criteria yield a fine-grained checklist.

Every limitation above has an eval trail, and each lists its current mitigation. This page shrinks as versions progress — the Chinese original is here.