ReviewGate is built for the "AI changed many files at once" reality, so it has an adaptive pipeline for diffs that exceed the model's context window. The design goal: rather mark the review as unfinished than let a big PR through as a clean PASS.
Mechanism
- Review units (
plan_units) — the input budget is the provider'smax_input_tokens(default 200k). A diff within budget is one unit (zero degradation for normal PRs); over budget, files are bin-packed by directory proximity into multiple units reviewed separately. - Fixed-overhead reservation — splitting reserves room for the system prompt and dimension focus (
plan_budget = budget − overhead), guaranteeing every unit can be sent on its first round. With the default 200k budget this is an invisible no-op. - Sampling pinned to 1 for multi-unit reviews — big PRs don't get multiplied by
--samples; multi-sampling is reserved for single-unit (normal) PRs where it stabilizes flaky recall. - Per-unit adaptive degradation — a unit's prompt starts with full new-file context; if over budget it falls back to diff-only; if a single file's diff alone exceeds the budget, that unit is skipped and marked
oversized— counted as incomplete, never silently passed. - Pre-flight check every agent round — during review the agent pulls context on demand (read_file / grep / navigation). If the conversation would exceed budget, the next round gracefully wraps up: findings gathered so far are kept, nothing crashes or truncates silently.
The guarantee: never a silent pass
If any unit is skipped, timed out, failed, or wrapped up early, the run always ends with incomplete = true plus explicit warnings (oversized / timed_out / failed), the verdict degrades to WARN (never PASS), and with --fail-on the process exits non-zero so CI cannot misread "unfinished" as "clean".
Stress tests (2026-06-27)
| Target | Lang | Size | Budget | Units | First-round over-budget | Findings | Verdict |
|---|---|---|---|---|---|---|---|
axios 847d89b | JS | 9 files | 4k | 8 | 0 | 13 | WARN + incomplete |
requests f8bec2f | Python | 19 files | 6k | 2 | 0 | 2 | WARN + incomplete |
axios HEAD~30..HEAD | JS | 55 files / ~5,000 lines | 16k | 8 | 0 | 12 | WARN + incomplete |
ripgrep HEAD~40..HEAD | Rust | 28 files | 12k | 4 | 0 | 6* | WARN + incomplete |
gin HEAD~80..HEAD | Go | 46 files / ~2,500 lines | 16k | 6 | 0 | 13** | BLOCK + incomplete |
curl HEAD~30..HEAD | C | 132 files / ~2,400 lines | 12k (tight) | 11 | 0 | 15*** | WARN + incomplete |
* Includes a real hallucinated-API catch (as_bytes_mut() doesn't exist on [u8]).
** Includes a real Slowloris DoS (a #nosec G112 suppression with no timeout set) and copy-paste residue referencing a nonexistent maskHeaders() — big PRs can BLOCK, not just WARN.
*** The honest counter-example: 12k budget is too tight for 132 dense C files; most units exhausted budget while fetching context, so real findings ≈ 0. Still safe — WARN + incomplete, not a silent pass. Match max_input_tokens to diff density (see tuning).
Takeaways
- Validated across 5 languages, up to 55 files / ~5,000 lines and 11 units: after unit splitting, first-round over-budget events dropped to zero at every scale.
- With adequate budget, splitting does not hurt recall — real findings (hallucinated APIs, copy-paste residue, a Slowloris DoS) surfaced, and big PRs can still BLOCK.
- With an under-sized budget the run degrades to "mostly unreviewed" — but says so, instead of impersonating a PASS.
The oversized-single-file path
When one file's diff alone exceeds the budget (nothing left to split), it is individually skipped and named. Tested on axios de1a8100 with a 2,467-line package-lock.json at an 8k budget: three lockfiles (~53k / 11k / 11k tokens each) were skipped one by one with per-file callouts, the small files in the same commit were reviewed normally with no false positives, and the run ended WARN, incomplete = true. When AI touches a huge generated file: no crash, no silent pass — you're told exactly which files went unreviewed, with the suggestion to split the change or raise max_input_tokens.
Tuning
| Knob | Effect |
|---|---|
max_input_tokens (provider config) | The input budget that decides when to split. Match it to diff density: too tight and units exhaust budget while fetching context (see the curl row). Small-context models can lower it; large models keep the 200k default. |
--samples N | Per-dimension sampling for single-unit PRs (union of runs, cost ×N); automatically pinned to 1 for multi-unit reviews. |
--timeout S | Wall-clock cap per dimension; a timed-out dimension is skipped, the rest are kept, and the run counts as incomplete. |
--fail-on block|warn|never | Which verdict makes CI exit non-zero; unfinished reviews degrade to WARN. |
Per-run raw records live in docs/evals/ (files named 2026-06-27__bigpr-*.md). 中文原版在这里。