Show model-judged code review results

kleb commented

2026-11-04 12:09:33 +01:00

Owner

The code review page showed title-pattern target hits, which count matching titles, not whether a finding is right. You asked for numbers that mean something, from the saved reviews and without rerunning them.

GPT-6.1 Sol High and Claude Opus 5.5 High now judge every saved finding of the 28 configurations (7 models, 4 efforts, 64 public pull requests). Their verdicts are frozen and scored offline. The page now shows, per configuration: known bugs found out of the 42 references both judges accept (solid when both judges credit a bug, faded when only one does), valid, disputed and invalid findings, and the original call time. A Method view lists the judging setup and its limits. The numbers are model-judged, not checked by a person, and the page says so.

The run is a new cohort with its own run ID; the existing title-pattern run and kReview reference stay unchanged. The evidence file holds hashes and counts only, no review text.

Two judge changes made Opus usable. Opus requests now carry the Claude Code identity block, because the pool's Claude Max OAuth credential answers Opus requests without it with HTTP 429. That works but is a gray area under the subscription terms; a real API key would be the clean route next time. Opus also miscopied 64-hex finding IDs, and a hash enum was too large to compile, so its findings use short aliases with an enum and are mapped back before validation.

Proven by replaying the scores twice offline with networking disabled (byte-identical), by bun test, typecheck and build, 475 pytest cases, and by checking the built page in a browser in light, dark and mobile layouts.

The code review page showed title-pattern target hits, which count matching titles, not whether a finding is right. You asked for numbers that mean something, from the saved reviews and without rerunning them. GPT-6.1 Sol High and Claude Opus 5.5 High now judge every saved finding of the 28 configurations (7 models, 4 efforts, 64 public pull requests). Their verdicts are frozen and scored offline. The page now shows, per configuration: known bugs found out of the 42 references both judges accept (solid when both judges credit a bug, faded when only one does), valid, disputed and invalid findings, and the original call time. A Method view lists the judging setup and its limits. The numbers are model-judged, not checked by a person, and the page says so. The run is a new cohort with its own run ID; the existing title-pattern run and kReview reference stay unchanged. The evidence file holds hashes and counts only, no review text. Two judge changes made Opus usable. Opus requests now carry the Claude Code identity block, because the pool's Claude Max OAuth credential answers Opus requests without it with HTTP 429. That works but is a gray area under the subscription terms; a real API key would be the clean route next time. Opus also miscopied 64-hex finding IDs, and a hash enum was too large to compile, so its findings use short aliases with an enum and are mapped back before validation. ![review-bugs-light.png](https://git.kleb.sh/attachments/475d77d7-5df4-49fd-878e-076255c7e06c) ![review-findings-dark.png](https://git.kleb.sh/attachments/1be1b678-2658-4ae7-92b2-811b85000ba4) ![review-mobile.png](https://git.kleb.sh/attachments/8837c882-b8aa-430a-b5ec-9cdba05ce018) Proven by replaying the scores twice offline with networking disabled (byte-identical), by bun test, typecheck and build, 475 pytest cases, and by checking the built page in a browser in light, dark and mobile layouts.

review-bugs-light.png

340 KiB

review-findings-dark.png

153 KiB

review-mobile.png

50 KiB

kleb added 1 commit

2026-11-04 12:09:33 +01:00

Show model-judged code review results ff4b8415d3

The review page showed title-pattern target hits, which do not measure
whether a finding is right. GPT-6.1 Sol High and Claude Opus 5.5 High now
judge every saved finding of the 28 configurations; their verdicts are
frozen and scored offline. The page reports known bugs found, valid,
disputed and invalid findings, and the original call time.

The judge sends the Claude Code identity block to Opus, because the pool's
Max OAuth credential otherwise answers 429. Opus findings use short
aliases with an enum, since it miscopies 64-hex IDs and a hash enum is too
large to compile. Rejected decisions no longer stop other dispatches.

kAgent commented

2026-11-04 12:09:33 +01:00

First-time contributor

kReview review

Verdict: no findings

No concrete defect was proved by the diff and repository evidence reviewed.

kReview checked only the PR diff, metadata, and supplied repository context at ff4b8415d3. It reports only issues supported by this material.

Estimated cost: ~$0.62 (386.1k in / 1.3k out / gpt-6.1-sol).

Run time: 1:08 total; this run 1:08.

## kReview review **Verdict:** no findings No concrete defect was proved by the diff and repository evidence reviewed. _kReview checked only the PR diff, metadata, and supplied repository context at `ff4b8415d3`. It reports only issues supported by this material._ _Estimated cost: ~$0.62 (386.1k in / 1.3k out / gpt-6.1-sol)._ _Run time: 1:08 total; this run 1:08._

kleb merged commit 7a0c530cba into main

2026-11-04 12:10:56 +01:00

kleb referenced this pull request from a commit

2026-11-04 12:10:57 +01:00

Merge pull request 'Show model-judged code review results' (#11) from feat/review-model-judged-results into main

Rows
Columns

Show model-judged code review results #11

kReview review