Field Note

AI code review needs system context

A mountain ridgeline at sunset
A mountain ridgeline at sunset.

AI code review can already find obvious defects, missing checks, suspicious branches, inconsistent patterns, and code that does not do what the ticket claims.

Good. That saves time.

It is also not where the most expensive enterprise mistakes usually live.

The code can be locally correct and still be wrong for the system.

A pull request is a narrow window

A reviewer sees the diff. The production system experiences the consequence.

A clean change may still duplicate customer state, bypass a pricing authority, create an unbounded retry loop, or introduce a new deployment dependency. None of those problems has to look bad at the line level.

Human reviewers miss this too, especially when a team is moving quickly or ownership is split across services. AI can help, but only if it gets more than the changed files.

The review system needs the contracts around the code: architecture decisions, API schemas, data ownership, security rules, deployment topology, known failure modes, and the tests that represent actual business behavior.

Without that, the model is reviewing syntax with better manners.

Give the reviewer invariants

The strongest review instructions are not style preferences. They are statements that must remain true.

Examples:

  • The browser never determines the price charged to the shopper.
  • Order state is owned by the order system.
  • Payment operations must be idempotent.
  • Customer data cannot enter application logs.
  • A failed downstream call cannot leave inventory reserved indefinitely.
  • Core platform files are not modified by an extension.

These are reviewable. They give the agent a way to connect an implementation detail to a system consequence.

They also force the organization to state what it believes about the architecture. That part is uncomfortable and valuable.

Let machines check machine-checkable things

AI should not replace a linter, type checker, dependency scanner, contract test, or unit test. Those tools are deterministic, fast, and much less likely to have a philosophical disagreement with the build.

Use them first. Then give their output to the AI reviewer along with the change.

This changes the job of the model. It no longer needs to guess whether the code compiles. It can focus on the parts that require synthesis:

  • Does the change conflict with an architecture decision?
  • Is the new dependency justified?
  • Are failure and recovery paths credible?
  • Did the implementation solve the requirement or route around it?
  • What operational assumption is hidden in the code?

That is a better use of the model and a better use of review time.

Review the review

AI-generated findings need their own quality loop. Track which findings were accepted, rejected, or missed. Look for noisy rules, recurring false confidence, and categories of defects that still reach production.

Do not measure success by comment volume. A reviewer that leaves thirty comments on naming and misses a broken authorization boundary is not thorough. It is distracting.

The useful output is a smaller set of findings connected to actual risk, with enough evidence that a person can verify them.

System context is maintained, not installed

Architecture guidance goes stale. Service ownership changes. Exceptions become normal. A repository can contain three documents that disagree and still pass every build.

So the context used for review needs an owner and a maintenance path. When an incident reveals a missing invariant, add it. When a decision changes, replace the old guidance. When a review finding becomes deterministic, move it into a test or policy check.

The review system should learn by becoming less dependent on interpretation over time.

AI can make review broader and faster. Give it bad system context and it will approve the wrong thing with excellent formatting.

The documentation finally has consequences. Maybe now we will keep it current.

Field Notes

More Field Notes.