Skip to main content
ariefrse.my
HomeSkillsProjectsAI WorkWritingContact
Resume
All writing

Note · Agentic engineering

Letting an AI hunt for the bugs humans skim past

Parity bugs are exactly what humans skim and agents grind through. The most valuable finding was the fix a human had to revert.

July 3, 2026 · 5 min read

A gap-finder audit is a focused agent task: compare every code path in surface A against its equivalent in surface B, and report behavioural gaps.

It works because parity bugs are the single thing humans are worst at. Comparing two hundred code paths for subtle disagreement is tedious in a way that makes attention slide off, and tedium is not a constraint an agent has.

On my HRMS the brief was "compare every API code path against its Filament admin-panel equivalent." Three findings shipped as fixes.

FindingConsequence
Payroll parity gapThe API path was missing claims reimbursement and unpaid-leave deductions. Two code paths computed different salaries for the same employee
Onboarding gapEmployees created through the API never received the welcome invite the admin flow sent
Exposed routesA resource controller had endpoints that were implemented but never meant to be exposed

The payroll one is the reason I keep doing this. Two code paths, same employee, different salary, and nothing crashes. Nobody finds that by reading.

The part that matters: the revert

One agent fix was technically sound and organisationally wrong.

Facing an approval deadlock, no approvers configured, it fell back to "any manager-role user can approve." No crash. Tests pass. Reasonable-looking code.

But approvers must be explicitly configured, never inferred from a role. That is not a technical fact about the system, it is a fact about how the company wants authority to work, and it is not written down anywhere the agent could read it.

Human review caught it, reverted the same day

That is the point of human-in-the-loop. Agents optimise for "no deadlock." Humans know which deadlocks are load-bearing.

Writing a good audit brief

Name the two surfaces precisely. API versus admin panel. Web versus mobile. Vagueness here produces findings about whitespace.

Ask for behavioural gaps, not style differences. Otherwise you get a hundred notes about naming conventions and none about payroll.

Require each finding to state three things: the gap, the user-visible consequence, and the proposed fix. Then reviewing is triage rather than re-derivation, which is the difference between an audit that saves time and one that generates homework.

What I learned

Agents and humans are good at opposite halves of auditing. The agent never gets bored grinding through hundreds of code-path comparisons. The human knows which of its findings, and which of its fixes, the business can actually live with.

I would trust neither alone.

agentsauditcode-reviewhuman-in-the-loop

Share

XLinkedInWhatsAppThreads

Related

  • The sweep that found managers could pay other teams' claimsThe scariest findings were not crashes or wrong numbers. They were correct-looking actions performed by the wrong person.
  • Teaching AI assistants to remember past mistakesAgents forget between sessions. The repo does not. Every regression a reviewer catches becomes a constraint the next session starts with.
  • An HR platform where seven in ten commits came from an agentSeven in ten commits came from an agent. Zero in ten decisions about what "correct" means for the business did.

© 2026 Muhammad Arief Roseli

GitHubLinkedInSnapInvois