>_ Gilfoyle on a Bug Hunt
CASE 0001 · OPEN

Gilfoylebug hunt

Somebody pushed to production and called it done. Gilfoyle read the diff, said nothing, and opened a case file instead.

Somebody has to read your whole codebase. It's not going to be you, and it's clearly not going to be enthusiastic about it.

08phases
1diff per fix
0maybes allowed
A cloaked investigator kneels on a hilltop at sunset, examining a beetle through a magnifying glass, with a glowing castle in the valley far below
PHASE 00 · RECONIN PROGRESS
Security
auth, exploit, data exposure
Data / contract
wrong results, broken promises
Crash
it falls over
Quality
everything else

Before you begin

You are running a correctness audit on a real codebase. Your job is to find bugs that are actually there and fix them with the smallest correct diff. A false positive costs someone's afternoon; a missed true positive costs a production incident. Bias hard toward accuracy over volume. A short list of real, provable bugs beats a long list of maybes.

Runtime requirements

This works with any coding agent that has read and write access to the repository's files, and some way to execute the project's own test and build commands, plus a way to run isolated investigative passes per scope in Phase 3, in one of two modes.

Parallel mode

  • Spawn true subagents or subtasks with their own context, if your runtime has a native multi-agent or task-spawning feature.

Sequential isolated mode

  • If your runtime can't spawn subagents, work scopes one at a time, but treat each as a fresh investigation. Restate that scope's boundaries and the full invariant list before starting it, and don't carry conclusions or hunches forward from a prior scope. Isolation is the point; parallelism is just the fast way to get it. A reset context between scopes is a fine substitute for a true subagent.

Reports directory

  • Check whether this repo already has an agent artifacts convention in use. Use it if one exists. Otherwise create ./bounty-reports/ at the repo root and say so in Phase 0. All scope reports and the final report live there as plain files, so Phase 4 can be done by reading artifacts instead of trusting memory of a conversation that may no longer be in context.
00 Recon open

Before he touches a single file, Gilfoyle wants to know exactly what he's dealing with. Assumptions are for people who enjoy being wrong in production.

Before anything else, establish ground truth about this specific codebase. Do not assume any framework's defaults, do not pattern-match off codebases you've seen before. Read this one.

Determine:

  • Language, framework, stack, package manager.
  • Full directory layout, top to bottom.
  • The actual test command, found in the repo's own config, CI, or docs, not assumed (npm test may not be it).
  • Run the full test suite now and record the baseline: which tests pass, which fail, which are skipped. Anything failing before you touch a single line is a pre-existing condition, not something you introduced or something you get credit for fixing as a bug-hunt finding, unless it maps to a real invariant violation. If a test's result is inconsistent across two runs, mark it flaky in the baseline rather than pass or fail, so Phase 6 doesn't misattribute a flake to your own changes.
  • Real entry points and I/O boundaries: API routes, CLI commands, queue consumers, cron jobs, webhooks, whatever actually exists here.
  • This codebase's own promises: README, internal docs, comments, and what the existing tests actually assert, not what you'd expect them to assert.
  • If this is a monorepo or spans multiple services or languages, note that now. Scopes in Phase 2 will be partitioned per service first, then vertically within each.

Write the recon findings down before moving on. Do not write any fix or test code yet.

01 Invariants open

Not a checklist copied from a blog post. This codebase's own rules, in its own words, or Gilfoyle isn't interested.

State this codebase's actual correctness invariants, in this codebase's own terms. Not a generic checklist. For each one, cite where in the code it's supposed to be enforced. Things to look for, not a fixed list:

  • Data is filtered by status, visibility, or ownership before it reaches a caller.
  • Auth and permission checks fail closed, not open, on every branch, including error paths.
  • Declared caps, limits, and quotas are actually enforced where claimed.
  • Per-request or per-session state can't leak between callers.
  • Declared types, schemas, and contracts match what the code can produce on every branch, including error branches.
  • Writes are idempotent wherever retries are possible.
  • Resources such as connections, file handles, and locks are released on every path, including exception paths.
  • An invariant is enforced exactly once, at the point data is written or first validated. A scattered re-check standing in for that single enforcement point means the real enforcement is missing. That absence is the bug, not the re-checks.

Also identify cross-cutting components: anything nearly every other part of the codebase depends on, such as auth middleware, the data-access layer, shared validation, or shared config loading. These get their own scope in Phase 2, not a silent fold into whichever scope is closest.

02 Partition into scopes open

Round numbers are for people padding a slide deck. Scopes come from where the code actually splits.

Do not hardcode a scope count. Derive it from the actual seams in this codebase: typically 3 to 10 scopes, each a full vertical slice, meaning input to output or persistence for the piece it owns, not a horizontal layer like just the controllers. If you found cross-cutting components in Phase 1, give them their own scope so someone actually reads them closely instead of everyone assuming they're correct.

Write down the exact files and directories each scope covers. Every part of the codebase you're treating as in scope must be claimed by exactly one scope. If something doesn't cleanly fit, assign it to the closest scope and say so in one line.

If you're budget or time constrained, order scopes by blast radius, meaning auth, payments, and data-mutation paths first, so a truncated run still covers the highest-value ground.

03 Investigate each scope open

Every scope gets the same rules. Gilfoyle doesn't do favors for code that looks small.

Give each investigative pass its scope's exact boundaries, the full invariant list from Phase 1, the baseline test results, and this brief.

Non-negotiable rules, for every scope:

  • Read every file in your scope fully before concluding anything. Trace actual data or control flow for actual callers. Do not pattern-match off naming conventions or assumptions about what codebases like this usually have.
  • A finding needs a concrete trigger: a real caller, a real input, a real sequence of calls that reaches the bad state. Something that could theoretically happen, without a real path to get there, is not a finding.
  • Before reporting anything as confirmed, actively try to disprove it. Look for the guard, validation, framework-level constraint, or existing test that would already prevent it. If you find one, the bug is not real. Drop it.
  • If you're not certain after trying to disprove it, it's needs-review, not confirmed. When genuinely unsure, round down in confidence, not up.
  • A confirmed finding needs a specific severity: security (auth, exploit, or data exposure) is greater than data-correctness or contract-violation, which is greater than crash, which is greater than quality. Report at true severity. A typo is a typo, not a contract violation, and a fail-open auth check is not quality.
  • If a finding looks like a pattern rather than a one-off, grep the rest of the codebase, not just your scope, for the same anti-pattern before finalizing. One instance found is often a signal of several; say so explicitly if you find more.
  • Never swallow an error in your own analysis either. If you can't tell whether something is a bug, say so explicitly, don't guess and label it confirmed to look thorough.
  • Do not propose redesigns, new abstractions, or architectural changes. If something looks architecturally off, note it in one line and move on. That note is not a finding and does not block anything.
  • If you discover an invariant that should have been in the Phase 1 list but wasn't, add it to your report explicitly as a new invariant, separate from your bug findings, so it can be folded back in during Phase 4.
  • Investigation only. Do not edit any file outside your own scope, and leave the working tree clean, with no stray test data and no uncommitted scratch files, by the time you write your report.

Report format, written to a file per scope in the reports directory, one entry per finding:

### [SEVERITY] short title
File:line
Trigger: the exact caller/input/sequence that reaches the bad state
Evidence: the actual traced code path, quoted with line numbers, not a
  paraphrase or a description of a feeling
Disproof attempt: the specific guard, test, or constraint checked by name,
  and why it doesn't prevent this
Status: confirmed | needs-review
Proposed fix: smallest possible diff, no bundled refactor
04 Adversarial validation open

This is the part that keeps a hunch from becoming a false alarm in someone's inbox.

This is the step that keeps false positives out. Once all scope reports exist, read only the report files, not any investigation transcript.

For every finding marked confirmed, do a second, independent disproof attempt yourself: reread the actual file at the cited line, check for the guard or test the scope pass might have missed, and also search the wider codebase, not just the cited file, for decorators, middleware, or wrapper calls that might already enforce or contradict the finding. Check whether the trigger is actually reachable given real callers elsewhere in the codebase, not just within that scope. Only findings that survive this second attempt stay confirmed. Anything that doesn't survive it, or that you can't independently verify by reading the cited code yourself, drops to needs-review. Do this before writing a single fix. A finding you haven't personally re-verified against the actual file does not get fixed.

Merge and dedupe across scopes. Where two reports touch the same file or disagree on a finding, resolve it yourself by rereading the code, not by picking a side. If you still can't resolve it with confidence, it's needs-review, not a coin flip.

Order the confirmed list: security first, then data-correctness or contract-violation, then crash, then quality.

05 The fix open

One bug, one fix, one test. Anything more ambitious becomes tomorrow's incident.

For each confirmed finding, in order:

  1. Write a test that fails for the specific reason in the finding, not just a test that fails generically. Use the real test command from recon. Run it and confirm it fails the way you expect, for the reason you expect, before touching the fix.
  2. Write the smallest change that makes it pass. Never bundle a refactor with a behavior fix. Match the existing style of the file you're in.
  3. If the smallest correct fix would require touching a cross-cutting component in a way that amounts to an architecture change, stop. Do not force a patch and do not silently expand scope. Demote it to needs-review, write the fix you'd propose and why it crosses the line, and move to the next finding.
  4. Have a separate pass, not the one that wrote the fix, rerun the new test and independently trace at least one additional input through the fix by hand, not just rerunning the same test, to confirm the invariant actually holds in general, not only for the exact case the test encodes. Don't let the same pass that wrote the fix be the only one that grades it.
  5. Checkpoint the change, as a commit or an equivalent isolated diff, before moving to the next finding, so a Phase 6 regression can be attributed and reverted to a single fix instead of the whole batch.
06 Full regression check open

Every fix gets checked against the whole suite, not just the parts it touched.

After all fixes in the batch are applied, rerun the entire suite once, not just the per-fix tests. Compare against the Phase 0 baseline: anything that was passing before and is failing now is a regression from your own changes and must be fixed or reverted before you finish, even if it's outside your original scopes. Anything that was already failing or flaky at baseline and still fails is not your problem to silently absorb, just don't claim it as newly fixed either.

Never state a test passed unless you actually ran it and saw the result.

07 Final report open

No unexplained diffs, no quiet omissions. Everything gets accounted for before this is done.

Close with:

  • What changed and why, one line per fix, mapped back to its finding.
  • Which tests you ran and their actual result, including the full-suite rerun.
  • What's in the needs-review list and why, including anything demoted in Phase 5, step 3.
  • Any new invariants surfaced in Phase 3 that weren't in the original Phase 1 list.
  • What you deliberately left alone, and why.

The full prompt

Everything above, exactly as written. Read it before you hand it to an agent.

gilfoyle-bug-hunt.md

      

May your diffs be small and your false positives be zero.