What can an interviewer see when a candidate directs an AI agent?

You can see four things: the prompts the candidate wrote, how they recovered when the agent went wrong, whether they checked the agent's output, and what the result actually does. Each one leaves a trace in the session record (the agent transcript, the git history and the test runs), so you can read the candidate's judgment instead of guessing it from the final code.

This post is about reading that record. If you want the wider interview format and a rubric you can copy, I covered that in how to interview developers who work with AI agents. Here I go one level down: which part of the record shows which behavior, and what the record cannot show.

Why is the final code not enough anymore?

A recent Hacker News thread on The Four Horsemen of Agentic Coding spent much of its length on one argument: do engineers still need to read the code agents write? Some commenters said no, at least for small projects. Others said large systems still need steady steering and review.

One commenter described asking an agent to make something faster and getting back functionally correct commits that still did not do what they wanted, for example a speedup that added real cost, or a migration that would be risky in production.

That is the interviewing problem in a single example. Code that looks right is cheap now. You are hiring the person who notices when "looks right" is not what was asked. Passing tests is necessary but not sufficient.

The prompts: what did the candidate ask for?

The agent transcript is the closest thing you get to the candidate's thinking. Read the first few prompts before anything else, and look for three things.

• Scope. Does the first prompt hand over the whole ticket, or one piece with a clear boundary?
• Constraints. Does it say what must not change, such as an interface, a response shape or a dependency?
• A definition of done. Does it say how the candidate will know the step worked?

On kodwai's developer side these behaviors are scored as two Direction signals, Spec Precision and Decomposition. You do not need those labels. You need to notice whether the agent was told what to build or left to guess.

One caution: a short prompt is not a red flag on its own. Someone who has just read the codebase can write a two-line prompt that is exact. Judge the prompt against what the candidate knew at that moment, not against its length.

The recovery: what happened after the agent went wrong?

Almost every session has a point where the agent takes a wrong turn: a failing test, a misread requirement, an approach that will not fit the existing code. The record makes that moment easy to find. Look for a failed test run followed by new prompts, or a commit that gets reverted or rewritten soon after.

Then read what the candidate did next. Strong recovery adds information: the error output, the constraint the agent missed, a narrower ask. Weak recovery repeats the same prompt and hopes, or throws the work away and starts over without saying what went wrong.

This is the Recovery signal. You cannot see it in the final code at all, because a good recovery leaves clean code behind.

The verification: what did the candidate check?

Verification shows up as timing. When did the candidate run the tests compared with when the agent changed things? After each meaningful edit, or once near the end? Did they ask for a test of the edge case the agent skipped, or try it by hand?

The git history helps here too. Small commits with a test run between them read very differently from one large commit just before time runs out.

This is Verification Rigor, and it is the part the Hacker News thread kept coming back to. If agents write most of the code, checking it is a large share of the job that is left.

The result: does it do what was asked?

Score the result the way you always have. Do the tests pass, and could your team maintain this code?

Then add one question: does it do what the problem asked, or something plausible next to it? Agents drift. A "faster" endpoint that got faster by serving stale data would pass a speed check and still miss the requirement.

On kodwai this is Intent Fidelity. In your own loop it is a walkthrough question such as "show me where this handles the empty case". A candidate who directed the work can point to it quickly.

What can't the record show?

Be honest about the gaps, because they are where interviewers over-read the data.

• Reading leaves no trace. A candidate can review a diff carefully and say nothing. If it matters to you, ask them to talk while they review, or ask about the diff afterwards.
• Quiet time is ambiguous. Five minutes without a prompt might be planning or might be stuck. Ask which.
• Tool familiarity skews speed. A candidate on an agent they do not use every day will spend some time on the tool itself. Keep that in mind when you compare timings.

The record is evidence to talk about, and it works best next to a short walkthrough with the candidate.

Where kodwai fits

The kodwai hiring track is built around this record. The hiring page says it in one line: "Run real-world coding challenges as private interviews. Measure how candidates work with AI agents: the prompts, the recovery, the verification, the result."

In practice, you create a custom interview project and send the candidate a one-link email invite. They solve it locally in Claude Code, on their own machine. Interviewers can watch live: prompts, commits, test runs, tools, time and API cost. After the session, kodwai scores the candidate against your team's own per-role rubric, with a written justification, strengths and weaknesses for each dimension. Your team adds its own scores and comments, and you can compare candidates side by side.

That hiring rubric is separate from kodwai's public developer score. On the developer side, a run is scored on three axes:

• Direction: how you steer, verify, and decompose.
• Outcome: what shipped, replayed and stress-tested to prove it holds.
• Lift: how far you beat a solo AI, not just that you passed.

The signals named in this post all sit under Direction. The score definition has the full list.

There are no ATS integrations and no published pricing. If you want to try it for a role, the details are on the kodwai hiring page.

If you would rather see a session from the candidate's chair first, run the starter challenge yourself. It is easy, backend and 60 minutes long:

npx @kodwai/cli@latest challenge bookshelf-rest-api

When you are done, submit it and read what each axis says about your own run:

npx @kodwai/cli@latest submit

Solving challenges is free, and every developer account gets 3 free scored submissions.