definition

The AI Collaboration Score, for coding agents.

kodwai's AI Collaboration Score is a 0 to 100 score for one coding session with an AI coding agent (Claude Code, Cursor, or Codex) that measures how well the developer directed the agent, across three axes: Direction, Outcome, and Lift.

kodwai gives this score to every challenge submission. It rewards how you steer the agent, check its work, and break the problem into steps. Your scores also place you on a public leaderboard.

By Hakan Karaagac, co-founder of kodwai Last updated 2026-09-22

The three axes

Direction, Outcome, and Lift, built from 11 signals. These are the labels the app shows on every scored session.

default weights

/ 100

Direction

50 pts

How you steer, verify, and decompose.

Spec Precision: Stating clear requirements and constraints before writing code.
Verification Rigor: Checking the AI's output, catching mistakes, pushing back.
Decomposition: Breaking the problem into ordered steps instead of one giant prompt.
Recovery: Redirecting effectively when the AI goes down the wrong path.
Intent Fidelity: The final solution matches what you actually asked for.
Engagement: Staying engaged and iterating, instead of pasting the spec and walking away.

Outcome

35 pts

What shipped, replayed and stress-tested to prove it holds.

Tests: Share of the challenge's tests your solution passes.
Code Quality: Clean, readable code without obvious smells.
Complexity: Reasonable structure and nesting.

Lift

15 pts

How far you beat a solo AI, not just that you passed.

Edge-Case Coverage: Handling subtle requirements that a careless single prompt would miss.
Lift over AI: How far you out-perform a solo AI on this challenge.

In the default profile Direction is worth 50 points, Outcome 35 and Lift 15. Other challenges use other splits, and a challenge with its own rubric scores Direction, a Challenge Rubric axis and Lift. On the default profile Direction carries the most weight, because it is the part a careless prompt cannot fake.

How a session is scored

kodwai is a platform where developers solve real coding challenges on their own machine with their own AI coding agent (Claude Code, Cursor, or Codex) and get scored on how well they direct the agent, across three axes: Direction, Outcome, and Lift.

You start a challenge with npx @kodwai/cli@latest challenge <slug>. The CLI asks which agent you will use, downloads PROBLEM.md, starter files and tests, sets up a git repo, and starts the timer.
You solve it on your own machine, in your own editor, with your own agent. There is no browser sandbox.
npx @kodwai/cli@latest submit packages your code, git history, test runs, agent transcript, and the time you took. The CLI only collects from the challenge workspace.
The session is scored on Direction, Outcome, and Lift. Every signal cites its own evidence from your transcript, commits, and test runs.
You get a score from 0 to 100 with a confidence interval, the evidence behind each signal, and a place on the public leaderboard.

Why passing tests is not enough

A one-shot “solve this” prompt can clear the tests, so tests alone say little about how someone works with an agent. Most of the score comes from how you direct the agent: whether you wrote down the spec, checked the agent's output, broke the work into steps, and recovered when it went wrong. Lift then measures how far you got beyond what the AI manages on its own.

Who it is for

Developers

Practice directing Claude Code, Cursor, or Codex on ticket-sized work and see which signals held you back. Your public profile at app.kodwai.com/developers/your-username is built to send to anyone, including a hiring manager instead of a take-home.

It is also practice for interview rounds where AI use is allowed, such as the AI-enabled coding interview Meta started rolling out in October 2025.

Hiring teams

Teams use kodwai's hiring track: custom interview projects, one link per candidate, the full session live (prompts, commits, test runs, tools, time, and API cost), and AI scoring against your own per-role rubric next to your team's manual scores. Candidates work in Claude Code on their own machine.

See kodwai for hiring

Try it on a real challenge

The starter challenge is Bookshelf REST API (easy, backend, 60 minutes). Create a free account, then run:

$npx @kodwai/cli@latest challenge bookshelf-rest-api

easybackend60 min

How kodwai compares

Honest comparisons that say where each product is the better choice.

Frequently asked questions

Is a high score the same as passing the tests?

No. Tests are one signal inside Outcome. Passing them matters, but most of the weight goes to Direction, because that is the part a careless prompt cannot fake. A solution that passes the tests without any steering, checking, or breaking the work into steps scores poorly on the axis that counts most.

Which coding agents can I use?

When you start a challenge, the CLI asks whether you will use Claude Code, Cursor, or Codex. You work in your own editor on your own machine.

Are the axes weighted the same on every challenge?

No. In the default profile Direction is worth 50 points, Outcome 35 and Lift 15. Other challenges use other splits: a debugging profile weights Direction higher, and a challenge with its own rubric scores Direction, a Challenge Rubric axis and Lift.

Can I see why I got my score?

Yes. Every signal points to its evidence in your transcript, commits, and test runs. Your score also comes with a confidence interval, so you can see how certain it is.

What does the CLI collect?

Only the challenge workspace: the code files, the challenge session's git history, and agent traces from the challenge time window. Nothing from your other projects.

Is it free?

Yes. Solving challenges, your score, your profile, and the leaderboard are free for developers.

Is this the score hiring teams see?

Not directly. kodwai's hiring track scores interview sessions against each team's own per-role rubric, 0 to 10 per dimension, with a written justification. That is a different number from the public 0 to 100 score described here.

Is the name unique to kodwai?

No. Other products use similar wording: Corrix, for example, offers an AI Collaboration Profile about how effectively people work with AI in general. On kodwai, the AI Collaboration Score always means the score for a session with an AI coding agent.

your turn

Get your own score.

Pick a challenge, solve it with your agent on your own machine, and see how you direct it. Free for developers.

Create a free account