What is an AI coding challenge, and what should a platform for one measure?

An AI coding challenge is a coding task you solve with an AI agent instead of without one. A good platform for it should tell you how you worked with the agent, not only whether the tests passed: what you asked for, what you checked, and what you did when it went the wrong way.

That sounds obvious, but the label covers very different things. This post sorts them into three formats and gives you five questions to ask before you spend an evening on any platform, including kodwai.

What counts as an AI coding challenge?

Three shapes show up. They test different skills, so it helps to know which one you are looking at.

1. A classic puzzle, now allowed to use AI. A short algorithm problem with a hidden test suite, in a browser editor. The AI part is incidental. LeetCode and similar sites are the standard here, with huge problem sets and communities, and they remain the right practice for a classic algorithm round.

2. Fix or beat the AI's code. You start from code an AI model already wrote and make it better. HackerEarth's VibeCode Arena is an example. Its event page, which I read on 2026-10-14, describes starting from code written by AI models and improving, debugging and optimizing it. Submissions are scored on functionality, performance, maintainability, security and elegance, and ranked on a leaderboard shared with the AI models. That tests how well you review and improve code. It is not the same as getting a task done with an agent of your own.

3. A ticket-sized task you solve with your own agent. You get a written spec, starter files and tests, a time limit, and you work in your own terminal with the agent you already use. This is the format kodwai uses.

Which five questions should you ask a platform?

Use these on any platform. They are about what you get, not about who is "best".

1. Where does the work happen? In a browser sandbox, or on your own machine with the agent and setup you actually use? Practice only transfers if the setup is close to real work.
2. How big is the task? A puzzle you finish in ten minutes, or a ticket with a spec, several files and tests, where decomposing the work matters?
3. What gets scored? Only whether the tests pass, or also how you got there: your prompts, your checks, your recovery after a wrong turn?
4. Can you see why you got the score? A bare number teaches you nothing. You want each part of the score tied to something you did.
5. What does it cost to try? Is there a free path you can run end to end before you commit, and what does it need from you?

How does kodwai answer them?

kodwai is a platform where developers solve real coding challenges on their own machine with their own AI coding agent (Claude Code, Cursor, or Codex) and get scored on how well they direct the agent, across three axes: Direction, Outcome, and Lift.

Here are the five answers, in order.

Where the work happens. On your machine, always. There is no browser sandbox. You pick a challenge and run one command:

npx @kodwai/cli@latest challenge bookshelf-rest-api

The CLI signs you in through the browser if needed, asks which agent you will use, downloads PROBLEM.md, starter files and tests, initializes a git repo and starts the timer. You need Node.js 20+ and git.

How big the tasks are. Ticket-sized. On 2026-10-14 the public challenge list had 15 challenges, from easy to hard, with time limits from 60 to 150 minutes, across backend, frontend, mobile, data, platform, security, search, realtime, distributed and AI work. bookshelf-rest-api is the starter: easy, backend, 60 minutes. secrets-vault-with-envelope-encryption is medium, security, 90 minutes. raft-lite-log-replication is hard, distributed, 150 minutes.

What gets scored. When you finish, you run:

npx @kodwai/cli@latest submit

That sends your code, git history, test runs, the agent transcript and the time taken. The CLI only collects from the challenge workspace, nothing from your other projects. The score is kodwai's AI Collaboration Score for coding agents, calibrated 0 to 100, across three axes:

• Direction: how you steer, verify, and decompose.
• Outcome: what shipped, replayed and stress-tested to prove it holds.
• Lift: how far you beat a solo AI, not just that you passed.

Passing tests is necessary but not sufficient. There are 11 signals under the three axes. Direction has six: Spec Precision, Verification Rigor, Decomposition, Recovery, Intent Fidelity and Engagement. Outcome has three and Lift has two. The score definition lists them all.

Why you got the score. Every signal cites its own evidence from your transcript, commits and test runs, and the score comes with a confidence interval instead of false precision. That is the part I would test first on any platform, including this one: open a result and check whether you can point at the moment in your session that moved a signal.

What it costs. Solving challenges, the score, the public profile and the leaderboard are free for developers. Every account gets 3 free submissions scored on kodwai's own Anthropic key. After that you connect your own Anthropic API key in Settings for unlimited submissions. The key is encrypted at rest and only used to score your own work. The free submissions are permanent once used, so pick your first one on purpose.

When is something else the better choice?

• You have a classic algorithm round coming up. Practice on LeetCode. kodwai does not test memorized algorithms, and the LeetCode comparison says where each one fits.
• You need a full enterprise assessment suite, or classic algorithm screening for a team. HackerRank is built for that. See the HackerRank comparison.
• You want to review and improve code an AI wrote, against a public leaderboard of models. The VibeCode Arena format above is built around exactly that, if you can find a current event.
• You use a different agent every day. kodwai names three agents in its CLI: Claude Code, Cursor and Codex. Other terminal tools may work, but those three are the ones it supports.

kodwai is also small and early. I would rather you know that before you pick it than after.

What should you do next?

If the third format sounds like the practice you want, run the starter challenge. It is 60 minutes, easy, and it ends with a score you can read against the five questions above.

npx @kodwai/cli@latest challenge bookshelf-rest-api

Or pick your own from the challenge list. If you want a walkthrough of the loop first, I wrote one in how to practice coding challenges with your AI coding agent.