melon is a coding agent that has to prove every claim it makes.

I wrote about the first six hours of this tool a few days after building it, back when it had no real name. It's called melon now, and it's a different size of thing. About 18k lines of Python, plus 5.7k lines of tests, and the test count went from 224 to 449. The code is public at github.com/Adityaadpandey/melon-harness, and the story of how it got built is in the next post.

The tagline is the thing I'd rather you judge it on: a coding agent that proves its work. Most agents tell you they're done. I wanted one that shows you the checks that passed, a second model's review of the real diff, and every file reference in the answer checked against your tree. Underneath it sits one rule I keep coming back to, that every token and every tool call should earn its place.

Decision 1: "done" is a status the code decides

A model saying "done" is a sentence. It costs nothing to produce and it can be wrong.

In melon every request is a task with a lifecycle, and the transitions are validated in code. A run can't jump from failed to verified, and each move is written to a trace. The final status is always one of verified, unverified, verification_failed, failed or done_no_changes. There is no bare "done".

To reach verified, a run that changed files has to clear three stages in order. Your checks run on the changed files only, never the whole suite. Then an independent reviewer on a different model reads the real diff. Then every path and path:line in the answer is looked up in the real tree. Fail the first and it goes back to the model as a repair request, up to two rounds, and then it escalates to the strong model.

Every failure path has a next step. Nothing ends in "the model said it was fine".

Decision 2: reviewers are cheap to fool, so a second agent tries to disprove them

This one started as a mistake. My early reviews read like confident fiction: plausible chains of reasoning about code that didn't behave that way.

So a finding isn't a result in melon, it's a claim. Each claim goes to a verifier whose job is to disprove it. A confirmed HIGH is then run in a throwaway copy of the change to see the defect actually happen. If it doesn't reproduce, it's capped at MEDIUM and the attempt is shown next to it. A finding that cites a file that doesn't exist is dropped before anyone reads it.

A refuted claim is remembered per repo for two weeks and shown to the next reviewers as already ruled out, and a changed file reopens it.

The cost is real. A review takes longer and spends more tokens. What I get for it is fewer findings that are more likely to be true, and I'd rather read five of those than twenty.

Decision 3: a hallucination check that isn't a model

The citation check is a few dozen lines of code. It looks up every file and line number in the final answer. A made-up file, or a line past the end of a real one, goes back for one correction, and whatever is still unverified shows up as a warning.

I like it because it's the cheapest thing I built and it catches a whole class of confident wrongness. A model would be a worse tool for it, since a model can also be confidently wrong about whether a file exists.

The same idea sits in front of edits. Editing a file the agent hasn't read this session is refused, and so is editing one that changed on disk after it was read.

Decision 4: I took rails out

melon started with a lot of guards. Call caps, aggressive clearing of old tool results, a scout before every task. I assumed each one saved tokens.

Then I measured it against a simple agent: seven plain tools, a tiny prompt, files read whole. A lot of my guards cost more than they saved. So pi mode is now the default, copied from how the pi coding agent works. It has a small tool set, no call caps, results cleared only near the window, and compaction only close to the limit. --no-pi brings the old guarded setup back. Safety and correctness stay on in both.

The measuring tool is a replay. python -m bench.context_replay scripts a "model" that makes the same calls in the same order whatever the harness does, so any difference between two runs comes from the harness. It needs no API key and gives identical numbers every run. That's how I could tell which rails were earning their place.

I keep the rule from that: every rail pays rent, or it goes.

Decision 5: only remember what can't be invented

An agent that remembers things about your repo is tempting, and it's also where drift starts. If it stores its own opinions about the codebase, wrong ones compound.

So melon stores two things only: what you told it, and what verified runs changed. A correction like "no, not this" is read next to the message it replied to and becomes a lesson. The lessons are written into your repo's AGENTS.md under a heading, so you can read them, edit them and commit them. Each verified run also records its request words and the files it changed, and search boosts files that fixed similar requests.

Decision 6: three layers of safety, and an undo

"Please don't touch my SSH keys" in a prompt is a request, and a model can forget a request. So the rules sit in code.

File tools are confined to the folder you started in, with paths resolved first, so .. and symlinks that point outside are refused. Shell commands are scanned for paths in your home, /etc or other users' folders before they run. On macOS an OS sandbox sits under both, so even a command built on the fly can't write outside the workspace or read ~/.ssh.

Two smaller decisions matter more than they look. A repo's own config can't turn on hooks, MCP servers or verify commands until you trust that exact file, and editing the file revokes the trust, so a cloned repo can't quietly start running code. And before each request, melon records the working tree as a commit object, so /undo puts back what the last request changed without touching your index or stash.

Decision 7: say what it didn't do

A harness that proves its work has to be just as clear about where it couldn't. The answer states what was reviewed, whether that was the working tree, three commits or a PR, and lists every fallback it took. It says when passing checks don't exercise the change, and it labels guesses [guess].

Here's a real run on tunyt's web app. The checks passed, the review said PASS, the result is verified, and the answer still opens with "Partly correct. I wouldn't ship it as is". It says it read the diff and the like path end to end, never ran it, and that this is from the code and not a reproduction.

That run was judged hard and routed to the strong model, which hit a rate limit. melon said so and finished on another provider. The cost report says 85% of the input was cached and 52% of it was earlier tool results being sent again.

The numbers, and how far to trust them

The system prompt is about 0.7k tokens, down from about 3k. A warm search index takes 1.2 seconds against 13.7 cold. In the replay, 75% of input was served from cache, and I want to be clear that 75% is an estimate, worked out from the longest prefix each request shares with the previous one. The 85% is from live provider usage in the screenshot, so that's the real one.

What it can't do yet

There's no image input, so you can't paste a screenshot. There's no structured JSON output for CI. The kernel-level sandbox is macOS only, and on Linux you get the confined tools and the shell checks. /undo reverts the last request but there's no rewind to an earlier message, no LSP diagnostics and no IDE extension.

Several of the results above were tuned on one real repo. I treat the defaults as measured there, not as universal. The planted-bug review bench has a handful of cases and two saved runs, so I'm not claiming a recall number. And no head-to-head benchmark against other harnesses has been run, so when I say melon is better for me, that's me using it every day and not a leaderboard.

It does ship its own bench, so you can point it at your own tasks with your own models and see.

I started with a tool that said it was done. I ended up with one that has to show its work first, and I trust that more.