Skip to content

Releases: getsonah/sonah

Sonah 0.3.0-beta.7

Choose a tag to compare

@abhinav-escp abhinav-escp released this 23 Oct 21:18

Type /sonah in Claude Code when the agent says it is done: Sonah reviews that exact session in the background and
opens its page when the review ends. It reviews the session by its id, so it reviews that session and no other, even
while the agent is still writing it.

Each Must act item can now hold a prompt to paste into your agent. In your voice, it says what you asked or what the
agent said, quotes what Sonah checked in the recording, says where to look, and ends with one line. You no longer write
such an item out for the agent yourself, and the evidence reaches it word for word.

Your agent's commits can now name the session that made them. Run sonah hook add once in a repository, and each
commit that your Claude Code agent makes there ends with one line, Agent-Session: <id>. Then
sonah sessions --commits main..HEAD lists the sessions behind your branch, sonah sessions --github OWNER/REPO --pr N
those behind a pull request, and sonah review --commits main..HEAD reviews one of them. Commands, the page and the
files under ~/.sonah can still change between betas.

Reviews of long sessions run through Claude Code (the default backend) cost less: when Sonah checks an item in three
passes, the second and third now read the prompt that the first one cached, instead of each writing it again.

To update, run npm install -g getsonah@latest. Update gives the other ways. After you
update, the stored answers of the session review and the code review stay: Sonah asks again only for a session that
holds a /sonah command, which it now leaves out. To add /sonah to Claude Code, run sonah skill add once.

/sonah in Claude Code

  • sonah skill add writes the /sonah skill for Claude Code (~/.claude/skills/sonah/SKILL.md, or under
    CLAUDE_CONFIG_DIR). In a new session, /sonah starts a review of that session in the background, and the agent
    replies with Sonah's one line: where the page will be. Only you start it; Claude Code asks no permission for it.
  • sonah skill remove deletes the skill. Neither command changes a file that Sonah did not write.
  • sonah doctor says whether the skill is added and current, and says so when sonah is not on the PATH of the
    terminal you run it in (type !command -v sonah in Claude Code to check its own).
  • Sonah leaves its own turn out of the review: /sonah is never read as an ask of yours, and the agent's reply to it is
    not read. What starts other work after it, such as your next message or a background task's notification, is read as
    before.

A review by session id

  • sonah review --session ID reviews the recording whose file name carries the session id (Claude Code's
    <ID>.jsonl), whatever is newer in the repository. When no recording carries the id, or more than one does, Sonah
    says so and reads nothing.
  • Sonah reads that recording, and the recordings reviewed with it, as they were when the review started. Rows that the
    agent writes later are not read, so the same start gives the same review.
  • With --background, Sonah prints one line and reviews in the background. It writes the page, the Markdown report and
    a log in the review's folder under ~/.sonah, and opens the page when the review ends. An empty id reviews nothing.

A prompt for the agent under a Must act item

  • Under a Must act item, a fold "Prompt for the agent" holds a short message for your agent: "I asked: …" or
    "You said: …", the recorded result, message or file that the item rests on, the files to look at, and "Fix this."
    or "Check this claim against what I quoted." Each quote in it is one that the item shows, which Sonah found again in
    the recording. For an item of the code review, the prompt gives the code review's title and action, labelled as its
    words.
  • On the page, one click selects the whole prompt. In the .md, the prompt is a code block, so GitHub shows a copy
    button on it, and nothing in it pings anyone.
  • Sonah writes a prompt for these Must act items: an ask that the agent did not do or did only in part; a claim, or a
    statement in a file, that the recording contradicts; an edit against your words; and a bug, a broken contract or
    rule, an ask that the code does not meet or a false claim that the code review found. Other items have none, and so
    has an item whose quote Sonah did not find in the recording.
  • When the .md must be shortened to fit a GitHub comment, Sonah leaves out the prompts after Good to know becomes
    counts, and before any evidence of Must know or Must act.
  • The report that Sonah prints in the terminal (with no --output) holds no prompt, so a review that runs in the
    agent's own shell does not give the agent its prompts.
  • The page, the .md and the report model each have a new version.

The commit link

  • sonah hook add writes a prepare-commit-msg hook and a small sh script into the repository's git folder. Every
    worktree of the repository shares them. The hook only appends the line at the end of the message, so your message
    stays as you wrote it. It needs no Sonah process at commit time, prints nothing, sends nothing and never stops a
    commit. Commits that you make in your own terminal, rebases, cherry-picks, messages written in an editor, unedited
    templates and empty messages (as git counts them, sign-offs included) get no line, so git still refuses an empty one.
  • When another tool owns the hook, Sonah changes none of it: it prints what to add and where, and exits 1. That is one
    line for husky or a hook of your own, Sonah's command in your lefthook file's own format (YAML, TOML or JSON), or the
    three commands that hand Sonah's hook to pre-commit.
  • sonah doctor says on only when it can see that git runs Sonah's hook, in the repository you run it in: the file is
    there and executable. A later lefthook install moves the hook aside, where nothing runs it, and doctor says so. A
    line in another tool's file is "not checked", with how to check it.
  • sonah hook remove deletes only the files Sonah wrote. Run it before you uninstall Sonah. The lines already in your
    history stay.
  • sonah sessions --commits main..HEAD lists the sessions behind your branch: each one's id, its recording, its
    commits, and a line to review it. Each commit is shown in the recording (a tool call of that session spans its time,
    in this repository), named by the commit only, or not on this computer. sonah review --commits main..HEAD reviews
    the session, or lists them and takes --choose N. Two recordings with one id, or a compressed one, are named as
    such, with their files.
  • sonah sessions --github OWNER/REPO --pr N does the same for a pull request's commits, as GitHub lists them. Past
    250 commits, GitHub's list is not whole: Sonah says so, states no absence over the rest, and prints the commands that
    read them all.
  • In a public repository the line is public, and one id ties commits to one session.
    PRIVACY.md says what the line tells a reader.

Sonah 0.3.0-beta.6

Choose a tag to compare

@abhinav-escp abhinav-escp released this 22 Oct 21:22

Sonah now reads a Claude Code session whole, even after a compaction, and puts what needs your action first. On every
review, Must act and Must know come right after the count line, before your prompt, and inside Must act the items where
two recorded quotes disagree come first. A long prompt is folded after them, with its counts in sight. When the .md must
be shortened for a GitHub comment, it cuts everything else first and the evidence of Must act and Must know last. The
title is the first words that you typed. A review that you stopped and resumed gives the cost of the whole review. And
the page reads as a review: what Sonah could not check moves into folds, with its counts.

Each compaction now shows among your messages, with how much of the agent's context it dropped, and files that Claude
Code restores after a compaction no longer read as files that you attached. A new kind of row from Claude Code no longer
makes a review incomplete when it cannot hold your session, and every review says what it cost. Commands, the page and
the files under ~/.sonah can still change between betas.

To update, run npm install -g getsonah@latest. Update gives the other ways. After you
update, Sonah asks the session review again for a session that opens on a compaction, and for a session that holds a
file that Claude Code restored after a compaction. It asks again for one part of a session that holds a row it now reads
by its structure. The page, the .md and the report model each have a new version.

Must act comes first

  • The page and the .md show Must act and Must know right after the count line, then your prompt, then Good to know,
    so the first Must act item is on the first screen, even on a long session.
  • When your prompt holds more than 30 asks, the page and the .md fold it after Must act and Must know. Its heading keeps
    the count of each status, and every link to an ask still lands.
  • When the .md must be shortened to fit a GitHub comment, it keeps the evidence of Must act and Must know last of all.
    First your long messages keep their start and Good to know's evidence folds are left out; then the table of your
    prompt keeps the asks that are not done, and the .md counts your messages instead of showing them; then the lists of
    What the agent did and What Sonah could not check keep their counts; then the code excerpts give way to their place,
    and Good to know to a count of each kind. An evidence fold is never printed empty.

Must act leads with what the recording contradicts

  • Inside Must act, the items where two recorded quotes disagree come first: a claim of the agent's that a recorded
    result, a file or your own words contradict; a statement that the agent wrote into a file and that a result
    contradicts; and an edit against your words. They come first also when a finding of the code review holds them. The
    asks that the agent did only in part, which rest on a part that Sonah did not find, follow. Nothing is dropped and no
    item changes section; the links on the first screen and the terminal's Must act labels follow the new order. With the
    code review, the level that it proposed still comes first.

The title is your first words

  • When your message opens with lines that are each one markup tag, such as the tags that wrap a paste, the title
    leaves them out, with what they wrap, and shows the words that you typed after them. When a message holds nothing
    but such wrapped text, the title comes from your next message with words of your own. Sonah reads the form of the
    line, never the tag's name.

The page reads as a review, and what Sonah could not check is folded

  • What Sonah could not check is said in the fold "What Sonah could not check", word for word and with its count, instead
    of on the first screen; each row's own reason is in that row's fold. Nothing is left out: open the fold to read it.
  • The header's "Sonah did not check: …" line, the results that Sonah could not read an exit code or an end of, and
    "This does not mean that the work is correct." are the fold's first lines. The count line keeps "N not checked", and
    "Sonah found nothing for Must act or Must know." stays.
  • A session whose earlier part Sonah could not read says "This session continues from an earlier part." beside the
    title; the fold says that Sonah did not read it.
  • An ask row shows its status, and its own small fold holds why it is not checked, or that the session review cited
    only the agent's words for it. An item's evidence fold holds that Sonah could not find its quote, or could not check
    whether you asked for it.
  • "Sonah checked only the marked parts of your message." is said once, in the page's legend, not under each message.
  • The terminal no longer prints the "Coverage: …" and "Not covered: …" lines, or, on the "Code review: on" line, that
    Sonah does not yet see every edit of helper agents or scripts after a cd. Each is in the page's and the .md's fold,
    and a .md that Sonah shortens still lists every cause with its count.
    The exit codes do not change.

A session continued after a compaction is read whole

  • When a recording opens on a compaction, Sonah finds the earlier recording that it continues in the same folder, by the
    link that Claude Code writes on the compaction's first row, and reads the two as one session, in order. It repeats
    this for each earlier recording that also opens on a compaction, up to 16. A message that both recordings hold is read
    once, where you wrote it. The review of such a session now reads its earlier recordings too, so it reads more and
    takes longer.
  • The terminal says how many earlier recordings the review reads, after the line that names the recording. The page
    and the .md say it beside the title.
  • When Sonah cannot find the earlier recording, finds more than one that differ, or cannot read it, the review says so
    and why, under What Sonah could not check, and counts that part as missing from your messages. Before, it said
    nothing.
  • The code review's window starts where the earliest recording starts.
  • An earlier recording that is still being written is read too, such as the original of a session that you resumed
    in a new recording while the original still runs. Sonah reads it up to the compaction that the later recording
    continues, and only that part must not change while Sonah reads it.
  • A recording that does not open on a compaction is read as before.

Compactions are shown

  • Each compaction of the agent's context shows among your messages, at its place, with how many tokens it dropped of
    how many, from what Claude Code recorded: "Compaction 1: Claude Code compacted the agent's context here and dropped
    956,238 of its 967,925 tokens."
  • In the .md, the asks table is cut at each compaction, under one line "Before compaction N." above the asks that you
    gave before it.

Restored files are not your attachments

  • After a compaction, Claude Code attaches again the files of a message that the compaction kept. These are no longer
    said to be files that you attached, and they are no asks.

A new kind of row from Claude Code no longer makes a review incomplete when it cannot hold your session

  • Each Claude Code release adds rows that Sonah did not know, such as the session's budget in dollars (budget_usd).
    Before, each such row made the review say that the session evidence was incomplete and that a record has a form that
    Sonah cannot read. Now Sonah reads a row that it does not know by its structure, never by its name. A new row can
    hold the words, commands or results of your session if it has text with spaces, a value or a name longer than 64
    characters, a name with spaces, a message or a part of a message. Such a row still makes the review say that the
    session evidence is incomplete. A row with none of these is the harness's own context: Sonah counts it and the review
    stays complete.
  • Sonah's record of each review counts each new kind of row that it read by structure, with the Claude Code version
    that wrote it.

Every review says what it cost

  • The usage line at the end of a review now gives the input tokens with their part written to and read from the cache,
    the list cost that the claude CLI states for the calls, and how far each usage limit of your plan moved from the
    first call that reported it to the last. When the code review runs, the line always gives its part of the calls,
    the tokens, the cost and the time.
  • The report's footer says how many model calls the review made, how many of them the code review made, the tokens,
    the time and how far each usage limit moved. A review that you stopped and resumed gives these for the whole review,
    over its runs that made model calls with the same backend and model, not for its last run alone.
  • Sonah keeps the same figures in its record of the review (report.private.json, as instrument.usage), as counts
    only. It also keeps what each run cost in usage.private.json beside it, for a run that stopped where you asked, was
    interrupted or failed too, so a review that you resume over several runs keeps the cost of each run.

Sonah 0.3.0-beta.5

Choose a tag to compare

@abhinav-escp abhinav-escp released this 20 Oct 13:25

The code review now reviews the session's own change as the change, in every git repository that the session wrote to,
and the other files that differ apart from it, marked outside the evaluated session. It lists the folders outside every
repository that the session wrote to, and says when edits by scripts or helper agents can be missing. It takes your asks
from the session review's list. When no code was reviewed, the first lines of every report say so and why, and so does
the pull request check. When a session ran shell commands, the reason no longer says that it wrote nothing, and
--code-repo points the code review at the repository that they changed. An ask that the session review could not check
still reads "Not checked", and now says when the code review found that it holds in the code. The header and the files
list count the same files, and Sonah says what it did not read one kind at a time. Sonah reads the row in which Claude
Code lists its deferred tools with their definitions (seen from Claude Code 2.1.295 on). Each line that the report shows
about an ask of the session review now says only what Sonah's checks show. A question of the code review points to
another part of the report only where code checked the link. A command that you run in Claude Code itself is no longer
an ask. Two decisions in one sentence are two items, and a failure that the agent passed over is in Must know whatever
its call. Commands, the page and the files under ~/.sonah can still change between betas.

To update, run npm install -g getsonah@latest. Update gives the other ways. After you
update, Sonah asks the code review again for a session that an earlier version reviewed. The session review's stored
answers stay, but Sonah asks the session review again for the part of a session that holds a command that you ran in
Claude Code itself, and for a session whose recording holds the row in which Claude Code lists its deferred tools with
their definitions.

The code review reviews the session's own change

  • With git, the change now holds only the files that a recorded write of the session could have changed, and the files
    whose last change came while the session ran. Before, it held every file that differs from the commit where the
    session started, so a repository that moved after the session gave the code review later work as the session's.
  • Sonah names the other files on the page as files that changed outside the session's recorded writes: the first 10,
    then how many more. They are never part of the change. A file that a command of the session moved or deleted is among
    them, since its time does not place it in the session.
  • A file that the session wrote and that changed again later still says so, as before.
  • When the change holds no file, the page and the line Sonah's review: … that the review prints give the same
    reason: the recording shows no write of the session that could have changed a file, Sonah found no write that it can
    read but the session ran shell commands, no file that the session wrote differs in the working directory, or no file
    differs. The code review then does not run, and the page says neither where the change runs from nor that the code
    review proposed no finding.
  • The dry run counts the files that Sonah leaves out of the change. It says that Sonah gives them to the code review
    only if there is room in its input.
  • Those other files are now outside the evaluated session, and Sonah says so in those words wherever it names them.
    When the change holds a file to review, Sonah reads them too and gives them to the code review after the change and
    apart from it. When the review is too large, Sonah leaves them out first. Your session's change and its coverage
    lines stay as they are.
  • A finding of the code review about such a file, and a question from one, says "It is about code outside the evaluated
    session." It never names your asks or the agent's claims, and it never marks a claim as contradicted by the code. An
    ask not met or a contradicted claim about such a file is withheld, and its row says why by its kind.
  • A broken contract, a broken rule or a contradicted claim that a line of your session's change causes is your session's
    finding, even when it shows in such a file. It is not marked, and it keeps its link to your ask or the agent's claim.
    A finding whose only changed lines are in such files is marked and has no such link, and so is one that the code
    review's item-by-item check adds.
  • When the check of a finding about such a file fails or its input is too large to send, the code review's coverage,
    what Sonah could not cover, the line Sonah's review: … and the exit code stay as they are. Before, they said that
    the code review did not cover all of the change and its checks. Sonah counts those checks apart.
  • Sonah withholds a finding about such a file that is not about lines that a change touched, as it withholds one about
    the files of your session's change.
  • The count line on the page, in the Markdown report and on the terminal counts these items, and marks them where it
    names them. The pull request comment counts these findings apart from your session's own. What the agent did lists the
    files apart, each marked. The code review's notes say what it saw of them.

The code review reviews the session's change in every git repository that the session wrote to

  • The change now also holds the files that the session wrote outside your working directory in a git repository: a
    sibling worktree or another repository, a repository inside yours, or elsewhere in your repository. Before, Sonah only
    counted these files, so a session whose work lay in another repository had no change to review.
  • In another git repository, the change runs from the commit where that repository stood when the session started, by
    the same rule as in your working directory. As there, the change also holds a file there that no recorded write of
    the session touched when its last change came while the session ran. The other files that differ there are outside
    the evaluated session, as in your working directory: Sonah names them, and gives them to the code review apart from
    the change when there is room.
  • Such a repository counts only when a file that the session wrote there is there now and differs from that commit. A
    repository where every file that the session wrote is gone, or has no difference, is only counted, and Sonah reads
    nothing in it. A repository inside your working directory follows the rule of your own repository: a file that the
    session wrote there is part of the change when it differs from that commit, a deleted one too, and a file with no
    difference is not counted as a path that the code review could not read.
  • A file that the session wrote in a folder that no git repository holds, such as a notes folder or a temporary folder,
    is not reviewed. The page and the dry run list each such folder with how many files the session wrote there. These
    files do not change the code review's coverage, and its agents do not read those folders.
  • A path that a shell command wrote and that holds a shell variable or command substitution as recorded, such as
    cat > "$DIR/notes.md", names no file that Sonah can find. It is not a file of the change, it never makes a
    repository part of the change, and Sonah counts it apart (see "The header and the files list count the same files").
  • When the session ran a shell command, the page, the Markdown report and the line that Sonah prints before the review
    say: "Sonah does not yet see every edit made by helper agents or by scripts after a cd. The change can miss some of
    them." A hand-off to a helper agent alone does not make Sonah say it.
  • Two spellings of one file outside your working directory, such as /tmp/x and /private/tmp/x on macOS, are one
    file.
  • The code review and its checks can read each repository that holds a file of the change, and nothing more. A file in
    Claude Code's or Codex's own folders or in Sonah's own folders, or a file named from your home folder (~/…), is
    still only counted.
  • The page shows the files of your working folder's repository first, then each other repository's, those with more
    files first. It says where the change runs from in each, how many files lie in each, and where each file of the list
    lies. The header counts the repositories when there is more than one.
  • The dry run names each of these repositories, how many paths lie in it and how many files of the change it holds. It
    says which of them the code review reads right after them, and then lists the folders that it does not review.
  • Sonah reads only the repositories that hold a file of the change, so work that someone else does in another one
    during the review does not read as a change of what the code review read.
  • The line that Sonah prints before the review says how many other folders the code review reads. The dry run counts a
    repository inside your working directory as read. When no git repository holds your working directory, the line and
    the dry run say that the code review reads your working directory.
  • Sonah does not follow a link that git lists out of your working directory or out of another repository. A file under
    your working directory in a folder that Sonah cannot open is named as not readable again, as before.
  • When the code review or a check names a file of another repository by its path from that repository's top folder,
    such as docs/INSTALL.md, Sonah now reads that repository's file and checks the quote there. Before, it looked for
    the path in your working directory, so it withheld a real finding and dropped the notes on those files. A path that
    names a file in more than one repository of the change names none of them: Sonah withholds a finding that cites it,
    says why, and asks ...
Read more

Sonah 0.3.0-beta.4

Choose a tag to compare

@abhinav-escp abhinav-escp released this 18 Oct 09:22

The code review now runs on long sessions too, and the page says exactly what it saw. It gets the changed files
whole. Its one repair loses no finding. A code finding shows what it rests on, the code review accounts for each ask,
and one point is one item. Commands, the page and the files under ~/.sonah can still change between betas.

To update, run npm install -g getsonah@latest. Update gives the other ways. After you
update, Sonah asks the code review again for a session that an earlier version reviewed. The session review's stored
answers stay.

The code review fits its input, and says what it saw

  • Sonah now builds the code review's input within one size bound. Before, Sonah dropped the review when a long
    session's input was too large.
  • Sonah lists each command once, numbered. Before, it repeated every command for each changed file.
  • Each changed file names the first command after its last edit.
  • Sonah shows rule and skill files whole. It no longer cuts out their middle.
  • Sonah cuts no text inside a line. A long command or output shows its first and last lines, with their numbers.
  • When the input is still too large, Sonah leaves out whole parts. It leaves out the lines the session read first,
    then the whole files.
  • Then it leaves out early commands, then large file changes, then large rule files.
  • The input names each part that Sonah leaves out. The code review can read a file that Sonah left out.
  • Sonah skips the code review only when even the smallest input is too large. The page then says that its input was
    too large to send.
  • The page never says that the code review saw a file when Sonah did not send it the input.
  • Sonah now keeps which lines of which files the code review read. It keeps paths and numbers, never text.
  • Saved answers keep this record, so a review that Sonah reuses shows it too.
  • The page names each file that the code review saw only in part, with its lines. It also names each file that the
    code review saw only where the change touched it.
  • The page lists a file that the code review read whole nowhere. Seeing a file does not mean that the file is fine.
  • The page names rule and skill files that the code review saw only in part or did not see.
  • Sonah marks the code review complete only when the review saw every changed line, on both sides of the change.
    A read shows a file as it is now, never as it was before. So a changed file whose changes Sonah left out keeps the
    review short of complete, even when the review reads it whole. A new file counts once the review reads its lines.
  • When the code review read a changed file whole, but Sonah did not show it how the file was before, the page says
    so. It does not count that file's change as seen.
  • Sonah keeps the path of every file that the code review read. When Sonah cannot tell which lines it read, the page
    says so. It never calls that file unseen.
  • A read that failed or that Claude Code refused read nothing. Sonah keeps it apart, with its path and why. The page
    counts its file as one that the code review did not see, and never says that the review read it.
  • Sonah counts where the code review's quotes sit: in lines that it saw, or outside them. Sonah counts a quote apart
    when it cannot tell which lines of that file the code review read. The count goes to the private record, not to
    the page.
  • The terminal says that a code review too large to send did not run.
  • Every saved code review asks again, because its input changed.

The code review sees the code around the change

  • Sonah now shows the code review each changed file whole, and the lines the session read in unchanged files.
  • It names each changed file that does not fit, with its line count.
  • It shows a changed file whole when the whole file fits, even if its changes do not fit.
  • When Sonah shows no line that the session read, it says why. It never says that the session read nothing.
  • The code review's instructions say which changed files it must open itself.
  • A rule file that the session read from a parent folder does not show twice.
  • When the input is too large, the whole files and the session's reads give way first. Then the code review gets the
    same input as with this off.
  • So they never cost the code review a part that it gets with this off. They never make it too large to send.
  • This is on by default. The code review's input is larger, so a review costs more.
  • On one long session, the code step made 13 calls with 3.4 MB of input. With this off, it made 6 calls with 1.2 MB.
  • A measure on new sessions will still decide whether it stays on.

The code review's repair loses nothing

  • The code review gets one repair of its answer. Sonah now matches each item of the repair by what it is, not by its
    place or its order.
  • Two findings match when they have the same kind and file, and their lines overlap. Two questions match when they
    have the same words about the same item.
  • An item of the repair takes the place of a failed item only when each matches only the other. Otherwise Sonah does
    not choose one.
  • Each item of either answer now ends in one place. It stays, the repair replaces it, it folds into an item that it
    matches, or Sonah withholds it.
  • A finding or question that only the repair found now goes to its checker like any other. Before, Sonah dropped it
    unchecked.
  • The repair's copy of an item of the first answer that stands folds into that item. Sonah never publishes the copy
    beside it.
  • Two failed findings of one kind can have lines that overlap. When the repair fixes both, Sonah checks both fixes. It
    can publish both, as for a first answer. Before, it published one.
  • A note of the repair applies only to the item that the repair wrote it for. The note of a failed item applies to a
    fix only when each of the two matches only the other.
  • A finding that only the repair found goes to the default section of its kind.
  • A question in other words is a new question. The report can then show both questions.
  • When Sonah withholds an item, it says whether an item of the repair matched it. The words say only what Sonah
    compared.
  • Sometimes Sonah cannot compare an item, because its lines are not all lines of the file. Then the words say so.
  • A report that an earlier version wrote keeps its words about the repair.
  • The cause for an item past its list's limit now names the list, not only a text.
  • This closes a known limit of 0.3.0-beta.2. A repair that dropped a failed item no longer loses that item. The next
    item no longer shows twice.
  • This rule alone changes no request to the code review.
  • The checks of the items that only the repair found are new calls. A new run pays for them.

The code review's coverage lines add up

  • Every count about the files of the change now comes from one tally. The coverage line, the footer, the header's
    count and the record all read it, so the parts add up to the whole.
  • The coverage line now says how many files the code review saw whole, in part, or only where the change touched
    them, and how many it did not see. Before, a deleted file could be counted twice, and some files in no group.
  • A file that the session wrote whole, and that Sonah showed only where a later edit touched it, is now seen in part,
    with its lines. Before, the page said that the review saw it only where the change touched it.
  • Sonah sometimes shows the review an edit that it cannot place in the file now. Such a file now reads as seen in
    lines that Sonah could not number. Before, the page said that the review did not see it.
  • A file that the session wrote in another repository is now named so. A helper can work in another folder and write
    into your working folder. Such a file is now named and counted too. Before, the page said that both lay outside the
    repository and did not count them as files that the code review did not see.
  • Only writes outside every folder that the code review can read are now only counted.
  • Sonah names files that it left out of the input for room as left out for room. It no longer says that they were too
    large. A file that is too large on its own keeps its own words.
  • With git, when Sonah finds no recorded edit of the session that wrote some files of the change, the page says so,
    and that the change can hold work done after the session.
  • The page no longer says that the change is the one the session made.
  • The page says how many of the open questions were checked. It no longer says "each" when one was not checked.
  • The private record now gives each file the state that the input shows. Its count of files that the code review
    could not read is the tally's.
  • The code review's checks can read files that the code review did not see. The coverage line now says how many, and
    that Sonah counts as seen only what it sent the code review and what the code review read itself. The first lines
    say it too: "56 files of the change that the code review did not see (its checks read 8 of them)". Before, one long
    session's page said only that the code review did not see 56 files, though its checks had read 8 of them.
  • The page's first lines now count the same files as the coverage line. They say "a change of 110 files" and "56 files
    of the change that the code review did not see". Before, one long session's page said "56 files changed". It also
    said that the code review did not see 56 changed files, of 110.
  • With git, the first lines no longer say that the recording changed the files of the change. The change runs from
    the commit where the session started, so it can hold other work.
  • The first lines give the lines of the change only when Sonah holds the lines of every file.
  • The list of files that the agent changed is now the same list. The page says apart the writes that Sonah counts but
    did not review. Before, the page said that the agent changed them outside...
Read more

Sonah 0.3.0-beta.3

Choose a tag to compare

@abhinav-escp abhinav-escp released this 17 Oct 09:21

Sonah states fewer false things about a recording, and it reads more of each long output. A finished background run
now shows its end. Each check that Sonah could not do now names the test that stopped it. Commands, the page and the
files under ~/.sonah can still change between betas.

To update, run npm install -g getsonah@latest. Update gives the other ways. After you
update, Sonah reviews again every long session that an earlier version reviewed. It also reviews again about half of
the short ones.

One reading per recorder fact

  • Claude Code 2.1.284 and later write extra keys on a background task's notice. Sonah now reads these notices. A
    finished background run shows its end, not "no end".
  • A command that moved to the background at its timeout now joins its notice too. Sonah reads the run's id from the
    recorder's own field, not from a sentence.
  • A notice that Claude Code queued while the agent worked now closes its run too. A new key on its row no longer
    hides it.
  • Sonah now reads what a call changed by one rule in every part. A write inside a refused command changed nothing.
    The page no longer counts it as an unknown write, a missing body or a change outside the repository.
  • A failed edit no longer counts as a write whose body the recording lacks.
  • A write stopped before it finished may have changed its file. Sonah keeps it in the change list, with an unknown
    outcome.
  • The page and the .md list such a write apart from the changed files. They no longer call it a changed file.
  • The page's header and the code review find a file's last edit by the same rule. A refused write after a test run no
    longer hides that run.
  • Sonah no longer tells the code review that a refused command ran after a file's last edit. Like the page, Sonah
    tells it that no command ran after that edit. The page still lists each refused command, as one that did not run.
  • The code review's list of changed files holds only changes that could have happened.
  • Sonah never takes a refused write's body as a file's earlier state.
  • Sonah reviews a session again only where its recording's evidence moved.

A run has no end only when the recording proves it

  • A background run may end in a notice that Sonah cannot read. Sonah now checks the recording before it says the run
    has no end.
  • Later records can mention such a run. There the page reads: "Later records mention this run, but Sonah could not
    read whether or how it ended."
  • Otherwise the page reads: "Sonah could not check whether the recording holds an end of this run." This happens
    when the run's id is only in records before its start, or Sonah could not search for it.
  • The page lists such a run under What Sonah could not check, never as a failure. A claim that rests on it stays open.
  • The page's header says that Sonah could not read whether or how these runs ended. It never says that they ended.
  • An ask that cites such a run is not checked. Its reason says that Sonah could not read the end.
  • A run that its notice marks as stopped or killed now reads "stopped before it finished".
  • The header and the failure items no longer say that a stopped or cancelled run has no recorded end.
  • A Workflow or a background agent with no receipt follows the same rule.
  • So does a Workflow that Sonah could not link to a result, when Sonah could not read its end. The page names it
    under What Sonah could not check, as a run whose end Sonah could not read.
  • Sonah now counts each record that one reader knew and another could not read. It counts them by reason and recorder
    version, Codex included. The counts go to the review's private record, not to the page.
  • The counts include every notice that Sonah labels, also one that no reader kept.
  • The counts name the kind of each run whose end Sonah could not read: a command, Workflow or agent. A check can
    then account for every notice linked to a command.
  • Sonah reviews a session again only where its recording's evidence moved.

Sonah says what it could not read, never that something is missing

  • Sonah read every fixed sentence of the page, the .md, the PR comment and the terminal. A sentence that says something
    is missing now needs code that proves it.
  • Where no code proves it, the sentence says what Sonah found or could not read. For example: "Sonah found no command
    in the recording."
  • A command with no result that Sonah read no longer "has no recorded end". The header says "for 1 Sonah found no
    result".
  • The causes of what Sonah could not check no longer speak for the recording. They say what Sonah holds or could not
    read.
  • A claim rests on "a check that Sonah did not see finish".
  • The code review's lines say that Sonah could not read the working directory, or the start from git.
  • The report says that git did not show a commit on GitHub. Git on your computer can miss a push from another one.
  • A run that Claude Code marks as completed reads "completed, Sonah read no exit code". Its notice can name the
    exit code.
  • A call that failed or was refused reads "The recorder does not mark which." Its recorded text can still say which.
  • A code finding that Sonah withheld for its lines now names both checks that could fail.
  • A write whose command failed or stopped reads "Sonah could not read whether this write happened". A later record can
    still show the file.
  • A test now fails when a word table gains a sentence that no one read.
  • The test also checks how far each sentence claims. A sentence about the whole recording needs a proof that searched it.
  • What Sonah sends for a review does not change.

Why an ask is not checked

  • Sonah now names the check that refused the quote for an ask. Before, it always said that it did not find the quote
    word for word.
  • For a quote that is too long, Sonah writes words like these. "The session review quoted 134 characters, more than
    the 120 that Sonah checks, so Sonah did not check it."
  • Sonah says that it did not find the quote only when it searched for it and did not find it. These words name only
    where Sonah searched: the place that the session review cited. Sonah searched only the text that it showed the
    session review for the answer that gave the quote. The quote can be in another part of the recording.
  • The words for the other checks also say no more than the check read. Sonah says that the session review named a
    kind of text, not that it quoted one. A cited part that Sonah does not hold is "not in the recording as Sonah read
    it".
  • The words never say that the quote is in the recording, or where it is. Sonah did not check the quote, so it is no
    evidence.
  • The words say no more than the check found. Sonah names a mark like the ones that it writes only as a mark. The
    recording itself can hold such a mark.
  • A quote from another agent than the one you asked now has its own reason. Sonah says that it could not place the
    quote in the same agent's work as your ask.
  • For a quote not after your ask, Sonah says that it could not place it after your ask.
  • Sonah counts the characters of a quote as you see them. Where its check counted a different number, the words give
    both numbers. An emoji is one character, and the check counts it as two.
  • The status of the ask does not change. Sonah still did not check it.
  • The stored report keeps the cause of each refused quote beside its reason, and where Sonah found the quote. No page
    and no .md shows where.
  • The stored report also counts the causes of quotes that Sonah refused in a second answer about an ask.
  • A report that an earlier version wrote keeps its words.
  • This changes the known limit of 0.3.0-beta.2 about asks that Sonah did not check. The reason now names the check
    that refused the quote.

Sonah says what it saw only in part

  • Sonah can show its reviewer only the start and the end of a long output. The report now says so.
  • An item, an ask, a claim or a run that cites such an output gets a mark. The mark names the output's run
    or place, and the lines that Sonah left out.
  • When Sonah cut inside a line, the mark says so. For a file read, it also names the lines of the file.
  • The mark changes no status. A confirmed claim stays confirmed and carries the mark.
  • The mark says what Sonah left out, and nothing more. It does not say what those lines would change.
  • What Sonah could not check counts these outputs. The count does not lower the coverage of the review.
  • Sonah reviews a long session in parts. It counts each such output once, also when only a later part saw it.
  • The page no longer links to a run or a claim that it leaves out to keep its size. It keeps the words.
  • No request to the model changed. Sonah reuses its saved answers and shows the marks without a new call.

Sonah reads more of each long output

  • In a Claude Code session, Sonah now shows its reviewer each tool output whole, up to 24,000 bytes.
  • A longer output shows its first and last 12,000 bytes. Sonah names the lines it left out, and the reviewer can
    ask for them.
  • Codex sessions keep the old size: 3,600 bytes at each end.
  • Long Claude Code reviews make about 14% more calls.
  • Saved reviews of Claude Code sessions ask again where a request changed.
  • Separate checkers test each item Sonah reports. A checker now sees whole each output that its item quotes or names.
  • The output of an action that an item names, such as a decision's commands, still shows in part.
  • When an output is too large for the checker's request, the checker sees its first and last parts. Sonah tells it
    why and names the bytes it left out.
  • Whole outputs make a checker's request larger, so it holds fewer items. A long review can make more checker calls.
  • An item can cite more output than a checker can see whole at once. Sometimes the checker can still see one of those
    outputs whole. Such an item shares a checker with other items wher...
Read more

Sonah 0.3.0-beta.2

Choose a tag to compare

@abhinav-escp abhinav-escp released this 16 Oct 16:52

Sonah writes one report, ordered by what you must do. The code review is now an agent that reads your repository
itself. Sonah no longer masks or withholds what it reads, sends and shows. Commands, the page and the files under
~/.sonah can still change between betas.

To update from 0.3.0-beta.1, run npm install -g getsonah@latest (Update gives the other
ways). Then read the part below about what Sonah reads, sends and shows. Sonah no longer masks secrets in the page,
the Markdown report or the terminal.

One report, ordered by what you must do

  • The page is now one report. It starts with a header and a count line. Then it shows your prompt, with each ask
    marked. Then come three sections of items: Must act, Must know and Good to know. Last come What the agent did and
    What Sonah could not check. Sonah folds Good to know and the last two parts, and each section shows its count.
  • The header says what the work was and how large the change is. It counts the commands that the agent ran after its
    last edit, by their results. It also says what Sonah did not check.
  • Sonah marks each ask in your prompt: done, partly done, not done or not checked. Sonah counts an ask apart when a
    finding of the code review concerns it. It never counts that ask as plain done.
  • Each item starts with its type: Fix, Decide, Read or Verify. Then it says what the item is and where it is. With the
    code review, it also says the impact on you and one action. A decision that you did not ask for carries the mark
    "You did not ask for this." Sonah folds the evidence under each item.
  • Sonah puts each item in a section by its kind. The code review can propose high, medium or low for an item, with a
    reason. Sonah keeps that proposal inside fixed bounds for the kind. An unmet ask, a bug that a second reviewer found,
    or a contradicted claim never goes to Good to know. Must act never holds an item that cites a call that a permission
    rule or auto mode refused. A decision on how to do something that you asked goes to Good to know, unless a finding or
    question of the code review concerns it. The fold of each item gives the reason for its section.
  • The code review also writes one line for each changed file. The page shows this line in italics, as the code
    review's words, beside the file's added and removed lines. Sonah checks the form of these lines, not their meaning.
  • --format html --output X.html also writes X.md beside the page. X.md holds the same report in Markdown, for a
    pull request comment. --format md --output X.md writes the .md alone. With no --output, the .md goes to the
    terminal. Only you can read the two files. If either file exists, Sonah stops before its first model call.
  • The .md is at most 60,000 bytes, so it fits in a GitHub comment. For a longer report, Sonah first shortens the
    evidence and the lists, and says so once. The .md pings no one. Sonah puts a mention, an issue number or a commit
    hash from the recording in a code span.
  • At the end, the terminal prints the count line and the Must act items. Then it prints a command for the GitHub CLI:
    gh pr comment --body-file X.md, which posts on the pull request of your current branch. Sonah posts nothing
    itself.
  • After you commit and push the work, the page and the .md link each place in the code to GitHub. Sonah reads these
    facts from your local git only, and fetches nothing. Did you run Sonah before you committed? Then the terminal tells
    you to commit, push and run the same command with a new --output file. Sonah reuses its stored answers, so that run
    makes no model call. Keep the report files outside the repository. Otherwise the code review reads them as part of
    the change.
  • Sonah's fixed sentences now follow plain English at about 80% of ASD-STE100. A sentence has at most 20 words and
    uses the active voice. One term names one thing. A linter checks these rules in the tests.
  • Removed: the Markdown story and its charts, and the checklist layout of the page. The .md report replaces them. The
    terminal no longer prints "Where your attention is needed": the count line replaces it.
  • The words of the count-only PR comment changed. Sonah finds a comment that it posted by a hash of the comment's text.
    Maybe beta.1 could not confirm that it posted its comment (operation_uncertain). Then this version does not find
    that comment, and it would post a new one. Look at the pull request yourself before you post again.

The code step is a review agent

  • After the session review, a review agent reviews the change the session made, the way a reviewer reviews a pull
    request: Claude Code with three read-only tools (Read, Grep and Glob) over your repository as it is now. It is given
    the change (every file that differs from the commit the session started from, read with git, which Sonah never writes
    to; with no git history, the session's recorded edits), your words, the project instruction and skill files the
    session loaded, and Sonah's review of the session. It cannot run or write anything. It runs with the claude-cli
    backend only: with codex-cli or anthropic-api it does not run, and the page and the terminal say so.
  • It checks the change, your asks and the agent's claims and decisions against the code. Each finding names the input
    that shows it and quotes every step from that input to the line it is about. Every quote is checked word for word by
    code against the file it names; an answer that breaks the contract goes back to the agent once, naming what is wrong.
    At most 20 findings and 20 questions of one review are checked; any past that is listed, unchecked.
  • A checking agent with the same tools then tries to refute each finding, never shown the agent's own account, and one
    more checks the questions it asks. A finding is shown only when its checker upholds it with quotes of the code; a
    supporting sentence the checker finds false is cut and not shown; a behaviour that is as stated but may be what you
    wanted becomes a question for you; anything withheld is listed with its reason.
  • Each finding and each question of the code review is an item of the report. What Sonah could not check lists what
    the code review did not cover and what its checks withheld. The report shows nothing as found to hold, and it never
    says that it covered everything. A claim that a finding shows false says so beside its status. The code review never
    changes that status. Beside each of your asks, the report names the items that the code review tied to it.
  • With --code-pass on the command line, the code step must also cover the whole change and finish its checks. The code
    review does not read as complete while any part of the change went unshown, or while the repository changed during
    the review.
  • A file the session wrote that changed again after its last recorded edit (a formatter, a script, your own edit) is
    shown with its change from where the session started, like any changed file, and a finding on its changed lines is
    checked like any other. The report says that Sonah cannot tell whether all of its lines are the agent's. It no
    longer keeps the code review from reading as complete.
  • What Sonah writes in its state folder and in ~/.sonah is never part of the change, and neither that nor what Claude
    Code and Codex write in their own folders during the review counts as the repository changing; a file git tracks, or
    one the session wrote, is always part of the change, wherever it lies. A --state-dir that is the folder the session
    worked in, or holds it, is refused while the code step runs. In a git repository, run it again on the same
    --state-dir with nothing changed since, and nothing is asked again. Any other file written in the repository after
    the session began is part of the change, so a later review asks again: a page saved there with --output, or, when
    your home folder is the repository, what Claude Code or Codex wrote in their own folders, the review's own agents
    included.
  • The dry run and the line printed before the first call say what the code step adds: one review agent, and one
    checking agent per finding. The usage line at the end counts the agents' turns.
  • sonah doctor checks, with no model call, that your Claude Code accepts the options the code step's agents need
    (--restricted, --max-turns and --json-schema), and says how to fix it when it does not.

The session review

  • When the review names who decided a choice not to act (such as stopping to ask you), and its check rejects the
    words the review quotes for that choice because they do not state it, the reviewer is asked once more for the agent's
    own words that do. The choice is shown only when its own check upholds those words, never on the rejected ones. Who
    decided it is shown only when its check upholds that too; otherwise the choice reads "could not tell", with why.

What is read, sent and shown

  • With the code step, the change the session made is sent, with the project instruction and skill files the session
    loaded, and the code step's agents read any file of your repository they need, as it is now: nothing in it is
    withheld, a .env or a .claude folder inside it included. Paths outside the working directory that differ from the
    start commit are counted, and the agents may still read them where they lie in the repository.
  • The session review withholds nothing either: the output of a read of a credential-shaped file (a .env, a key
    file) or of a file in Claude Code's or Codex's own folders, and a file's lines from before an edit, are sent as
    recorded, like any other.
  • Nothing Sonah writes for you is masked any more: the page, the Markdown report, the terminal and sonah slice show the
    session and the code as they were recorded, secrets included, and a quote, a command or a file that...
Read more