Skip to content

Drive the application in tests, and fix the two things that found - #17

Merged
bokuweb merged 1 commit into
mainfrom
e2e
Aug 23, 2026
Merged

Drive the application in tests, and fix the two things that found#17
bokuweb merged 1 commit into
mainfrom
e2e

Conversation

@bokuweb

@bokuweb bokuweb commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Yes — gpui can do this, and it found two bugs immediately.

Everything in this port that was wrong in a way a unit test could not see was
found by running the app and reading its log, and half of it was never seen at
all: I cannot drive this machine's window, so the shelf screens, the spread and
the chat panel all shipped "verified by reading".

gpui opens a headless window in tests, with a real element tree, and delivers
clicks and keystrokes through the same dispatch a reader's go through. So the
shell can be tested the way it is used:

  • a real SQLite library in a temporary directory
  • real PDFs that pdfium really reads
  • a stand-in CLI that prints recorded JSONL, like the other agent tests
  • ⌘⇧S pressed, not called; the question typed into the composer and
    sent with the return key
cx.simulate_input("where is it?");
cx.simulate_keystrokes("enter");

The five tests

The library appearing; a book opening to its first page; the spread key; a book
remembering how it was read; and the whole loop from typing a question to a
citation resolving to a page.

Two failed on the first run, and both were real

No key did anything until the reader clicked something. Keys are dispatched
along the focus path, and nothing claimed focus when the window opened. The key
that focuses the search field is itself a key, so it could not be what started
them off — the reader had to click some part of the shell before ⌘B, ⌘⇧S or
the arrows would work at all. The shell now takes focus as the window opens.

Reopening a book in the same session forgot where the reader was. The place
is written to the database and read back from the list of books the shell
holds — but only the database was being updated. The next launch restored it
correctly, which is exactly what made this look like it worked. Affects the
page as much as the layout.

Both had been in every build you have used.

What it still cannot do

Look. It says which page the reader is on, which pages share a row, and what a
citation resolved to — not whether the result is legible. The screenshots are
still the only way to see that, and they need the app to hold focus.

Two notes on the harness, in case they read as odd:

  • PEDRO_LIBRARY_PATH is process-wide, so a test that sets it holds a mutex for
    as long as it is looking at that library.
  • An answer is revealed over frames — that is what makes it read as writing — so
    a test has to let those frames happen. written_out does that, and panics
    rather than hanging if one never finishes.

301 tests. RUSTFLAGS="-D warnings" cargo clippy --locked --workspace --all-targets --all-features exits 0, and so does the matching cargo test.

Everything in this port that was wrong in a way a unit test could not see was
found by running the application and reading its log, and half of it was
never seen at all — I cannot drive this machine's window, so shelves,
spreads and the chat panel all shipped verified by reading.

gpui opens a headless window in tests, with a real element tree, and delivers
clicks and keystrokes through the same dispatch a reader's go through. So the
shell can be tested the way it is used: a real SQLite library in a temporary
directory, real PDFs that pdfium really reads, a stand-in CLI that prints
recorded JSONL, and ⌘⇧S pressed rather than called.

The five tests here cover the library appearing, a book opening, the spread
key, a book remembering how it was read, and the whole loop from typing a
question to a citation resolving to a page.

Two of them failed on first run, and both were real:

No key did anything until the reader clicked something. Keys are dispatched
along the focus path and nothing claimed focus when the window opened — and
the key that focuses the search field is itself a key, so it could not be
what started them off. The shell now takes focus as the window opens.

Reopening a book in the same session forgot where the reader was. The place
was written to the database but not to the list of books the shell holds, and
reopening reads that list. The next launch restored it, which is exactly what
made this look like it worked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@bokuweb
bokuweb merged commit 2406436 into main Aug 23, 2026
2 checks passed
@bokuweb

bokuweb commented Aug 23, 2026

Copy link
Copy Markdown
Owner Author

Four more tests, covering the rest of what shipped without ever being seen.

  • a book put on a shelf, and the sidebar drawing it as a section of its own
  • a shelf opening the panel it is asked in, and the composer changing to "Ask across these books…"
  • a question put to a shelf coming back with a source that names which book
  • the search box showing passages in place of the books

The shelf question is the one worth having. Two books on a shelf, a question typed and sent with the return key, an answer quoting the second book — and the citation resolves to keys.pdf page 2. Nothing but looking the quotation up could have told it that, which is the design decision from #5 that I have never been able to check until now.

One failed, and the fixture was at fault

The shelving test said "the other book left the library". It had: the library holds one book, named two.pdf.

That is the store working as designed — two files of identical bytes are one book, so that re-adding a file keeps its marks and its place and only takes the newer name. My fixture had given both books the same content. Books now say their own name on every page.

Worth writing down because it is the kind of thing a test can only teach you once: a content-addressed library makes "two books" a thing a fixture has to mean, not just say.

Where this leaves it

Nine end-to-end tests, 304 in the workspace. RUSTFLAGS="-D warnings" clippy and test both exit 0.

Still no substitute for looking — none of this says whether the result is legible — but the shelf flow, the spread key, the ask loop and the search are now checked by something other than me reading them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant