Each directory in this repo is a separate research project carried out by an LLM tool - usually Claude Code or OpenAI Codex. Every single line of text and code was written by an LLM.
This repo follows the approach pioneered by Simon Willison's simonw/research - see his post Code research projects with async coding agents like Claude Code and Codex for details on how this works.
See HOWTO.md for the workflow and for how to construct research prompts.
Prompts and links to session transcripts are included in the PRs that added each report, or in the commits.
OpenAI Models Speedtest (2026-07-20 19:19)
OpenAI Models Speedtest is a browser-based tool that benchmarks the streaming response performance of OpenAI models via API calls, measuring metrics like time to first token (TTFT), total latency, tokens per second, and cost. Users input an API key, choose models/modes, and set spending limits; the tool shuffles and tests requests across selected model/mode pairs to ensure fair comparison, retrying rate-limited responses where possible. Pricing fields are editable due to frequent OpenAI changes, and the tool’s restrictive Content Security Policy enhances security—connections are only made to OpenAI’s API, with script execution governed by strict hashing. No backend or build step is required; you can run the benchmark locally or through the demo interface.
Key Features:
- Benchmarks TTFT, throughput, and cost across model streaming modes, including priority processing.
- Editable pricing/adaptive cost cap; partial results preserved if spending limits are hit.
- Secure-by-design: only API calls, tight CSP, no backend.
- Project repo and documentation available for further customization.
python-office-mcp-server vs go-ooxml for Office P&L workflows (2026-07-07 22:22)
For business workflows involving frequent Word and Excel document creation and analysis, python-office-mcp-server proved much more ergonomic and feature-rich than go-ooxml. The Python tool, leveraging mature libraries, enables rapid generation of polished P&L reports, delivers rich document inspection features (including Markdown/JSON outputs for LLMs), and offers agent-oriented abstractions for workflows such as patching, commenting, and auditing. In contrast, go-ooxml’s lower-level API works for basic OOXML tasks but requires substantial custom coding to match the business-level utility of the Python-based option—especially for LLM enablement or user-facing automation. Governance and open-source health for both tools are early-stage, but python-office-mcp-server is more recently active and better suited for immediate, practical use in a tech sales/PM environment, pending license clarity.
Key findings:
- python-office-mcp-server excels at workflow-level tasks, returns LLM-friendly outputs, and is productive for non-developers; it has more contributors/stars, though a missing license must be addressed.
- go-ooxml is viable for Go-centric platforms but is not as ergonomic or mature for business document tasks; it has a permissive license but is less active and single-author.
- Either tool should be treated as a prototype or internal toolkit rather than a supported product; neither is suitable for immediate production use without additional evaluation and governance.
browser-ai-steering (2026-07-06 22:03)
Browser AI steering tools are rapidly evolving to meet the needs of AI agents that must interact with modern websites in both transactional (e.g., banking, e-commerce) and exploratory (API mapping, frontend cloning) contexts. The landscape now balances agent ergonomics, browser fidelity, security posture, and inspection power: tools like Playwriter excel at leveraging a real user’s Chrome session for SSO- and extension-dependent flows, while Wirebrowser specializes in runtime JavaScript and API reverse-engineering. Highly agent-friendly CLIs such as agent-browser and scriptable harnesses like Browser Harness offer flexibility for coding agents, whereas infrastructure-oriented solutions like Steel Browser supply session lifecycle and scaling. The dominant trade-offs are between reproducibility/safety (sterile browser environments with fixed toolsets) and maximum capability (direct access to real user sessions and low-level runtime introspection).
Key Findings:
- Playwriter + Wirebrowser is the strongest pairing for maximum capability across both transactional steering and API mapping, provided strict security controls are enforced.
- Playwright MCP + Wirebrowser is recommended for enterprise environments that require standardized, isolated browser automation.
- Tools split into those optimized for execution (agent control, repeatability) versus those optimized for understanding (runtime instrumentation, API causality), so combining both is often necessary.
- All tools handling sensitive accounts require human-in-the-loop approval, audit logging, and isolated/dedicated browser profiles for safe use.
- For complex web automation, no single tool covers the full spectrum—purpose-built combinations outperform one-size-fits-all solutions.
Rodney and Showboat for research projects (2026-07-06 20:32)
For research workflows requiring both web interaction and reproducible evidence, Rodney and Showboat—two command-line tools from Simon Willison—offer complementary strengths. Rodney (GitHub) enables persistent browser-driven inspection and extraction from JavaScript-rich or stateful web pages, capturing rendered content, screenshots, accessibility data, and UI workflows that go beyond simple HTML fetches. Showboat (GitHub) creates executable Markdown transcripts that log commentary, commands, and output side-by-side—making research steps transparent, rerunnable, and easy to verify or review. Integrating Rodney for browser interaction and Showboat for evidence capture transforms ad-hoc investigations into durable, replayable artifacts, enhancing reproducibility and auditability without altering existing folder or note conventions.
Key Findings:
- Rodney provides persistent Chrome session control, DOM extraction, visual evidence capture, accessibility tree inspection, and script-friendly assertion checks.
- Showboat offers executable Markdown documents, reproducibility verification, code extraction, in-context images, and error-tolerant logging.
- Using both tools together ensures explorations are browsable, verifiable, and easily reviewed—bridging the gap between loose chat logs and formal proof transcripts.
- Existing research structures (notes, README, scripts) are complemented rather than replaced, with practical folder layouts and clear guidance for evidence management.
Passkey Tester (2026-07-05 19:43)
Passkey Tester is a single-page web tool for evaluating browser password managers and authenticators using WebAuthn passkey registration and authentication flows. Users can test how different platforms handle passkey ceremonies, inspect attestation details, and review metadata stored client-side, without access to private keys. The tool supports configurable authentication requirements and exposes credential data for educational and debugging purposes, but emphasizes that robust attestation validation must occur server-side. For hands-on testing and inspection, the artifact is available at the GitHub demo.
Key Features:
- Tests
navigator.credentials.createandnavigator.credentials.getWebAuthn flows. - Metadata and attestation details are viewable via localStorage—only public outputs are accessible.
- Designed to help developers explore authenticator properties; not intended for production security validation.
Pipeline self-test: does the publishing plumbing of this repo work? (2026-07-05 16:01)
Focusing on validating its own automation workflow, this project ensures that the repository's publishing pipeline functions as intended with a master default branch. Key adaptations from simonw/research were made to support master branch triggers and links. All essential steps—from automated README indexing, AI-generated summaries (using GitHub Models), bot-generated commits, to Pages deployment—are confirmed through observable artifacts, including a live demo page and workflow run logs. The process is designed to avoid redundant workflow retriggers and facilitate clear verification.
Key findings:
- End-to-end pipeline functions correctly, verified by live artifacts: README indexing, AI summary generation, note injection, and GitHub Pages deployment.
- Adaptation for master branch required only minimal workflow and script changes.
- Automation prevents workflow self-retriggering, ensuring efficiency.
This README uses cogapp to automatically generate project descriptions.
A GitHub Action runs cog -r -P README.md on every push to master and commits any changes to the README or new _summary.md files, then builds and deploys the GitHub Pages site.
To update locally: GITHUB_TOKEN=$(gh auth token) uv run --with-requirements requirements.txt cog -r -P README.md
The script discovers all project subdirectories, sorts them by the first commit that touched each folder's README.md (newest first), and for each folder either reuses the cached _summary.md or generates a new one with llm -m github/gpt-4.1.
To regenerate a specific project's description, delete its _summary.md file and run cog again.