bookmark-organiser

A personal utility to organise my Pinboard bookmarks.

It takes a JSON export of bookmarks as input, runs a series of transformations as part of a workflow, and outputs the updated bookmarks in the same JSON format.

Usage

nix develop
uv run prefect server start --background
uv run prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api
uv run bookmark-processor <input.json> <output.json>

Workflow Steps

The tool runs the following tasks and transformations in a prefect workflow:

1. Liveness check

• Checks whether the URL is still accessible, and adds a "not-live" tag if the liveness check fails.
• Attempts a GET request first, with some timeout and retry logic.
• If this fails, falls back to a headless browser using Playwright.

2. Extract content

• Raw HTML source isn't suitable for text processing, so we try and clean this up somewhat.
• Uses Beautiful Soup to perform basic stripping.

3. Summarises article content

• Auto-summarisation of the article content, to be used as the bookmark description.
• Uses the llm Python API to call a configured LLM.

4. Suggest tags

• Auto-suggest tags based on the article content.
• Uses llm as above.

5. Lint tags

• Ensure bookmarks have at least one category (see category definitions) in their tags.
• Allows control of tags through allow/block lists.

Development

This project served as an opportunity to spend more time developing with LLMs.

Tooling used included:

• Aider
• Google Gemini 2.5 Pro
• Google Gemini 2.5 Flash

High-level workflow

The high-level approach I used while working with Aider was as follows:

1. Iterated with LLM to define PLAN.md.
2. Switched to a new git branch.
3. Instructed the LLM to follow the plan.
4. Heavily iterated with the LLM to achieve functionality.
5. Committed manually when necessary.

If you review the commit history, commits from Aider have the Co-authored-by trailer in the commit message to distinguish from manual commits I've made alongside working with the LLM. I'd typically rebase commits on a local branch prior to merging (e.g., this workflow) but here I feel it's valuable to preserve the commit history generated as I was working with Aider. For those unfamiliar with Aider, it auto-commits after each change by default.

Lessons learned

As this was my first end-to-end project written in combination with an LLM, this is a recap of what I learned from the experience:

Start with a plan

• This allowed me to think about my requirements and implementation constraints.
• Initial progress by the LLM was rapid.
• The plan needs updated as requirements evolve, else the LLM gets confused.

Tool usage is essential

• Enables removing yourself from the change → test → iterate loop.
• Having the LLM (or you!) write tests therefore accelerates progress.
• Aider doesn't support MCP (yet), but can still use tools.

Define conventions

• The LLM had no point of reference as to which libraries to use.
  • E.g., it would take dependencies on libraries that I didn't want it to use.
  • E.g., it would take dependencies which were unnecessary.
• The LLM had no point of reference as to coding style.
  • Defining your preferred conventions is likely helpful, but:
  • Auto formatting via tool usage (nix fmt in this repository) is preferable.
• Read more on defining conventions for agents, including which tools are available.
  • The format for doing this is tool specific currently - e.g., Aider recommends CONVENTIONS.md but is flexible.

Context management

• While writing good prompts still matters, ensuring the LLM has adequate context is also important.
  • Tools have different approaches to this - e.g., Aider has its repository map and manual /add command.
• Regardless of tooling, providing adequate context to requested changes matters for the quality of result.

Code review is essential

• As impressive as current generation LLMs are, they still make mistakes.
  • Tool usage can eliminate syntax errors, but logical errors can persist.
  • Logical errors in tests can be particularly problematic, as the LLM uses those to justify it's implementation decisions.
• Undesirable changes are also surprisingly common.
  • E.g., the LLM is tasked with making a change, but refactors other code unnecessarily and without being asked. I've had an LLM swap out dependencies (e.g., Selenium for Playwright) in the middle of an unrelated change.
• Budget time for reviewing code thoroughly, and give feedback to the LLM.
  • Update your conventions each time you identify undesirable behaviour.

Human⭤LLM workflow

• Further to the above point on code review, it's important to think about the workflow between you and the LLM.
• Aider auto-commits after each change by default, so there's a natural point for code review to occur.
• In addition to reviewing code, it's also important to review progress against your stated plan or goals.

Notes and caveats

As expected, this experience has also highlighted opportunities for further exploration:

Model usage

• The decision to use the Gemini models was influenced by looking for a balance of cost and performance.
  • Leaderboards (e.g., Aider polygot) generally have 2.5 Pro near the top and 2.5 Flash around the middle.
  • Usage (e.g., OpenRouter Programming) generally has 2.5 Pro and 2.5 Flash at or near the most popular models.
• Gemini 2.5 Flash was capable with sufficient prompting and context. 2.5 Pro was able to solve some problems where 2.5 Flash was struggling.
• I have no point of comparison in similar models, and would be interested in gaining experience with:
  • Claude Opus 4 and Claude Sonnet 4
  • DeepSeek R1 and DeepSeek V3
  • o3/o4-mini and GPT-4.1
• The approach of separating code reasoning and editing seen in Aider's --architect mode seems sensible, and in line with my original aim of finding a balance between performance (e.g., the more expensive models such as 2.5 Pro, Opus 4, o3) and cost (e.g., middle-ground models like 2.5 Flash, Sonnet 4, GPT-4.1).

Tooling

• The decision to use Aider was due to a combination of factors:
  • It's model agnostic. Tools tied to a specific model provider may be powerful, but limiting.
  • It's open-source.
  • I didn't want to change my editor, so a terminal-based tool was a good fit.
  • I wanted to start with a tool with a slower iteration cycle, before trying more fully autonomous agents.
• For future exploration, I'm interested in gaining experience with tools which:
  • Integrate with my editor: aidermacs and emigo.
  • Could replace my editor: Cursor, Windsurf, and zed.
  • Coexist with my editor: Amp, Claude Code, Gemini CLI and OpenAI Codex.