Skip to content

ThroughlineTech/worklog

Repository files navigation

Throughline Worklog

A self-hosted service that harvests my git, ticket, and PR history from every machine I work on into one SQLite dataset, then answers "what was I actually working on in March" through a web UI and a CLI.

Read the portfolio case study.

Why it exists

I work across a lot of repos on a few machines, and after a while the honest answer to "what did you ship this quarter" is a shrug. Memory is unreliable, timesheets are worse, and the real record - commits, tickets, pull requests - is scattered across GitHub, a self-hosted Plane instance, and whichever laptop happened to be open that week. Some of it was never pushed anywhere.

This pulls all of it into one place and lets me ask questions of it: which projects were active in a window, which ones overlapped, how many distinct days I actually worked, what a given project's history looks like end to end. It reconstructs the record from evidence instead of asking me to remember.

Architecture

Four harvesters feed one store. A single query core sits on top of it, and both front ends call that same core - the query logic is written once.

  always-on hub (a Proxmox LXC)
  ------------------------------------------------------------------
   harvesters (python, on systemd timers)
     git_mirror : git fetch bare mirrors of every GitHub repo   (primary; no dev machine needed)
     git_ssh    : ssh each dev machine, scan ~/src for local-only commits + provenance
     github     : REST -> PRs, issues, repo metadata
     plane      : Plane REST -> tickets, state transitions
                         |
                         v
                    worklog.db   (SQLite - the single portable artifact)
                         |
                    query core   <- one implementation
                    /          \
              CLI            FastAPI + a build-free SPA
                             (published over a Cloudflare Tunnel, behind a login)

Two things about that shape are deliberate. GitHub mirrors are the primary git source, so a harvest works with every dev machine powered off; the SSH scan exists to catch what mirrors cannot see - unpushed branches, repos that were never pushed at all, and which machine holds a given commit. And commits are deduplicated by SHA across all sources, with repo identity resolved from the normalized remote URL, so the same work seen from three machines counts once.

Beyond querying, there is a report layer and a deep-dive generator that turn the corpus into prose (via an LLM gateway) for project write-ups.

Install

Worklog requires Python 3.13. From a clone:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt
cp worklog.toml.example worklog.toml

On Windows PowerShell, activate with .venv\Scripts\Activate.ps1 and copy the config with Copy-Item worklog.toml.example worklog.toml.

Edit the new worklog.toml for your machines, GitHub accounts, author identities, project mapping, timezone, and hours factors. The runtime filename is deliberately ignored by git; worklog.toml.example is the versioned template and should remain generic.

Initialize an empty SQLite database, inspect the CLI, and optionally start the local web UI:

python -m worklog.init_db worklog.db
python -m worklog --help
python -m worklog.web --db worklog.db

The UI is then available at http://127.0.0.1:8000. A fresh database uses the documented default admin password; change it from the admin page before exposing the service.

Queries work without network credentials once the database contains data. Harvesting needs a dedicated SSH key and/or GitHub and Plane credentials. On the hub those secrets live only in /etc/worklog/.env (mode 0600), never in either TOML file. The supported keys and deployment shape are documented in docs/WORKFLOW.md; worklog/secrets.py is the loader.

CLI verbs:

verb what it answers
recent which projects were active in the last N days
project one project's full history, bucketed
simultaneous what overlapped with what, as a co-occurrence matrix
hours the hours model output
timeline gantt rows: project x day intensity
search commits and tickets matching text
harvest run the configured harvesters
deep-dive generate a long-form project write-up from the corpus

The web UI is the same query core over HTTP: python -m worklog.web --db worklog.db.

Deployment

The hub is a Proxmox LXC running the harvesters on systemd timers, with the web UI published through a Cloudflare Tunnel behind a login. The provisioning, systemd, backup, and tunnel scripts are in hub/, and each has a runbook: DEPLOY-APP.md, DEPLOY-SYSTEMD.md, DEPLOY-BACKUPS.md, DEPLOY-TUNNEL.md.

Testing

python -m pytest

512 collected tests; 9 cases skip unless you point them at a live hub or a token. The one worth pointing at is the acceptance oracle: a set of independently-derived facts about my actual 2026 H1 history - union commit count, distinct active days, the per-machine split, reconciled hours - that the harvest has to reproduce. It runs in two layers, a hermetic gate on fixture data in CI and a live gate against a real two-machine harvest on the hub. It is the difference between "the tests pass" and "the numbers are right", and it caught real dedup and exclusion bugs during the build. See docs/ORACLE-GATE.md.

Status and scope

This is a personal tool, built for one operator's setup, and it is not a turnkey product. It assumes a specific shape of world: an always-on hub that can SSH to your dev machines, a GitHub account or three, a self-hosted Plane instance. There is no installer and no multi-user story - authentication is a single admin credential. You could adapt it, but you would be adapting it, not installing it.

It is published because the engineering is worth reading: the harvest and dedup design, the oracle gate, and the way the operational side (systemd units, backups, tunnel, LXC provisioning) is versioned alongside the code rather than living in someone's shell history.

The full product spec, including the phase-by-phase build log, is in docs/WORKFLOW.md.

License

MIT - see LICENSE.

About

Self-hosted evidence pipeline for git, GitHub, and Plane history.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages