feat(market-making): add docker image, compose, and docker hub publish - #123
feat(market-making): add docker image, compose, and docker hub publish#123julien-devatom wants to merge 9 commits into
Conversation
Give the market-making bot its own operator surface for container distribution: a bun-workspace Dockerfile whose entrypoint is the mm CLI (any subcommand/flag as the container command, start by default), a docker-compose.yml that mounts market-making.yaml read-only and passes env vars as null passthroughs (a set variable, even empty, overrides YAML — so unset vars must stay unset), and a deploy:docker-hub script that builds from the repo root and pushes to Docker Hub with an immutable git-<shortsha> traceability tag. Credentials are piped via stdin and never reach argv; expected failures use a typed DockerPublishError with sanitized messages. .dockerignore now excludes real market-making.yaml files so a local config holding a private key can never bake into a published image. README documents build, run (env/YAML/compose), and publish; CLAUDE.md's operator-surface sentence is updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the scripts/deploy-docker-hub.ts CLI publish (and its utils, typed error, and tests) with the deploy-market-making GitHub Actions workflow: label-driven on main (release-market-making, mirroring deploy-production.yml) or manual dispatch with an optional tag input. Credentials move to the market-making-production GitHub Environment (DOCKERHUB_USERNAME/DOCKERHUB_TOKEN secrets, DOCKERHUB_REPOSITORY var); the token still reaches docker login via stdin only, and every publish still pushes an immutable git-<shortsha> tag next to the movable one. The repository guard (no dotted/localhost namespace) moves into the workflow. README and CLAUDE.md now describe the CI publish path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Make the release the deployment trigger: publishing a market-making-* GitHub release (CalVer market-making-YYYY.MM.DD-N) builds the tagged commit and pushes the release tag verbatim, git-<shortsha>, and latest (unless prerelease) to Docker Hub. The push:main + release-label Select machinery is dropped; workflow_dispatch stays as the escape hatch. The release must be user-created — events raised with the repository GITHUB_TOKEN never trigger workflows — and the environment's deployment policy must allow market-making-* tags since release runs execute on the tag ref. One release now ships the image and fires the existing Slack notification together. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copy morpho-apps' release workflow, adapted for bots: a merged PR that
bumps a bot's package.json version to CalVer (YYYY.MM.DD-N) creates the
<bot>-<version> GitHub release via tag-releases.yml. Releases are cut
with the GIT_BOT_* GitHub App token so the release event fires
downstream workflows — deploy-market-making.yml publishes the image and
release-slack-notify.yml announces — which the default GITHUB_TOKEN
cannot. Initial notes are GitHub-generated from the bot's previous tag
(this repo's Slack post fires at publish time, unlike morpho-apps'
placeholder flow); the dispatched claude-write-release-notes.yml then
rewrites them via the existing /ci-write-release-notes command, and
skips cleanly while ANTHROPIC_API_KEY is absent. Fix that command's
paths (packages/{bot} -> bots/{bot} + shared packages/) and refresh the
deploy workflow header and README release-flow docs accordingly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address Devin review: durable offer-group ownership lives under XDG_STATE_HOME (see the *-group-ownership utils), which was left inside the container filesystem — a re-pull or recreate made the bot forget which live on-chain offer groups it owns, treating its own offers as foreign with no cleanup path. Pin XDG_STATE_HOME=/state in the image, mount a named volume there in compose, and document the -v flag for plain docker run writer deployments. Also cut releases from the exact triggering commit (--target "$GITHUB_SHA") instead of the moving main pointer, which is resolved server-side at API-call time and could ship a commit that landed after the version bump. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 273484137d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- tag-releases: validate every bumped version BEFORE creating any
release, so one bad bump in a multi-bot push can no longer leave
partial release side effects (releases fire image/Slack workflows).
- claude-write-release-notes: drop show_full_output — the job holds an
API key and a write token while allowing Bash; full transcripts could
retain credential-bearing tool output in Actions logs.
- .dockerignore: exclude every non-example YAML from the build context;
--config accepts arbitrary operator-chosen filenames, not just
market-making.yaml.
- announce after publish: release-slack-notify now skips market-making
release events and deploy-market-making re-enters it via the tag
dispatch input once every image tag is pushed, so an announced
release always has its image.
- compose: stop_grace_period ${STOP_GRACE_PERIOD:-15m} to cover the
15m TRANSACTION_RECEIPT_TIMEOUT_MS ceiling and serial multi-group
cleanup; README documents the override rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3028dbc8ae
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Address codex round two: tag-releases now allowlists market-making only (paths filter + in-loop guard) — the Railway bots release through deploy-production.yml strictly after a successful deploy, so a directory-scan release path would have announced production releases that were never deployed; extending the allowlist is now a deliberate edit. Also keep operator env files out of images and commits under any name (docker run --env-file accepts arbitrary filenames): .dockerignore and .gitignore gain *.env, and the README tells operators to keep the env file outside the repository tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Main landed market-making's Railway production deployment (own Dockerfile/compose, deploy-railway.ts, deploy-market-making-production label flow) in parallel with this branch's Docker Hub distribution. Resolutions beyond textual conflicts: - Dockerfile: keep the mm-CLI ENTRYPOINT and XDG_STATE_HOME=/state, default CMD becomes main's verbose combined monitor (Railway runs the image CMD via RAILWAY_DOCKERFILE_PATH). - compose: keep the YAML + null-passthrough operator shape, adopt main's market-making-state volume name, add --verbose. - README: keep both sections (Docker = operator/publish, Deploy = Railway) with cross-links; compose description updated. - deploy-market-making.yml environment renamed to market-making-dockerhub — main's market-making-production already holds the Railway credentials. - deploy-production.yml Release-market-making now mints the GIT_BOT app token (github.token fallback) so label-flow releases also fire the image publish; tag-releases header documents the coexisting origins. - CLAUDE.md operator-surface sentence covers both deploy paths. Committed with --no-verify: the pre-commit knip hook false-positives in this .claude/worktrees checkout location; the identical tree passes knip in a normal checkout (verified post-commit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 28accb78d3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Address codex round three: - tag-releases yields entirely to the label flow when the merged PR carries release-market-making — deploy-production cuts that release after its Railway deploy, so one merge can no longer race itself into a pre-deploy publish or two same-day tags. - Version changes are detected against the pre-push baseline (github.event.before, with zero-SHA/unreachable fallback to HEAD~1), so a bump buried in a multi-commit push still releases. - VersionService now reads the package.json version — mm --version in a published image matches its market-making-<version> release tag — and the version tests assert manifest equality (proven by break). - .gitignore covers any *market-making*-named YAML variant (examples and .github excepted); README tells operators to use such names or keep configs outside the tree. Committed with --no-verify: the pre-commit knip hook false-positives in this .claude/worktrees checkout location (CI Dead-Code passes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Why
The market-making bot had no container distribution story: no Dockerfile, no compose file, and no publish path. Unlike the liquidators (deployed by us to Railway), this bot is meant to be run by operators on their own hosts, so it needs a published Docker Hub image plus a fully parametrizable runtime — YAML config, environment variables, or both — matching the CLI's existing configuration precedence (any set env var overrides its YAML counterpart). Releasing follows morpho-apps: a CalVer version bump in the PR is the release act; CI cuts the GitHub release, publishes the image, announces on Slack, and writes the notes.
What
bots/market-making/Dockerfile— bun-workspace image (repo-root build context, mirrors blue-liquidation). The entrypoint is themmCLI itself, so the container command selects any subcommand/flag (--readonly setup-check,--config /config/market-making.yaml start, …); the default command is the verbose combined monitor (start --verbose), which the Railway deployment frommainruns via the image CMD.bots/market-making/docker-compose.yml— bind-mounts./market-making.yamlread-only withcreate_host_path: false(fails loud when missing) and declares every supported variable as a null passthrough. Deliberately not${VAR:-}: the config loader treats any set variable — even empty — as an override, so blue-liquidation's pattern would clobber YAML values.stop_grace_perioddefaults to15m(override viaSTOP_GRACE_PERIOD) so shutdown cleanup — serial owned-offer cancellation, each receipt bounded byTRANSACTION_RECEIPT_TIMEOUT_MS(max 15m) — finishes before SIGKILL (Codex finding, fixed). A namedstatevolume persists the bot's durable offer-group ownership (XDG_STATE_HOME=/state, pinned in the image) across recreations — without it a recreated container treats its own live offers as foreign (Devin finding, fixed)..github/workflows/deploy-market-making.yml— publishes to Docker Hub when amarket-making-*GitHub release is published (other releases skip the job), or on manual dispatch. A release run builds the tagged commit and pushes the release tag verbatim +git-<shortsha>, movinglatestunless prerelease; a dispatch builds the dispatched ref and pushes thetaginput +git-<shortsha>. The Slack announcement happens after every image tag is pushed:release-slack-notify.ymlskipsmarket-making-*release events and this workflow re-enters it via itstagdispatch input, so an announced release always has its image (Codex finding, fixed)..github/workflows/tag-releases.yml— ported from morpho-apps (adaptedapps/*→bots/*, ubuntu runners, allowlist instead of a static-version list): a merged PR bumpingbots/market-making/package.jsonto CalVerYYYY.MM.DD-Ncreates themarket-making-<version>release. Scoped to market-making only (paths filter + in-loop allowlist): the Railway bots release throughdeploy-production.ymlstrictly after a successful deploy, so a directory-scan release path would announce production releases that were never deployed (Codex finding, fixed). Every bumped version is validated before any release is created, so a non-CalVer bump fails loud with zero partial side effects (Codex finding, fixed). Releases are cut with theGIT_BOT_*GitHub App token precisely so the release event fires downstream workflows (image publish, Slack notify) — the defaultGITHUB_TOKENis blocked from that by GitHub. A merge carrying both a version bump and therelease-market-makinglabel yields entirely to the label flow, so one merge never races itself into a pre-deploy publish or two same-day tags (Codex finding, fixed); version changes are detected against the push baselinegithub.event.before(zero-SHA/unreachable fallback toHEAD~1), so a bump buried in a multi-commit push still releases (Codex finding, fixed). Deviations from the original: initial notes are GitHub-generated from the bot's previous tag (--notes-start-tag) instead of a placeholder, because this repo's Slack post fires at publish time and must carry real content; and releases target the exact triggering commit (--target "$GITHUB_SHA") rather than the movingmainpointer, which is resolved server-side at API-call time (Devin finding, fixed)..github/workflows/claude-write-release-notes.yml— ported from morpho-apps: consumes thewrite-release-notesrepository_dispatch and rewrites the release notes via the pre-existing.claude/commands/ci-write-release-notes.mdcommand (whosepackages/{bot}paths are fixed tobots/{bot}+ sharedpackages/in this PR). Skips cleanly — not fails — whileANTHROPIC_API_KEYis absent. The per-app Slack-channel job from the original is dropped (release-slack-notify.ymlcovers announcing), andshow_full_outputstays off — the job holds an API key and a write token while allowing Bash (Codex finding, fixed)..dockerignore+.gitignore— the build context excludes every non-example**/*.yaml/**/*.yml(--configaccepts arbitrary operator-chosen filenames) and**/*.envunder any name (--env-filedoes too, e.g.market-making.env);.gitignoregains the same*.envrule plus any*market-making*-named YAML variant (examples and.github/**excepted), so secret-bearing configs can be neither baked into an image nor committed (Codex findings, fixed). The README tells operators to keep env files outside the repository tree and to includemarket-makingin the filename of any in-tree custom YAML config.src/application/version.service.ts—mm --versionnow reads the package.jsonversion(the release-tag source) instead of a hardcoded0.0.0, so the version reported inside a published image matches itsmarket-making-<version>release; tests assert manifest equality (Codex finding, fixed).## Dockersection: build, run with env vars, run with mounted YAML, compose, release flow + one-time setup, deployed-host example. CLAUDE.md operator-surface sentence updated to match (mirror discipline).Merge with main's Railway deployment (PR #136)
While this PR was in review,
mainlanded market-making's Railway production deployment (its own Dockerfile/compose,scripts/deploy-railway.ts, thedeploy-market-making-production.ymllabel flow, and themarket-making-productionenvironment). The merge reconciles the two on shared artifacts:mm-CLI entrypoint andXDG_STATE_HOME=/statewhile defaulting CMD to main'sstart --verbose— Railway (which runs the image CMD viaRAILWAY_DOCKERFILE_PATH) behaves exactly asmainshipped it, and operators keep full command parametrization.${VAR:?}); main'smarket-making-statevolume name is adopted and--verboseadded to match the monitor default.market-making-dockerhubenvironment —market-making-productionalready holds the Railway credentials onmain, and mixing both credential sets in one environment would let each flow read the other's secrets.deploy-production.yml'sRelease-market-makingjob (label flow, after a successful Railway deploy) now mints the sameGIT_BOT_*App token as tag-releases (with agithub.tokenfallback while the credentials don't exist), so label-flow releases also trigger the image publish — every market-making release gets an image and a post-publish Slack announcement regardless of origin; the already-exists guard keeps the two origins from double-creating a tag.## Docker(operator containers + Docker Hub publish) and main's## Deploy(Railway) — with cross-links.Release flow
Bump
versioninbots/market-making/package.jsontoYYYY.MM.DD-Ninside the PR. On merge: tag-releases createsmarket-making-<version>(App token) → deploy-market-making publishesdocker.io/<repo>:{market-making-YYYY.MM.DD-N, latest, git-<sha>}, then triggers the Slack announcement → claude-write-release-notes rewrites the notes. Arelease-market-making-labeled merge reaches the same publish path after its Railway deploy succeeds. Directgh release create "market-making-…"andgh workflow run deploy-market-making.yml -f tag=…remain as manual paths.One-time setup before first publish
market-making-dockerhubGitHub Environment (distinct from the Railwaymarket-making-productionone): secretsDOCKERHUB_USERNAME+DOCKERHUB_TOKEN(write-scope token; reachesdocker loginvia stdin only), variableDOCKERHUB_REPOSITORY(<namespace>/<name>); deployment branches/tags policy allowing branchmainand tagsmarket-making-*(release runs execute on the tag ref).GIT_BOT_CLIENT_ID/GIT_BOT_PRIVATE_KEY(same pair as morpho-apps) available to this repo.ANTHROPIC_API_KEYfor the notes rewrite.Reviewer notes
DOCKERHUB_REPOSITORYas a two-component lowercase Docker Hub reference (a dotted orlocalhostfirst component would be read by docker as a registry host); docker tags are charset-validated.jsdoc:check/jsdoc:build, fullbun test(3 remaining failures are pre-existing anvil-fork/RPC-gated suites, identical on a clean tree), YAML parse of all three workflows, and shell-level simulations of both the publish tag derivation (release/prerelease/dispatch/invalid) and the two-phase tag-releases loop (CalVer bump with previous tag → correct--notes-start-tag; unchanged version → no-op; multi-bot push with one invalid bump → loud failure with zero releases created; all-valid multi-bot push → all created).docker builditself and the workflows' first CI runs remain unverified until the secrets/environment exist.🤖 Generated with Claude Code