feat: packages worker temporal scaffold#4145
Conversation
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
|
|
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
This reverts commit affb6bd. Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
09c2822 to
4099d64
Compare
There was a problem hiding this comment.
Pull request overview
This PR converts packages_worker from an idle/health-probe process into a Temporal-backed ServiceWorker, scaffolding an initial (placeholder) npm ingest workflow + activity + cron schedule and wiring a dedicated Temporal namespace via CROWD_PACKAGES_TEMPORAL_NAMESPACE. It also updates build/deploy assets (Dockerfile, compose service env, builder definition) and applies non-functional formatting changes in the GitHub repos enricher.
Changes:
- Replace
packages-workerentrypoint withServiceWorkerTemporal worker startup and register annpm-registry-ingestTemporal Schedule. - Add minimal Temporal workflow/activity scaffold for npm (
npmHello+sayHiNpm) and export wiring for worker registration. - Update Docker/build + compose/env plumbing for the packages worker (namespace/task queue env, Dockerfile changes, builder env file); minor formatting-only enricher edits.
Reviewed changes
Copilot reviewed 18 out of 20 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| services/apps/packages_worker/src/workflows/index.ts | Exports npmHello workflow for worker bundling/imports. |
| services/apps/packages_worker/src/service.ts | Introduces ServiceWorker instance configured for Temporal-only operation. |
| services/apps/packages_worker/src/npm/workflows.ts | Adds npmHello workflow calling proxied npm activities. |
| services/apps/packages_worker/src/npm/schedule.ts | Creates Temporal Schedule npm-registry-ingest for npmHello. |
| services/apps/packages_worker/src/npm/activities.ts | Adds placeholder npm activity sayHiNpm. |
| services/apps/packages_worker/src/enricher/updateEnrichedRepos.ts | Formatting/import-order adjustments only. |
| services/apps/packages_worker/src/enricher/runEnrichmentLoop.ts | Formatting-only refactors (line wrapping). |
| services/apps/packages_worker/src/enricher/fetchLightRepo.ts | Formatting-only refactors (line wrapping). |
| services/apps/packages_worker/src/db.ts | Import ordering change only. |
| services/apps/packages_worker/src/config.ts | Formatting-only refactor of token parsing chain. |
| services/apps/packages_worker/src/bin/packages-worker.ts | Switches packages-worker startup to Temporal ServiceWorker + schedule registration. |
| services/apps/packages_worker/src/activities.ts | Re-exports npm activity for worker activity registration. |
| services/apps/packages_worker/package.json | Adds Temporal/archetype deps and wires namespace/taskqueue into start/dev scripts. |
| scripts/services/packages-worker.yaml | Sets task queue + namespace env for compose service. |
| scripts/services/docker/Dockerfile.packages-worker.dockerignore | Adds ignore patterns intended to reduce Docker build context. |
| scripts/services/docker/Dockerfile.packages-worker | Updates base image to bookworm-slim and narrows pnpm install scope; copies archetypes explicitly. |
| scripts/builders/packages-worker.env | Adds builder definition for packages-worker image. |
| pnpm-lock.yaml | Lockfile updates for new deps/workspace links (plus incidental metadata churn). |
| docs/adr/README.md | Adds ADR-0003 index entry for Temporal ServiceWorker decision. |
| backend/.env.dist.local | Adds CROWD_PACKAGES_TEMPORAL_NAMESPACE default for local runs. |
| backend/.env.dist.composed | Adds CROWD_PACKAGES_TEMPORAL_NAMESPACE placeholder for composed envs. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: anilb <epipav@gmail.com>
Signed-off-by: anilb <epipav@gmail.com>
| WORKDIR /usr/crowd/app | ||
| RUN npm install -g corepack@latest && corepack enable pnpm && corepack prepare pnpm@9.15.0 --activate | ||
|
|
||
| COPY ./pnpm-workspace.yaml ./pnpm-lock.yaml ./ |
|
|
||
| WORKDIR /usr/crowd/app | ||
| RUN npm install -g corepack@latest && corepack enable pnpm && corepack prepare pnpm@9.15.0 --activate | ||
| RUN npm install -g corepack@latest && corepack enable pnpm && corepack prepare pnpm@9.15.0 --activate && apt update && apt install -y ca-certificates --no-install-recommends && rm -rf /var/lib/apt/lists/* |
Signed-off-by: anilb <epipav@gmail.com>
Note
Medium Risk
Introduces Temporal schedules and a new worker runtime path for packages-worker; production impact is limited while workflows are placeholders, but misconfigured namespace/task queue could prevent jobs from running.
Overview
packages-worker stops being an idle process with file-based health probes and becomes a Temporal worker via
@crowd/archetype-worker, aligned with other crowd workers.On startup it registers a daily schedule (
npm-registry-ingest, cron15 3 * * *) that runs a placeholdernpmHelloworkflow callingsayHiNpmon thepackages-workertask queue—scaffolding for future npm registry ingest, not real ingest yet.Configuration and deploy wire a dedicated packages Temporal namespace (
CROWD_PACKAGES_TEMPORAL_NAMESPACEin local env and compose), setCROWD_TEMPORAL_TASKQUEUE/ namespace inpackages-workerscripts and Docker Compose, and add Temporal client/workflow plus archetype dependencies. The Docker image moves to bookworm-slim, installs build deps via apt, copies only archetypes/libs/packages_worker, uses a dockerignore, and adds a builder env file for the OCI image.The github-repos-enricher path is unchanged in behavior aside from a tighter worker loop (
while (nextIdx < validUrls.length)instead of unboundedwhile (true)) and minor formatting/logging tweaks.Reviewed by Cursor Bugbot for commit 818b7b9. Bugbot is set up for automated code reviews on this repo. Configure here.