Skip to content

feat: add custom Composer repository support for extension version checking#588

Open
pariweshsubedi wants to merge 6 commits intoFriendsOfShopware:mainfrom
pariweshsubedi:feature/custom-composer-repositories
Open

feat: add custom Composer repository support for extension version checking#588
pariweshsubedi wants to merge 6 commits intoFriendsOfShopware:mainfrom
pariweshsubedi:feature/custom-composer-repositories

Conversation

@pariweshsubedi
Copy link
Copy Markdown

@pariweshsubedi pariweshsubedi commented Mar 9, 2026 •
edited
Loading

Closes #587

  • Add support for configuring custom Composer repositories (e.g. Packeton, Private Packagist, Satis) per shop to detect extension updates from private package sources
  • Support Composer v1 (inline) and v2 (lazy provider) repository formats with none, HTTP Basic, and Bearer token authentication
  • Credentials are encrypted at rest using the existing APP_SECRET encryption

How it works

  1. Shop scrape runs as usual (fetch from shop API → check Shopware Store)
  2. After scrape completes, if the shop has custom repos configured, a composer-check job is enqueued
  3. The composer-check job reads unresolved extensions from shop_extension, fetches versions from each Composer repo, and updates matching rows with latestVersion and changelog

Changes

  • Schema: New composer_repositories JSONB column on shop table (migration 0011)
  • API: Zod-validated create/update endpoints with encrypted credentials
  • Queue: New composer queue + composer-check job type + dedicated worker (concurrency 2)
  • Scrape job: Inline composer check removed, replaced with job enqueue
  • Service: composer-repo.service.ts — extracted Composer repo fetching logic
  • Frontend: Repository management UI with auth type selection in Edit Shop

Testing performed

  • make lint passes with no new warnings
  • Manually tested adding, editing, and removing custom Composer repositories in the Edit Shop UI
  • Tested all three auth types (none, HTTP Basic, Bearer token) — correct fields appear for each
  • Verified repositories persist after page reload with URLs and auth types preserved
  • Verified credentials (passwords, tokens) are not returned to the frontend after saving
  • Verified saving an empty repository list works correctly

Suggested tests

  • Verify extension versions are detected from a Composer v1 (inline) repository
  • Verify extension versions are detected from a Composer v2 (lazy provider) repository
  • Verify credentials are not returned to the frontend after saving
  • Verify scraping still works for shops with no custom repositories configured
  • Test with invalid repository URLs and verify graceful error handling

@shyim
Copy link
Copy Markdown
Member

shyim commented Mar 11, 2026

I would like to add a proper queue before and migrate to that before, we add those things as next

@pariweshsubedi
Copy link
Copy Markdown
Author

pariweshsubedi commented Mar 11, 2026

Makes sense! Just to make sure, by "proper queue" do you mean moving the scrape jobs (both the hourly cron and the manual refresh)?

I'll hold off on this PR until the queue is in place. Is there an issue tracking the queuing feature?

@shyim
Copy link
Copy Markdown
Member

shyim commented Mar 11, 2026

I would like to parallelize using queue the scrapes. Still not sure which of those NPM packages we're gonna will use at the end: https://bullmq.io/

I expect to be done this week.

shyim
shyim requested changes Mar 11, 2026
View reviewed changes
api/src/modules/shop/jobs/shop-scrape.job.ts Outdated
return timeDifference >= 24 * 60 * 60 * 1000;
}

interface ComposerPackageVersion {
Copy link
Copy Markdown
Member

@shyim shyim Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move this composer foo into an own typescript file.

Copy link
Copy Markdown
Author

@pariweshsubedi pariweshsubedi Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracted all Composer repository logic to api/src/modules/shop/composer-repo.service.ts

api/src/modules/shop/jobs/shop-scrape.job.ts Outdated
const data = (await resp.json()) as ComposerPackagesJson;

// Composer v2 lazy provider: metadata-url + available-packages
if (data["metadata-url"] && data["available-packages"]?.length) {
Copy link
Copy Markdown
Member

@shyim shyim Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

available-packages is optional in composer spec and does not require it.

Copy link
Copy Markdown
Author

@pariweshsubedi pariweshsubedi Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

available-packages is now treated as optional. If it's present, we fetch only those packages. When absent, we fall through to process inline packages from packages.json.

@pariweshsubedi pariweshsubedi requested a review from shyim March 13, 2026 06:57
@pariweshsubedi
feat: add custom Composer repository support for extension version ch…
962f098 
…ecking

Allow shops to configure private Composer repositories (e.g. Packeton,
Private Packagist, Satis) so that extension updates from custom package
sources are detected during scraping. Supports Composer v1 and v2 formats
with none, HTTP Basic, and Bearer token authentication. Credentials are
encrypted at rest.
@pariweshsubedi
chore: use generic example for custom composer repo source
d2a20e7
@pariweshsubedi
chore: extract Composer repo operations into composer-repo.ts
b556fe6
@pariweshsubedi
chore: move composer-repo.ts to shop module root and fix formatting
ed58e83
@pariweshsubedi
chore: use layer suffix for composer-repo
785829a
@pariweshsubedi
feat: move composer repo check to dedicated BullMQ queue
b2e01cd
@pariweshsubedi pariweshsubedi force-pushed the feature/custom-composer-repositories branch from a27e7cb to b2e01cd Compare March 20, 2026 13:18
@pariweshsubedi
Copy link
Copy Markdown
Author

pariweshsubedi commented Mar 20, 2026

@shyim I added in some changes to move composer checks to the queue as well + some refactor. Can you have a look ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shyim shyim Awaiting requested review from shyim

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Support custom Composer repositories for extension version checking

2 participants

@pariweshsubedi @shyim