Parallel deploys can cross-contaminate app bundles via cached App Management signed upload URLs

[MitchLillie]

Summary

When running multiple shopify app deploy --path=... processes concurrently for different apps in the same organization, app versions can be created from the wrong app bundle. In our production stress test, a generated app's local bundle was correct, but the released production version rendered another generated app's UI and static asset manifest.

The strongest suspected root cause is that Shopify CLI's App Management client caches appRequestSourceUploadUrl responses for 59 minutes using variables that only include organizationId and sourceExtension. The app ID/API key passed to generateSignedUploadUrl is ignored. This means concurrent deploys in the same org can upload different bundles to the same signed URL, and whichever upload App Management reads last can be used for another app's appVersionCreate.

Impact

Concurrent deploys of multiple apps in one organization can produce app versions containing another app's bundle/config/assets.

This is especially dangerous for automation or CI workflows that deploy multiple generated/test apps in parallel using --path, because each CLI invocation appears isolated locally but can share the same cached remote upload URL.

Observed behavior

Using this repo, we ran a guarded production static-asset stress test with 10 generated apps.

You can repro this by asking the LLM for parallel deploys. The local source and built bundle were not contaminated. The wrong content appeared after upload/deploy.

Relevant CLI code

In /packages/app/src/cli/utilities/developer-platform-client/app-management-client.ts:

async generateSignedUploadUrl({organizationId}: MinimalAppIdentifiers): Promise<AssetUrlSchema> {
  const variables = {
    sourceExtension: 'BR' as SourceExtension,
    organizationId: gidFromOrganizationIdForShopify(organizationId),
  }
  const result = await this.appManagementRequest({
    query: CreateAssetUrl,
    variables,
    cacheOptions: {cacheTTL: {minutes: 59}},
  })
  return {
    assetUrl: result.appRequestSourceUploadUrl.sourceUploadUrl,
    userErrors: result.appRequestSourceUploadUrl.userErrors,
  }
}

Issues with this implementation:

1. generateSignedUploadUrl accepts MinimalAppIdentifiers, but only destructures organizationId.
2. apiKey and id are ignored.
3. The GraphQL variables are only sourceExtension and organizationId.
4. The response is cached for 59 minutes.
5. CLI-kit GraphQL request cache keys are derived from query + variables + CLI version + optional cacheExtraKey.
6. No cacheExtraKey is supplied.
7. Therefore, all App Management deploys in the same org using Brotli can receive the same cached signed upload URL.

The test currently asserts this caching behavior in packages/app/src/cli/utilities/developer-platform-client/app-management-client.test.ts:

expect(appManagementRequestDoc).toHaveBeenCalledWith({
  query: CreateAssetUrl,
  token: 'token',
  variables: {
    sourceExtension: 'BR' as SourceExtension,
    organizationId: 'gid://shopify/Organization/213141',
  },
  unauthorizedHandler: {
    handler: expect.any(Function),
    type: 'token_refresh',
  },
  cacheOptions: {
    cacheTTL: {minutes: 59},
  },
})

Why this explains the observed behavior

The deploy flow is roughly:

1. Build app bundle locally at ./<app>/.shopify/deploy-bundle.br.
2. Request a signed upload URL.
3. Upload the local bundle to that URL.
4. Call App Management appVersionCreate with that URL as sourceUrl.

If concurrent CLI processes in the same org reuse one cached signed URL:

1. App 9 uploads bundle 9 to the shared URL.
2. App 10 uploads bundle 10 to the same shared URL.
3. App 9 calls appVersionCreate(sourceUrl: sharedUrl).
4. App Management reads sharedUrl after app 10 overwrote it.
5. App 9's version is created from app 10's bundle.

This matches the observed case where app 9's local source and local deploy bundle were correct, but production app 9 rendered APP 10.

Expected behavior

Each deploy should upload to an isolated source location. Deploying multiple apps concurrently in the same organization should not allow one app's bundle to be consumed by another app's version creation.

Suggested fix

Do not cache signed upload URLs.

Suggested implementation:

async generateSignedUploadUrl({organizationId}: MinimalAppIdentifiers): Promise<AssetUrlSchema> {
  const variables = {
    sourceExtension: 'BR' as SourceExtension,
    organizationId: gidFromOrganizationIdForShopify(organizationId),
  }
  const result = await this.appManagementRequest({
    query: CreateAssetUrl,
    variables,
  })
  return {
    assetUrl: result.appRequestSourceUploadUrl.sourceUploadUrl,
    userErrors: result.appRequestSourceUploadUrl.userErrors,
  }
}

Signed upload URLs represent mutable write destinations and should be treated as unique per upload attempt, not cached metadata.

Alternative mitigation if caching is required

If caching must remain for some reason, include a unique cache key per deploy/upload attempt, such as app ID/API key plus command run ID or a fresh UUID:

cacheOptions: {
  cacheTTL: {minutes: 59},
  cacheExtraKey: `${app.id}-${process.env.COMMAND_RUN_ID}-${randomUUID()}`,
}

However, this effectively disables useful caching. Removing caching is simpler and safer.

Server-side hardening ideas

Even with the CLI fixed, App Management could prevent this class of bug by:

• Generating a unique object key for every appRequestSourceUploadUrl call.
• Returning an opaque upload/source ID instead of a reusable mutable URL.
• Having appVersionCreate atomically consume/copy the uploaded source once.
• Rejecting reused source URLs.
• Accepting and verifying bundle SHA256/size before creating the app version.

Workaround

Until fixed, avoid deploying multiple apps concurrently with shopify app deploy --path=... in the same organization. Our stress-test runner now deploys generated apps serially by default.