Skip to content

Feat/uepr 535 introduce script for migrating packages to monorepo#505

Open
KManolov3 wants to merge 3 commits into
developfrom
feat/uepr-535-introduce-script-for-migrating-packages-to-monorepo
Open

Feat/uepr 535 introduce script for migrating packages to monorepo#505
KManolov3 wants to merge 3 commits into
developfrom
feat/uepr-535-introduce-script-for-migrating-packages-to-monorepo

Conversation

@KManolov3
Copy link
Copy Markdown
Contributor

TODO:

  • Test more extensively for corner cases

Resolves

Part of https://scratchfoundation.atlassian.net/browse/UEPR-535

Proposed Changes

Add two scripts - add-repo.sh and refresh-gh-workflows to assist in adding new packages to monorepo.

Reason for Changes

We want to expand the scratch-editor monorepo with other packages that make sense to be there (e.g. scratch-paint, scratch-audio, ...)

Test Coverage

This is for development use, and the results of it should always undergo manual developer review, so I'm unsure if automated tests make sense in terms of effort.

@KManolov3 KManolov3 requested a review from kbangelov March 27, 2026 14:15
@KManolov3 KManolov3 requested a review from a team as a code owner March 27, 2026 14:15
run: npm publish --access=public --tag="$NPM_TAG" --ignore-scripts --workspace=@scratch/task-herder

- name: Publish scratch-media-lib-scripts
run: |
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those were some leftovers from the publish update PR, the build and token set happen earlier currently.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 27, 2026

Test report for task-herder

28 tests  ±0   28 ✅ ±0   0s ⏱️ ±0s
 7 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 725c499. ± Comparison against base commit ace70ed.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 27, 2026

Test report for scratch-media-lib-scripts

20 tests  ±0   20 ✅ ±0   0s ⏱️ ±0s
 1 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 725c499. ± Comparison against base commit ace70ed.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 27, 2026

Test report for scratch-svg-renderer

  1 files  ±0   60 suites  ±0   0s ⏱️ ±0s
124 tests ±0  124 ✅ ±0  0 💤 ±0  0 ❌ ±0 
276 runs  ±0  275 ✅ ±0  1 💤 ±0  0 ❌ ±0 

Results for commit 725c499. ± Comparison against base commit ace70ed.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 27, 2026

Test report for scratch-render

  1 files  ±0   55 suites  ±0   2s ⏱️ ±0s
209 tests ±0  209 ✅ ±0  0 💤 ±0  0 ❌ ±0 
279 runs  ±0  279 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 725c499. ± Comparison against base commit ace70ed.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 27, 2026

Test report for scratch-gui

  2 files  ±0   63 suites  ±0   10m 47s ⏱️ +37s
399 tests ±0  391 ✅ ±0  8 💤 ±0  0 ❌ ±0 
417 runs  ±0  409 ✅ ±0  8 💤 ±0  0 ❌ ±0 

Results for commit 725c499. ± Comparison against base commit ace70ed.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 27, 2026

Test report for scratch-vm

    1 files  ±0    886 suites  ±0   1m 59s ⏱️ -1s
1 916 tests ±0  1 916 ✅ ±0   0 💤 ±0  0 ❌ ±0 
5 218 runs  ±0  5 188 ✅ ±0  30 💤 ±0  0 ❌ ±0 

Results for commit 725c499. ± Comparison against base commit ace70ed.

This pull request removes 2 and adds 2 tests. Note that renamed tests count towards both.
test/unit/blocks_control.js > wait ‑ Wait block ended too early: 10 < 10 - 16.666666666666668
test/unit/blocks_control.js > wait ‑ Wait block ended too late: 10 > 10 + 333.3333333333333
test/unit/blocks_control.js > wait ‑ Wait block ended too early: 12 < 10 - 16.666666666666668
test/unit/blocks_control.js > wait ‑ Wait block ended too late: 12 > 10 + 333.3333333333333

♻️ This comment has been updated with latest results.

Comment thread scripts/add-repo.sh
-not -path '*/.git/*' \
-not -path '*/node_modules/*' \
-exec grep -Il "$REPO_NAME" {} \; 2>/dev/null | while read -r file; do
sed -i '' -e "s:\(require(\|from \|resolve(\|node_modules\)\(['\"/]\)${REPO_NAME}\(['\"/]\):\1\2${NPM_ORGANIZATION}/${REPO_NAME}\3:g" "$file"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sed -i '' -e seems to be a macOS-only syntax. Consider using perl -pi -e

Comment thread scripts/add-repo.sh

# Rewrite all paths in the cloned repo to live under a subdirectory
# Handles repos with and without submodules
move_repository_subdirectory() {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was taken from the build-monorepo script. We don't (yet) have a case of repo with submodules.

Comment thread scripts/add-repo.sh
) "${PACKAGE_PATH}/package.json" | sponge "${PACKAGE_PATH}/package.json"
fi

### Step 5: Update root workspaces list ###
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

@kbangelov kbangelov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me overall! Left a few small comments.

Comment thread scripts/add-repo.sh
Comment on lines +53 to +63
--source-branch)
SOURCE_BRANCH="$2"
shift 2
;;
--org)
GITHUB_ORG="$2"
shift 2
;;
--cache-dir)
BUILD_CACHE="$2"
shift 2
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should consider adding checks for whether the second argument exists and whether it is not another flag, which would lead to nonsense instead of error.

if [ -z "$2" ] || [[ "$2" == --* ]]; then
      echo "Error: --source-branch requires a value" >&2
      usage 1
  fi

Comment thread scripts/add-repo.sh
fi

if [ -d "$PACKAGE_PATH" ]; then
echo "Error: ${PACKAGE_DIR} already exists in the monorepo." >&2
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice coverage!

Comment thread scripts/add-repo.sh
Comment on lines +372 to +391
for DEP in $DEPS; do
npm install --force --save --save-exact \
"${NPM_ORGANIZATION}/${DEP}" -w "${NPM_ORGANIZATION}/${PACKAGE}" \
|| package_replacement_error "$PACKAGE" "$DEP"
done
for DEP in $DEVDEPS; do
npm install --force --save-dev --save-exact \
"${NPM_ORGANIZATION}/${DEP}" -w "${NPM_ORGANIZATION}/${PACKAGE}" \
|| package_replacement_error "$PACKAGE" "$DEP"
done
for DEP in $OPTDEPS; do
npm install --force --save-optional --save-exact \
"${NPM_ORGANIZATION}/${DEP}" -w "${NPM_ORGANIZATION}/${PACKAGE}" \
|| package_replacement_error "$PACKAGE" "$DEP"
done
for DEP in $PEERDEPS; do
npm install --force --save-peer --save-exact \
"${NPM_ORGANIZATION}/${DEP}" -w "${NPM_ORGANIZATION}/${PACKAGE}" \
|| package_replacement_error "$PACKAGE" "$DEP"
done
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this might lead to a situation where the npm install fails but package_replacement_error just logs the error and continues.

Copy link
Copy Markdown
Contributor

@adzhindzhi adzhindzhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

run: npm publish --access=public --tag="$NPM_TAG" --ignore-scripts --workspace=@scratch/task-herder

- name: Publish scratch-media-lib-scripts
run: |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Comment thread scripts/add-repo.sh
echo "***ERROR***"
echo "Could not replace a dependency with the local monorepo version."
echo "Failed to replace $3 in $1#$2" | tee -a "monorepo.errors.log"
#exit 1 # uncomment this to make it a fatal error
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nicpick: Should we actually uncomment this? Or maybe make it a flag that is passed to the script? Because I think right now we could get to a point where something fails to install correctly, but the script still succeeds. Which makes it easier for the person running the script to miss it?

github-actions Bot pushed a commit that referenced this pull request May 14, 2026
…and #505

Replaces the script work of two prior PRs that independently solved
"add an existing scratchfoundation repo to the monorepo":

scripts/add-repo.sh — collapses add-to-monorepo.sh (#434) and add-repo.sh
(#505) into a single canonical entry point. Takes the CLI shape and
auto-CI-regen from #505 and the source-branch auto-detect (develop ->
main -> master), normalization (sort-package-json), and clean-tree
pre-flight from #434.

scripts/update-gha-workflows.ts — replaces the CircleCI-era
build-gha-workflows.ts and workspace-template.yml. Regenerates
.github/path-filters.yml and incrementally updates publish.yml.

Open review comments on #505 are addressed:

- sed -i portability bug: rewrites now use perl -pi -e (works on macOS,
  NixOS, Ubuntu/Debian/Arch, WSL). Perl is added to the explicit prereq
  check next to git-filter-repo, sponge, and jq.
- Arg parser now rejects flag-stealing: --source-branch --org foo errors
  instead of silently using --org as the branch name.
- npm-install failures hard-fail by default. --continue-on-error opts
  into the prior soft-fail behavior; failures are still logged to
  add-repo.errors.log when continuing.
- The submodules-aware branch in move_repository_subdirectory keeps a
  TODO comment marking that it has never been exercised against a real
  submodules-bearing repo and should be run carefully when one lands.

No repo additions are included; this PR is script-tooling only.
cwillisf added a commit that referenced this pull request May 14, 2026
Replaces the script work of two prior PRs that independently solved
"add an existing scratchfoundation repo to the monorepo":

scripts/add-repo.sh — collapses add-to-monorepo.sh (#434) and add-repo.sh
(#505) into a single canonical entry point. Takes the CLI shape and
auto-CI-regen from #505 and the source-branch auto-detect (develop ->
main -> master), normalization (sort-package-json), and clean-tree
pre-flight from #434.

scripts/update-gha-workflows.ts — replaces the CircleCI-era
build-gha-workflows.ts and workspace-template.yml. Regenerates
.github/path-filters.yml and incrementally updates publish.yml.

Open review comments on #505 are addressed:

- sed -i portability bug: rewrites now use perl -pi -e (works on macOS,
  NixOS, Ubuntu/Debian/Arch, WSL). Perl is added to the explicit prereq
  check next to git-filter-repo, sponge, and jq.
- Arg parser now rejects flag-stealing: --source-branch --org foo errors
  instead of silently using --org as the branch name.
- npm-install failures hard-fail by default. --continue-on-error opts
  into the prior soft-fail behavior; failures are still logged to
  add-repo.errors.log when continuing.
- The submodules-aware branch in move_repository_subdirectory keeps a
  TODO comment marking that it has never been exercised against a real
  submodules-bearing repo and should be run carefully when one lands.

Cross-workspace dep handling: rewires both bare-name and already-prefixed
@scratch/<name> deps in the new package to exact-pinned current monorepo
versions. Matches the existing convention in scratch-gui/scratch-vm/
scratch-render. npm's workspace: protocol would be the ideal here but
npm does not actually support it (npm/cli#8845 - EUNSUPPORTEDPROTOCOL
at install time).

Workspaces array ordering: the new package is inserted just after the
latest existing workspace it depends on, so 'npm run --workspaces build'
builds its deps first. Falls back to position 0 (prepend) when the new
package has no monorepo deps. This is a "last dep wins" heuristic, not
a full topo sort; a real topo sort can be added later by extending
update-gha-workflows.ts if needed.

bash 3.2 compatibility: macOS ships bash 3.2 at /bin/bash, which lacks
the 'mapfile' builtin. The find/grep output is collected via a 'while
IFS= read -r' loop so the script runs on macOS as well as Linux.

No repo additions are included; this PR is script-tooling only.
github-actions Bot pushed a commit that referenced this pull request May 14, 2026
…and #505

Replaces the script work of two prior PRs that independently solved
"add an existing scratchfoundation repo to the monorepo":

scripts/add-repo.sh — collapses add-to-monorepo.sh (#434) and add-repo.sh
(#505) into a single canonical entry point. Takes the CLI shape and
auto-CI-regen from #505 and the source-branch auto-detect (develop ->
main -> master), normalization (sort-package-json), and clean-tree
pre-flight from #434.

scripts/update-gha-workflows.ts — replaces the CircleCI-era
build-gha-workflows.ts and workspace-template.yml. Regenerates
.github/path-filters.yml and incrementally updates publish.yml.

Open review comments on #505 are addressed:

- sed -i portability bug: rewrites now use perl -pi -e (works on macOS,
  NixOS, Ubuntu/Debian/Arch, WSL). Perl is added to the explicit prereq
  check next to git-filter-repo, sponge, and jq.
- Arg parser now rejects flag-stealing: --source-branch --org foo errors
  instead of silently using --org as the branch name.
- npm-install failures hard-fail by default. --continue-on-error opts
  into the prior soft-fail behavior; failures are still logged to
  add-repo.errors.log when continuing.
- The submodules-aware branch in move_repository_subdirectory keeps a
  TODO comment marking that it has never been exercised against a real
  submodules-bearing repo and should be run carefully when one lands.

Cross-workspace dep handling: rewires both bare-name and already-prefixed
@scratch/<name> deps in the new package to exact-pinned current monorepo
versions. Matches the existing convention in scratch-gui/scratch-vm/
scratch-render. npm's workspace: protocol would be the ideal here but
npm does not actually support it (npm/cli#8845 - EUNSUPPORTEDPROTOCOL
at install time).

Workspaces array ordering: the new package is inserted just after the
latest existing workspace it depends on, so 'npm run --workspaces build'
builds its deps first. Falls back to position 0 (prepend) when the new
package has no monorepo deps. This is a "last dep wins" heuristic, not
a full topo sort; a real topo sort can be added later by extending
update-gha-workflows.ts if needed.

bash 3.2 compatibility: macOS ships bash 3.2 at /bin/bash, which lacks
the 'mapfile' builtin. The find/grep output is collected via a 'while
IFS= read -r' loop so the script runs on macOS as well as Linux.

No repo additions are included; this PR is script-tooling only.
cwillisf added a commit that referenced this pull request May 14, 2026
Replaces the script work of two prior PRs that independently solved
"add an existing scratchfoundation repo to the monorepo":

scripts/add-repo.sh — collapses add-to-monorepo.sh (#434) and add-repo.sh
(#505) into a single canonical entry point. Takes the CLI shape and
auto-CI-regen from #505 and the source-branch auto-detect (develop ->
main -> master), normalization (sort-package-json), and clean-tree
pre-flight from #434.

scripts/update-gha-workflows.ts — replaces the CircleCI-era
build-gha-workflows.ts and workspace-template.yml. Regenerates
.github/path-filters.yml and incrementally updates publish.yml.

Open review comments on #505 are addressed:

- sed -i portability bug: rewrites now use perl -pi -e (works on macOS,
  NixOS, Ubuntu/Debian/Arch, WSL). Perl is added to the explicit prereq
  check next to git-filter-repo, sponge, and jq.
- Arg parser now rejects flag-stealing: --source-branch --org foo errors
  instead of silently using --org as the branch name.
- npm-install failures hard-fail by default. --continue-on-error opts
  into the prior soft-fail behavior; failures are still logged to
  add-repo.errors.log when continuing. Scoped to the per-dep rewire
  loop; the final lockfile install always hard-fails by design.
- The submodules-aware branch in move_repository_subdirectory keeps a
  TODO comment marking that it has never been exercised against a real
  submodules-bearing repo and should be run carefully when one lands.

Cross-workspace dep handling: rewires both bare-name and already-prefixed
@scratch/<name> deps in the new package to exact-pinned current monorepo
versions. Matches the existing convention in scratch-gui/scratch-vm/
scratch-render. npm's workspace: protocol would be the ideal here but
npm does not actually support it (npm/cli#8845 - EUNSUPPORTEDPROTOCOL
at install time).

Workspaces array ordering: the new package is inserted just after the
latest existing workspace it depends on, so 'npm run --workspaces build'
builds its deps first. Falls back to position 0 (prepend) when the new
package has no monorepo deps. This is a "last dep wins" heuristic, not
a full topological sort; it relies on the existing workspaces array
already being in valid build order. A real topo sort can be added later
(e.g. by extending update-gha-workflows.ts) if it ever becomes necessary.

bash 3.2 compatibility: macOS ships bash 3.2 at /bin/bash, which lacks
the 'mapfile' builtin. The find/grep output is collected via a 'while
IFS= read -r' loop so the script runs on macOS as well as Linux.

Bare-clone alternates path: bare clones store the alternates file at
objects/info/alternates, not .git/objects/info/alternates. The previous
path was a no-op; the working disconnection comes from --dissociate in
the clone command. Path corrected so the line is defensive belt-and-
braces rather than dead code.

update-gha-workflows.ts: throws with a clear error if npm query returns
a workspace not present in the root "workspaces" array (previously
would silently misorder via indexOf returning -1). The comment in
resolveWorkspaces is corrected to reflect that the function returns
Map insertion order, with the caller re-sorting into declared order.

No repo additions are included; this PR is script-tooling only.
github-actions Bot pushed a commit that referenced this pull request May 14, 2026
…and #505

Replaces the script work of two prior PRs that independently solved
"add an existing scratchfoundation repo to the monorepo":

scripts/add-repo.sh — collapses add-to-monorepo.sh (#434) and add-repo.sh
(#505) into a single canonical entry point. Takes the CLI shape and
auto-CI-regen from #505 and the source-branch auto-detect (develop ->
main -> master), normalization (sort-package-json), and clean-tree
pre-flight from #434.

scripts/update-gha-workflows.ts — replaces the CircleCI-era
build-gha-workflows.ts and workspace-template.yml. Regenerates
.github/path-filters.yml and incrementally updates publish.yml.

Open review comments on #505 are addressed:

- sed -i portability bug: rewrites now use perl -pi -e (works on macOS,
  NixOS, Ubuntu/Debian/Arch, WSL). Perl is added to the explicit prereq
  check next to git-filter-repo, sponge, and jq.
- Arg parser now rejects flag-stealing: --source-branch --org foo errors
  instead of silently using --org as the branch name.
- npm-install failures hard-fail by default. --continue-on-error opts
  into the prior soft-fail behavior; failures are still logged to
  add-repo.errors.log when continuing. Scoped to the per-dep rewire
  loop; the final lockfile install always hard-fails by design.
- The submodules-aware branch in move_repository_subdirectory keeps a
  TODO comment marking that it has never been exercised against a real
  submodules-bearing repo and should be run carefully when one lands.

Cross-workspace dep handling: rewires both bare-name and already-prefixed
@scratch/<name> deps in the new package to exact-pinned current monorepo
versions. Matches the existing convention in scratch-gui/scratch-vm/
scratch-render. npm's workspace: protocol would be the ideal here but
npm does not actually support it (npm/cli#8845 - EUNSUPPORTEDPROTOCOL
at install time).

Workspaces array ordering: the new package is inserted just after the
latest existing workspace it depends on, so 'npm run --workspaces build'
builds its deps first. Falls back to position 0 (prepend) when the new
package has no monorepo deps. This is a "last dep wins" heuristic, not
a full topological sort; it relies on the existing workspaces array
already being in valid build order. A real topo sort can be added later
(e.g. by extending update-gha-workflows.ts) if it ever becomes necessary.

bash 3.2 compatibility: macOS ships bash 3.2 at /bin/bash, which lacks
the 'mapfile' builtin. The find/grep output is collected via a 'while
IFS= read -r' loop so the script runs on macOS as well as Linux.

Bare-clone alternates path: bare clones store the alternates file at
objects/info/alternates, not .git/objects/info/alternates. The previous
path was a no-op; the working disconnection comes from --dissociate in
the clone command. Path corrected so the line is defensive belt-and-
braces rather than dead code.

update-gha-workflows.ts: throws with a clear error if npm query returns
a workspace not present in the root "workspaces" array (previously
would silently misorder via indexOf returning -1). The comment in
resolveWorkspaces is corrected to reflect that the function returns
Map insertion order, with the caller re-sorting into declared order.

No repo additions are included; this PR is script-tooling only.
cwillisf added a commit that referenced this pull request May 14, 2026
Replaces the script work of two prior PRs that independently solved
"add an existing scratchfoundation repo to the monorepo":

scripts/add-repo.sh — collapses add-to-monorepo.sh (#434) and add-repo.sh
(#505) into a single canonical entry point. Takes the CLI shape and
auto-CI-regen from #505 and the source-branch auto-detect (develop ->
main -> master), normalization (sort-package-json), and clean-tree
pre-flight from #434.

scripts/update-gha-workflows.ts — replaces the CircleCI-era
build-gha-workflows.ts and workspace-template.yml. Regenerates
.github/path-filters.yml and incrementally updates publish.yml.

Open review comments on #505 are addressed:

- sed -i portability bug: rewrites now use perl -pi -e (works on macOS,
  NixOS, Ubuntu/Debian/Arch, WSL). Perl is added to the explicit prereq
  check next to git-filter-repo, sponge, and jq.
- Arg parser now rejects flag-stealing: --source-branch --org foo errors
  instead of silently using --org as the branch name.
- npm-install failures hard-fail by default. --continue-on-error opts
  into the prior soft-fail behavior; failures are still logged to
  add-repo.errors.log when continuing. Scoped to per-dep package.json
  rewrite failures in the rewire step; the final lockfile install
  always hard-fails by design.
- The submodules-aware branch in move_repository_subdirectory keeps a
  TODO comment marking that it has never been exercised against a real
  submodules-bearing repo and should be run carefully when one lands.

Cross-workspace dep handling: rewires both bare-name and already-prefixed
@scratch/<name> deps in the new package to exact-pinned current monorepo
versions. Matches the existing convention in scratch-gui/scratch-vm/
scratch-render. npm's workspace: protocol would be the ideal here but
npm does not actually support it (npm/cli#8845 - EUNSUPPORTEDPROTOCOL
at install time).

Workspaces array ordering: the new package is inserted just after the
latest existing workspace it depends on, so 'npm run --workspaces build'
builds its deps first. Falls back to position 0 (prepend) when the new
package has no monorepo deps. This is a "last dep wins" heuristic, not
a full topological sort; it relies on the existing workspaces array
already being in valid build order. A real topo sort can be added later
(e.g. by extending update-gha-workflows.ts) if it ever becomes necessary.

bash 3.2 compatibility: macOS ships bash 3.2 at /bin/bash, which lacks
the 'mapfile' builtin. The find/grep output is collected via a 'while
IFS= read -r' loop so the script runs on macOS as well as Linux.

Bare-clone alternates path: bare clones store the alternates file at
objects/info/alternates, not .git/objects/info/alternates. The previous
path was a no-op; the working disconnection comes from --dissociate in
the clone command. Path corrected so the line is defensive belt-and-
braces rather than dead code.

update-gha-workflows.ts: throws with a clear error if npm query returns
a workspace not present in the root "workspaces" array (previously
would silently misorder via indexOf returning -1). The comment in
resolveWorkspaces is corrected to reflect that the function returns
Map insertion order, with the caller re-sorting into declared order.
The npm query exec call now requests a 64 MiB maxBuffer; Node's
default 1 MiB is fine for the current repo size but would silently
truncate as the monorepo grows.

Path anchoring: BUILD_TMP, BUILD_CACHE, and any user-supplied
--cache-dir value are resolved to absolute paths up front. A
CWD-relative default would have put add-repo.tmp outside the repo
(and outside the .gitignore pattern) if the script were run from a
subdirectory of the monorepo.

No repo additions are included; this PR is script-tooling only.
github-actions Bot pushed a commit that referenced this pull request May 14, 2026
…and #505

Replaces the script work of two prior PRs that independently solved
"add an existing scratchfoundation repo to the monorepo":

scripts/add-repo.sh — collapses add-to-monorepo.sh (#434) and add-repo.sh
(#505) into a single canonical entry point. Takes the CLI shape and
auto-CI-regen from #505 and the source-branch auto-detect (develop ->
main -> master), normalization (sort-package-json), and clean-tree
pre-flight from #434.

scripts/update-gha-workflows.ts — replaces the CircleCI-era
build-gha-workflows.ts and workspace-template.yml. Regenerates
.github/path-filters.yml and incrementally updates publish.yml.

Open review comments on #505 are addressed:

- sed -i portability bug: rewrites now use perl -pi -e (works on macOS,
  NixOS, Ubuntu/Debian/Arch, WSL). Perl is added to the explicit prereq
  check next to git-filter-repo, sponge, and jq.
- Arg parser now rejects flag-stealing: --source-branch --org foo errors
  instead of silently using --org as the branch name.
- npm-install failures hard-fail by default. --continue-on-error opts
  into the prior soft-fail behavior; failures are still logged to
  add-repo.errors.log when continuing. Scoped to per-dep package.json
  rewrite failures in the rewire step; the final lockfile install
  always hard-fails by design.
- The submodules-aware branch in move_repository_subdirectory keeps a
  TODO comment marking that it has never been exercised against a real
  submodules-bearing repo and should be run carefully when one lands.

Cross-workspace dep handling: rewires both bare-name and already-prefixed
@scratch/<name> deps in the new package to exact-pinned current monorepo
versions. Matches the existing convention in scratch-gui/scratch-vm/
scratch-render. npm's workspace: protocol would be the ideal here but
npm does not actually support it (npm/cli#8845 - EUNSUPPORTEDPROTOCOL
at install time).

Workspaces array ordering: the new package is inserted just after the
latest existing workspace it depends on, so 'npm run --workspaces build'
builds its deps first. Falls back to position 0 (prepend) when the new
package has no monorepo deps. This is a "last dep wins" heuristic, not
a full topological sort; it relies on the existing workspaces array
already being in valid build order. A real topo sort can be added later
(e.g. by extending update-gha-workflows.ts) if it ever becomes necessary.

bash 3.2 compatibility: macOS ships bash 3.2 at /bin/bash, which lacks
the 'mapfile' builtin. The find/grep output is collected via a 'while
IFS= read -r' loop so the script runs on macOS as well as Linux.

Bare-clone alternates path: bare clones store the alternates file at
objects/info/alternates, not .git/objects/info/alternates. The previous
path was a no-op; the working disconnection comes from --dissociate in
the clone command. Path corrected so the line is defensive belt-and-
braces rather than dead code.

update-gha-workflows.ts: throws with a clear error if npm query returns
a workspace not present in the root "workspaces" array (previously
would silently misorder via indexOf returning -1). The comment in
resolveWorkspaces is corrected to reflect that the function returns
Map insertion order, with the caller re-sorting into declared order.
The npm query exec call now requests a 64 MiB maxBuffer; Node's
default 1 MiB is fine for the current repo size but would silently
truncate as the monorepo grows.

Path anchoring: BUILD_TMP, BUILD_CACHE, and any user-supplied
--cache-dir value are resolved to absolute paths up front. A
CWD-relative default would have put add-repo.tmp outside the repo
(and outside the .gitignore pattern) if the script were run from a
subdirectory of the monorepo.

No repo additions are included; this PR is script-tooling only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants