Switch-to-dev script should add packages instead of linking those

[pomek]

We shouldn't link packages because during yarn install, all these packages will be removed (and installed from NPM).

I had some issues with that during testing the new test environment on CI. See: ckeditor/ckeditor5-paragraph@c118504. After switching to yarn add file:, all issues disappeared.

Following command should be called instead:

yarn add file:../ckeditor5-dev/packages/ckeditor5-dev-docs \
  file:../ckeditor5-dev/packages/ckeditor5-dev-tests \
  file:../ckeditor5-dev/packages/ckeditor5-dev-webpack-plugin \ 
  file:../ckeditor5-dev/packages/ckeditor5-dev-env \
  file:../ckeditor5-dev/packages/ckeditor5-dev-utils \
  file:../ckeditor5-dev/packages/jsdoc-plugins \
  -W

Unfortunately, this solution comes with other bug: all files are copied. It means, if you changed anything in the dev package, you need to install it once again.

[ma2ciek]

Maybe we could use yarn link?

[pomek]

The main goal is to use changes from ckeditor5-dev in the CKEditor 5 project.

Proposals:

(1) Use yarn workspaces

• CKE 5 package.json - add a workspace to ../ckeditor5-dev/packages/*
• (Perhaps we need to remove all node_modules/ directories from CKE 5 Dev).
• yarn install
• Revert changes in package.json in CKE 5.

(2) Replace versioned packages in package.json with link/portal

• Iterate over all CKEditor 5 packages (including CF & Internal) and replace versions for ckeditor5-dev-* packages with a protocol (link or portal).
• (Perhaps we need to remove all node_modules/ directories from CKE 5 Dev).
• Update packages that exist in the ckeditor5-dev repository.
• yarn install
• Revert all changes.

[przemyslaw-zan]

Using yarn workspaces to link the repository located in parent directory of the workspace root is not supported by yarn:

• Can workspace be in parent directory of root? yarnpkg/yarn#6368
• Question: multiworkspace monorepo yarnpkg/yarn#6483

In order to use workspaces we could use mrgit to synchronise the ckeditor5-dev repository inside the external directory. I've tested this approach, and most things work out of the box. There are some problems with automatic tests being launched from the root of cke5:

This can be circumvented by using -f option and skipping tests from ckeditor5-dev. Launching ckeditor5-dev tests from its own nested repository works as expected. This issue could probably be ironed out.

Another problem with this approach is that during the testing phase, we want to use ckeditor5-dev version available on npm, not the one from the master branch. This would require manual removal of that repository during each testing phase, both from the directory, as well as the mrgit.json file, before proceeding with the usual testing phase preparations.

Second option, Replace versioned packages in package.json with link/portal would require from each user one time setup, where they would define paths to the repositories which they want to link. Then, after calling switch-to-dev script, all of the packages from the listed repositories would be used as dependencies wherever possible. Then dependencies are updated, and the packages are linked.

One disadvantage to this approach is the fact that calling yarn install again would break the links. This can be circumvented by simply calling switch-to-dev script instead of yarn install, as it installs the dependencies as well.

[pomek]

It turned out we should fix #12887 before resolving the issue.

[pomek]

Recent highlights/updates regarding the issue:

• As the first option ((1) Use yarn workspaces) couldn't be used, we tried to implement (2).
• While implementing PoC, we found a blocker: The serveTranslations() function throws when using the linked package #12887.
• After resolving it (with some troubles), the same error (The 'compilation' argument must be an instance of Compilation) comes directly from webpack sources.
• We decided to implement the first proposed solution, but instead of leaving the root directory of CKEditor 5, we aim to clone ckeditor5-dev directly in the external/ directory.
• Manual checks seem to work.
• Now, we need to improve the workflow.
  • Support for checking out tags in mrgit (Support for tags cksource/mrgit#148) – it is needed to check out a repository to a tag when the testing phase starts.
  • Rewrite the development environment guide to introduce mrgit again.
  • Remove the existing mentions and usages of the switch-to-dev script.

[pomek]

To sum up what we decided:

• We decided to clone CKEditor 5-related repositories (ckeditor5-dev, linters, etc.) into the external/ directory.
• To switch between tagged and master commits easily, we decided to improve mrgit – https://github.com/cksource/mrgit/releases/tag/v2.0.0.
• The switch-to-dev-dev.sh script is no longer needed.
  • You can use mrgit instead.

[pomek]

Covered by #12970.