Fix watch mode to listen to changes below the "longest common directory prefix" of relevant files, rather than only files below process.cwd(), while keeping event filtering intact

[milesrichardson]

Description

This fixes a regression in --watch mode which was introduced in #9009. The previous behavior before #9009 was to watch all relevant files with Chokidar, but when #9009 switched to @parcel/watcher, the behavior changed to listen for all events below process.cwd() and then filter them for relevant files. This is a good approach, but it's a regression for some configurations that validly refer to files outside of the current working directory (e.g. documents: "../some-other/*.graphql"). Configurations with such paths continue to build correctly, but the bug was that any files referenced by those paths would not be included in watch mode.

The fix is to listen to events on all file changes below the "longest common directory prefix" of all relevant files, instead of only listening to events on all file changes below process.cwd(). An alternate fix would be to add something like config.watchDirectoryRoot, but that would still be a regression (or at least, it would be a breaking change) from pre-#9009, since there should be no reason previously valid configurations should need to add an extraneous config value to work as they were before.

Note this does not alter the logic related to filtering file change events, once they are received, so e.g. config.watchPattern will continue to work as expected. This only changes the root directory given to parcelWatcher.subscribe(). Filtering continues to happen after receiving events in the ParcelWatcher.SubscribeCallback function.

Related #9266

Maybe related #9233

Type of change

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Test from this PR

I've tested this in my own repository. I also tested it in this repository by adding a dev-test-outer-dir with a *.graphql file in it, adding it as a document to one of the stanzas in dev-test/codegen.ts, and adding a yarn watch:examples script which is the watch mode equivalent of yarn generate:examples. I didn't see any way to properly test the watching logic, so this was the best I could do without a much larger PR. At least this way we can manually check for regressions in the future by seeing if yarn watch:examples or yarn generate:examples incorrectly output a diff from HEAD. Please let me know if you have any better ideas.

Here's how you can test it, after pulling this branch:

yarn
yarn build
yarn generate:examples

Running git status should show no changes. Then, you can test watch mode:

yarn watch:examples

You should see output which includes:

Watching for changes in /path/to/repo/on/your/machine/graphql-code-generator...

This is because the repo root is the "longest common directory prefix" of ./dev-test and ./dev-test-outer-dir.

While this is running, edit (save) the file dev-test-outer-dir/githunt/current-user.query.graphql, which is "outside" of ./dev-test/codegen.ts. This should trigger a rebuild:

touch dev-test-outer-dir/githunt/current-user.query.graphql

Running git status should show no changes (unless of course you actually made a meaningful change to the .graphql file, rather than just touching/saving it).

Click here for a more involved test where you actually `cd` into `dev-test` and update `codegen.ts` to use relative paths

The problem with the test described above

The problem with the approach above is that process.cwd() is always going to be the repository root, so you can't really tell if it's working correctly or just falling back to process.cwd(), since the listener root will be the repo root in all cases. (Note: The original motivating issue for this bug was in a yarn workspaces repo, where running a command with yarn workspace changes the current working directory to the workspace, i.e. the sub-package containing our graphql configs.)

Try a more detailed test

I have a commit on another branch that updates all ./dev-test paths referenced inside of ./dev-test to instead reference ./ (and also contains a change to call prettier from ../node_modules/.bin/prettier). To test the changes here, you can do the following:

First, checkout the branch with these changes from my fork and peek at the diff:

git remote add fork-9267 https://github.com/milesforks/graphql-code-generator.git
git fetch fork-9267
git checkout fork-9267/from-within-dev-test-dir
git show HEAD

Then, cd into dev-test and run the generator while the current working directory is dev-test instead of the repo root:

cd dev-test
node ../packages/graphql-codegen-cli/dist/cjs/bin.js --require dotenv/config --config ./codegen.ts dotenv_config_path=.env --watch

You should see watch mode listening for changes in the repo root:

ℹ Watching for changes in /path/to/repo/on/your/machine/graphql-code-generator...

(Give it a second for the prettier changes to apply before running git status again.)

This is because of the referenced '../dev-test-outer-dir/githunt/**/*.graphql' document. You can remove that:

# Go back to repo root
cd ../

# Apply this patch to remove the line referencing ../dev-test-outer-dir (you can copy paste everything below into your terminal)
git apply << 'EOF'
diff --git a/dev-test/codegen.ts b/dev-test/codegen.ts
index bbd161262..2c12a5351 100644
--- a/dev-test/codegen.ts
+++ b/dev-test/codegen.ts
@@ -55,7 +55,7 @@ const config: CodegenConfig = {
     },
     './githunt/graphql-declared-modules.d.ts': {
       schema: './githunt/schema.json',
-      documents: ['./githunt/**/*.graphql', '../dev-test-outer-dir/githunt/**/*.graphql'],
+      documents: ['./githunt/**/*.graphql'],
       plugins: ['typescript-graphql-files-modules'],
     },
     './githunt/typed-document-nodes.ts': {
EOF

And then run the watch mode again:

cd dev-test
node ../packages/graphql-codegen-cli/dist/cjs/bin.js --require dotenv/config --config ./codegen.ts dotenv_config_path=.env --watch

This time, you should see it watching for changes in dev-test, because there are no documents referenced outside of that directory (since you just removed the only one of them with that git apply command):

  ℹ Watching for changes in /path/to/repo/on/your/machine/graphql-code-generator/dev-test...

This testing shows the PR working as expected.

(end of more detailed test)

Test Environment:

• OS: MacOS
• @graphql-codegen/cli: This PR
• NodeJS: v18.10.0

Checklist:

• I have followed the CONTRIBUTING doc and the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation [none required?]
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works [need to be run manually with yarn watch:examples]
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules [not applicable]

Further comments

PR Structure

This PR is separated into two commits. The first commit (4e171d2 at time of writing) contains the actual fix, and all the code is in ./packages/graphql-codegen-cli/src/utils/watcher.ts. The second commit (0a64f6b at time of writing) is a bit messier, and it contains the addition of the yarn watch:examples script and dev-test-outer-dir directory.

Safety Mechanism

If the path about to be returned by findHighestCommonDirectory is not accessible (according to fs.access, which is the recommended best practice for checking accessibility), then it will return process.cwd() instead, effectively falling back to current behavior.

Note that falling back to current behavior would still be a regression from pre-#9009. Perhaps it would be worthwhile to also add a config.watchRootPath option to explicitly override this, and to prefer that when it's set, while otherwise using findHighestCommonDirectory (with the process.cwd() fallback if it's about to return an inaccessible path). But I wanted to keep this PR small, and I think this is the safest code that fixes the regression introduced in #9009.

longestCommonPrefix logic

For the longestCommonPrefix function, obviously this is a common leetcode problem, so I adapted the solution from this medium post, which seemed to work fine in my testing.

findHighestCommonDirectory and micromatch.scan logic

The logic in findHighestCommonDirectory maps each file in files (which can be both relative and absolute paths or micromatch patterns) to micromatch.scan(file).base. This is only partially documented, but in my testing, it seems to be what we want: the .base value is the longest directory before any wildcard segment. I included some examples in a comment above the code that uses it.

[changeset-bot]

🦋 Changeset detected

Latest commit: 0a64f6b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@graphql-codegen/cli	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[milesrichardson]

Force pushed some minor typo fixes. This PR is ready for review.

All relevant workflows are passing when run on my fork.

[saihaj]

Thank you for the fix and explanation! I agree we should't add a new config option. Tested this locally and work as expected.

[milesrichardson]

amazing! thanks for the merge 👍