The development server doesn't reflect node_modules updates done after running the server at least once

[MisRob]

Observed behavior

When Kolibri is run with devserver or devserver-hot, changes made to a package in kolibri/node_modules done after running the server at least once don't propagate to the app. This seems to be the main cause of problems with linking the KDS package locally to Kolibri described in #10809 (however can be reproduced separately outside of linking context as demonstrated below)

Expected behavior

devserver/devserver-hot propagates node_modules updates to the app

User-facing consequences

None

Steps to reproduce

1. Set up a fresh Kolibri repository and development environment by following Getting started (may be unnecessary for reproducing, however I wanted to eliminate as many influences as possible)
2. Run the development server with yarn run devserver/yarn run devserver-hot
3. Stop the development server
4. Make some changes to a package in Kolibri's node_modules that should reflect in UI in an obvious way (e.g. delete a significant part of a KDS component in node_modules/kolibri-design-system
5. Run yarn run devserver/yarn run devserver-hot
6. Open Kolibri in a private browser window
7. See UI not being updated as expected

Comments

• When node_modules package update is done before running the development server for the first time, that is when skipping steps to reproduce 2 and 3, updates propagate as expected (I think it still wouldn't work for further updates done later on, however I didn't check it)
• When Kolibri run with yarn run build and yarn run python-devserver, updates propagate as expected

Context

• Kolibri 0.16.0a14.dev0+git.124.g726134e8
• Google Chrome Version 113.0.5672.92 (Official Build) (64-bit)
• Ubuntu 20.04.6 LTS

[rtibbles]

Seems like both this and #10809 are manifestations of this webpack issue: webpack/webpack#11612

The comment at the end supplies a resolution - although one thing that applying this resolution will do is slow down the development server as currently webpack is using updates to package.json files and yarn.lock as the source of truth for busting the cache (as an optimization as node_modules can be very large). I am happy to make this change (with an optional configuration flag for local packages), but would want others to test out the performance impact it has on their machines before merging.

[MisRob]

although one thing that applying this resolution will do is slow down the development server

My guess would be that the majority of the team and most of the time doesn't need to link KDS. Would it be possible to have the symlinks settings values set via environment variables, for example? So that every developer could change the value to whatever is needed depending on their current tasks.

[MisRob]

Or rather, just having one environment variable, DEV_WITH_KDS or something like that. If truthy, then webpack config would use the settings that allows linking to work as expected

[MisRob]

@rtibbles Also posted what looks to be a possible alternative to using yarn link to Slack. Even if we didn't go that way, I think that just having a separate command yarn run devserver-with-kds that would use a different webpack config (still based on the current one, just with a few overrides) may be even better than an environment variable I suggested above.