NIFI-12999: Revert "Bump webpack-dev-middleware and karma-webpack (#8547)"

[sardell]

Link to Jira: https://issues.apache.org/jira/browse/NIFI-12999

This PR reverts commit 69f1a3b (PR #8547). This commit on main is currently causing local builds of NiFi to fail due to a dependency in nifi-registry's package-lock.json requiring a version of Node.js higher than the one we use to build nifi-registry. cc @scottyaslan

Summary

NIFI-00000

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 21

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for flagging this issue @sardell.

As the previous PR has been merged for NIFI-12935, this PR should have a new Jira issue for tracking.

To the substance of the issue, however, can you clarify how this is a problem for a local build as opposed to GitHub Actions build? I see that package-lock.json references Node 18, so it is unclear how this works in automated builds. It also works on recent local builds, so is the issue limited to running Registry unit tests? If it is possible to rescope the changes, as opposed to just reverting that would be ideal, but I understand it may be more complicated.

[sardell]

@exceptionfactory Thanks for taking a look. I created https://issues.apache.org/jira/browse/NIFI-12999 to track the issue.

To the substance of the issue, however, can you clarify how this is a problem for a local build as opposed to GitHub Actions build?

That is a great question, and I'm not 100% sure. Maybe there is some global caching going on with those deps in GitHub Actions or some configuration that causes them to fail silently in that environment, but I'm not sure. I would have to investigate further.

It also works on recent local builds, so is the issue limited to running Registry unit tests? If it is possible to rescope the changes, as opposed to just reverting that would be ideal, but I understand it may be more complicated.

The issue happens to me whenever I try to build the project with maven (used ./mvnw clean install initially and mvn clean install at the root level to see that error. I also cannot cd into nifi-registry's ui and manually install npm deps either. If others aren't running into this issue on main, I'll gladly close this and chalk it up to a local issue I need to investigate. I found this issue late last night and wanted to have a PR ready to discuss in the morning due to the potential impact a failing build on main would have.

EDIT: In regard to rescoping the changes, we could do that but registry has many outdated deps, specifically Angular 11, that would require much bigger changes than a simple update with npm.

[sardell]

@exceptionfactory I've confirmed with @koccs that he's able to reproduce the issue building the latest from main locally.

[scottyaslan]

Node 18, so it is unclear how this works in automated builds. It also works on recent local builds, so is the issue limited to running Registry unit tests? If it is possible to rescope the changes, as opposed to just reverting that would be ideal, but I understand it may be more complicated.

@sardell Is it possible to update the version of node that this requires? Possibly in the pom.xml? Or maybe even update the registry's package.json such that we don't fail because of it? I know we don't want to support npm versions less than the minimum but I think newer versions of npm are fine right?

I am also a bit confused. I tried to run ./mvnw clean install from /nifi/nifi-registry and that command seems not to be supported. When I run mvn clean install -DskipTests everything builds fine locally for me.

➜ main git:(NIFI-12956) ✗ pwd
/Users/jaslan/Development/nifi/nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main
➜ main git:(NIFI-12956) ✗ node -v
v16.13.2
➜ main git:(NIFI-12956) ✗ npm -v
8.1.2
➜ main git:(NIFI-12956) ✗ npm ci
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE package: 'karma-webpack@5.0.1',
npm WARN EBADENGINE required: { node: '>= 18' },
npm WARN EBADENGINE current: { node: 'v16.13.2', npm: '8.1.2' }
npm WARN EBADENGINE }
npm WARN deprecated @npmcli/move-file@1.1.2: This functionality has been moved to @npmcli/fs
npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility
npm WARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated chokidar@2.1.8: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated html-webpack-plugin@3.2.0: 3.x is no longer supported
npm WARN deprecated @angular/http@4.4.7: Package no longer supported. Use @angular/common instead, see https://angular.io/guide/deprecations#angularhttp
npm WARN deprecated @angular/flex-layout@11.0.0-beta.33: This package has been deprecated. Please see https://blog.angular.io/modern-css-in-angular-layouts-4a259dca9127
npm WARN deprecated core-js@2.6.12: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.

nifi-registry@0.0.5-SNAPSHOT preinstall
npm --ignore-scripts install npm-force-resolutions && npm-force-resolutions

npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE package: 'karma-webpack@5.0.1',
npm WARN EBADENGINE required: { node: '>= 18' },
npm WARN EBADENGINE current: { node: 'v16.13.2', npm: '8.1.2' }
npm WARN EBADENGINE }

up to date, audited 1427 packages in 1s

179 packages are looking for funding
run npm fund for details

13 vulnerabilities (1 moderate, 6 high, 6 critical)

To address issues that do not require attention, run:
npm audit fix

To address all issues (including breaking changes), run:
npm audit fix --force

Run npm audit for details.

added 1426 packages, and audited 1427 packages in 17s

179 packages are looking for funding
run npm fund for details

13 vulnerabilities (1 moderate, 6 high, 6 critical)

To address issues that do not require attention, run:
npm audit fix

To address all issues (including breaking changes), run:
npm audit fix --force

Run npm audit for details.
➜ main git:(NIFI-12956) ✗ npm run test

nifi-registry@0.0.5-SNAPSHOT test
karma start karma.conf.js --single-run

webpack was not included as a framework in karma configuration, setting this automatically...
karma-webpack does not currently support customized filenames via
webpack output.filename, if this is something you need consider opening an issue.
defaulting [name].[contenthash].js to [name].js.
Webpack bundling...
Browserslist: caniuse-lite is outdated. Please run:
npx update-browserslist-db@latest
Why you should do it regularly: https://github.com/browserslist/update-db#readme
(node:17323) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see webpack/loader-utils#56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.
(Use node --trace-deprecation ... to show where the warning was created)

04 04 2024 13:31:13.449:INFO [karma-server]: Karma v6.4.1 server started at http://localhost:9876/
04 04 2024 13:31:13.450:INFO [launcher]: Launching browsers Chrome with concurrency unlimited
04 04 2024 13:31:13.452:INFO [launcher]: Starting browser Chrome
04 04 2024 13:31:19.888:INFO [Chrome 123.0.0.0 (Mac OS 10.15.7)]: Connected on socket tzL8nUgjmubwyQ2GAAAB with id 48750448
04 04 2024 13:31:23.668:WARN [web-server]: 404: /assets/images/registry-logo-web-app.svg

NfRegistryCreateBucket Component isolated unit tests
✓ should create a new bucket and keep the dialog open
✓ should create a new bucket and close the dialog
✓ should close the dialog

NfRegistryManageUser Component
✓ should FAIL to get user by id and redirect to workflow perspective
✓ should toggle to remove the manage policies privileges for the current user
✓ should toggle to create the manage policies privileges for the current user
✓ should toggle to create the manage proxy privileges for the current user
✓ should toggle to create the manage tenants privileges for the current user
✓ should toggle to remove the manage bucket privileges for the current user
✓ should toggle to update the manage policies privileges for the current user
✓ should remove user from group
✓ should FAIL to get user by id and redirect to admin users perspective
✓ should sort groups by column
✓ should destroy the component
✓ should have a defined component
✓ should open a modal dialog UX enabling the addition of the current user to a group(s)
✓ should toggle to remove the manage tenants privileges for the current user
✓ should fail to update user name (409)
✓ should toggle to update the manage tenants privileges for the current user
✓ should toggle to remove the manage proxy privileges for the current user
✓ should toggle to update the manage bucket privileges for the current user
✓ should toggle to update the manage proxy privileges for the current user
✓ should toggle to create the manage bucket privileges for the current user
✓ should update user name
✓ should redirect to users perspective

NfRegistry Component
✓ should create component

NfRegistryManageGroup Component
✓ should remove user from group
✓ should toggle to update the manage proxy privileges for the current group
✓ should update group name
✓ should toggle to remove the manage policies privileges for the current group
✓ should toggle to remove the manage proxy privileges for the current group
✓ should toggle to create the manage tenants privileges for the current group
✓ should toggle to update the manage tenants privileges for the current group
✓ should toggle to remove the manage tenants privileges for the current group
✓ should redirect to users perspective
✓ should toggle to create the manage policies privileges for the current group
✓ should toggle to update the manage policies privileges for the current group
✓ should FAIL to get user by id and redirect to admin users perspective
✓ should FAIL to get user by id and redirect to workflow perspective
✓ should destroy the component
✓ should toggle to create the manage proxy privileges for the current group
✓ should have a defined component
✓ should fail to update group name (409)
✓ should toggle to remove the manage bucket privileges for the current group
✓ should open a modal dialog UX enabling the addition of the current user to a group(s)
✓ should toggle to create the manage bucket privileges for the current group
✓ should sort users by column
✓ should toggle to update the manage bucket privileges for the current group

NfRegistryExplorer Component
✓ should have a defined component
✓ should destroy the component
ERROR LOG: 'DEPRECATION: An asynchronous function called its 'done' callback more than once. This is a bug in the spec, beforeAll, beforeEach, afterAll, or afterEach function in question. This will be treated as an error in a future version. Seehttps://jasmine.github.io/tutorials/upgrading_to_Jasmine_4.0#deprecations-due-to-calling-done-multiple-times for more information.
(in spec: NfRegistryUsersAdministration Component should have a defined component)
Error
Note: This message will be shown only once. Set the verboseDeprecations config property to true to see every occurrence.'
ERROR: 'DEPRECATION: An asynchronous function called its 'done' callback more than once. This is a bug in the spec, beforeAll, beforeEach, afterAll, or afterEach function in question. This will be treated as an error in a future version. Seehttps://jasmine.github.io/tutorials/upgrading_to_Jasmine_4.0#deprecations-due-to-calling-done-multiple-times for more information.
(in spec: NfRegistryUsersAdministration Component should have a defined component)
Error
Note: This message will be shown only once. Set the verboseDeprecations config property to true to see every occurrence.'
ERROR LOG: 'DEPRECATION: An asynchronous function called its 'done' callback more than once. This is a bug in the spec, beforeAll, beforeEach, afterAll, or afterEach function in question. This will be treated as an error in a future version. Seehttps://jasmine.github.io/tutorials/upgrading_to_Jasmine_4.0#deprecations-due-to-calling-done-multiple-times for more information.
(in spec: NfRegistryWorkflowAdministration Component should have a defined component)
Error
Note: This message will be shown only once. Set the verboseDeprecations config property to true to see every occurrence.'
ERROR: 'DEPRECATION: An asynchronous function called its 'done' callback more than once. This is a bug in the spec, beforeAll, beforeEach, afterAll, or afterEach function in question. This will be treated as an error in a future version. Seehttps://jasmine.github.io/tutorials/upgrading_to_Jasmine_4.0#deprecations-due-to-calling-done-multiple-times for more information.
(in spec: NfRegistryWorkflowAdministration Component should have a defined component)
Error
Note: This message will be shown only once. Set the verboseDeprecations config property to true to see every occurrence.'

NfRegistryUsersAdministration Component
✓ should open a dialog to create a new group
✓ should destroy the component
✓ should have a defined component
✓ should open a dialog to create a new user

NfRegistryGridListViewer Component
✓ should destroy the component
✓ should have a defined component

NfRegistryCreateNewGroup Component isolated unit tests
✓ should make a call to the api to create a new group and close the dialog.
✓ should make a call to the api to create a new group and keep the dialog open.
✓ should cancel the creation of a new group

NfRegistryAdministration Component
✓ should destroy the component
✓ should have a defined component

NfRegistryAddUser Component isolated unit tests
✓ should make a call to the api to create a new user and close the dialog
✓ should make a call to the api to create a new user and keep the dialog open
✓ should cancel the creation of a new user

NfRegistryAddUsersToGroup Component isolated unit tests
✓ should cancel the creation of a new user
✓ should determine if all users are selected
✓ should make a call to the api to add selected users to the group
✓ should select all groups.
✓ should toggle all groups checked properties to false.
✓ should sort groups by column
✓ should determine all user groups are not selected
✓ should deselect all groups.
✓ should toggle all groups checked properties to true.

NfRegistryAddUserToGroups Component isolated unit tests
✓ should cancel the addition of the user to any group
✓ should make a call to the api to add user to selected groups
✓ should select all groups.
✓ should determine if all groups are not selected
✓ should sort groups by column
✓ should toggle all groups checked properties to false.
✓ should determine if all groups are selected
✓ should toggle all groups checked properties to true.
✓ should deselect all groups.

NfRegistryWorkflowAdministration Component
✓ should destroy the component
✓ should open a dialog to create a new bucket
✓ should have a defined component

NfRegistry Auth Guard Service NfRegistryResourcesAuthGuard isolated unit tests
✓ should navigate to test url (registry security not configured)
✓ should navigate to login (error loading current user)
✗ should navigate to test url (registry security configured and we know who you are)
Uncaught TypeError: self.nfRegistryApi.getRegistryConfig is not a function thrown

Error: Timeout - Async function did not complete within 1000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)

Chrome 123.0.0.0 (Mac OS 10.15.7) NfRegistry Auth Guard Service NfRegistryResourcesAuthGuard isolated unit tests should navigate to test url (registry security configured and we know who you are) FAILED
Uncaught TypeError: self.nfRegistryApi.getRegistryConfig is not a function thrown
Error: Timeout - Async function did not complete within 1000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)
✓ should navigate to login

NfRegistry Auth Guard Service NfRegistryLoginAuthGuard isolated unit tests
✓ should navigate to test url
✓ should navigate to base nifi-registry url

NfRegistry Service w/ Angular testing utils
✓ should execute a delete action on a bucket.
✓ should filter droplets by name.
✓ should execute a delete action on a user.
✓ should execute a manage action on a bucket.
✓ should execute a delete action on a group.
✓ should filter buckets by name.
✓ should execute a manage action on a user.
✓ should execute a manage action on a group.
✓ should execute the delete droplet action.
Chrome 123.0.0.0 (Mac OS 10.15.7) ERROR
An error was thrown in afterAll
TypeError: Cannot read properties of undefined (reading 'pipe')
Chrome 123.0.0.0 (Mac OS 10.15.7) ERROR
An error was thrown in afterAll
TypeError: Cannot read properties of undefined (reading 'pipe')
✓ should delete all selected buckets.
✓ should retrieve the snapshot metadata for the given droplet.
✓ should filter users and groups by name.
✓ should filter writable buckets.
✓ should open the Import Versioned Flow dialog.
✓ should open the Import New Flow dialog.
✓ should delete all selected users and groups.
✓ should open the Export Version dialog.

Chrome 123.0.0.0 (Mac OS 10.15.7): Executed 108 of 245 (1 FAILED) ERROR (11.915 secs / 11.71 secs)

=============================== Coverage summary ===============================
Statements : Unknown% ( 0/0 )
Branches : Unknown% ( 0/0 )
Functions : Unknown% ( 0/0 )
Lines : Unknown% ( 0/0 )

➜ main git:(NIFI-12956) ✗

[scottyaslan]

@sardell Running mvn clean install -DskipTests -PjsUnitTests from /nifi/nifi-registry also builds successfully for me.

Can you please provide more details on your environment and how you are running this to see this failure?

[sardell]

I'm simply building the project from the root with ./mvnw clean install -T2C (also tried without the parallel build). When I try to install npm dependencies by running npm run ci at nifi/nifi-registry/nifi-registry-core/nifi-registry-web-ui/src/main like the build does, I receive a dependency mismatch error.

nifi-registry-web-ui/src/main on  NIFI-11520-with-backend [$] via  v16.19.0
❯ npm run ci

> nifi-registry@0.0.5-SNAPSHOT ci
> npm ci --ignore-scripts

npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: @angular/http@4.4.7
npm ERR! Found: @angular/core@11.2.14
npm ERR! node_modules/@angular/core
npm ERR!   @angular/core@"11.2.14" from the root project
npm ERR!   peer @angular/core@"11.2.14" from @angular/animations@11.2.14
npm ERR!   node_modules/@angular/animations
npm ERR!     @angular/animations@"11.2.14" from the root project
npm ERR!     peer @angular/animations@"^11.0.0 || ^12.0.0-0" from @angular/material@11.2.13
npm ERR!     node_modules/@angular/material
npm ERR!       @angular/material@"11.2.13" from the root project
npm ERR!       2 more (@covalent/core, @nifi-fds/core)
npm ERR!     2 more (@angular/platform-browser, @nifi-fds/core)
npm ERR!   10 more (@angular/cdk, @angular/common, @angular/flex-layout, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @angular/core@"4.4.7" from @angular/http@4.4.7
npm ERR! node_modules/@angular/http
npm ERR!   peer @angular/http@"^2.0.0||^4.0.0" from angular2-jwt@0.2.3
npm ERR!   node_modules/angular2-jwt
npm ERR!     angular2-jwt@"0.2.3" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: @angular/core@4.4.7
npm ERR! node_modules/@angular/core
npm ERR!   peer @angular/core@"4.4.7" from @angular/http@4.4.7
npm ERR!   node_modules/@angular/http
npm ERR!     peer @angular/http@"^2.0.0||^4.0.0" from angular2-jwt@0.2.3
npm ERR!     node_modules/angular2-jwt
npm ERR!       angular2-jwt@"0.2.3" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /Users/sardell/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/sardell/.npm/_logs/2024-04-05T12_58_50_575Z-debug-0.log

[exceptionfactory]

Thanks for providing additional details @sardell. As an additional point of reference, I am able to build nifi-registry-web-ui, but NPM reports warnings, not errors, for the version mismatch:

[INFO] --- frontend:1.14.2:npm (npm-install) @ nifi-registry-web-ui ---
[INFO] Running 'npm run ci' in /Users/exceptionfactory/projects/nifi/nifi-registry/nifi-registry-core/nifi-registry-web-ui/target/frontend-working-directory
[INFO] 
[INFO] > nifi-registry@0.0.5-SNAPSHOT ci
[INFO] > npm ci --ignore-scripts
[INFO] 
[INFO] npm WARN EBADENGINE Unsupported engine {
[INFO] npm WARN EBADENGINE   package: 'karma-webpack@5.0.1',
[INFO] npm WARN EBADENGINE   required: { node: '>= 18' },
[INFO] npm WARN EBADENGINE   current: { node: 'v16.13.2', npm: '8.1.2' }
[INFO] npm WARN EBADENGINE }

[sardell]

@exceptionfactory Thanks for sharing. I investigated a little more and I think I found the issue. There's a configuration property for npm called engine-strict. When set to true, version mismatches will cause a failure. I looked in my .npmrc file and saw mine was set to true. Rebuilding now after removing that property. If successful, I'll close this PR and issue.