Production Build with Vaadin 24 is very slow on a large project

[simasch]

Description of the bug

The production build with Vaadin 24 takes around 40 minutes compared to 5 minutes with Vaadin 23.

The project has around 400 Routes.

Expected behavior

It shouldn't take longer than with previous versions.

Minimal reproducible example

Unfortunately, I cannot share the project as it's not opensource.open source

Versions

Vaadin 24.1.3

[mcollovati]

First findings:

• During production build, class scanning is performed twice in Vaadin 24
• Byte code scanning performs the same both on V23 and V24, but V24 has an additional recursive step to collect reachable classes for every entry point. This seems to be slow, especially if the entry point can reach hundreds of classes.

[mcollovati]

A first improvement consists in pre-compiling the regular expression used to detect visitable classes.
A simple test on a dummy project with 400 routes shows a significant build time reduction

8:17:37.387 [INFO] --- vaadin-maven-plugin:24.1.3:build-frontend (default) @ spring-skeleton ---
08:17:37.723 [INFO] Reflections took 292 ms to scan 127 urls, producing 4866 keys and 25879 values
08:17:37.743 [INFO] Scanning classes to find frontend configurations and dependencies...
08:17:46.729 [INFO] Visited 1377 classes. Took 8985 ms.

08:20:59.760 [INFO] --- flow-maven-plugin:24.2-SNAPSHOT:build-frontend (default) @ spring-skeleton ---
08:20:59.963 [INFO] Reflections took 196 ms to scan 127 urls, producing 4866 keys and 25880 values
08:20:59.969 [INFO] Scanning classes to find frontend configurations and dependencies...
08:21:02.292 [INFO] Visited 1370 classes. Took 2322 ms.

[mcollovati]

Looking at log files for a very slow build, it seems that also AbstractUpdateImports.process() may take a long time during particular circumstances

[gpsfl]

Is it possible to also manually exclude classes / package names? In our project it scans about 11000 classes most of which have nothing to do with vaadin. It takes 80s

[simasch]

That's a very good question @gpsfl

In my project it is
[INFO] Visited 16572 classes. Took 94647 ms

[mshabarov]

Vaadin 24 and 23 use a slightly different algorithm to validate the license during production build.
Vaadin 23 checks node_modules for installed components, whereas Vaadin 24 uses class scanning, because node_modules may not exist due to using pre-compiled bundle (new feature in V24).
This leads to class scanning is performed twice in Vaadin 24.
The possible solution might be either to cache the class scanning results and use it during license validation, or bring back old approach for license validation by scanning node_modules if Flow sees that a new build is triggered (i.e. node modules are installed anyway).

[knoobie]

It's probably more related to your Mac vs Jenkins 😅

[simasch]

@knoobie Indeed I'm still waiting for the build to finish on Jenkins. So far 25 minutes elapsed :-(

[simasch]

Unfortunately, it was indeed my Mac. On the Jenkins server still the same

[mcollovati]

@simasch we just released Flow 24.1.13 which has an improvement for production build.

If you mind getting it a try, you can follow Mikhail's instruction from this comment #17234 (comment)

[simasch]

@mcollovati Unfortunately no change. The one with over 1 hour is with Flow 24.1.13

[mcollovati]

That is somehow surprising, compared with local results 🤔
Do you mind providing the logs of the build with debug info (even via DM if you prefer)?

[simasch]

https://www.dropbox.com/scl/fi/wf1io4gpaz0zwi3iz6lu9/build.log.zip?rlkey=fdniyas7uga8ci634khfswh5j&dl=0

[mcollovati]

@simasch just a confirmation: given the times in the logs, I suppose the build is from your local machine, not Jenkins. Is it so?

[simasch]

Yes, that's right.

[mcollovati]

But it looks like it is not running the latest Flow plugin

[INFO] --- vaadin:24.1.10:build-frontend (default) @ tosca-develop ---
[DEBUG] Loading mojo com.vaadin:vaadin-maven-plugin:24.1.10:build-frontend from plugin realm ClassRealm[plugin>com.vaadin:vaadin-maven-plugin:24.1.10, parent: jdk.internal.loader.ClassLoaders$AppClassLoader@251a69d7]
[DEBUG] Configuring mojo execution 'com.vaadin:vaadin-maven-plugin:24.1.10:build-frontend:default' with basic configurator -->

[mcollovati]

Or better, it is running both the plugins

[INFO] --- flow:24.1.13:build-frontend (default) @ tosca-develop ---
...
[DEBUG] Start updating imports file and chunk files.
[DEBUG] Found 437 modules, 437 scripts and 437 css.
[DEBUG] Start collecting scanned JS modules and scripts.
[DEBUG] JS modules and scripts collected in 681 ms.
[DEBUG] Start sorting imports to lazy and eager.
[DEBUG] Imports sorting took 62 ms.
[DEBUG] Start generating lazy loaded chunks.
[DEBUG] Lazy chunks generation took 3707 ms.
...
[DEBUG] Imports and chunks update took 4463 ms.
....
[INFO] --- vaadin:24.1.10:build-frontend (default) @ tosca-develop ---
...
[DEBUG] Start updating imports file and chunk files.
[DEBUG] Found 437 modules, 437 scripts and 437 css.
[DEBUG] Start collecting scanned JS modules and scripts.
[DEBUG] JS modules and scripts collected in 730 ms.
[DEBUG] Start sorting imports to lazy and eager.
[DEBUG] Imports sorting took 41 ms.
[DEBUG] Start generating lazy loaded chunks.
[DEBUG] Lazy chunks generation took 701675 ms.

[mcollovati]

@simasch it looks like that flow:24.1.13:build-frontend performs much better for lazy chunk generation.

[simasch]

After configuring the plugin correctly the build time is down to 6 minutes! Great job!

[mcollovati]

Great to hear! Thank you so much for helping to test the fix and providing the logs 🙇

[mshabarov]

If no objections, I'd propose to close this issue. The tested Flow version will be available in the next maintenance release for V24.1.

[simasch]

Thank you very much for the effort to fix this.