Cancel build of outdated federation artifact builds when new bundles are already built

[dr460nf1r3]

For which library do you have a feature request?

native-federation

Information

Ongoing builds should be stopped when new builds are already done. This prevents a) wasting resources, b) chained and unnecessary page reloads.

This is especially critical because of this unresolved issue, which is consequently ignored #559.

Consider the following log output:

 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-QDGD4Y4U.js | bootstrap | 621.94 kB | 

Application bundle generation complete. [5.890 seconds]

Page reload sent to client(s).

 INFO  Building federation artefacts
 INFO  [Federation SSE] Client connected. Active connections: 1
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-HXHSLSBS.js | bootstrap | 621.94 kB | 

Application bundle generation complete. [2.585 seconds]

Page reload sent to client(s).

 INFO  Building federation artefacts
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-QDGD4Y4U.js | bootstrap | 621.94 kB | 

Application bundle generation complete. [1.334 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-HXHSLSBS.js | bootstrap | 621.94 kB | 

Application bundle generation complete. [34.011 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
 INFO  Done!
 INFO  [Federation SSE] Client connected. Active connections: 1
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
 INFO  Done!
 INFO  [Federation SSE] Client connected. Active connections: 1
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
 INFO  [Federation SSE] Client connected. Active connections: 1

Describe any alternatives/workarounds you're currently using

No response

I would be willing to submit a PR to fix this issue

• Yes
• No

[wamasimba]

I have the impression that this issue is related to the previous one , I somewhat improved the situation by setting the rebuildDelay to 120000 (yes two minutes), but this by no means an acceptable solution, of course.

[Aukevanoost]

Hi @dr460nf1r3

I'd like to know a bit more about the context, a small repo that shows the issue would be nice. But these are the things i am particularly interested in.

• What commands are you using when you startup your project?
• Is this the host of a micro frontend?
• Are you using Nx, SSR, SSG or anything else?
• Does this happen if you run a single project or multiple projects concurrently?
• Is your workspace a monorepo or polyrepo?

If you want to help and re-create the issue, but cant share your current repo. here are some starter repo's:

• https://github.com/Aukevanoost/native-federation-examples-ng
• https://github.com/manfredsteyer/mf-nf-poc/tree/main
• https://github.com/manfredsteyer/nf-test (for frameworks)

You can check it out and start adding configuration similar to your setup to see what breaks the setup. This helps us a ton to investigate what went wrong :)

[dr460nf1r3]

I can give the following information right now:

• nx serve shell / nx serve mfe
• This was a micro frontend, and not the host itself
• Nx is used, no SSR, no SSG
• So far witnessed it only when running both shell/mfe, but I also only develop using this setup.
• It is an Nx mono repo with a lot of apps in it

[Aukevanoost]

Hi @dr460nf1r3

Thanks for the info!

I upgraded the artifact caching once more to avoid rebundling of artifacts if the packages didn't change using a checksum, would you mind checking if this fix works for your project setup? PR #981

This is not a complete fix but it might help! if you turn on "verbose" in your angular.json, you can see what time it takes to perform each subtask.

A next step for us might be to explore if builder runs can be cancelled if the next one is requested.

Cheers

[dr460nf1r3]

Thanks for the update @Aukevanoost ! I've tested this briefly (by naively replacing my node_modules version of @softarc/native-federation with the version built from your PR), and it looks like it is a lot quicker than before. I've added verbose to the nx config:

  "serve-original": {
      "executor": "@angular-devkit/build-angular:dev-server",
      "configurations": {
        "production": {
          "buildTarget": "project:esbuild:production"
        },
        "development": {
          "buildTarget": "project:esbuild:development"
        }
      },
      "defaultConfiguration": "development",
      "options": {
        "port": 4207,
        "publicHost": "http://localhost:4207",
        "verbose": true
      },
      "continuous": true
    }

And now receive this in the console on rebuilds:

{
  "added": [],
  "modified": [
    "modifiedfilereference.*"
  ],
  "removed": []
}

I did not get any timers, where do I have to enable verbose to see them?

[Aukevanoost]

Great stuff! I think you have to do it at "esbuild" or "build". It will take the options from that target anyway if I recall correctly.

[dr460nf1r3]

Runs between 5-6 seconds on one change I tested multiple times.

 INFO  Building federation artefacts
 DBG!  5s:712898542.000ms - To bundle all mappings and exposed.
 DBG!  8s:328478792.000ms - To bundle all dependencies
Initial chunk files   | Names         |  Raw size
styles.css            | styles        | 313 bytes | 
main.js               | main          | 288 bytes | 
polyfills.js          | polyfills     | 121 bytes | 

                      | Initial total | 722 bytes

Lazy chunk files      | Names         |  Raw size
bootstrap-IBPKIRS3.js | bootstrap     | 640.31 kB | 

Application bundle generation complete. [4.182 seconds]

Watch mode enabled. Watching for file changes...
NOTE: Raw file sizes do not reflect development server per-request transformations.
  ➜  Local:   http://localhost:4207/
  ➜  press h + enter to show help
 INFO  [Federation SSE] Client connected. Active connections: 1
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-L5KQPWYI.js | bootstrap | 640.29 kB | 

Application bundle generation complete. [2.032 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
 DBG!  5s:963753792.000ms - To bundle all mappings and exposed.
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
 INFO  [Federation SSE] Client connected. Active connections: 1

Before (took the commit where you added timers as reference, interestingly we can also see the problem of this issue):

 INFO  Building federation artefacts
 DBG!  9s:606283875.000ms - To bundle all mappings and exposed.
 DBG!  10s:225404375.000ms - To bundle all dependencies
Initial chunk files   | Names         |  Raw size
styles.css            | styles        | 313 bytes | 
main.js               | main          | 288 bytes | 
polyfills.js          | polyfills     | 121 bytes | 

                      | Initial total | 722 bytes

Lazy chunk files      | Names         |  Raw size
bootstrap-IBPKIRS3.js | bootstrap     | 640.31 kB | 

Application bundle generation complete. [6.742 seconds]

Watch mode enabled. Watching for file changes...
NOTE: Raw file sizes do not reflect development server per-request transformations.
  ➜  Local:   http://localhost:4207/
  ➜  press h + enter to show help
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-L5KQPWYI.js | bootstrap | 640.29 kB | 

Application bundle generation complete. [1.856 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
 DBG!  5s:978155666.000ms - To bundle all mappings and exposed.
 INFO  Done!
 INFO  [Federation SSE] Client connected. Active connections: 1
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-IBPKIRS3.js | bootstrap | 640.31 kB | 

Application bundle generation complete. [0.823 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-QSIZK4W2.js | bootstrap | 640.31 kB | 

Application bundle generation complete. [0.578 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-IBPKIRS3.js | bootstrap | 640.31 kB | 

Application bundle generation complete. [0.654 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
 DBG!  10s:295197542.000ms - To bundle all mappings and exposed.
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 DBG!  11s:127212875.000ms - To bundle all mappings and exposed.
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
 INFO  [Federation SSE] Client connected. Active connections: 1
 DBG!  11s:473022000.000ms - To bundle all mappings and exposed.
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
 INFO  [Federation SSE] Client connected. Active connections: 1
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-L5KQPWYI.js | bootstrap | 640.29 kB | 

Application bundle generation complete. [1.191 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
 DBG!  6s:466301792.000ms - To bundle all mappings and exposed.
 INFO  Done!
 INFO  [Federation SSE] Event 'federation-rebuild-complete' broadcast to 1 clients
 INFO  [Federation SSE] Client disconnected. Active connections: 0
 INFO  [Federation SSE] Client connected. Active connections: 1

The times are almost the same, which is interesting. I'm not sure where this comes from, to be honest.
Some more context to the double builds: this happens when I hit CTRL + S to trigger a rebuild in Webstorm, and Webstorm then decides to reformat files (because it is an on-save action).

[Aukevanoost]

The times are almost the same, which is interesting. I'm not sure where this comes from, to be honest.

Right, for the sequential builds I think you're referring to this right?

 DBG!  5s:963753792.000ms - To bundle all mappings and exposed.

We'll have to take a look if we can fix that, since this is hard to keep track of. If you want to rebuild after files changed, the rebuilding does include the shared libraries of your monorepo (shared mappings).

[Aukevanoost]

If you're using the newer (caching) builder, this is what the format should look like:

$ ng serve mfe1
 DBG!  00:00:308.040ms - To load the federation config.
 INFO  [Federation SSE] Local reloader initialized with endpoint /@angular-architects/native-federation:build-notifications
 INFO  Building federation artefacts
 DBG!  00:00:911.841ms - [build artifacts] - To bundle all mappings and exposed.

[dr460nf1r3]

It seems I was not using the PR version even tough I replaced the files - pnpm link did a better job. Now I have the following problem:

 DBG!  00:05:283.118ms - [build artifacts] - To bundle all mappings and exposed.
 WARN  Could not purge cache, metadata file 'meta-browser-shared.json' could not be found.

What information do you need?

Edit: the cache path in node_modules/.cache/native-federation/project_name is empty after this happens, but there are many JSONs in node_modules/.cache/native-federation. And at the same time, the project was seemingly successfully built in dist.

[Aukevanoost]

Ah interesting! I know enough, this is an error on my part. It's looking for the metadata file which doesn't exist yet.

It was supposed to cancel the cache purge if the metadata file didn't exist (which is created on the first run so obviously it's not there). I now added the return statement and it should work now.

Thanks for the heads up, you don't need to but it helps for clarity to remove the node_modules/.cache folder.

[dr460nf1r3]

Purging the cache folder did not change the outcome (did this before, and now again to confirm).

[Aukevanoost]

Yess! I meant the cache purge in native-federation (in the purgeCacheFolder method) Here's the change i just committed:
fcd305e#diff-f1ea8a410d86bdb32a74790867ce4e3cbb2fbeab875b91152a865fa992691092

[dr460nf1r3]

Here are the results:

Page reload sent to client(s).
 INFO  Building federation artefacts
 DBG!  00:06:515.443ms - [build artifacts] - To bundle all mappings and exposed.
 INFO  Done!
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-4HEU3YZF.js | bootstrap | 640.23 kB | 

Application bundle generation complete. [0.445 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
 DBG!  00:06:651.442ms - [build artifacts] - To bundle all mappings and exposed.
 INFO  Done!
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-XNRGNODX.js | bootstrap | 640.31 kB | 

Application bundle generation complete. [0.293 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
{
  "added": [],
  "modified": [
    "filereference"
  ],
  "removed": []
}
Initial chunk files   | Names     |  Raw size
main.js               | main      | 288 bytes | 

Lazy chunk files      | Names     |  Raw size
bootstrap-JNSJ4I3Q.js | bootstrap | 640.31 kB | 

Application bundle generation complete. [0.598 seconds]

Page reload sent to client(s).
 INFO  Building federation artefacts
 DBG!  00:08:304.051ms - [build artifacts] - To bundle all mappings and exposed.
 INFO  Done!
 DBG!  00:07:804.131ms - [build artifacts] - To bundle all mappings and exposed.
 INFO  Done!