sap-ui-cachebuster-info.json isn't generated for library project

[ljheric]

Expected Behavior

sap-ui-cachebuster-info.json should be generated for ui5 library project when the generateCachebusterInfo task is included in ui5 build command of package.json

Current Behavior

sap-ui-cachebuster-info.json is not generated for ui5 library project when the generateCachebusterInfo task is included in ui5 build command of package.json

Steps to reproduce the issue

1. Create a ui5 library project manually or web ide.
2. add option '--include-task=generateCachebusterInfo' in the ui5 build command in build script of package.json.
3. run 'npm run build-for-deploy'.

Context

• UI5 Module Version (output of ui5 --version when using the CLI): 2.0.3
• Node.js Version: v10.15.0
• npm Version: 6.4.1
• OS/Platform: macOS 10.15.4

Affected components (if known)

• ui5-builder
• ui5-server
• ui5-cli
• ui5-fs
• ui5-project
• ui5-logger

[petermuessig]

Hi @ljheric,

for what purpose do you need the cachebuster information for libraries? This cachebuster information is only useful for UI5 applications. The UI5 framework is using the application cachebuster information to redirect the application resources but for libraries this information is never been used. Can you please elaborate a bit more on your use-case?

THX and BR,
Peter

[RandomByte]

Closing due to lack of information from reporter.

[lennart-m]

Hi, I recently learned how to use cache buster and I just came across this issue as well. My custom library is built without sap-ui-cachebuster-info.json, and the documentation also states that generateCachebusterInfo will never run for libraries (https://sap.github.io/ui5-tooling/pages/Builder/#standard-tasks).
So if I have an app that consumes a custom library (that I wrote), is there no way to make that library run with the cache buster mechanism?

I'm currently writing an app to run on cloud foundry (using UI5 from CDN) but I would be interested in how to handle custom libraries for on-prem scenarios as well. Thanks a lot!

edit:
I just checked on our on-prem system. Using parameter sap-ui-appCacheBuster=/path/to/my/lib I get a dynamically generated cache buster info file. Though I didn't get UI5 to consume this file.

[petermuessig]

Hi @lennart-m ,

for libraries we are typically benefit from the runtime cachebuster concept which adds a cachebuster token in the URL after the resources path segment. For the SAPUI5 distribution, we can do this in general, as we have use the latest timestamp of all inlcuded UI5 libraries in the distribution. In your case, if your have a custom library, it is more interesting to know how you finally refer to it: is it a separate CF application or is it packaged in your application?

For the on-prem system it is more a general feature of the UI5 BSP respository. This repository is generating the cachebuster info file independent from the project type.

But going one step back: for applications we need a mechanism to load all application related resources with a prefix. And those resources could be in any path - therefore we need that index to match them and shift them. For libraries, we typically have a namespace and we know where the library resources are. So in this case, for libraries we can make use of the namespace to URI mapping and add a timestamp segment to the library resource URL, e.g.:

  sap.ui.loader.config({

    paths: {
      'my/lib/path': 'resources/~1266261621~/my/lib/path'
    },

You could control that also on your own and easily shift the path for the library resources which isn't easily possible for applications. IMO, this provides you with much more flexibility and it will ensure that "all" library resources are shifted properly whereas in the appcachebuster concept there are some cases which cannot be handled properly as we need to intercept native apis and UI5 apis.

HTH and BR,
Peter

[lennart-m]

Hi @petermuessig, thanks for your reply.
I created a proof of concept application for using AppCacheBuster together with a library and while doing so I think I started to understand some of the problems involved in using cache buster for libraries. My question was not only about apps running on BTP CF, but also about self-hosted UI5 apps (served by express, for example). But in my apps the library is mostly a separate application.

Anyway, here it is:
https://5c4ed75dtrial-dev-my-demo-approuter.cfapps.eu10.hana.ondemand.com/app/index.html
Code: https://github.com/lennart-m/ui5-lib-cachebuster-demo
There I added a bit of documentation.

Short form:

• the folders ui-app and ui-lib contain the UI modules; the app consumes the library. The library provides some JS (base controller, button handler), CSS (rotated button) and a message bundle.
• everything is deployed in a single MTA
• routing is configured in approuter/xs-app.json; the app is served at /app, the library is served at /lib
• I created the lib's cachebuster info file by hand
• AppCacheBuster is used for both the app and the library. Most resources are loaded at tokenized URLs; app as well as lib. This has to be configured in the app's index.html. The handling of tokenized URIs is just a dummy (remove token from the URI and forward un-tokenized URI to HTML5 app repository)
• In order to make AppCacheBuster handle the library files I had to make sure the library resources' paths don't start with /resources (that's why it's served at /lib)
• UI5 standard libraries are also loaded with cache buster functionality (resources/sap-ui-cachebuster/sap-ui-core.js)

Result: all the library are being loaded using cache tokens; JS, XML, CSS, properties. I use the filter -/~.+~/ in chrome network to find resources without tokenized URLs

Now I still have a few questions:

• You mentioned that some library resources cannot be handled by app cache buster; which are those?
• When should we generate the cachebuster info file at all? On SAP onprem it's handled by the ui5_ui5 ICF handler or by Fiori cachebuster. In which scenarios is the sap-ui-cachebuster-info.json file even being used?
• Related to the question above; when serving UI5 apps/libs by a custom web server (e.g. express.static), we probably have to implement validation of cache tokens for requested resources (i.e. if resource at /<token>/path/to/file really has token <token>) ourselves. Is that correct?
• Do you think it is possible to somehow automate your suggestion about shifting the library URI? I am thinking about dynamically adding a resourceRoots config (or your sap.ui.loader.config snippet) into index.html at build time (similar to how sap-ui-core.js is replaced with sap-ui-custom.js for self-contained apps). Therefore it would be necessary to somehow determine the library's token at build time which is probably the hardest part. Also this would make it impossible to update a library without also updating the consuming apps' index.html; so that's probably not a good idea at all.

An alternative to the suggestion above (adding hard-coded cache token to index.html): it might be a better idea to mimic the cache busting from UI5 standard libraries. I'm thinking about a new task generateCachebusterScript that can be activated when building libraries. This task creates a new JS file at /resources/my/lib/namespace/sap-ui-cachebuster-init.js; this script sets the resource root for my.lib.namespace to resources/<token: build timestamp>/my/lib/namespace (or some other configurable path). But only if the developer has not explicitly set the library's resource root.

When it comes to loading this script from within an application I'm not sure how to do it best. It could be done by some application build task that injects a link to this library cachebuster script (again the path must be configurable) into the app's index.html.

I hope this all makes sense and I would be glad if you were able to help me better understand this. Thanks a lot!
Lennart

[lennart-m]

I just made an update to my demo, using a similar approach to setting the library's resource root on start as it's used in resources/sap-ui-cachebuster/sap-ui-core.js.
For this I created a custom task ui5-task-generatelibcachebuster in my demo app. It creates a simple JavaScript ("cachebuster loader") that just sets the resource root for my library to the maximum timestamp of any library resource (so any file change in the library will make the library's resource root change).

In the app's index.html this script is loaded before sap-ui-core.js (so pre-loading the library should also work).

Now the library's files are loaded using a token URL; JS, CSS, i18n.

I am wondering if this might be a helpful addition to the standard UI5 tooling tasks, or alternatively a standalone task hosted as npm package. However, I'm not so sure about how to make this task work in general. It works fine for my demo, but the developer consuming a library currently cannot set a custom resource root. For this the cachebuster loader script would probably at runtime have to read the resource roots configured in the bootstrap, and override those with the token, if present.

[petermuessig]

Hi @lennart-m ,

shame on me to reply so late. I completely missed to reply to your first reply already. In general, the app cachebuster only handles SCRIPT and LINK tags as well as the URI properties of controls. There are several ways around those APIs which can finally lead to URLs not been adopted, e.g. if your would use a String property for a URL and render it as IMG src or something like this. Unfortunately, the app cachebuster is not 100% water tight and will never be. Therefore, I personally would prefer to get rid of it - also because of the dirty way it adopts the URLs.

Basically, I would prefer a clear separation between applications and libraries when it comes to cachebusting. Unfortunately, the runtime cachebuster for libraries is mainly built for the libraries provided by the OpenUI5/SAPUI5 distribution and it is not able to handle custom libraries. In your case, having some kind of sap-ui-cachebuster-loader.js might be one option to do so - and indeed, I would do it similar like you sketched it to configure the resource roots. Also that it will be loaded from the library itself. The only downside I see is the additional request initially which comes per library - but without rebuilding the application or having a common project providing the script for all custom libraries this can't be avoided. Somewhere the dynamic have to be - or you implement it via Service Workers - this might also be an option and you can check whether the resources are up-to-date. Since IE11 is out of the game starting with UI5 1.88 this might be also worth to think about.

But your idea is also good and it is worth to have a custom task for this. In some of the productive SAP scenarios, the platforms are handling the configuration of the resource roots. E.g. in the FLP case, the FLP receives the resource roots configuration from the AppIndex and then configures the resourceRoots. But this requires a backend component.

HTH and BR,
Peter

[codeworrior]

+1 for getting rid of the AppCachebuster in general :-)

[vobu]

might also help to talk to @lemaiwo on this cachebuster indexing task: https://github.com/lemaiwo/ui5-task-cachebuster-indexing

[BenReim]

Another alternative approach to the app cachebuster that would also work for libraries could be the useage of the self-contained build, together with a custom task to include a timestamp / hash within the bundle filename.

However, to my knowledge currently the tooling allows only for the bundling of JavaScript files, but not for instance for CSS resources: ui5-community/ui5-ecosystem-showcase#168