Handling of Javascript libraries in context of debian

[lhw]

Question summary

While packaging netdata for debian we are currently and have been running into the situation that we need to verify and provide a non-minimized version of every js library and css file with the package. As per debian standards minimized version count as compiled source even if it still plaintext. And I do agree with that decision.

Well... there are a lot of them, 33 to be precise. This causes an immense amount of work every time we release a new upstream version as we have to check each and every file for changes and if there are any we need to find the corresponding non-minimized version, sometimes even from cherry-picked git commits. It feels like we spend most of our time for this package with trying to find the corresponding information.
(https://salsa.debian.org/debian/netdata/tree/master/debian/missing-sources)

For the latest release we kind of lost interest in doing this work and just plain ignore all the errors that spring up in the build process from debian but that is not a good solution.
(https://salsa.debian.org/debian/netdata/blob/master/debian/source/lintian-overrides)

We would really love a better solution to this. Maybe provide the JavaScript libraries non-minified version and minify them for redistributables? Or provide a list of changes or some kind of registry for the used files. But I am all up for suggestions. I know most other distributions packages will not be affected by this as much as debian but maybe there are more people interested in this as well.

OS / Environment

debian/any

Component Name

JavaScript Libraries/CSS

[Ferroin]

I have a feeling people who do security auditing would be very interested in this too. Personally, I'm in favor of the first option you listed (have the files in full-form in the repo, minify them during the build if possible).

[FedericoCeratto]

I recommend shipping non-minified and unmodified libraries and ideally consider decreasing the quantity and size of the libraries.

(As a side note, I wonder if it is worth performing minification for the Netdata use-case. It's often used on local, high-bandwidth connections, there are infrequent page reloads and most of the transferred data is to populate the charts)

[ktsaou]

hm... I was not aware of this problem. Thank you @lhw for reporting it!

Well, I think we could provide both versions of the files.

I think we should not provide just the unminified version of them, because this would require people installing from source to have a toolchain to minify them.

I would prefer the files to be distributed minified, since a lot of users install netdata in IoT environments that network is not that abundant.

So, a good balance seems to provide both versions.
Would this be better for you?

---

PS: @cakrit I see there are a few files that are not needed any more. @gmosx has removed support for js libraries that were not that useful in netdata (they didn't support all chart types, they were extremely slow, etc). Please check dashboard.js for the js libraries it uses and remove the rest.

[FedericoCeratto]

Providing both versions would work. A clear naming scheme e.g. foo.version-number.js and foo.version-number.min.js would be helpful.

[gmosx]

I added #5086 to remove unused external JS libraries.

[Ferroin]

Unless I'm mistaken, this should be indirectly fixed by the new dashboard.

Longer term, I'm hoping to have a separate package for the dashboard, which should simplify this significantly.

[Ferroin]

Te aforementioned split dashboard package may be done as part of the followup work after the installer overhaul.

@ilyam8 Other than the above bit, this can be collected for product to look at.

[ilyam8]

Closing this feature request. We will re-evaluate the request internally.