Optimize the resource images size

[mingrammer]

• Optimize the size of existing resource images
• Add optimization processing in the autogen.sh

[bdsoha]

There are many ways to reduce the overall footprint of the project (both the production artifact and the git repository).

1. Do not include static web assets in the repository, instead have generate them on demand (during CI @cbrown/743 mkdocs site #748) or published to a separate repository dedicated to documentation. The list below include files that do not need to be under source control:
   • /diagrams/website/
   • /diagrams/docs/nodes/
2. Optimize all images using a package like PIL or something similar. As well as restricting the upload of images larger than 256x256 using a custom action.yml.
3. Prefer the use of .svg images and only generate images on demand (during the creation of a diagram) Use native SVG icons when outputting SVG diagrams #866.
4. Publish core functionality as part of the diagrams package and split provider resources into extra packages such as diagrams[aws], diagrams[onprem,k8s], or diagrams[all] to include them all.

[gabriel-tessier]

One more idea copied from #764 :

• Create a different project something like diagrams-custom-resources that will only accept icons and can be called with Custom node and the url hosted by Github.

[bdsoha]

One more idea copied from #764 :

• Create a different project something like diagrams-custom-resources that will only accept icons and can be called with Custom node and the url hosted by Github.

@gabriel-tessier How would you handle air-gapped or offline networks?

[Collinbrown95]

Quick observation: it appears that the .png files for this project are duplicated in https://github.com/mingrammer/diagrams/tree/master/website/static/img/resources and https://github.com/mingrammer/diagrams/tree/master/resources.

As @bdsoha suggested above, we could copy the contents of https://github.com/mingrammer/diagrams/tree/master/resources to the docs folder at CI time in order to build the docs site, but otherwise have only a single copy of the /resources folder at the top level. E.g. something like cp -r resources docs/img/resources && mkdocs build -c -f mkdocs.yml.

I added this change in #748. The .png files now only exist in the top-level resources folder and are copied into the docs folder during the CI workflow.

[gabriel-tessier]

@bdsoha

How would you handle air-gapped or offline networks?

if you want to access the icon offline you can always git clone the repository in a local dir and follow the doc:
https://diagrams.mingrammer.com/docs/nodes/custom#custom-with-local-icons

[bdsoha]

@gabriel-tessier The documentation describes a use-case in which you want to include an icon not distributed with the packages. What you are suggesting would require the package consumer to import all the image resources to the air-gapped network and then overwrite the Node classes one-by-one.
IMO, that is very close to re-write of the package itself.

It would be better to include an option of online=True, in which case icons are downloaded on-demand from the network. Alongside an option to download the (complete set of) resource and use them from a pre-defined local directory.

[bdsoha]

I just ran a test using the optimize-images resources/ from PyPi with the following results:

----------------------------------------
   Processed 1699 files (23.6 MB) in 58.2s (29.2 f/s).
   Optimized 1582 files.
   Average savings: 1.9 KB per optimized file
   Total space saved: 2.9 MB / 12.3%

[Kochise]

See #866

[bdsoha]

@Kochise Using SVG images is not always an option.

[Kochise]

@bdsoha Yeah, I know, but it helps.

[bdsoha]

@bdsoha Yeah, I know, but it helps.

I update my comment above.
👍

[Kochise]

It was your point 3 already, I just linked to the other issue to aggregate the requests.

But indeed, SVG do help, like, by a lot.

[bdsoha]

Corrected... 😁