[ENH] - Create multi-platform images

[mathbunnyru]

What docker image(s) is this feature applicable to?

all-spark-notebook, base-notebook, datascience-notebook, minimal-notebook, pyspark-notebook, r-notebook, scipy-notebook, tensorflow-notebook

What changes are you proposing?

It would be great to merge amd64 image with aarch64 image with the same tag into a single multi-platform image

How does this affect the user?

It would allow users not to specify aarch64- prefix.
Also, it will make make build/<somestack>-notebook work as expected, because it will always use correct parent image.

Anything else?

Implementation detail - this should probably be done in a separate reusable workflow, after pushing all images to DockerHub.
It would take a look which tags were pushed for amd64 and which for aarch64 and it would merge these tags.

[nathancarter]

I would love to see this. Just to give an example of why it's important:

Just got an M1 Mac and tried to re-use a VS Code devcontainer I have that's based on jupyter/datascience-notebook. A build script in one of my projects, which used to run just fine, now runs out of memory, because the AMD devcontainer is being run on an ARM chip and thus has to go through some semi-supported emulation business that I don't fully understand, except to note that out-of-memory errors are documented as something to be expected. I have to physically go to my PC to run that build script...kind of the opposite of the point of Docker. While I guess I could add the aarch64- prefix to the FROM line of my Dockerfile, that will make the project work on my Mac but not my PC.

In other words, things got worse when I bought a better computer. And there's no workaround. I'm not complaining; I'm just trying to clarify the significance of @mathbunnyru's issue. Thanks for opening it!

[mathbunnyru]

Thank you for sharing your use case.

I think right now you can use your have Dockerfile look like FROM $BASE_CONTAINER and have this BASE_CONTAINER set to different things for Mac and PC, if it's possible.
I know it's not ideal, but if it works, it's not that bad.

[msemelman]

Not sure if it completely fits current building setup, but maybe buildx could be used to simplify this:

https://docs.docker.com/build/buildx/

[mathbunnyru]

Not sure if it completely fits current building setup, but maybe buildx could be used to simplify this:

https://docs.docker.com/build/buildx/

Unfortunately, docker buildx didn't work for us and created many issues and they were leading to bugs and very unpleasant things.
So far, current solution only has this issue.
You can see the details here and in the relevant issues: #1703

[mathbunnyru]

@nathancarter @consideRatio @tomharvey @msemelman this is fixed now, please, take a look.

[tomharvey]

Massive thanks @mathbunnyru - my workflows just got a lot simpler!

[consideRatio]

Wieee great work on this @mathbunnyru!!!!!