Images are broken for M1 Mac Docker desktop >= 4.1.1

[atong01]

M1 Mac running macOS 12.1 Docker for Mac 4.3.1 amd64 image. Current solutions are to use arm64 images or downgrade Docker for Mac to 4.0.1

I suspect this is something upstream so have filed an issue here: docker/for-mac#6097
but posting here for others that may have the same issue.

What docker image you are using?

jupyter/minimal-notebook scipy-notebook also reproduces the error

What complete docker command do you run to launch the container (omitting sensitive values)?

docker run --platform=linux/amd64 -p 8888:8888 jupyter/minimal-notebook

What steps do you take once the container is running to reproduce the issue?

Example:

1. Visit http://localhost:8888
2. Start a python notebook

What do you expect to happen?

Notebook should boot with python kernel

What actually happens?

Jupyter server crashes with

Operation not permitted (src/thread.cpp:309)
Operation not permitted (src/thread.cpp:309)
qemu: uncaught target signal 6 (Aborted) - core dumped

Full details:

docker run -p 8888:8888 jupyter/minimal-notebook
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
WARNING: Jupyter Notebook deprecation notice https://github.com/jupyter/docker-stacks#jupyter-notebook-deprecation-notice.
Entered start.sh with args: jupyter notebook
Executing the command: jupyter notebook
[I 21:32:40.973 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
[W 2021-12-15 21:32:44.288 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2021-12-15 21:32:44.289 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2021-12-15 21:32:44.290 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2021-12-15 21:32:44.290 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2021-12-15 21:32:44.334 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.9/site-packages/jupyterlab
[I 2021-12-15 21:32:44.334 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 21:32:44.361 NotebookApp] Serving notebooks from local directory: /home/jovyan
[I 21:32:44.362 NotebookApp] Jupyter Notebook 6.4.6 is running at:
[I 21:32:44.362 NotebookApp] http://4b0325865eae:8888/?token=760d18fe8ecd1a926f954c3b66f89758e04a619ab017f5c6
[I 21:32:44.363 NotebookApp]  or http://127.0.0.1:8888/?token=760d18fe8ecd1a926f954c3b66f89758e04a619ab017f5c6
[I 21:32:44.363 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 21:32:44.382 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///home/jovyan/.local/share/jupyter/runtime/nbserver-8-open.html
    Or copy and paste one of these URLs:
        http://4b0325865eae:8888/?token=760d18fe8ecd1a926f954c3b66f89758e04a619ab017f5c6
     or http://127.0.0.1:8888/?token=760d18fe8ecd1a926f954c3b66f89758e04a619ab017f5c6
[I 21:32:52.640 NotebookApp] 302 GET /?token=760d18fe8ecd1a926f954c3b66f89758e04a619ab017f5c6 (172.17.0.1) 6.880000ms
[I 21:32:59.344 NotebookApp] Creating new notebook in
[I 21:32:59.386 NotebookApp] Writing notebook-signing key to /home/jovyan/.local/share/jupyter/notebook_secret
Operation not permitted (src/thread.cpp:309)
qemu: uncaught target signal 6 (Aborted) - core dumped

[mathbunnyru]

I do agree that this looks like an upstream issue of one version worked fine and the newer one fails.
Let's wait for the answer in the docker project.

[richtong]

Sigh, they closed the bug as "best efforts only" how sad, tried to replace the qemu with v6, they are on v5, but it is not QEMU but something else above that level.

[massongit]

The build of datascience-notebook for arm64 is failed in the following process to install the Julia packages:

docker-stacks/datascience-notebook/Dockerfile

Lines 99 to 110 in ace92d3

	# Add Julia packages.
	# Install IJulia as jovyan and then move the kernelspec out
	# to the system share location. Avoids problems with runtime UID change not
	# taking effect properly on the .local folder in the jovyan home dir.
	RUN julia -e 'import Pkg; Pkg.update()' && \
	julia -e 'import Pkg; Pkg.add("HDF5")' && \
	julia -e 'using Pkg; pkg"add IJulia"; pkg"precompile"' && \
	# move kernelspec out of home \
	mv "${HOME}/.local/share/jupyter/kernels/julia"* "${CONDA_DIR}/share/jupyter/kernels/" && \
	chmod -R go+rx "${CONDA_DIR}/share/jupyter" && \
	rm -rf "${HOME}/.local" && \
	fix-permissions "${JULIA_PKGDIR}" "${CONDA_DIR}/share/jupyter"

How about preparing a datascience-notebook without Julia and supporting arm64 for it?

#1551

[mathbunnyru]

I think it works now with Docker 4.4.2 🎉
I checked it on my M1 Mac with and without -e DOCKER_STACKS_JUPYTER_CMD=notebook.

So, i think, we can close this issue as resolved one.

[ndtreviv]

😭

$ docker run -p 8888:8888 -v $(pwd):/home/ubuntu/work jupyter/tensorflow-notebook
Unable to find image 'jupyter/tensorflow-notebook:latest' locally
latest: Pulling from jupyter/tensorflow-notebook
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.

So I go here and download the Dockerfile, try to build it locally, but no:

 => ERROR [2/2] RUN mamba install --quiet --yes     'tensorflow' &&     mamba clean --all -f -y &&     fix-permissions "/opt/conda" &&     fix-permissions "/home/jovyan"            10.4s
------
 > [2/2] RUN mamba install --quiet --yes     'tensorflow' &&     mamba clean --all -f -y &&     fix-permissions "/opt/conda" &&     fix-permissions "/home/jovyan":
#6 10.12 Encountered problems while solving:
#6 10.12   - nothing provides requested tensorflow
#6 10.12
------
executor failed running [/bin/bash -o pipefail -c mamba install --quiet --yes     'tensorflow' &&     mamba clean --all -f -y &&     fix-permissions "${CONDA_DIR}" &&     fix-permissions "/home/${NB_USER}"]: exit code: 1

[mathbunnyru]

@ndtreviv we don't provide aarch64 image for jupyter/tensorflow-notebook.
And currently jupyter/tensorflow-notebook is a manifest list with only one image for linux/amd64.
To make it run amd64 version under your Mac, just add --platform linux/amd64 to the docker run options.