-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shell detection failure inside docker image on ARM Macbook #55
Comments
If you have an idea what could be the reason, I'd be happy to look deeper inside |
Further info by @ramirezfranciscof in aiidalab/aiidalab-docker-stack#202 (comment) Is it possible that shellingham somehow gets confused when the architecture the docker image was built for does not match the architecture of the host OS? |
I managed to get a "more minimal" example, if that helps. This is the
Then I build it with root@baseimage_test:/# python3
Python 3.10.0 (default, Oct 5 2021, 23:49:26) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shellingham
>>> shellingham.detect_shell()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/site-packages/shellingham/__init__.py", line 24, in detect_shell
raise ShellDetectionFailure()
shellingham._core.ShellDetectionFailure If I build without the root@baseimage_test:/# python3
Python 3.10.0 (default, Oct 6 2021, 00:09:42) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shellingham
>>> shellingham.detect_shell()
('bash', '/bin/bash') |
I don't have an ARM machine to test this out, so you'll probably need to debug this mostly on your own. Note that macOS and Linux are likely using different implementations (macOS uses the I'd probably start with doing something like >>> import shellingham.posix.proc
>>> print(shellingham.posix.proc.get_process_mapping()) and see if there's anything like a shell in there. If not, I'd manually break the loop apart and see where the parsing code went wrong. The fact that this does not happen if you use a native container seems to also indicate that this is something related to the cross-arch translation; maybe a process of Python built against Intel can't map its pid correctly to native ARM? No idea to be honest. |
Thanks for the hints @uranusjr ! Indeed, there is nothing that looks like a shell in the process mapping
I should note that there is also no shell in the output of
Finally, here is an example of a
P.S. This is just for future reference: it turns out that even on my machine, using the same container, the shell detection error is not always raised. After this, activating the conda environment again did not remove the error, however - it now persisted. |
Maybe. I don't really understand this behaviour either, probably due to some kind of magic in Docker or the Linux kernel. But in any way, if the OS is not reporting the existence of a shell, there's really nothing we can do… The application using shellingham is supposed to provide a reasonable default because this kind of oddities do happen, and shell detection can only do so much. This can probably be better explained by some Docker or OCI or Linux on ARM expert, and I am really none of those. |
git clone git@github.com:HenryFBP/trading-bot.git
cd trading-bot/
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run -it -v $(pwd):/mnt python:3.7-slim bash
# from inside the running Docker shell
cd /mnt
pip install pipenv
pipenv --python /usr/local/bin/python install
pipenv shell I get this even trying to "abstract" away the arm64 and stick to good ol x86_64 (amd64) through emulation
|
That fixes it for some reason? |
Shellingham is never called if you set |
Alright I finally have a chance to look into this. So for this specific environment combination (an Intel image running in Docker on ARM Mac), the parent process is (interestingly) hooked to a different TTY from the Python process itself. I suspect this is due to some simulation implementation detail that leaked into the container. So this is “easily” amendable by removing the TTY check, but I’m hesitant to just do that since it makes the proc implementation quite a bit slower. An alternative approach would be to do some refactoring and make process look up lazier, so we only access process IDs that are related to the current process. I’m not particularly motivated to do this myself (especially considering that setting |
We have a docker container, for which we're running into a shell detection failure:
For some reason, this only happens when running the docker container on M1 Macbooks (on Intel Macbooks, the error in the container does not occur). Observations:
os.name
isposix
For some reason, the shell detection for posix returns
None
shellingham/src/shellingham/posix/__init__.py
Lines 82 to 90 in 325c643
Steps to reproduce:
The text was updated successfully, but these errors were encountered: