Nix: Environment isn't passed to controllers started with artiq_ctlmgr #7
Comments
|
@b-bondurant @lriesebos have you seen this issue? |
|
@drewrisinger not that I'm aware of, but I'll take a look |
|
@sbourdeauducq @dnadlinger is this a known bug/do you have workarounds? My guess is that don't work because their python environment isn't wrapped properly (i.e. /nix/store/...-python-3.7.../bin/python doesn't have the PYTHONPATH of the downstream modules e.g. ARTIQ available) |
The correct Python executable should have something with |
|
@sbourdeauducq ack, that's what flagged me to the python env wrapping. Yeah, here's the issue^^. The PATH for
|
Just for completeness: I am not running ARTIQ using the Nix package; rather, I just use it to prepare all the dependencies for then installing ARTIQ/… in development mode into two different venvs for our two trap nodes. This is a very suboptimal setup (among other things, it discards all the reproducibility benefits), but was the easiest way I could find to get all the executable wrappers installed. (Just setting PYTHONPATH in a Nix shell derivation to get "development mode" works for me personally, but would force all the other people to use |
I would prefer this solution over adding package manager specific code into ARTIQ. |
|
Ok. I finally had time to track down this issue. High-level summary:The wrapped python environment (i.e. the one created with PatchPatch file below (apply with ``git am``)DetailsMinimal reproduction steps$ mkdir -p test-artiq-comtools-env/repository
$ cd test-artiq-comtools-envFiles: device_db.pyartiq_minimal.nixTesting
Tracking down the bugNix wraps python executables to provide the proper environment & variables, e.g. to add the We can look at this wrapping from within Outputting the wrapper script files$ cat $(which artiq_ctlmgr)
#! /nix/store/hrpvwkjz04s9i4nmli843hyw9z4pwhww-bash-4.4-p23/bin/bash -e
export NIX_PYTHONPREFIX='/nix/store/x21yjqv9nccaca730p40iggcp5w2lsrk-python3-3.7.6-env'
export NIX_PYTHONEXECUTABLE='/nix/store/x21yjqv9nccaca730p40iggcp5w2lsrk-python3-3.7.6-env/bin/python3.7'
export NIX_PYTHONPATH='/nix/store/x21yjqv9nccaca730p40iggcp5w2lsrk-python3-3.7.6-env/lib/python3.7/site-packages'
export PYTHONNOUSERSITE='true'
exec "/nix/store/d8cgxms01c7ifx59yg5bcmcx1vha97cv-python3.7-artiq-comtools-1.1/bin/artiq_ctlmgr" "$@"
$ cat /nix/store/d8cgxms01c7ifx59yg5bcmcx1vha97cv-python3.7-artiq-comtools-1.1/bin/artiq_ctlmgr
#! /nix/store/hrpvwkjz04s9i4nmli843hyw9z4pwhww-bash-4.4-p23/bin/bash -e
export PATH='/nix/store/r94aa2gj4drkhfvkm2p4ab6cblb6kxlq-python3-3.7.6/bin:/nix/store/d8cgxms01c7ifx59yg5bcmcx1vha97cv-python3.7-artiq-comtools-1.1/bin:/nix/store/1fpcnywm092zvgbs6j6kcmc7ln2v06x3-python3.7-sipyco-1.1/bin:/nix/store/qz0j7ynd2fv73hmkv8nfrj04xd4h7dcj-python3.7-numpy-1.18.1/bin:/nix/store/8dmmabfb8g3xcpsacnh4ipflkmai2465-python3.7-chardet-3.0.4/bin'${PATH:+':'}$PATH
export PYTHONNOUSERSITE='true'
exec -a "$0" "/nix/store/d8cgxms01c7ifx59yg5bcmcx1vha97cv-python3.7-artiq-comtools-1.1/bin/.artiq_ctlmgr-wrapped" "$@"
$ cat /nix/store/d8cgxms01c7ifx59yg5bcmcx1vha97cv-python3.7-artiq-comtools-1.1/bin/.artiq_ctlmgr-wrapped
#!/nix/store/r94aa2gj4drkhfvkm2p4ab6cblb6kxlq-python3-3.7.6/bin/python3.7
# -*- coding: utf-8 -*-
import sys;import site;import functools;sys.argv[0] = '/nix/store/d8cgxms01c7ifx59yg5bcmcx1vha97cv-python3.7-artiq-comtools-1.1/bin/artiq_ctlmgr';functools.reduce(lambda k, p: site.addsitedir(p, k), ['/nix/store/d8cgxms01c7ifx59yg5bcmcx1vha97cv-python3.7-artiq-comtools-1.1/lib/python3.7/site-packages','/nix/store/1fpcnywm092zvgbs6j6kcmc7ln2v06x3-python3.7-sipyco-1.1/lib/python3.7/site-packages','/nix/store/qz0j7ynd2fv73hmkv8nfrj04xd4h7dcj-python3.7-numpy-1.18.1/lib/python3.7/site-packages','/nix/store/zj3yq7398qah5cnw75yi9h207ssfb5w9-python3.7-aiohttp-3.6.2/lib/python3.7/site-packages','/nix/store/bbicl709z65vh33nhjipm5apmsv81li8-python3.7-attrs-19.3.0/lib/python3.7/site-packages','/nix/store/8dmmabfb8g3xcpsacnh4ipflkmai2465-python3.7-chardet-3.0.4/lib/python3.7/site-packages','/nix/store/idl2413784psk7gs5b24971cc7dc2la9-python3.7-multidict-4.7.4/lib/python3.7/site-packages','/nix/store/j7k2d3rflf7yqzkglvaicvc7xyhxmahg-python3.7-async-timeout-3.0.1/lib/python3.7/site-packages','/nix/store/9skch6zxx8g1nxwb2kwjhlb9c5msahcz-python3.7-yarl-1.4.2/lib/python3.7/site-packages','/nix/store/py5py6kkf8yc3284i6v1zh1lraym2nr4-python3.7-idna-2.8/lib/python3.7/site-packages'], site._init_pathinfo());
import re
import sys
from artiq_comtools.artiq_ctlmgr import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())By combining this with the test output from Patching the BugI see 3 possible solutions to fixing this issue.
Solution details:Effectively, this looks like prepending the However, this is a little complicated b/c the $NIX_PYTHONPATH isn't declared until "runtime" (i.e. creating the wrapped python environment happens after building e.g. the buildPythonPackage {
pname = "artiq-comtools";
...
makeWrapperArgs = [ "--prefix PATH : $NIX_PYTHONPREFIX/bin" ];
}won't work, because ConclusionApplying the patch above should work. Tested with:
|
|
Thanks! |
Summary
Starting the ARTIQ
no_hardwareexamples fails on Ubuntu (Nix).Steps to Reproduce
Use the following
my-shell.nixfile with latest versions ofnixos-20.03&artiq-full:EDIT: taken from ARTIQ manual, extra python packages are needed for correct python packages for
no_hardware/device_db.pyso it (shouldn't) fail.(Assuming ARTIQ Nix derivations & binaries are installed as specified in manual).
Get the ARTIQ repo w/ examples & start
Error Message (short, what I think is relevant)
Error Message (full, removing machine name)
Full Traceback
Just discovered this bug, and haven't had time to debug/isolate it fully. Running
python -c "import artiq"from the same shell works fine.This is an issue to me because it also prevents other controllers from being started by
artiq_ctlmgron the same machine.The text was updated successfully, but these errors were encountered: