You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While configuring a JupyterHub server on a cluster, I encountered several issues related to environment variables not being properly exported on the compute nodes by the batchspawner.
Expected behaviour
In the default batch script, a line is responsible for exporting the variables (e.g. #SBATCH --export={{keepvars}} for slurm). The keepvars variable is basically a concatenation of two coma separated lists of variables req_keepvars and req_keepvars_extra.
The former is not configurable and already contains some required variables.
The latter should be configurable and is expected to provide the user with a convenient way to pass additional environment variables to the batch script.
Actual behaviour
The attribute BatchSpawnerBase.req_keepvars_extra does not seem to be used when producing the submitted batch script. The extra variables are not present in the list of exported environment variables.
Workaround
Overriding the default batch script in the config by hardcoding the variables to export does the trick but I am not comfortable with that approach. In the future, the idea would be to load the shell script from a file that should be agnostic of those variables.
(In other words, I want to define a list of additional environment variables in the config, and keep {keepvars} at the top of a .sh file read in the jupyterhub config)
Proposed fix
I managed to get the expected behaviour by changing this segment of code:
#batchspawner.py: line 169
req_keepvars_extra = Unicode(
help="Extra environment variables which should be configured, "
"added to the defaults in keepvars, "
"comma separated list.",
)
and adding .tag(config=True):
req_keepvars_extra = Unicode(
help="Extra environment variables which should be configured, "
"added to the defaults in keepvars, "
"comma separated list.",
).tag(config=True)
How to reproduce
Install a jupyterhub server on a cluster
Configure it to use a batch spawner that inherits from BatchSpawnerBase
Try to export additional environment variables using c.BatchSpawnerBase.req_keepvars_extra = "YOUR,VARS,HERE"
Start the server (in my case using systemctl) and see it fail when attempting to spawn a new batch job (checking the produced script reveals that the environment variables do not contain the extra variables)
The text was updated successfully, but these errors were encountered:
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋
Bug description
While configuring a JupyterHub server on a cluster, I encountered several issues related to environment variables not being properly exported on the compute nodes by the batchspawner.
Expected behaviour
In the default batch script, a line is responsible for exporting the variables (e.g.
#SBATCH --export={{keepvars}}
for slurm). Thekeepvars
variable is basically a concatenation of two coma separated lists of variablesreq_keepvars
andreq_keepvars_extra
.The former is not configurable and already contains some required variables.
The latter should be configurable and is expected to provide the user with a convenient way to pass additional environment variables to the batch script.
Actual behaviour
The attribute
BatchSpawnerBase.req_keepvars_extra
does not seem to be used when producing the submitted batch script. The extra variables are not present in the list of exported environment variables.Workaround
Overriding the default batch script in the config by hardcoding the variables to export does the trick but I am not comfortable with that approach. In the future, the idea would be to load the shell script from a file that should be agnostic of those variables.
(In other words, I want to define a list of additional environment variables in the config, and keep
{keepvars}
at the top of a .sh file read in the jupyterhub config)Proposed fix
I managed to get the expected behaviour by changing this segment of code:
and adding
.tag(config=True)
:How to reproduce
c.BatchSpawnerBase.req_keepvars_extra = "YOUR,VARS,HERE"
The text was updated successfully, but these errors were encountered: