Handle commas in environment variables

[PhracturedBlue]

Version of Singularity:

3.8.0-1.el7

Expected behavior

using singularity exec --env LDFLAGS='-L/some/path -Wl,-rpath,/some/path' image.sif should pass the LDFLAGS variable properly

Actual behavior

LDFLAGS is truncated at the comma due to --env splitting variables by comma.
I am aware that I can pass variables using --env-file or SINGULARITYENV_* but for my use case this is very inconvenient

What OS/distro are you running

$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"

How did you install Singularity

yum install singularity from rpel

[DrDaveD]

Please provide a full reproducer that anybody can run by copy & paste. I don't think it is supported to pass environment variables after the exec command like that at all so I'm not sure what you're actually trying.

[PhracturedBlue]

I typoed the command above and forgot the --env switch
I am running a command like:

singularity exec --env LDFLAGS="-L/opt/gcc/9.4/lib64 -Wl,-rpath,/opt/gcc/9.4/lib64" --bind /opt centos-7.9.sif /opt/gcc/9.4/bin/g++ myapp.cpp

to ensure that my application is linked against the proper standard libraries and has the needed rpath

[DrDaveD]

I see, and that results in these warnings

WARNING: Ignore environment variable "-rpath\\": '=' is missing
WARNING: Ignore environment variable "/opt/gcc/9.4/lib64": '=' is missing

and LDFLAGS is only set to the part before the comma. What you'd like is a feature like #6008 recently merged from @cclerget but for --env instead of --bind.

Meanwhile a workaround could be to replace the --env LDFLAGS... with

--env-file <(echo LDFLAGS=\"-L/opt/gcc/9.4/lib64 -Wl,-rpath,/opt/gcc/9.4/lib64\")

which takes advantage of bash process substitution to create a "file" where the separator is newlines instead of commas. I also don't see why using

SINGULARITYENV_LDFLAGS="-L/opt/gcc/9.4/lib64 -Wl,-rpath,/opt/gcc/9.4/lib64" singularity exec ...

would be inconvenient.

[PhracturedBlue]

It is inconvenient to use --env-file or SINGULARITY_ENV because I am running singularity via ansible and I do not have an easy way to set the environment variables at the proper level of script hierarchy. I can manage, it is just a lot of ugly scripting.

I would indeed like a feature like #6008 for env variables. What is the proper way to request such a thing?

[DrDaveD]

I don't understand; why is --env-file with <() or SINGULARITYENV on the singularity command line any less convenient than --env? Aren't they both a single command line that ansible could run?

singularity is an open source project, so to get code updates the best way to do it is either submit a pull request or hire someone to do it. The person who did #6008 works for a company that accepts contracts for updates to singularity, you could try contacting them if it's important to you.

[PhracturedBlue]

I really can't go into explaining the complexities of our workflow. If it were easy to do, I wouldn't have asked.

I will either submit a pull request, or develop a workflow to work-around this issue. Feel free to close the ticket.

[DrDaveD]

Maybe you're saying that ansible makes it more difficult to use anything that requires a shell to expand. So this might be nice to have but I think most people would rather use a workaround than invest the time to write the singularity modification PR. I'll close the issue for now as you request.