setenv not working with <

[IceyMu]

Hello,

When I try to read the value of a file into an environment variable using setenv and < I get back an empty environment variable e.g

setenv AWS_ACCESS_KEY_ID < /etc/xrootd/access_key

Doesn't change the environment variable AWS_ACCESS_KEY_ID.

I can however read from a file into a variable using set and then copy that variable to the environment variable e.g

set ACCESS_KEY < /etc/xrootd/access_key
setenv AWS_ACCESS_KEY_ID = $ACCESS_KEY

Makes the environment variable AWS_ACCESS_KEY_ID have the value of the contents of /etc/xrootd/access_key

setenv also works if I were to pass the value directly e.g

setenv AWS_ACCESS_KEY_ID = 1234

I am using xrootd 5.5.1 from the epel repository.

Thanks

[amadio]

Can you please explain to us where this setenv command is? If that's a ~/.tcshrc or similar, I don't see how this could be a problem with XRootD.

[IceyMu]

The setenv is in the xrootd configuration file. I am trying to use setenv as described in section 4.4 of the configuration file syntax manual.

The configuration I am trying to use is:

set redirector = elephant103.heprc.uvic.ca

all.manager $(redirector):1095


# Debugging output
#xrd.trace all
#xrootd.trace all
#ofs.trace all
#sec.trace all
#pss.trace all
#http.trace all


all.export /

# The names used here must match what is going to be passed as server_type
if named s3_proxy
  # Proxy setup
  all.role proxy server
  ofs.osslib libXrdPss.so
  pss.origin http://elephant102.heprc.uvic.ca:9000
  pss.localroot /bucket

  # XRootD doesn't seem to like reading files directly into environment variables
  set ACCESS_KEY < /etc/xrootd/access_key
  set SECRET_KEY < /etc/xrootd/secret_key

  setenv AWS_ACCESS_KEY_ID = $ACCESS_KEY
  setenv AWS_SECRET_ACCESS_KEY = $SECRET_KEY
else if named standalone
  all.role server
  oss.localroot /data
fi

# TLS Setup
xrd.tls /etc/grid-security/xrd/xrdcert.pem /etc/grid-security/xrd/xrdkey.pem
xrd.tlsca certdir /etc/grid-security/certificates
xrootd.tls all

if $redirector
  all.role manager
  http.desthttps yes
else
  xrootd.seclib libXrdSec.so
  sec.protocol gsi -dlgpxy:request \
                   -exppxy:=creds \
                   -ca:verify \
                   -crl:try \
                   -gmapopt:nomap \
                   -vomsat:require \
                   -vomsfun:default
fi

xrootd.seclib libXrdSec.so
http.secxtractor libXrdHttpVOMS.so

if exec xrootd
  xrd.protocol http libXrdHttp.so
fi

# Authorization configuration
ofs.authorize
acc.audit deny grant
acc.authdb /etc/xrootd/Authfile

# TPC Setup
xrootd.chksum adler32

ofs.tpc fcreds ?gsi =X509_USER_PROXY
ofs.tpc require client gsi
ofs.tpc scan stderr
ofs.tpc echo
ofs.tpc ttl 300 600
ofs.tpc xfr 100
ofs.tpc autorm
ofs.tpc pgm /etc/xrootd/tpc.sh

http.exthandler xrdtpc libXrdHttpTPC.so
http.header2cgi Authorization authz

# Macaroon setup
http.exthandler xrdmacaroons libXrdMacaroons.so
macaroons.secretkey /etc/xrootd/macaroon-secret
all.sitename test_sitename
ofs.authlib libXrdMacaroons.so

[abh3]

The setenv exists n the config file. You can do a setenv as explained in the documentation: https://xrootd.slac.stanford.edu/doc/dev55/Syntax_config.htm#_Toc520499869

…

On Wed, 8 Feb 2023, Guilherme Amadio wrote: Can you please explain to us where this `setenv` command is? If that's a `~/.tcshrc` or similar, I don't see how this could be a problem with XRootD. -- Reply to this email directly or view it on GitHub: #1899 (comment) You are receiving this because you were assigned. Message ID: ***@***.***>

[amadio]

Thank you both for the explanations. I will have a look tomorrow how to fix this one.

[xrootd-dev]

Actually, I assigned myself to fix this as I wrote the code in the first place. Let me if that is OK with you so that we both don't work on it. Andy

…

On Wed, 8 Feb 2023, Guilherme Amadio wrote: Thank you both for the explanations. I will have a look tomorrow how to fix this one. -- Reply to this email directly or view it on GitHub: #1899 (comment) You are receiving this because you are subscribed to this thread. Message ID: ***@***.***> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

[amadio]

Hi Andy, I can fix this one, leave it for me. It looks as if the code below is not going to override an environment variable if it's already set:

https://github.com/xrootd/xrootd/blob/master/src/XrdOuc/XrdOucStream.cc#L1458-L1460

@IceyMu Can you please confirm that AWS_ACCESS_KEY_ID is not set when you launch the server?

[IceyMu]

I've tried running my configs on a new virtual machine and this makes the config work with setenv and < together.

[abh3]

Actually, the "if" (if that is what you are talking about) is merely saying if the envar is not set or if it is set but the new value differs from the old value then it is set. The latter part of the clause (i.e. set it only if it is a different) is needed to not loose memory as config files can be reprocessed multiple times. So, I'd say something else is going on. The amount of memory lost is small but it dirties up valgrind output if not minimized.

…

On Mon, 13 Feb 2023, Guilherme Amadio wrote: Hi Andy, I can fix this one, leave it for me. It looks as if the code below is not going to override an environment variable if it's already set: https://github.com/xrootd/xrootd/blob/master/src/XrdOuc/XrdOucStream.cc#L1458-L1460 @IceyMu Can you please confirm that `AWS_ACCESS_KEY_ID` is not set when you launch the server? -- Reply to this email directly or view it on GitHub: #1899 (comment) You are receiving this because you were assigned. Message ID: ***@***.***>

[abh3]

Then I would surmise that the problem was the virtual machine not the code and this ticket can be closed, right?

…

On Mon, 13 Feb 2023, IceyMu wrote: I've tried running my configs on a new virtual machine and this makes the config work with `setenv` and `<` together. -- Reply to this email directly or view it on GitHub: #1899 (comment) You are receiving this because you were assigned. Message ID: ***@***.***>

[amadio]

Actually, the "if" (if that is what you are talking about) is merely
saying if the envar is not set or if it is set but the new value differs
from the old value then it is set.

Indeed, after I commented I went through the code a bit more carefully and saw that it it's working for both set and setenv. I wrote a simple config file and ran with GDB and saw that it works as intended.