Poudriere env passing doesn't handle embedded quotes with spaces right #669

michael-o · 2019-03-04T08:54:25Z

Consider the following in login.conf:

    :setenv=MAIL=/var/mail/$,BLOCKSIZE=K,\
HTTP_PROXY=http\c//194.145.60.1\c9400,\
HTTPS_PROXY=http\c//194.145.60.1\c9400,\
FTP_PROXY=ftp\c//194.145.60.1\c9400,\
NO_PROXY="localhost .siemens.net .siemens.com .siemens.de":\
        :charset=UTF-8:\
        :lang=de_DE.UTF-8:\

Running poudriere gives me:

+ exec env -i 'LIBEXECPREFIX=/usr/local/libexec/poudriere' 'PATH=/usr/local/libexec/poudriere:/net/home/osipovmi/bin:/usr/local/bin:/net/home/osipovmi/bin:/usr/local/bin:/sbin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/net/home/osipovmi/bin:/sbin:/usr/sbin' 'POUDRIERE_VERSION=3.3.0' 'POUDRIEREPATH=/usr/local/bin/poudriere' 'HTTP_PROXY=http://194.145.60.1:9400' 'NO_PROXY="localhost' .siemens.net .siemens.com '.siemens.de"' 'FTP_PROXY=ftp://194.145.60.1:9400' 'SCRIPTPATH=/usr/local/share/poudriere/jail.sh' 'SCRIPTPREFIX=/usr/local/share/poudriere' sh /usr/local/share/poudriere/jail.sh
env: .siemens.net: No such file or directory

One cannot reasonably provide proxy excludes. While fetch and curl allow a comma-separated list, this cannot be used in login.conf There is no way to escape both. If this cannot be fixed, and I do believe that this is only possible with bash, the script shall use sed(1) to replace spaces with commas:

$ echo $NO_PROXY | sed 's/ /,/g'
"localhost,.siemens.net,.siemens.com,.siemens.de"

The text was updated successfully, but these errors were encountered:

michael-o · 2019-03-04T12:11:58Z

Opened also an upstream issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236204

bdrewery · 2019-03-04T19:35:33Z

I see a bug here but it shouldn't block you.

# NO_PROXY="localhost .siemens.net .siemens.com .siemens.de"
# echo $NO_PROXY
localhost .siemens.net .siemens.com .siemens.de

Looks good.
Compared to login.conf:

# grep setenv /etc/login.conf
        :setenv=MAIL=/var/mail/$,BLOCKSIZE=K,NO_PROXY="localhost .siemens.net .siemens.com .siemens.de":\

# echo $NO_PROXY
"localhost .siemens.net .siemens.com .siemens.de"

Note that the quotes literally are in there. That's not what you want.

Just remove the quotes in login.conf and it works as expected:

# grep setenv /etc/login.conf
        :setenv=MAIL=/var/mail/$,BLOCKSIZE=K,NO_PROXY=localhost .siemens.net .siemens.com .siemens.de:\

# echo $NO_PROXY
localhost .siemens.net .siemens.com .siemens.de

bdrewery · 2019-03-04T19:42:00Z

I cannot reproduce the problem here with any combination of quotes in NO_PROXY. It's probably not worth fixing anyway since I don't see a need to passthrough literal quotes.

bdrewery · 2019-03-04T19:48:36Z

Oh #670 is needed to reproduce this.

michael-o · 2019-03-05T08:44:10Z

#670 is not necessary to reproduce this.

This needs to be reopened, quotes have been removed:

root@deblndw011x - ~
2052 # grep NO /etc/login.conf
NO_PROXY=localhost .siemens.net .siemens.com .siemens.de:\

root@deblndw011x - ~
2053 # echo $NO_PROXY
localhost .siemens.net .siemens.com .siemens.de

root@deblndw011x - ~
2054 # poudriere jail -l
env: .siemens.net: No such file or directory

root@deblndw011x - ~
2055 # sh -x $(which poudriere) jail -
+ LC_ALL=C
+ unset SHELL
+ SAVED_TERM=xterm
+ unset TERM
+ POUDRIERE_VERSION=3.3.1
+ SETX=''
+ getopts Ae:Nvx FLAG
+ shift 0
+ [ 2 -lt 1 ]
+ realpath /usr/local/bin/poudriere
+ POUDRIEREPATH=/usr/local/bin/poudriere
+ [ /usr/local/bin/poudriere '!=' /usr/local/bin/poudriere ]
+ [ /usr/local '=' /usr/local/bin/poudriere ]
+ POUDRIEREPREFIX=/usr/local
+ LIBEXECPREFIX=/usr/local/libexec/poudriere
+ SCRIPTPREFIX=/usr/local/share/poudriere
+ CMD=jail
+ shift
+ CMD_ENV=''
+ [ -n '' ]
+ [ -n '' ]
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere'
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin'
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin POUDRIERE_VERSION=3.3.1'
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin POUDRIERE_VERSION=3.3.1 POUDRIEREPATH=/usr/local/bin/poudriere'
+ [ -n '' ]
+ [ -n '' ]
+ env
+ sed -Ee 's,^([^=]*)=(.*),\1 \2,'
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin POUDRIERE_VERSION=3.3.1 POUDRIEREPATH=/usr/local/bin/poudriere HTTP_PROXY=http://194.145.60.1:9400'
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin POUDRIERE_VERSION=3.3.1 POUDRIEREPATH=/usr/local/bin/poudriere HTTP_PROXY=http://194.145.60.1:9400 NO_PROXY=localhost .siemens.net .siemens.com .siemens.de'
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ read envvar envvalue
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin POUDRIERE_VERSION=3.3.1 POUDRIEREPATH=/usr/local/bin/poudriere HTTP_PROXY=http://194.145.60.1:9400 NO_PROXY=localhost .siemens.net .siemens.com .siemens.de FTP_PROXY=ftp://194.145.60.1:9400'
+ read envvar envvalue
+ read envvar envvalue
+ SCRIPTPATH=/usr/local/share/poudriere/jail.sh
+ CMD_ENV=' LIBEXECPREFIX=/usr/local/libexec/poudriere PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin POUDRIERE_VERSION=3.3.1 POUDRIEREPATH=/usr/local/bin/poudriere HTTP_PROXY=http://194.145.60.1:9400 NO_PROXY=localhost .siemens.net .siemens.com .siemens.de FTP_PROXY=ftp://194.145.60.1:9400 SCRIPTPATH=/usr/local/share/poudriere/jail.sh SCRIPTPREFIX=/usr/local/share/poudriere'
+ : 022
+ umask 022
+ : sh
+ exec env -i 'LIBEXECPREFIX=/usr/local/libexec/poudriere' 'PATH=/usr/local/libexec/poudriere:/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/kde4/bin:/sbin:/usr/sbin' 'POUDRIERE_VERSION=3.3.1' 'POUDRIEREPATH=/usr/local/bin/poudriere' 'HTTP_PROXY=http://194.145.60.1:9400' 'NO_PROXY=localhost' .siemens.net .siemens.com .siemens.de 'FTP_PROXY=ftp://194.145.60.1:9400' 'SCRIPTPATH=/usr/local/share/poudriere/jail.sh' 'SCRIPTPREFIX=/usr/local/share/poudriere' sh /usr/local/share/poudriere/jail.sh -
env: .siemens.net: No such file or directory

bdrewery · 2019-03-05T18:50:40Z

Ah thanks.

bdrewery · 2019-03-05T19:33:01Z

This impacts everything we pass through even $PATH if it has spaces.
I found a simple and reliable fix.

This also reduces the command line length. This works by using env -S FOO=${FOO}. Issue #669

Fixes #669

This also reduces the command line length. This works by using env -S FOO=${FOO}. Issue #669

bdrewery · 2019-03-05T20:41:46Z

Thanks for your persistence here. It's fixed now in git.

# echo $NO_PROXY
localhost .siemens.net .siemens.com .siemens.de
# src/bin/poudriere jail -l
JAILNAME          VERSION                         ARCH          METHOD                TIMESTAMP           PATH
...
# NO_PROXY="foo\j bar" src/bin/poudriere jail -l
JAILNAME          VERSION                         ARCH          METHOD                TIMESTAMP           PATH
...

Also fixed other variables with the same problems:

# PATH="/foo b\$,\\xar:/bin:/usr/bin" src/bin/poudriere api 'echo $PATH'
/root/git/poudriere3:/foo b$,\xar:/bin:/usr/bin:/sbin:/usr/sbin

koobs · 2020-12-27T05:15:00Z

I think I may be seeing this or a related issue in:

poudriere-devel-3.3.99.20200326_2

Edit: This was running a testport (sudo poudriere testport -j 12amd64 -p koobs www/py-httpbin)

I had FETCH_ARGS=-"Fpr4" set in /usr/local/etc/poudriere.d/make.conf to bypass the lack of happy-eyeballs support in libfetch which was causing fetches of DISTFILES in python ports to hang forever.

Setting FETCH_ARGS didn't seem to be working:

[12amd64-koobs] [2020-12-27_05h07m50s] [parallel_build:] Queued: 5  Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Tobuild: 5   Time: 00:00:14
        [01]: www/py-flask              | py37-Flask-1.1.2          fetch           (00:00:05 / 00:00:06)
        [02]: devel/py-gevent           | py37-gevent-20.9.0        fetch           (00:00:05 / 00:00:06)

Removing the quotes (FETCH_ARGS=-Fpr4) made it work:

[12amd64-koobs] [2020-12-27_05h11m19s] [parallel_build:] Queued: 5  Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Tobuild: 5   Time: 00:00:13
        [01]: www/py-flask              | py37-Flask-1.1.2          run-depends     (00:00:00 / 00:00:06)
        [02]: devel/py-gevent           | py37-gevent-20.9.0        build-depends   (00:00:03 / 00:00:06)

bdrewery · 2021-08-03T18:06:13Z

I think I may be seeing this or a related issue in:
poudriere-devel-3.3.99.20200326_2
Edit: This was running a testport (sudo poudriere testport -j 12amd64 -p koobs www/py-httpbin)

I had FETCH_ARGS=-"Fpr4" set in /usr/local/etc/poudriere.d/make.conf to bypass the lack of happy-eyeballs support in libfetch which was causing fetches of DISTFILES in python ports to hang forever.

This issue is about poudriere reading its own environment, not related to make.conf.

It appears that make passes quotes in variables verbatim. Don't use quotes.

koobs · 2021-08-09T12:22:35Z

Thanks Bryan

bdrewery changed the title ~~NO_PROXY cannot be passed, command fails~~ Poudriere env passing doesn't handle embedded quotes right Mar 4, 2019

bdrewery changed the title ~~Poudriere env passing doesn't handle embedded quotes right~~ Poudriere env passing doesn't handle embedded quotes with spaces right Mar 4, 2019

bdrewery closed this as completed Mar 4, 2019

bdrewery reopened this Mar 5, 2019

bdrewery added this to the 3.3.2 milestone Mar 5, 2019

bdrewery added the bug label Mar 5, 2019

bdrewery added a commit that referenced this issue Mar 5, 2019

Fix passthrough of special environment with spaces/quotes.

692f5b7

This also reduces the command line length. This works by using env -S FOO=${FOO}. Issue #669

bdrewery added a commit that referenced this issue Mar 5, 2019

poudriere: Fix all environment to be space/quote safe.

bda7c1c

Fixes #669

bdrewery added a commit that referenced this issue Mar 5, 2019

Fix passthrough of special environment with spaces/quotes.

598f9c3

This also reduces the command line length. This works by using env -S FOO=${FOO}. Issue #669

bdrewery closed this as completed in 5496a14 Mar 5, 2019

koobs reopened this Dec 27, 2020

bdrewery closed this as completed Aug 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Poudriere env passing doesn't handle embedded quotes with spaces right #669

Poudriere env passing doesn't handle embedded quotes with spaces right #669

michael-o commented Mar 4, 2019

michael-o commented Mar 4, 2019

bdrewery commented Mar 4, 2019

bdrewery commented Mar 4, 2019

bdrewery commented Mar 4, 2019

michael-o commented Mar 5, 2019

bdrewery commented Mar 5, 2019

bdrewery commented Mar 5, 2019

bdrewery commented Mar 5, 2019 •

edited

Loading

koobs commented Dec 27, 2020 •

edited

Loading

bdrewery commented Aug 3, 2021

koobs commented Aug 9, 2021

Poudriere env passing doesn't handle embedded quotes with spaces right #669

Poudriere env passing doesn't handle embedded quotes with spaces right #669

Comments

michael-o commented Mar 4, 2019

michael-o commented Mar 4, 2019

bdrewery commented Mar 4, 2019

bdrewery commented Mar 4, 2019

bdrewery commented Mar 4, 2019

michael-o commented Mar 5, 2019

bdrewery commented Mar 5, 2019

bdrewery commented Mar 5, 2019

bdrewery commented Mar 5, 2019 • edited Loading

koobs commented Dec 27, 2020 • edited Loading

bdrewery commented Aug 3, 2021

koobs commented Aug 9, 2021

bdrewery commented Mar 5, 2019 •

edited

Loading

koobs commented Dec 27, 2020 •

edited

Loading