Simple bash command fails during build

[neon12345]

Due to a failed build I have stripped down a simple Dockerfile that should run an ls command on a bash shell to list some directories, but instead fails. I tried to alter the RUN command but nothing seems to help.

The Dockerfile contains:

FROM debian:stable

RUN ["/bin/bash", "-c", "ls -d /usr/bin/[a-Z]*"]

I run a debian stable clean install with:

ii  docker-ce                             5:18.09.3~3-0~debian-stretch      amd64        Docker: the open-source application container engine
ii  docker-ce-cli                         5:18.09.3~3-0~debian-stretch      amd64        Docker CLI: the open-source application container engine

Doing a build will have the unexpected result:

Sending build context to Docker daemon  8.704kB
Step 1/2 : FROM debian:stable
 ---> c04b519eaefa
Step 2/2 : RUN ["/bin/bash", "-c", "ls -d /usr/bin/[a-Z]*"]
 ---> Running in 4629f06f6b47
ls: cannot access '/usr/bin/[a-Z]*': No such file or directory
The command '/bin/bash -c ls -d /usr/bin/[a-Z]*' returned a non-zero code: 2

Also manually running the command on a normal shell works with:

/bin/bash -c "ls -d /usr/bin/[a-Z]*"

and fails as expected with:

/bin/sh -c "ls -d /usr/bin/[a-Z]*"

Thanks,

Alex

[seemethere]

That command doesn't seem to work at all in a debian:stable container:

❯ docker run --rm -it debian:stable /bin/bash -c "ls -d /usr/bin/[a-Z]*"
ls: cannot access '/usr/bin/[a-Z]*': No such file or directory

What exactly are you trying to do?

[neon12345]

In a regular debian bash shell the [a-Z]* expands to the directories that match the regular expression. I have this in a deb installer script that fails to install because of this not working.

[thaJeztah]

Running the same command on a debian VM gives the same error;

ls -d /usr/bin/[a-Z]*
ls: cannot access '/usr/bin/[a-Z]*': No such file or directory

The problem is that that regex is invalid (a-Z should be either a-z, A-Z or a-zA-Z). Changing it to a-zA-Z would make it work;

docker run --rm -it debian:stable /bin/bash -c "ls -d /usr/bin/[a-zA-Z]*"

/usr/bin/addpart     /usr/bin/chattr		    /usr/bin/debconf-copydb	      /usr/bin/env	  /usr/bin/hostid     /usr/bin/link		 /usr/bin/mesg	   /usr/bin/perl	/usr/bin/routel		   /usr/bin/sha256sum  /usr/bin/toe		     /usr/bin/whereis
/usr/bin/apt	     /usr/bin/chcon		    /usr/bin/debconf-escape	      /usr/bin/expand	  /usr/bin/i386       /usr/bin/linux32		 /usr/bin/mkfifo   /usr/bin/perl5.24.1	/usr/bin/rtstat		   /usr/bin/sha384sum  /usr/bin/touch		     /usr/bin/which
/usr/bin/apt-cache   /usr/bin/chfn		    /usr/bin/debconf-set-selections   /usr/bin/expiry	  /usr/bin/iconv      /usr/bin/linux64		 /usr/bin/namei    /usr/bin/pg		/usr/bin/runcon		   /usr/bin/sha512sum  /usr/bin/tput		     /usr/bin/who
/usr/bin/apt-cdrom   /usr/bin/chrt		    /usr/bin/debconf-show	      /usr/bin/expr	  /usr/bin/id	      /usr/bin/lnstat		 /usr/bin/nawk	   /usr/bin/pinky	/usr/bin/savelog	   /usr/bin/shred      /usr/bin/tr		     /usr/bin/whoami
/
....

Closing, because this is not a bug, but feel free to continue the conversation

[neon12345]

The regex is working fine for me on a debian stable install with bash. But I can live with that.

[thaJeztah]

What version of bash is that running? (so that I can try reproduce); in any case, I don't think it's a bug in the docker codebase (as it doesn't sit in between there)

[neon12345]

The package was installed on many machines with a new debian stable net install without problems:

ii  bash                                 4.4-5                          amd64        GNU Bourne Again SHell
GNU bash, Version 4.4.12(1)-release (x86_64-pc-linux-gnu)

I agree it is most probably not a code bug in docker but I would also not expect version pitfalls with something labeled "debian:stable".

[thaJeztah]

Actually, looks like this is depending on the LC_COLLATE, which by default is POSIX (and case-sensitive), but case-insensitive when using en_US;

docker run -it --rm debian:stable
root@236884b89852:/# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
root@236884b89852:/$ ls -d /usr/bin/[a-Z]*
ls: cannot access '/usr/bin/[a-Z]*': No such file or directory

Install the locales package, generate the en_US locale, and set it as the active one;

root@236884b89852:/$ apt-get update && apt-get install -y locales
root@236884b89852:/$ sed -i 's/^# *\(en_US.UTF-8\)/\1/' /etc/locale.gen
root@236884b89852:/$ locale-gen en_US.UTF-8
root@236884b89852:/$ export LC_COLLATE="en_US.UTF-8"

Matching is now case-insensitive;

root@236884b89852:/$ ls -d /usr/bin/[a-Z]*

/usr/bin/addpart	/usr/bin/cut			  /usr/bin/fallocate  /usr/bin/link		 /usr/bin/od		/usr/bin/sdiff		   /usr/bin/toe
/usr/bin/apt		/usr/bin/debconf		  /usr/bin/find       /usr/bin/linux32		 /usr/bin/pager		/usr/bin/select-editor	   /usr/bin/touch
/usr/bin/apt-cache	/usr/bin/debconf-apt-progress	  /usr/bin/flock      /usr/bin/linux64		 /usr/bin/partx		/usr/bin/sensible-browser  /usr/bin/tput

...


root@236884b89852:/$ ls -d /usr/bin/[A-Z]*
/usr/bin/b2sum			 /usr/bin/deb-systemd-helper	   /usr/bin/getent     /usr/bin/locale		  /usr/bin/partx	 /usr/bin/sdiff		    /usr/bin/tic
/usr/bin/base32			 /usr/bin/deb-systemd-invoke	   /usr/bin/getopt     /usr/bin/localedef	  /usr/bin/passwd	 /usr/bin/select-editor     /usr/bin/timeout
/usr/bin/base64			 /usr/bin/delpart		   /usr/bin/gpasswd    /usr/bin/logger		  /usr/bin/paste	 /usr/bin/sensible-browser  /usr/bin/toe
...

root@236884b89852:/$ ls -d /usr/bin/[A-z]*
/usr/bin/b2sum			 /usr/bin/deb-systemd-helper	   /usr/bin/getent     /usr/bin/locale		  /usr/bin/partx	 /usr/bin/sdiff		    /usr/bin/tic
/usr/bin/base32			 /usr/bin/deb-systemd-invoke	   /usr/bin/getopt     /usr/bin/localedef	  /usr/bin/passwd	 /usr/bin/select-editor     /usr/bin/timeout
/usr/bin/base64			 /usr/bin/delpart		   /usr/bin/gpasswd    /usr/bin/logger		  /usr/bin/paste	 /usr/bin/sensible-browser  /usr/bin/toe
/usr/bin/basename		 /usr/bin/diff			   /usr/bin/gpgv       /usr/bin/logname		  /usr/bin/pathchk	 /usr/bin/sensible-editor   /usr/bin/touch
...

[neon12345]

Ok, this is odd and definitely good to know, plus it works for "de_DE.UTF-8" too. :-/

Thanks for investigating this.