reimplement logging without referencing file descriptors

[fwyzard]

reimplenent the verbose and debug messages without referencing /dev/stdout or
/dev/stderr, which may be unavailable (e.g. running inside a container)

[cmsbuild]

A new Pull Request was created by @fwyzard (Andrea Bocci) for branch master.

@cmsbuild, @smuzaffar, @iahmad-khan, @davidlange6 can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

external issue cms-sw/cmsdist#2587

[fwyzard]

please test

[fwyzard]

@smuzaffar , is anything holding this up ?

[smuzaffar]

nothing , just that did not find much time to go through the changes. Only way to test these is to put in DEVEL IBs and see how it goes

[cmsbuild]

Pull request #78 was updated.

external issue cms-sw/cmsdist#2587

[fwyzard]

OK, I've migrated also git-cms-merge-topic and git-cms-sparse-checkout and I've tested in a container where /dev/stderr does not work.

[davidlt]

Not having /dev/stdout and /dev/stderr is very unlikely. Which container backend doesn't bind-mount these?

[fwyzard]

They exist, but if you enter the container from a local shell (instead of e.g. over ssh) they point to devices that the local user does not have access to:

fwyzard@fool:~$ lxc exec slc6 -- /usr/bin/sudo -i -u fwyzard
fwyzard@slc6:~$ ls -l /dev/stderr 
lrwxrwxrwx 1 root root 15 Dec  1 13:26 /dev/stderr -> /proc/self/fd/2
fwyzard@slc6:~$ echo 'Hello world'
Hello world
fwyzard@slc6:~$ echo 'Hello world' > /dev/stderr 
-bash: /dev/stderr: Permission denied

[davidlt]

Could you point to bash documentation about this?
I found that on Bash 3.X (OS X) it doesn't work: ./test2.sh: line 6: exec: {verbose}: not found
I couldn't find anything on Bash manual on this.

[smuzaffar]

correct, looks like exec {verbose}> /dev/null type of commands are not working on OS X

[ktf]

too nice to be true... ;-)

[fwyzard]

from man bash and https://www.gnu.org/software/bash/manual/bash.html#Redirections

Each redirection that may be preceded by a file descriptor number may instead be preceded by a word of the form {varname}. In this case, for each redirection operator except >&- and <&-, the shell will allocate a file descriptor greater than or equal to 10 and assign it to varname. If >&- or <&- is preceded by {varname}, the value of varname defines the file descriptor to close.

The use of exec for redirections is documented in many online articles, e.g. http://www.linuxjournal.com/content/bash-redirections-using-exec .

[davidlt]

I knew that you can do that with exec using file descriptor numbers, but man pages didn't specify that {varname}, but it's is mentioned in info pages. Good to know.

[fwyzard]

Going through the history of the bash documentation, looks like this was introduced (or at least documented) in bash 4.1

[davidlt]

I guess, you are on very old LXC. This looks to fixed almost 3 years ago.

[fwyzard]

I guess, you are on very old LXC. This looks to fixed almost 3 years ago.

I'm using LXD 2.5, released on October 26th: https://github.com/lxc/lxd/releases/tag/lxd-2.5 .

[davidlt]

I am running LXC 2.0.5 (Fedora 25). Create Debian container:

[davidlt@pccms205 ~]$ sudo lxc-attach -n davidlt2
root@davidlt2:/# ls /dev/
console  core  fd  full  hugepages  initctl  log  lxc  mqueue  null  ptmx  pts  random  shm  stderr  stdin  stdout  tty  tty1  tty2  tty3  tty4  urandom  zero
root@davidlt2:/# file /dev/stderr
/dev/stderr: symbolic link to /proc/self/fd/2
root@davidlt2:/# file /dev/stdout
/dev/stdout: symbolic link to /proc/self/fd/1

[fwyzard]

As I posted above, I do see /dev/stdout (and /dev/stderr):

fwyzard@fool:~/src/bash$ lxc exec slc6 -- /usr/bin/sudo -i -u fwyzard
fwyzard@slc6:~$ file /dev/stdout
/dev/stdout: symbolic link to `/proc/self/fd/1'
fwyzard@slc6:~$ file /dev/stderr
/dev/stderr: symbolic link to `/proc/self/fd/2'

but the shell is not able to write to them

fwyzard@slc6:~$ echo 'Hello world' > /dev/stdout
-bash: /dev/stdout: Permission denied
fwyzard@slc6:~$ echo 'Hello world' > /dev/stderr
-bash: /dev/stderr: Permission denied

probably because the underlying /dev/pts is not available inside the container

fwyzard@slc6:~$ file /proc/self/fd/1
/proc/self/fd/1: broken symbolic link to `/dev/pts/2'
fwyzard@slc6:~$ file /proc/self/fd/2
/proc/self/fd/2: broken symbolic link to `/dev/pts/2'

[fwyzard]

By the way, does it make a difference if you use a non-privileged container ?

[cmsbuild]

Pull request #78 was updated.

external issue cms-sw/cmsdist#2587

[fwyzard]

I've committed a workaround for older versions of BASH, can you give it a try ?

[davidlt]

I cannot do unprivileged container. Fedora does not package cgmanager, which is gateway for cgroup management from user perspective. I would need to build missing pieces myself.

[fwyzard]

I've tried, and I can write to /dev/stdout and /dev/stderr if I log in as root in a privileged container.

As a standard user in a privileged container, or as root in an unprivileged container, I cannot.

[fwyzard]

please test, comment, or merge ?