WARN running lsof (cron job)

[christrotter]

This doesn't seem to affect functionality (has been happening for a while), but our 1.7.1 hub is showing this in the grid extras logfile:

31 Mar 2015 15:35:07 [main] WARN  seleniumgridextras.ExecuteCommand  - Problems in running lsof -sTCP:LISTEN -i TCP:3000
java.io.IOException: Cannot run program "lsof": error=2, No such file or directory

Cronjob:

*/5 * * * * bash -i -c 'cd /srv/SeleniumGridExtras; java -jar SeleniumGridExtras.jar' >> /srv/SeleniumGridExtras/log/log.out 2>&1

Happy to help diagnose if it's deemed important. To me, looks like the lsof command has a typo? "-sTCP" should be "-s TCP" ?

[smccarthy]

@christrotter This seems like the correct command. What is the operating system you are using? Does it have lsof installed? I verified that command works on my Ubuntu 14.04 LTS and my OSX 10.10.4 (Yosemeti).

Btw, Here is where that command comes from :
https://github.com/groupon/Selenium-Grid-Extras/blob/master/SeleniumGridExtras/src/main/java/com/groupon/seleniumgridextras/PortChecker.java#L122

[christrotter]

It's a CentOS 6.5 VM, and yeah, it has lsof installed. Maybe Java on CentOS/RHEL doesn't like the missing space between 's' and 'TCP'?

If I run the commands manually (first no space, then with space) I get the same results successfully.

[root@serverName ~]# lsof -sTCP:LISTEN -i TCP
COMMAND    PID    USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
rpcbind   1291     rpc    8u  IPv4  11698      0t0  TCP *:sunrpc (LISTEN)
rpcbind   1291     rpc   11u  IPv6  11703      0t0  TCP *:sunrpc (LISTEN)
rpc.statd 1311 rpcuser    8u  IPv4  11818      0t0  TCP *:50506 (LISTEN)
rpc.statd 1311 rpcuser   10u  IPv6  11826      0t0  TCP *:46992 (LISTEN)
cupsd     1387    root    6u  IPv6  12056      0t0  TCP localhost:ipp (LISTEN)
cupsd     1387    root    7u  IPv4  12057      0t0  TCP localhost:ipp (LISTEN)
sshd      1521    root    3u  IPv4  12589      0t0  TCP *:ssh (LISTEN)
sshd      1521    root    4u  IPv6  12598      0t0  TCP *:ssh (LISTEN)
master    1598    root   12u  IPv4  12797      0t0  TCP localhost:smtp (LISTEN)
master    1598    root   13u  IPv6  12799      0t0  TCP localhost:smtp (LISTEN)
java      1765    root   10u  IPv6  15649      0t0  TCP *:hbci (LISTEN)
java      1778    root   18u  IPv6  15704      0t0  TCP *:krb524 (LISTEN)
xinetd    2057    root    5u  IPv6  18301      0t0  TCP *:6556 (LISTEN)
[root@serverName ~]# lsof -s TCP:LISTEN -i TCP
COMMAND    PID    USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
rpcbind   1291     rpc    8u  IPv4  11698      0t0  TCP *:sunrpc (LISTEN)
rpcbind   1291     rpc   11u  IPv6  11703      0t0  TCP *:sunrpc (LISTEN)
rpc.statd 1311 rpcuser    8u  IPv4  11818      0t0  TCP *:50506 (LISTEN)
rpc.statd 1311 rpcuser   10u  IPv6  11826      0t0  TCP *:46992 (LISTEN)
cupsd     1387    root    6u  IPv6  12056      0t0  TCP localhost:ipp (LISTEN)
cupsd     1387    root    7u  IPv4  12057      0t0  TCP localhost:ipp (LISTEN)
sshd      1521    root    3u  IPv4  12589      0t0  TCP *:ssh (LISTEN)
sshd      1521    root    4u  IPv6  12598      0t0  TCP *:ssh (LISTEN)
master    1598    root   12u  IPv4  12797      0t0  TCP localhost:smtp (LISTEN)
master    1598    root   13u  IPv6  12799      0t0  TCP localhost:smtp (LISTEN)
java      1765    root   10u  IPv6  15649      0t0  TCP *:hbci (LISTEN)
java      1778    root   18u  IPv6  15704      0t0  TCP *:krb524 (LISTEN)
xinetd    2057    root    5u  IPv6  18301      0t0  TCP *:6556 (LISTEN)

So perhaps whatever is wrapping that command/returned info is angry about a missing space when living on CentOS? Dunno!

[smccarthy]

Thanks for the quick response. I can try and debug this some more later.

[christrotter]

No worries. It has yet to affect functionality.

[smccarthy]

I did a quick install of CentOS 6.5 and downloaded java and Selenium Grid Extras 1.7.1 and I got the same error. Are you running Selenium-Grid-Extras as root? From this post , http://www.linuxquestions.org/questions/linux-newbie-8/lsof-command-not-found-centos-906882/ , someone says lsof is only available to root and if you want other people to use it, to use :
/usr/sbin/lsof /path/to/some/file

If SeleniumGridExtras is not ran as root, can you try to log in with that other user and use the lsof command?

Thank you.

[christrotter]

It's running as root. If I'm logged on as root and do a crontab -e, that's where SGE is executed. (as per the documentation, IIRC)

For giggles, I created a user 'selenium' and tested - got denials.

[selenium@serverName root]$ lsof -sTCP:LISTEN -i TCP
[selenium@serverName root]$ lsof -s TCP:LISTEN -i TCP
[selenium@serverName root]$ lsof
COMMAND     PID      USER   FD      TYPE DEVICE SIZE/OFF     NODE NAME
init          1      root  cwd   unknown                          /proc/1/cwd (readlink: Permission denied)
init          1      root  rtd   unknown                          /proc/1/root (readlink: Permission denied)
init          1      root  txt   unknown                          /proc/1/exe (readlink: Permission denied)
init          1      root NOFD                                    /proc/1/fd (opendir: Permission denied)
kthreadd      2      root  cwd   unknown                          /proc/2/cwd (readlink: Permission denied)
kthreadd      2      root  rtd   unknown                          /proc/2/root (readlink: Permission denied)
kthreadd      2      root  txt   unknown                          /proc/2/exe (readlink: Permission denied)
kthreadd      2      root NOFD                                    /proc/2/fd (opendir: Permission denied)
migration     3      root  cwd   unknown                          /proc/3/cwd (readlink: Permission denied)
migration     3      root  rtd   unknown                          /proc/3/root (readlink: Permission denied)
<snip>

I ran lsof looking for Selenium:

[root@serverName ~]# lsof | grep Selenium
bash       1754      root  cwd       DIR              253,0     4096     654083 /srv/SeleniumGridExtras
bash       1754      root    1w      REG              253,0 27415094     654086 /srv/SeleniumGridExtras/log/log.out
bash       1754      root    2w      REG              253,0 27415094     654086 /srv/SeleniumGridExtras/log/log.out
bash       1754      root  255w      REG              253,0 27415094     654086 /srv/SeleniumGridExtras/log/log.out
java       1765      root  cwd       DIR              253,0     4096     654083 /srv/SeleniumGridExtras
java       1765      root  mem       REG              253,0 54780777     654154 /srv/SeleniumGridExtras/SeleniumGridExtras-1.7.1-SNAPSHOT-jar-with-dependencies.jar
java       1765      root    1w      REG              253,0 27415094     654086 /srv/SeleniumGridExtras/log/log.out
java       1765      root    2w      REG              253,0 27415094     654086 /srv/SeleniumGridExtras/log/log.out
java       1765      root    6r      REG              253,0 54780777     654154 /srv/SeleniumGridExtras/SeleniumGridExtras-1.7.1-SNAPSHOT-jar-with-dependencies.jar
java       1765      root    7w      REG              253,0    30109     655377 /srv/SeleniumGridExtras/log/grid_extras.log
java       1778      root  cwd       DIR              253,0     4096     654083 /srv/SeleniumGridExtras
java       1778      root  mem       REG              253,0 54780777     654154 /srv/SeleniumGridExtras/SeleniumGridExtras-1.7.1-SNAPSHOT-jar-with-dependencies.jar
java       1778      root    6r      REG              253,0 54780777     654154 /srv/SeleniumGridExtras/SeleniumGridExtras-1.7.1-SNAPSHOT-jar-with-dependencies.jar
java       1778      root    8wW     REG              253,0        0     654242 /srv/SeleniumGridExtras/log/grid_hub.log.lck
java       1778      root   10w      REG              253,0 26957375     654098 /srv/SeleniumGridExtras/log/grid_hub.log
java       1778      root   11w      REG              253,0  1780143     655730 /srv/SeleniumGridExtras/log/grid_extras.log.2015-07-13

[smccarthy]

after I did a sudo yum instal lsof , I did not see that WARN in my grid_extras.log .
When I do yum list | grep lsof , I see:
lsof.x86_64 4.82-4.e16 @base

I did a sudo yum erase lsof , than reran the selenium grid extras jar and I see the exact same warning message as you. I will try to run it the exact same way as you (inside of that crontab).

I think I figured it out. I guess crontab doesn't keep the PATH (not sure how it finds java). Either way, I added the PATH to the "crontab -e", like :
http://stackoverflow.com/questions/2388087/how-to-get-cron-to-call-in-the-correct-paths
http://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work

Please try this and let us know.

[smccarthy]

Closing, since no response back after > 6 months. If still an issue, please respond back or open a new issue. Thanks.