bdutil doesn't work with Ubuntu images

[dennishuo]

Reported on StackOverflow, I'm able to reproduce the problem at the latest 1.2.1 release. There appear to be multiple issues leading to the incompatibility.

[dennishuo]

86c7b09 fixes the missing /usr/local/share/google directory; at head we see:

dhuo-ubuntu-m:  No boot process configuration tool found.

This means "which insserv" and "which chkconfig" both failed running as root. Various Ubuntu forum threads indicate insserv residing in /usr/lib/insserv/insserv, and needing to symlink into /sbin before usage:

sudo ln -s /usr/lib/insserv/insserv /sbin/insserv

Applying this fix to configure_startup_processes.sh appears to allow the deployment to proceed at least to the NFS-setup steps.

[dennishuo]

The second failure is on setup_master_nfs.sh with:

dhuo-ubuntu2-m: ./hadoop-env-setup.sh: line 531: /proc/fs/nfsd/nfsv4gracetime: No such file or directory
dhuo-ubuntu2-m: 'overwrite_file_with_strings /proc/fs/nfsd/nfsv4gracetime 10' attempt 1 failed! Sleeping 10.
dhuo-ubuntu2-m: ./hadoop-env-setup.sh: line 531: /proc/fs/nfsd/nfsv4gracetime: No such file or directory
dhuo-ubuntu2-m: 'overwrite_file_with_strings /proc/fs/nfsd/nfsv4gracetime 10' attempt 2 failed! Sleeping 10.
dhuo-ubuntu2-m: ./hadoop-env-setup.sh: line 531: /proc/fs/nfsd/nfsv4gracetime: No such file or directory
dhuo-ubuntu2-m: 'overwrite_file_with_strings /proc/fs/nfsd/nfsv4gracetime 10' attempt 3 failed! Sleeping 10.
dhuo-ubuntu2-m: ./hadoop-env-setup.sh: line 531: /proc/fs/nfsd/nfsv4gracetime: No such file or directory
dhuo-ubuntu2-m: 'overwrite_file_with_strings /proc/fs/nfsd/nfsv4gracetime 10' attempt 4 failed! Sleeping 10.
dhuo-ubuntu2-m: ./hadoop-env-setup.sh: line 531: /proc/fs/nfsd/nfsv4gracetime: No such file or directory

This is a neat quirk of Ubuntu; whereas on plain Debian, running service nfs-kernel-server stop still retains the /proc/fs/nfsd files for edit, on Ubuntu the files disappear when the service is stopped, and are no longer editable.

# service nfs-kernel-server stop
[ ok ] Stopping NFS kernel daemon: mountd nfsd.
[ ok ] Unexporting directories for NFS kernel daemon....
# ls -l /proc/fs/nfsd
total 0
-r--r--r-- 1 root root 0 May 15 00:42 export_features
-r--r--r-- 1 root root 0 May 15 00:42 exports
-rw------- 1 root root 0 May 15 00:42 filehandle
-rw-r--r-- 1 root root 0 May 15 00:42 max_block_size
-rw------- 1 root root 0 May 15 00:42 nfsv4gracetime
-rw------- 1 root root 0 May 15 00:42 nfsv4leasetime
-rw------- 1 root root 0 May 15 00:42 nfsv4recoverydir
-r--r--r-- 1 root root 0 May 15 00:42 pool_stats
-rw------- 1 root root 0 May 15 00:42 pool_threads
-rw-r--r-- 1 root root 0 May 15 00:42 portlist
-r--r--r-- 1 root root 0 May 15 00:42 reply_cache_stats
-r--r--r-- 1 root root 0 May 15 00:42 supported_krb5_enctypes
-rw------- 1 root root 0 May 15 00:42 threads
-rw------- 1 root root 0 May 15 00:42 unlock_filesystem
-rw------- 1 root root 0 May 15 00:42 unlock_ip
-rw------- 1 root root 0 May 15 00:42 versions

Compared to Ubuntu:

root@dhuo-ubuntu2-m:/home/dhuo# ls -l /proc/fs/nfsd
total 0
-r--r--r-- 1 root root 0 May 15 01:06 export_features
-r--r--r-- 1 root root 0 May 15 01:06 exports
-rw------- 1 root root 0 May 15 01:06 filehandle
-rw-r--r-- 1 root root 0 May 15 01:06 max_block_size
-rw------- 1 root root 0 May 15 01:06 nfsv4gracetime
-rw------- 1 root root 0 May 15 01:06 nfsv4leasetime
-rw------- 1 root root 0 May 15 01:06 nfsv4recoverydir
-r--r--r-- 1 root root 0 May 15 01:06 pool_stats
-rw------- 1 root root 0 May 15 01:06 pool_threads
-rw-r--r-- 1 root root 0 May 15 01:06 portlist
-r--r--r-- 1 root root 0 May 15 01:06 reply_cache_stats
-r--r--r-- 1 root root 0 May 15 01:06 supported_krb5_enctypes
-rw------- 1 root root 0 May 15 01:06 threads
-rw------- 1 root root 0 May 15 01:06 unlock_filesystem
-rw------- 1 root root 0 May 15 01:06 unlock_ip
-rw------- 1 root root 0 May 15 01:06 versions
root@dhuo-ubuntu2-m:/home/dhuo# service nfs-kernel-server stop
 * Stopping NFS kernel daemon                                                                                                                                                                                                         [ OK ] 
 * Unexporting directories for NFS kernel daemon...                                                                                                                                                                                   [ OK ] 
root@dhuo-ubuntu2-m:/home/dhuo# ls -l /proc/fs/nfsd
total 0
root@dhuo-ubuntu2-m:/home/dhuo#

[dennishuo]

It appears there's no easy way to modify nfsv4gracetime for Ubuntu, aside from editing /etc/init.d/nfs-kernel-server, as described in this serverfault.com post and in our case the edited gracetime wouldn't have survived across reboots anyhow. Since it's just a best-effort setting to speed up boot time, it's probably not appropriate to try to deeply modify /etc/init.d/nfs-kernel-server and instead just print a warning if the file is non-existent for editing.

[dennishuo]

For posterity, the root cause of the existence/nonexistence of nfsv4gracetime is that in Debian, /etc/init.d/nfs-kernel-server's "stop" command ends like this:

    if mountpoint -q $PROCNFSD_MOUNTPOINT
    then
        $PREFIX/sbin/exportfs -f
    fi
    ;;

Compared to Ubuntu's ending like this:

    if mountpoint -q $PROCNFSD_MOUNTPOINT
    then
        $PREFIX/sbin/exportfs -f
        umount $PROCNFSD_MOUNTPOINT
    fi
    ;;

So the extra umount $PROCNFSD_MOUNTPOINT seems to unfortunately clean up the procfs bits.

[dennishuo]

The fix has been tested successfully in several setups now and pull request has been sent: #30

[dennishuo]

Merged into master: 84fdd46