tips and tricks

nadyawilliams edited this page May 14, 2014 · 1 revision

Table of Contents

Q: How do I get some needed RPM package and install it onto my cluster without reinstalling all the nodes?

A: based on a post from Phil P. using Atlas and HPL as examples:

(atlas and HPL are in the rocks HPC roll:

(the above will build and install atlas and hpl on your front end) (the following will get it everywhere on your cluster)

 cp ../../RPMS/<arch>/hpl*.rpm ../../RPMS/<arch>/atlas*.rpm /home/install/contrib/4.1/x86_64/RPMS
 cd /home/install
 rocks-dist dist
 cluster-fork "/boot/kickstart/cluster-kickstart; exit"

(in this particular case since it is an update of RPMS only, you could replace the last line with:

 cluster-fork 'rpm -Uvh --force /home/install/rocks-dist/lan/x86_64/RedHat/RPMS/hpl*.rpm /home/install/rocks-dist/lan/x86_64/RedHat/RPMS/atlas*.rpm'

You'd still need to modify your extend-compute.xml to make sure that your compute nodes get these RPM packages in case they are reinstalled during a hard boot or a manual shoot-node.

Q. How do I increase NFS efficiency on the head node and for compute nodes?

A. (taken from Rocks-Discuss threads and other sources):

a. Add 'RPCNFSDCOUNT=64' in /etc/sysconfig/nfs

b. Remove the --ghost option from all entries in /etc/auto.master

c. Modify /etc/auto.master on line for /home auto.home. Remove --timeout 1200, added rsize=32768,wsize=32768,hard,intr.

d. Reload nfs config info (service autofs reload)

Q. What files should survive a frontend re-install/upgrade?

A. Recent versions of Rocks allow you to create a Restore Roll, which you then burn to a CD and supply during a re-install or upgrade of your frontend. Take a minute to read this page, if you haven't already.

Basically, you edit the file /export/site-roll/rocks/src/roll/restore/src/system-files/, and add your local files. Then

cd /export/site-roll/rocks/src/roll/restore
make roll

and burn the resulting *.iso image to a CD. Don't forget to label it, especially with the date!

Ok, so which files should you include? Here's my list, hopefully others will add stuff that I've forgotten. I've included some comments to help jog your memory. All of my users use tcsh instead of bash, so this is pretty tcsh-centric.


NAME	= restore-system-files

# these are the files that will be restored when this roll is supplied
# during installation
FILES	= /etc/passwd /etc/shadow /etc/gshadow /etc/group \
	  /etc/exports /etc/auto.home /etc/motd
# Save yourself
FILES  += /export/site-roll/rocks/src/roll/restore/src/system-files/
# Save contributed RPMs.  WARNING: if they include a kernel RPM it may be 
# installed instead of the kernel RPM you expect.  This was true for a 
# CentOSplus kernel I needed only for a NAS node.  So this will happen for a 
# re-install, but might not happen for an upgrade (it should take the newest
# kernel RPM, right?).
FILES  += /home/install/contrib/5.0/x86_64/RPMS/*
FILES  += /home/install/contrib/5.0/x86_64/SRPMS/*
# X-windows settings
FILES  += /etc/X11/xorg.conf /etc/skel/.Xdefaults
# Customizations for the shells (csh, tcsh, bash)
FILES  += /etc/skel/.*
FILES  += /etc/csh.cshrc 
# Profile files, for each application (basically $path additions and setenv calls)
FILES  += /etc/profile.d/less.csh /etc/profile.d/mpich.csh \
	  /etc/profile.d/ncarg.csh /etc/profile.d/pave.csh \
	  /etc/profile.d/rip.csh /etc/profile.d/smoke.csh \
# System-wide mail file
FILES  += /etc/postfix/ /etc/postfix/transport /etc/aliases
# Local mail and crontabs for all users
FILES  += /var/spool/cron/* /var/spool/mail/*
# Samba 
FILES  += /etc/samba/smb.conf /etc/samba/passdb.tdb /etc/krb5.conf 
# Any customized autofs maps
FILES  += /etc/auto.mm5 /etc/
# I use the PGI compilers, and distribute this file via kickstart too
FILES  += /etc/ \
# PBS-related stuff
FILES  += /opt/torque/pbs_environment

I found that after a re-install of Rocks 5.0.3, not all the files I asked for were restored! I suspect it has to do with the order things get installed. For example, /etc/samba/smb.conf was not restored, and none of my additions to /etc/profile.d/ were installed.

After a re-install/upgrade, you can find a copy of all of the files in the restore roll in /upgrade (for example, /upgrade/etc/samba/smb.conf). So you can copy them (for a re-install) or integrate the changes (for an upgrade) into the new versions of those same files. For example:

# cp /upgrade/etc/profile.d/* /etc/profile.d

Q: What can I do about a fuzzy screen following installation?

A. This is due to an unsupported video card. Boot into rescue mode and enter the following line at the command prompt

# system-config-display --reconfig --set-driver=vesa --set-depth=24 --set-resolution=1024x768 --set-videoram=0

reboot and this should correct the problem, albeit with limited graphics. I ran into this using a DELL 6450 server with an on board graphics card

(This was culled from the CENTOS forum. here. posted by user AlanJBarlett.

Bart 17:14, 8 September 2008 (EDT)

Q: How do I use a USB attached DVD drive to install the frontend?

A. Append driverload=usb-storage to the build command when you first boot the frontend.

category:Cluster setup