engineering_tools

A repository of scripts, configuration useful for the PCDS team

Push updates

git push -u origin master

Add Tag

git tag -a R{tag} -m '{comment}'

git push -u origin R{tag}

Creating a new release

# Clone the source code into a new folder
git clone https://github.com/pcdshub/engineering_tools.git R{tag}
# Enter repository
cd R{Tag}
# checkout tag number
git checkout tags/R{tag}

Updating latest

# Go to latest checkout
cd engineering_tools
# Pull latest from master branch
git pull origin master

The scripts

ami_offline_psana	usage: ami_offline_psana options We will run ami_offline OPTIONS: -u user (needs to be able to log into the psananeh/feh, if not on psana already) -e EXPNUMBER -R rebinning (binned to 640x640) -n no timetool plugin
archive-status	usage: archive-status [-h] PV Return the status of the specified PV in the archiver. OPTIONS: -h, --help: Show the help message and exit.
camViewer	usage: /reg/g/pcds/engineering_tools/latest/scripts/camViewer options start the viewer for controls cameras OPTIONS: -c|--cam camera name as in camera list or gige # -m|--main bring up the edm screen -r|--reboot reboot the IOC -l|--list print list of cameras -w|--wait # (wait for # hours to ask to renew, default 2 and 12 for GIGEs) -u|--update # update rate limit (default 5) -H|--hutch: use a specific hutches camviewer config file -e|--enable disable camera ioc -d|--disable disable camera ioc -a|--acquire start acquiring images -s|--stop stop acquiring images -n|--cycle cycles acquisition (first stops then starts)
check_host	Usage: /reg/g/pcds/engineering_tools/latest/scripts/check_host HOSTNAME Display host info and run some checks.
configdb_readxtc	usage: configdb_readxtc options We will run configdb_readxtc OPTIONS: -u user (needs to be able to log into the psananeh/feh) -e expnumber
daq_control	daq_control COMMAND TARGET COMMAND : { start, stop, restart, status } TARGET : { daq, ami } COMMAND : ami TARGET : { [0], 1 }
daq_waitwin	Waits for the LCLS-I daq windows to load, then exits.
detector_totals.py	Generates a report for the detector group. Reports contains the number of events per detector type gathered from all experiments in a run period. For example, detector_totals.py --run_period 20 generates the detector total report for run 20
dev_conda	Source this to activate a pcds conda environment. By default, this activates the latest environment. Use export PCDS_CONDA_VER=VERSION before running to pick a different env. Pick up EPICS environment variable settings just in case user did not
eloggrabber	usage: eloggrabber options start the eloggrabber, by default look at current exp OPTIONS: -e pass in an experiment to look at -x instrument logbook -c controls logbook -u username
epicsArchChecker	usage: epicsArchChecker [-h] [-w] [-s] filepath Checks epicsArch files for mismatches of PVs and aliases, missing files, and unconnected PVs. positional arguments: filepath Full filepath of the file to check e.g /reg/g/pcds/dist/pds/xpp/misc/epicsArch.txt optional arguments: -h, --help show this help message and exit -w, --warnings Displays: -Pvs and Aliases duplicated. -Pvs with no alias and aliases no Pvs. -s, --status Displays Pvs not connected.
get_curr_exp	usage: get_curr_exp options OPTIONS: -l add live status -i/H information for hutch (override autodetection)
get_hutch_name	Returns the hutch name based on the host it is run on. See `get_info` for more information.
get_info	usage: get_info [-h] [--run] [--exp] [--live] [--ended] [--hutch HUTCH] [--station STATION] [--getHutch] [--gethutch] [--getstation] [--getbase] [--getinstrument] [--getcnf] [--files_for_run FILES_FOR_RUN] [--nfiles_for_run NFILES_FOR_RUN] [--setExp SETEXP] optional arguments: -h, --help show this help message and exit --run get last run --exp get experiment name --live ongoing? --ended ended --hutch HUTCH get experiment for hutch xxx --station STATION optional station for hutch with two daqs, e.g. cxi and mfx --getHutch get hutch (uppercase) --gethutch get hutch (lowercase) --getstation get hutch station (for multiple daqs) --getbase get base daq name (hutch_station if multiple daqs, otherwise hutch) --getinstrument get instrument (HUTCH_station if multiple daqs, otherwise hutch) --getcnf get cnf file name --files_for_run FILES_FOR_RUN get xtc files for run --nfiles_for_run NFILES_FOR_RUN get xtc files for run --setExp SETEXP set experiment name
get_lastRun	usage: get_lastRun options OPTIONS: -l add live status -i/H information for hutch (override autodetection)
grep_ioc	usage: grep_ioc KEYWORD [hutch] hutch can be any of: xpp, xcs, cxi, mfx, mec, xrt, aux, det, fee, hpl, icl, las, lfe, tst, thz, all If no hutch is specified, all hutches will be searched
grep_pv	GREP SEARCHES THROUGH ALL IOCs IN /reg/d/iocData/ FOR PVs THAT MATCH GIVEN KEYWORD/HANDLE. Ideally, find_pv should be used as it gives a lot more information (but can be slow sometimes)
hdf5_to_gif.py	Converts images in hdf5 files created with h5_img_collect to gifs. Specify the path with -p and and how long each frame should last (ms) with -d. Will save to cwd with as {original_filename}.gif
image-saver.py	Uses h5_img_collect to save images from a camera in an hdf5 format. Command line arguments -c, -n, -p, and -f to specify camera name, number of images, path to save hdf5 file to, and name to save hdf5 file as. Also can use -g switch to open a GUI with a button that when pressed takes images with specified parameters - can be pressed multiple times. The number of images and the label on each button can be changed within the gui.
iocmanager	iocmanager [hutch] Control status of all IOCs running in a particular hutch in an interactive GUI. Current hutch is used if not provided.
ioctool	usage: ioctool <ioc>|<pv> [option] Script that returns information about an ioc given its name or a PV it hosts default option is 'name', list of options: status : print power status of machine, try to ping interfaces name : returns the name of the ioc dir : returns the path to the directory the ioc is running from cddir :opens the directory the ioc is running from (start with "source" before calling script with this option) cfg : returns the the file name of the ioc .cfg (or st.cmd) less: opens the ioc .cfg (or st.cmd) in less cat : opens the ioc .cfg (or st.cmd) in cat data : returns the path of the appropriate iocData directory if it exists autosave : opens the most recent autosave file in less archive : opens the most recent archive file in less log : opens the most recent log file in less telnet : starts a telnet session with the ioc
ipmConfigEpics	usage: ipmConfigEpics [-b boxname] [-H hutch] [-d] [-h] [-l] -b: specify boxname to view -H: specify a hutch to use, overriding the automated selection -d: fix issues with Bld Damage (likely camera IOC w/plugins on same machine) -h: display this help text -l: list available boxnames
makepeds	usage: makepeds options Make a pedestal file for offline use OPTIONS: -u user (needs to be able to log into the psananeh/feh) -r runnumber for pedestal -e EXPNAME in case you do not want pedestals for the ongoing experiment -J make pedestals for Jungfrau (default only cspad/EPIX detectors) -j make pedestals for Jungfrau - 3 run version(default only cspad/EPIX detectors) -O make pedestals for Opals (default only cspad/EPIX detectors) -Z make pedestals for Zyla (default only cspad/EPIX detectors) -p TEXT: add to elog post -c EVTCODE X use events with eventcode X set -n # : if you have created a noise file, then write pixel mask file for pixels with noise above #sigma -N # : use this number of events (default 1000) -D : dark run for XTCAV -L : lasing off run for XTCAV -v STR: validity range (if not run#-end, e.g. 123-567 or 123-end) -l: do NOT send to batch queue -F : use the FFB (default if no experiment is passed) -g : run on an FFB batch node -f : full epix10k charge injection run -C # : if noise filecreated, write pixel mask file for pixels with noise below xxx (currently integer only...) -m # : write pixel mask file for pixels with pedestal below xxx (currently integer only...) -x # : write pixel mask file for pixels with pedestal above xxx (currently integer only...) -i start calibman. -r 0: show all darks, -r n: show runs (n-25) - 25 -d give path for alternative calibdir
makepeds_psana	usage: makepeds_psana options Make a pedestal file OPTIONS: -r runnumber for pedestal -e EXPNAME in case you do not want pedestals for the ongoing experiment -H HUTCH in case you do not pass an experiment name -Z pedestal for zyla -O make pedestals for Opals -J pedestal for jungfrau (needs first of set of 3 runs!) -D : dark run for XTCAV -L : lasing off run for XTCAV (-b specifies the number of assumed bunches, def 1) -l : donot send to batch queue -F : use FFB -f : full epix10k charge injection run -v STR: validity range (if not run#-end, e.g. 123-567 or 123-end) -N # : use this number of events (default 1000) -n # : if noise filecreated, write pixel mask file for pixels with noise above xxx (currently integer only...) -C # : if noise filecreated, write pixel mask file for pixels with noise below xxx (currently integer only...) -m # : write pixel mask file for pixels with pedestal below xxx (currently integer only...) -x # : write pixel mask file for pixels with pedestal above xxx (currently integer only...) -c EVTCODE X use events with eventcode X set -i start calibman. -r 0: show all darks, -r n: show runs (n-25) - 25 -d give path for alternative calibdir -t : test, do not deploy. -y : when specify cuts for status mask, apply those for epix100.
motor-expert-screen	usage: motor-expert-screen options MOTOR_PV_BASENAME Start an EDM for the specified motor. Attempts to choose the correct type. OPTIONS: -h shows the usage information
motor-typhos	usage: motor-typhos options MOTOR_PV_BASENAME Start a typhos screen for the specified motor. Attempts to choose the correct type. OPTIONS: -h shows the usage information
motorInfo	usage: motorInfo MOTOR_PV (motor_pv_2/autosave/archive/pmgr_diff/pmgr_save) OPT If given two motors, compare their settings If given autosave as second argument, compare the current settings to the autosaved values: differences will be printed If given archive, the archive values will be printed for the last week. If only the base PV is given, extra arguments will be needed OPTIONS: -h shows the usage information -f fields to use as a comma separated list (default: use all autosave values) -s start time for archiver info (YYYY/MM/DD HH:MM:SS) -e end time for archiver info (YYYY/MM/DD HH:MM:SS)
pcds_conda	Source this to activate a pcds conda environment. By default, this activates the latest environment. Use export PCDS_CONDA_VER=VERSION before running to pick a different env. Pick up EPICS environment variable settings just in case user did not
pkg_release	Checks out a package from the pcdshub github at a particular tag. Does not update "latest" softlinks, these are inconsistent between packages. Make sure your tag exists before running.
pmgr	pmgr [hutch] [--debug] [--applyenable] --debug : Displays the debug button, which prints out any edits made --applyenable : Displays the apply all button, which applies settings to all motors
pydev_env	Source this file to activate a development environment based on the latest shared environment and on past calls to pydev_register
pydev_register	Use this script to register development packages so that they will be available when you source pydev_env
pyps-deploy	usage: pyps-deploy [-h] -r RELEASE -c CONDA [--repo REPO] [--app-bin APP_BIN] app Sets up a pyps/apps deployment for a particular github python package. This will create an executable under .../pyps/apps/APP-NAME/RELEASE/APP-NAME and repoint the symbolic link at .../pyps/apps/APP-NAME/latest to the new release folder. positional arguments: app Name of the app to deploy optional arguments: -h, --help show this help message and exit -r RELEASE, --release RELEASE App version -c CONDA, --conda CONDA Conda environment name --repo REPO Clone this repo and mask the environment package. Use this when you have only a small change that does not need a full environment release. --app-bin APP_BIN Use in conjunction with --repo arg when the launcher is not in the bin directory
questionnaire_tools	usage: questionnaire_tools [-h] [-f FROMEXP] [-t TOEXP] [-r READEXP] [-c] [-d ADD_DEVICE] [-l] [-p PRINT_DEVICE] [--dev] [--experimentList] [--propList] optional arguments: -h, --help show this help message and exit -f FROMEXP, --fromExp FROMEXP experiment to copy from -t TOEXP, --toExp TOEXP experiment to copy to -r READEXP, --readExp READEXP experiment to read CDS tag from -c, --copy_CDS copy data from CDS tab -d ADD_DEVICE, --add_device ADD_DEVICE name of device to be added -l, --list_devices list device to be added -p PRINT_DEVICE, --print_device PRINT_DEVICE print data for device --dev connect to dev database --experimentList list of experiments --propList list of proposals
restartdaq	usage: restartdaq options OPTIONS: -w sort windows after start -p select partition (same as used last) -s silent (do not email jana)
serverStat	usage: serverStat servername options Script to check status of servers & reboot/power cycle them SIGNATURE: serverStat SERVERNAME [command] default command is 'status', list of commands: status : print power status of machine, try to ping interfaces on : power machine on off : power machine off cycle : power cycle machine, wait a few second in off state reset : reset machine (ideally try that before power cycling) console: open the ipmi console where possible expert : display info and run checks on server
startami	usage: startami options we are starting another ami session here OPTIONS: -s: stop the ami client current running on this machine -c: config file you'd like to use (i.e. cxi_test.cnf)
stopami	Kill an AMI process running in the current hutch.
stopdaq	Stop the daq in the current hutch.
takepeds	usage: takepeds Takes a run with dark images for use in pedestals, and posts to the elog.
verify-hutch	usage: verify-hutch hutch Verifies that the passed argument is a known hutch, exit 0 for success and exit 1 for failure.
wheredaq	Discover what host is running the daq in the current hutch, if any.
wherepsana	Usage: where_psana [-h] [-c CONFIG] [-d DETAIL] Checks where we have shared memory servers for psana running and could run psana jobs. Optional arguments: -h Show usage -c Pick a specific DAQ config file rather than automatically selecting current hutch's file -d Also show information about dss node mapping