owsim: an LSST triggered observing over-write simulator

In June of 2018, the Large Synoptic Survey Telescope (LSST) project issued a call for whitepapers requesting input on observing strategy, together with a set of reference results for different strategies, as simulated by LSST’s opsim application. The project has previously released and supports an application for analyzing such results, MAF. Owsim is a tool for generating minor modifications to the reference results so some types of minor changes in strategy may be evaluated using the same tool (MAF) without the need to produce complete opsim simulations. The specific modification included in the initial version of owsim is for triggered follow-up of transient events, but some elements of the package have more general application.

Workflow

The basic stages in the workflow are:

1. Obtain a cable of LSST field ids and coordinates;
2. generate a list of random events on which to trigger (with coordinates and times);
3. generate a set of LSST visits to be used to follow up each event;
4. calculate or simulate conditions for each visit; and
5. insert these follow-up visits in an existing opsim output database, removing pre-existing visits that collide with the follow-up ones.

Individual steps

Getting LSST field ids and coordinates

The Makefile included in this package can retrieve field ids and coordinates from the LSST github repository and format them for use by other program in this package:

make data/munged/fieldID.txt

Generating random trigger events

The random_events.py program generates random events. It can be called thus:

python $OWSIM_DIR/python/random_events.py events.conf events.txt

The events.conf file provides most of the parameters needed for event generation; an example can be found in $OWSIM_DIR/etc/events.conf. The program will produce events.txt, a text table of events.

Note that random_events.py can be used to generate multiple episodes (instances of the survey) in a single execution.

Generating a table of LSST follow-up visits

The followup.py program generates a set of visits to follow up on a set of events. It can be called thus:

python $OWSIM_DIR/python/followup.py \
  followup.conf \
  events.txt \
  ${OWSIM_DIR}/data/munged/fieldID.txt \
  visits.txt

The followup.conf file provides most of the parameters needed for visit generation; an example can be found in $OWSIM_DIR/etc/followup.conf. events.txt is the text table generated by random_events.py. The program will produce visits.txt, a table of LSST visits.

Batching visit generation

To parallelize generation of visits, the events file may be broken up into batches of events, and events for each batch can then be generated in parallel.

The unility for generating batches can be run thus:

python $OWSIM_DIR/python/batch_eventsn.py \
  events.txt \
  10 \
  3 \
  'events_%i.txt'

This will break the events into 10 roughly equal sets, where the breaks between the latest event in one set are at least 3 days before the first event in the next one. followup.py can then be run on each events file, and the results combined:

for BATCHID in {1..10}; do
python $OWSIM_DIR/python/followup.py \
  followup.conf \
  events_${BATCHID}.txt \
  ${OWSIM_DIR}/data/munged/fieldID.txt \
  visits_${BATCHID}.txt &
done

When all of these complete, the visits files can then be combined:

cat <(head -1 visits_1.txt) <(cat visits_*.txt | sort -u | grep -v start) > visits.txt

Get conditions for each visit

The visitcond.py program calculates or simulations conditions for each visit, where "conditions" refers to parameters like seeing, sky brightness, and cloud level. It can be called thus:

python $OWSIM_DIR/python/visitcond.py visitcond.conf visits.txt conditions.txt

The visitcond.conf file provides most of the parameters needed for conditions calculation; an example can be found in $OWSIM_DIR/etc/visitcond.conf. visits.txt is the text table produced by followup.py. The program will produce conditions.txt, a text table with conditions for each visit.

Insert new visits into an existing database, replacing colliding visits

The owsched.py program reads a reference database of opsim output and a table of visits with conditions, and writes a new database that includes the new visits, removing any overlapping visits in the reference database. It can be called thus:

python $OWSIM_DIR/python/owsched.py \
  reference_opsim4_output.db \
  conditions.txt \
  overwritten.db

Where reference_opsim4_output.db is the opsim4 database into which the new visits should be inserted and conditions.txt is a text table of the visits to be insterted (with conditions).