Introducing pythonized Omega scans

[alurban]

@duncanmmacleod I've made a first pass at pythonizing the long-used Omega scan code, which is currently written Matlab. The table below points to a couple of test pages I stood up for direct comparison against the current production code. (All tests were run on GPS time 1187008882.0, the time of GW170817.)

Observatory	Test output	Production page
H1	Link	Link
L1	Link	Link

The test pages were configured to show four channels (GDS-CALIB_STRAIN, CAL-DELTAL_EXTERNAL_DQ, CAL-PCALY_RX_PD_OUT_DQ, and CAL-PCALY_TX_PD_OUT_DQ) for H1 and L1 as a proof of concept, but this can easily be extended to Virgo and KAGRA (see below).

I've tried to port over functionality that people like from the production Omega scans as much as possible. This currently includes viewing blocks of channels; buttons that toggle between types of timeseries and Q-transforms; and an about page with the runtime configuration (see "How was this page generated?" at the very bottom). All functionality related to eventgrams, including plots and SNR calculations, has yet to be implemented and may require an independent PR in GWPy.

The code is also written in serial; parallelizing it is a short-timescale development goal.

Design choices I made include:

• Color-coding timeseries plots according to the GWO standards (accomplished by using the GW_OBSERVATORY_COLORS dict from gwpy.plotter.colors)
• Color-coding JavaScript panel objects the same way using the context attribute
• Configuring channels in contextual blocks (e.g. GDS-CALIB_STRAIN in a block called Strain, all CAL channels in a block called Calibration) similar to other DetChar tools
• Toggle between plot types with obnoxiously large color-coded buttons, rather than links
• Whitening with an ASD measured on 512 seconds of data using GWPy's TimeSeries.whiten()

Note, the observatory color-coding will also handle GEO, Virgo, KAGRA, and LIGO-India.

I expect that whitening in particular might be a contentious issue. I had tried several different things and this worked best, so I now welcome comments and opinions.

Imminent development goals (possibly in future PRs):

• Implement eventgrams [now done]
• Show SNR in the table of properties for each channel [now done]
• Implement multithreading (currently only done while reading in data from frames)
• add unit tests

This pull request fixes #85.

[alurban]

@duncanmmacleod, The pipeline failure reported here seems to be a compatibility issue between gwdetchar-scattering and GWPy:

3.60s$ .travis/test-exe.sh gwdetchar-scattering --help
Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.7.14/bin/gwdetchar-scattering", line 40, in <module>
    from gwpy.utils.compat import OrderedDict
ImportError: No module named compat

The check I just added to .travis.yml, running gwdetchar-omega --help, exits with code 0.

[duncanmmacleod]

@alurban, the CI failure has been addressed in #87, please rebase your feature branch against the upstream master branch and force push to rerun the CI. Thanks.

[alurban]

@duncanmmacleod, thanks much! I'm working on another commit I want to push before lunch, so I'll rebase before pushing that and let you know when the PR is updated.

[alurban]

@duncanmmacleod, I've now reproduced all functionality from the Matlab Omega scans, including the features mentioned above and the following new bells and whistles:

• A threshold on the expected false alarm rate from white Gaussian noise for each channel, which drastically reduces the number of channels scanned per run
• Pages that update-as-you go, such that the output HTML is generated once at the very beginning, then written over at the end of every user-configured contextual block of channels, then finalized after all channels have been run
• Those channels that do not meet the false alarm rate threshold (set by the user at runtime with a default of .001 Hz) are removed from the analysis
• Config files built from the standard LIGO channel lists, which I'll post in another PR over in ligo-channel-lists

I have a few other ideas in the short term, but perfect is the enemy of good enough, so I'll attack those in future issues and pull requests. Other developers are welcome to join in the fun as well.

tl;dr: I believe this code is ready for primetime, so please go ahead and merge.