Command-line module for gwdetchar.scattering #298

alurban · 2019-04-07T14:47:47Z

This PR implements a command-line module for scattering investigations, which can be invoked via

$ python -m gwdetchar.scattering [OPTIONS] GPSTIME

Changes include:

Move gwdetchar.scattering into a proper sub-module with its own __init__.py
Add a module gwdetchar/scattering/__main__.py, which projects scattering fringe frequencies of all the default optic motion channels
For those whose projected fringes are above a given threshold, produce plots comparing these to a Q-scan of h(t)
Include unit tests for the above
Add help text pointing to the new module from gwdetchar-scattering output
Typo fix to gwdetchar.lasso.tests

This fixes #296.

cc @duncanmmacleod, @jrsmith02, @andrew-lundgren

coveralls · 2019-04-07T15:05:28Z

Coverage increased (+0.1%) to 95.636% when pulling a5fe2fb on alurban:scattering-module into fd8de27 on gwdetchar:master.

alurban · 2019-04-08T04:54:52Z

cc @jrsmith02, @andrew-lundgren, @duncanmmacleod

For testing and benchmarking, I've run this module on a recent time with known scattering. The command line used is like so:

$ python -m gwdetchar.scattering -i L1 1238510178

This identified 1 channel whose predicted fringe frequency (4th harmonic) strays above 15 Hz, which is L1:SUS-ETMX_M0_DAMP_L_IN1_DQ:

This process completed in 18.858 seconds.

duncanmmacleod · 2019-04-09T09:41:38Z

I have no particular objections about this change, but I think having two different command-line interfaces to scattering:

python -m gwdetchar.scattering, and
gwdetchar-scattering

might be confusing. I, at least, would expect those to do the same thing, but clearly they will not. As long as this is documented well (in the --help) messages, this is probably not a problem, but I thought I would mention it.

jrsmith02 · 2019-04-09T09:42:42Z

This looks great to me, not to be greedy, but one thing we use quite frequently is the ability to overlay the fringes on the time-frequency plot. Might the user optionally choose overlay, to get one plot with both the omega scan and the predicted fringe frequencies shown together? (In this case, the blue line color should be excluded from the time series curves.) Other than that, thank you so much, I love this work and will use it often!

alurban · 2019-04-09T15:45:50Z

@duncanmmacleod, that's definitely fair, I've added short descriptions to the docstring of both utilities that informs the user what each is used for.

@jrsmith02, as discussed on slack, we'll add in the overlay feature for a future release, and go with what we already have for now.

alurban · 2019-04-09T19:18:47Z

@jrsmith02, I changed my mind and coded up the overlay plot now, example is here:

Since plotting eats up neither memory nor time in this case, I've decided to just make both plots for every channel whose scattering fringes are predicted to cross above 15 Hz.

codecov-io · 2019-04-12T16:43:13Z

Codecov Report

Merging #298 into master will increase coverage by 0.14%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #298      +/-   ##
==========================================
+ Coverage   95.53%   95.67%   +0.14%     
==========================================
  Files          21       23       +2     
  Lines        1522     1571      +49     
==========================================
+ Hits         1454     1503      +49     
  Misses         68       68

Flag	Coverage Δ
#Linux	`95.67% <100%> (+0.14%)`	⬆️
#python27	`95.35% <100%> (+0.15%)`	⬆️
#python35	`94.97% <100%> (+0.16%)`	⬆️
#python36	`95.1% <100%> (+0.16%)`	⬆️
#python37	`95.04% <100%> (+0.16%)`	⬆️

Impacted Files	Coverage Δ
gwdetchar/scattering/plot.py	`100% <100%> (ø)`
gwdetchar/scattering/core.py	`100% <100%> (ø)`
gwdetchar/scattering/__init__.py	`100% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2aed2d4...e105e47. Read the comment docs.

codecov-io · 2019-04-12T16:43:16Z

Codecov Report

Merging #298 into master will increase coverage by 0.13%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #298      +/-   ##
==========================================
+ Coverage   95.53%   95.67%   +0.13%     
==========================================
  Files          21       23       +2     
  Lines        1522     1569      +47     
==========================================
+ Hits         1454     1501      +47     
  Misses         68       68

Flag	Coverage Δ
#Linux	`95.67% <100%> (+0.13%)`	⬆️
#python27	`95.35% <100%> (+0.14%)`	⬆️
#python35	`94.84% <100%> (+0.03%)`	⬆️
#python36	`94.96% <100%> (+0.02%)`	⬆️
#python37	`94.9% <100%> (+0.03%)`	⬆️

Impacted Files	Coverage Δ
gwdetchar/scattering/plot.py	`100% <100%> (ø)`
gwdetchar/scattering/core.py	`100% <100%> (ø)`
gwdetchar/scattering/__init__.py	`100% <100%> (ø)`
gwdetchar/lasso/plot.py	`100% <0%> (ø)`	⬆️
gwdetchar/io/html.py	`98.25% <0%> (+0.01%)`	⬆️
gwdetchar/io/datafind.py	`96.43% <0%> (+0.27%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2aed2d4...cfa2c1a. Read the comment docs.

duncanmmacleod

Just a few minor things, in general this will be very useful.

Additional change: please add python -m gwdetchar.scattering --help or similar to the CI configuration.

duncanmmacleod · 2019-04-15T13:11:25Z

bin/gwdetchar-scattering

+
+This tool identifies time segments when evidence for scattering is strong,
+to compare projected fringes against spectrogram measurements for a specific
+time please use the command-line module: `python gwdetchar.scattering --help`


Suggested change

time please use the command-line module: `python gwdetchar.scattering --help`

time please use the command-line module: `python -m gwdetchar.scattering --help`

duncanmmacleod · 2019-04-15T13:13:10Z

gwdetchar/scattering/__main__.py

+        output = os.path.join(
+            args.output_dir,
+            '%s-%s-%s-{}.png' % (
+                channel.replace(':', '-'), gps, args.duration)


I think this needs to be a little more specific:

Suggested change

channel.replace(':', '-'), gps, args.duration)

channel.replace('-', '_').replace(':', '-', 1), gps, args.duration)

duncanmmacleod · 2019-04-15T13:15:40Z

gwdetchar/scattering/core.py

-    velocity.override_unit('m/s')  # just so multiplication works
+    velocity = type(series)(numpy.zeros(series.size))
+    velocity.__array_finalize__(series)
+    velocity[:] = savgol_filter(series.value, 5, 2, deriv=1)


Either use numpy.empty or

velocity = savgol_filter(series.value, 5, 2, deriv=1).view(type(series)) velocity.__array_finalize__(series)

duncanmmacleod · 2019-04-15T13:15:55Z

gwdetchar/scattering/plot.py

@@ -0,0 +1,151 @@
+# coding=utf-8
+# Copyright (C) Alex Urban (2018-)


Suggested change

# Copyright (C) Alex Urban (2018-)

# Copyright (C) Alex Urban (2018-2019)

duncanmmacleod · 2019-04-15T13:16:09Z

gwdetchar/scattering/plot.py

+
+def _format_timeseries(ax, gps, fringe, multipliers=(1, 2, 4, 8),
+                       linewidth=1, thresh=None):
+    """Helper tool to format a `~gwpy.timeseries.TimeSeries plot axis


Suggested change

"""Helper tool to format a `~gwpy.timeseries.TimeSeries plot axis

"""Helper tool to format a `~gwpy.timeseries.TimeSeries` plot axis

Added gwdetchar.scattering.plot module Added unit test for scattering plot Use a Savitzky-Golay filter to take derivatives Document separate command-line interfaces for scattering

alurban · 2019-04-15T14:31:45Z

Thanks for reviewing this @duncanmmacleod, I've addressed your comments and added a coverage test for gwdetchar.scattering.

duncanmmacleod

LGTM, thanks for addressing my comments.

alurban added feature-request scattering labels Apr 7, 2019

alurban added this to the 0.5.0 milestone Apr 7, 2019

alurban self-assigned this Apr 7, 2019

alurban changed the title ~~Command-line module for gwdetchar.scattering~~ WIP: Command-line module for gwdetchar.scattering Apr 7, 2019

alurban force-pushed the scattering-module branch 3 times, most recently from ff2c089 to 23bcc31 Compare April 7, 2019 21:43

alurban requested review from duncanmmacleod and jrsmith02 April 8, 2019 04:41

alurban force-pushed the scattering-module branch from f278f27 to 84a42ae Compare April 8, 2019 04:47

alurban changed the title ~~WIP: Command-line module for gwdetchar.scattering~~ Command-line module for gwdetchar.scattering Apr 8, 2019

alurban force-pushed the scattering-module branch from 74091e6 to d820777 Compare April 8, 2019 20:08

alurban force-pushed the scattering-module branch from 97cfd94 to 04787b0 Compare April 9, 2019 15:43

alurban force-pushed the scattering-module branch from 04787b0 to f0d1f58 Compare April 9, 2019 15:49

alurban force-pushed the scattering-module branch 2 times, most recently from b138269 to 7e7f9a1 Compare April 11, 2019 01:38

alurban mentioned this pull request Apr 11, 2019

More scattering upgrades #301

Merged

alurban force-pushed the scattering-module branch 3 times, most recently from a5fe2fb to e105e47 Compare April 12, 2019 16:12

duncanmmacleod requested changes Apr 15, 2019

View reviewed changes

alurban requested a review from duncanmmacleod April 15, 2019 13:44

Alex L. Urban and others added 2 commits April 15, 2019 08:56

Flesh out gwdetchar.scattering with its own __init__.py and __main__.py

549b376

Added gwdetchar.scattering.plot module Added unit test for scattering plot Use a Savitzky-Golay filter to take derivatives Document separate command-line interfaces for scattering

Address review comments

a963c06

alurban force-pushed the scattering-module branch 2 times, most recently from f3489bd to c437a39 Compare April 15, 2019 14:07

Add gwdetchar.scattering to coverage

f523920

alurban force-pushed the scattering-module branch from c437a39 to f523920 Compare April 15, 2019 14:29

duncanmmacleod approved these changes Apr 15, 2019

View reviewed changes

alurban merged commit e7eb207 into gwdetchar:master Apr 15, 2019

alurban deleted the scattering-module branch April 15, 2019 16:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Command-line module for gwdetchar.scattering #298

Command-line module for gwdetchar.scattering #298

alurban commented Apr 7, 2019 •

edited

Loading

coveralls commented Apr 7, 2019 •

edited

Loading

alurban commented Apr 8, 2019 •

edited

Loading

duncanmmacleod commented Apr 9, 2019

jrsmith02 commented Apr 9, 2019

alurban commented Apr 9, 2019 •

edited

Loading

alurban commented Apr 9, 2019

codecov-io commented Apr 12, 2019

codecov-io commented Apr 12, 2019 •

edited

Loading

duncanmmacleod left a comment

duncanmmacleod Apr 15, 2019

duncanmmacleod Apr 15, 2019

duncanmmacleod Apr 15, 2019

duncanmmacleod Apr 15, 2019

duncanmmacleod Apr 15, 2019

alurban commented Apr 15, 2019

duncanmmacleod left a comment

	time please use the command-line module: `python gwdetchar.scattering --help`
	time please use the command-line module: `python -m gwdetchar.scattering --help`

	channel.replace(':', '-'), gps, args.duration)
	channel.replace('-', '_').replace(':', '-', 1), gps, args.duration)

		@@ -0,0 +1,151 @@
		# coding=utf-8
		# Copyright (C) Alex Urban (2018-)

	# Copyright (C) Alex Urban (2018-)
	# Copyright (C) Alex Urban (2018-2019)

	"""Helper tool to format a `~gwpy.timeseries.TimeSeries plot axis
	"""Helper tool to format a `~gwpy.timeseries.TimeSeries` plot axis

Command-line module for gwdetchar.scattering #298

Command-line module for gwdetchar.scattering #298

Conversation

alurban commented Apr 7, 2019 • edited Loading

coveralls commented Apr 7, 2019 • edited Loading

alurban commented Apr 8, 2019 • edited Loading

duncanmmacleod commented Apr 9, 2019

jrsmith02 commented Apr 9, 2019

alurban commented Apr 9, 2019 • edited Loading

alurban commented Apr 9, 2019

codecov-io commented Apr 12, 2019

Codecov Report

codecov-io commented Apr 12, 2019 • edited Loading

Codecov Report

duncanmmacleod left a comment

Choose a reason for hiding this comment

duncanmmacleod Apr 15, 2019

Choose a reason for hiding this comment

duncanmmacleod Apr 15, 2019

Choose a reason for hiding this comment

duncanmmacleod Apr 15, 2019

Choose a reason for hiding this comment

duncanmmacleod Apr 15, 2019

Choose a reason for hiding this comment

duncanmmacleod Apr 15, 2019

Choose a reason for hiding this comment

alurban commented Apr 15, 2019

duncanmmacleod left a comment

Choose a reason for hiding this comment

alurban commented Apr 7, 2019 •

edited

Loading

coveralls commented Apr 7, 2019 •

edited

Loading

alurban commented Apr 8, 2019 •

edited

Loading

alurban commented Apr 9, 2019 •

edited

Loading

codecov-io commented Apr 12, 2019 •

edited

Loading