Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scripts for dealing with preparation of offline gracedb upload files #4534

Merged
merged 7 commits into from
Jan 17, 2024
Merged

Scripts for dealing with preparation of offline gracedb upload files #4534

merged 7 commits into from
Jan 17, 2024

Conversation

GarethCabournDavies
Copy link
Contributor

The PR #4438 was getting too long and complicated to review, and there was an obvious split in order to make this more simple.

Here I separate off the changes for new executables being used.

These are for:

  • The creation of the XML for the specific event to be uploaded
  • The script for uploading the single event
  • A wrapper script around bayestar

The first two of these are basically a split of the existing 'upload many events from this XML' script so that the files can be prepared in advance, and then uploaded later.

The wrapper around bayestar is so that we can specify the output filename and use them in the workflow

The changes to existing files are mainly so that we can reuse existing code without repetition, and a minor change so that the XML uploads from offline match the convention used in online

@GarethCabournDavies GarethCabournDavies mentioned this pull request Oct 16, 2023
Merged
@GarethCabournDavies GarethCabournDavies self-assigned this Oct 16, 2023
@GarethCabournDavies GarethCabournDavies mentioned this pull request Oct 16, 2023
Closed
@GarethCabournDavies
Copy link
Contributor Author

See #4535 for associated workflow changes

@GarethCabournDavies
Copy link
Contributor Author

@titodalcanton note that these changes are the ones already reviewed in #4438

titodalcanton
titodalcanton reviewed Dec 20, 2023
View reviewed changes
pycbc/io/live.py Outdated Show resolved Hide resolved
@titodalcanton titodalcanton mentioned this pull request Dec 20, 2023
Merged
titodalcanton
titodalcanton reviewed Dec 22, 2023
View reviewed changes
pycbc/io/live.py Outdated Show resolved Hide resolved
titodalcanton
titodalcanton approved these changes Jan 17, 2024
View reviewed changes
Copy link
Contributor

@titodalcanton titodalcanton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No further comments.

titodalcanton
titodalcanton reviewed Jan 17, 2024
View reviewed changes
pycbc/io/live.py Outdated Show resolved Hide resolved
@titodalcanton titodalcanton enabled auto-merge (squash) January 17, 2024 13:01
@titodalcanton titodalcanton added the v23_release_branch PRs applied to the v2.3.X release branch or to be cherry-picked if merging to master label Jan 17, 2024
@titodalcanton titodalcanton merged commit 4784e77 into gwastro:master Jan 17, 2024
33 checks passed
spxiwh pushed a commit to spxiwh/pycbc that referenced this pull request Feb 2, 2024
@GarethCabournDavies @titodalcanton @spxiwh
Scripts for dealing with preparation of offline gracedb upload files (g…
c1b48f2 
…wastro#4534)

* Scripts for dealing with preparation of offline gracedb upload files

* Update bin/all_sky_search/pycbc_make_bayestar_skymap

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>

* reinstate SNR timeseries being saved into HDF

* TDC comments

* TDC comments 2

* Remove unneeded import

---------

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>
Co-authored-by: Tito Dal Canton <tito.dalcanton@ijclab.in2p3.fr>
@spxiwh spxiwh removed the v23_release_branch PRs applied to the v2.3.X release branch or to be cherry-picked if merging to master label Feb 2, 2024
spxiwh added a commit that referenced this pull request Feb 2, 2024
@spxiwh @titodalcanton
@GarethCabournDavies @tdent @maxtrevor
Additions for v2.3.4 release (#4616)
1c1a91a 
* Fix error querying Virgo frame type (#4523)

* Fix error querying Virgo frame type

* Typo

* Typo

* Implement Ian's suggestion

* Make it work

* Use an actual DeprecationWarning

* Scripts for dealing with preparation of offline gracedb upload files (#4534)

* Scripts for dealing with preparation of offline gracedb upload files

* Update bin/all_sky_search/pycbc_make_bayestar_skymap

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>

* reinstate SNR timeseries being saved into HDF

* TDC comments

* TDC comments 2

* Remove unneeded import

---------

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>
Co-authored-by: Tito Dal Canton <tito.dalcanton@ijclab.in2p3.fr>

* Use vetoed times followup in example (#4597)

* Use vetoed times followup in example

* Fix statistic files double-definition

* Rationalize some calls to waveform properties (#4540)

* rationalize some calls to waveform properties

* Only allow explictly listed names

* don't try to change function name

* g

g

* Add the ability to choose how to sort injections in minifollowups (#4602)

* Add the ability to choose how to sort injections in minifollowups

* Minor cleanup

* Minor refactor

* Tom's comments

* Update example search workflow

* Further thoughts and suggestions

* Tom's suggestion

* Fix bug and Tom's suggestion

* Standardise logging: bin/all_sky_search and pycbc/events (#4576)

* add level to default logging

* Decrease logging level for debug information in coinc_findtrigs

* decrease logging level for some information in sngls_findtrigs

* add named logger in pycbc/events/cuts.py

* add named logger in pycbc/events/significance.py

* REVERT ME: an example of usage

* CC fix

* Revert "REVERT ME: an example of usage"

This reverts commit eb647e0.

* Use init_logging throughout all_sky_search

* give pycbc/events modules named loggers

* missed the fact that the level was set to warn, not info, as default before

* CC

* missed an import

* start moving verbose argparse into common facility

* add common_options to all_sky_search executables

* set --verbose default value to zero

* pycbc not imported in some codes

* CC

* rename function to be more decriptive

* O4b idq offline update (#4603)

* Start offline dq workflow rework

* Modified stat

* Rewrote dq workflow

* Added note for future suggested changes to dq workflow

* Finished first draft of offline workflow

* Fixed a comment

* Removed unneeded argument to make dq workflow

* Made bin templates executable

* WOrking version of offline dq workflow

* Reverting non-functional changes in stat.py

* linting

* Reverted more non-functional changes

* Reverted low-latency related features

* Rearrange imports

* Addressed Ian's comments

* Simplified masking in dq workflow

* Modify dq workflow to avoid using numpy arrays

* Use vetoed times followup in example (#4597)

* Use vetoed times followup in example

* Fix statistic files double-definition

* Addressed more comments from Tom

* Addressed Gareth's comments on parser arguments

* Don't allow dq stat to uprank candidates

* Modify dq trigger rate plots to not use dq trigger rates below 1

* Address Tom's most recent comment

* Readded [Min 1] to dq plot's y-axis label

* Pass bank plotting tags to dq template bin plots

* Update bin/plotting/pycbc_plot_dq_flag_likelihood

Co-authored-by: Thomas Dent <thomas.dent@usc.es>

* Update bin/workflows/pycbc_make_offline_search_workflow

Co-authored-by: Thomas Dent <thomas.dent@usc.es>

* Use abs() correctly

* Break up 2 try/ecept cases

---------

Co-authored-by: Gareth S Cabourn Davies <gareth.cabourndavies@ligo.org>
Co-authored-by: Thomas Dent <thomas.dent@usc.es>

* Bumping version number

---------

Co-authored-by: Tito Dal Canton <tito.dalcanton@ijclab.in2p3.fr>
Co-authored-by: Gareth S Cabourn Davies <gareth.cabourndavies@ligo.org>
Co-authored-by: Tito Dal Canton <tito@dalcanton.it>
Co-authored-by: Thomas Dent <thomas.dent@usc.es>
Co-authored-by: maxtrevor <65971534+maxtrevor@users.noreply.github.com>
bhooshan-gadre pushed a commit to bhooshan-gadre/pycbc that referenced this pull request Mar 4, 2024
@GarethCabournDavies @titodalcanton @bhooshan-gadre
Scripts for dealing with preparation of offline gracedb upload files (g…
0d45c08 
…wastro#4534)

* Scripts for dealing with preparation of offline gracedb upload files

* Update bin/all_sky_search/pycbc_make_bayestar_skymap

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>

* reinstate SNR timeseries being saved into HDF

* TDC comments

* TDC comments 2

* Remove unneeded import

---------

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>
Co-authored-by: Tito Dal Canton <tito.dalcanton@ijclab.in2p3.fr>
lpathak97 pushed a commit to lpathak97/pycbc that referenced this pull request Mar 13, 2024
@GarethCabournDavies @titodalcanton @lpathak97
Scripts for dealing with preparation of offline gracedb upload files (g…
525c086 
…wastro#4534)

* Scripts for dealing with preparation of offline gracedb upload files

* Update bin/all_sky_search/pycbc_make_bayestar_skymap

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>

* reinstate SNR timeseries being saved into HDF

* TDC comments

* TDC comments 2

* Remove unneeded import

---------

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>
Co-authored-by: Tito Dal Canton <tito.dalcanton@ijclab.in2p3.fr>
acorreia61201 pushed a commit to acorreia61201/pycbc that referenced this pull request Apr 4, 2024
@GarethCabournDavies @titodalcanton @acorreia61201
Scripts for dealing with preparation of offline gracedb upload files (g…
21e13df 
…wastro#4534)

* Scripts for dealing with preparation of offline gracedb upload files

* Update bin/all_sky_search/pycbc_make_bayestar_skymap

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>

* reinstate SNR timeseries being saved into HDF

* TDC comments

* TDC comments 2

* Remove unneeded import

---------

Co-authored-by: Tito Dal Canton <tito@dalcanton.it>
Co-authored-by: Tito Dal Canton <tito.dalcanton@ijclab.in2p3.fr>
@GarethCabournDavies GarethCabournDavies deleted the offline_gracedb_prep_scripts branch April 8, 2024 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@titodalcanton titodalcanton titodalcanton approved these changes

Assignees

@GarethCabournDavies GarethCabournDavies

Labels
offline search
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@GarethCabournDavies @titodalcanton @spxiwh