Organizing Issues

[troyraen]

Groups of tasks:

Automate code deployment and testing

• Pull resource management out of code, into config yamls. Setup Cloud Build. Setup Terraform.
• Change the testid keyword for production instances from False to None #97
• Integration Testing #82
• Use config files to name the resources used in modules/scripts #80
• Create GCP Setup Script #65

Pipeline design and content

• Put the purity filter back into the pipeline #112
• Add xmatch with Abril 2020 CV catalog #104
• Calculate DIA Object statistics  #102
• Pipeline Design #101
• Port the Salt2 Beam code into a Cloud ~Function~ Run #98
• Implement a variable stars classifier #89
• Create a galactic events filter #84
• Create a semantically compressed Pub/Sub stream #83
• Use TNS to add the IAU name and other info to the alert packets #76

Improve code stability.

• Improve Logging and System Monitoring #109
• Unit Testing #81
• Enforce linting and black formatting #75

Databases: Create object table. Optimize tables for common queries.

• Calculate DIA Object statistics  #102
• SQL queries and database structures #50

Example use cases. Tutorials.

• How do we allow users to implement a Watchlist? #57
• Create an object page on the website #55

Pipeline admin and technicalities

• Calculate cost/alert programmatically #103
• Update broker_utils dependencies #113
• Fix Codacy issues #99
• Change the testid keyword for production instances from False to None #97
• Potential issues at LSST scale #96
• Redesign night conductor #95
• Towards metadata organization: store/query/view #92
• broker_utils fails to publish an alert dict containing cutouts to Pub/Sub  #88
• Organize and track broker versions #79
• Move user tutorials and pgb-utils to a dedicated repo #78
• Refactor consumer simulator to use streaming pull #74
• Disable checking of PRs that always fails. #73
• Develop Security Model + Analysis #52
• Use multiple HTTP request threads #18

Develop products based on specific needs/requests of outside groups (users and surveys)

• Participate in ELAsTiCC #107
• Submit classifications to TNS #56

Bugs

• pgb-utils won't install in Python 3.9 environment #67
• Consumer simulator pulling fewer messages than expected #66
• Decat BigQuery INVALID_ARGUMENT #61
• Salt2 Minuit RuntimeError #46

[troyraen]

This served its purpose. The issues are now all labeled and organized in the Broker Pipeline project board.