Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: NotNullViolation in nightly notification status #1653

Open
3 tasks
cris-oddball opened this issue Feb 16, 2024 · 1 comment
Open
3 tasks

BUG: NotNullViolation in nightly notification status #1653

cris-oddball opened this issue Feb 16, 2024 · 1 comment

Comments

@cris-oddball
Copy link

cris-oddball commented Feb 16, 2024

Description

While working on SAWarnings in #1132, a pre-existing but unnoticed bug was found in app/celery/reporting_tasks.py::create_nightly_notification_status for the nightly stats query.

The logs report
psycopg2.errors.NotNullViolation: null value in column "notification_count" of relation "ft_notification_status" violates not-null constraint whenever the nightly process runs.

  • Ticket is understood, and QA has been contacted (if the ticket has a QA label).

Steps to Reproduce

  1. Observe the Cloudwatch logs for any environment within an hour of midnight ET to see the following when the nightly process runs:

305245763-9a6381e8-711e-45fb-b39d-1962060834c2.png

Impact/Urgency

Unknown

Expected Behavior

  • The Cloudwatch logs do not report a NotNullViolation when the nightly notification status runs

QA Considerations

  • verify this is not seen when deployed to Perf/Staging

Additional Info & Resources

This does not need to wait until night time. The cron can be adjusted to run shortly after a deploy.

app/dao/fact_notification_status_dao.py L88

image (26).png

We have seen multiple iterations of this failure. For notification_count, It's not in the parameter list. Suspect it's being stripped out somewhere.
image (27).png

For bst_date it appears the whole row was null
image.png

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants