Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix missing ds error #34562

Closed
wants to merge 4 commits into from
Closed

Fix missing ds error #34562

wants to merge 4 commits into from

Conversation

Charl1996
Copy link
Contributor

@Charl1996 Charl1996 commented May 7, 2024

Technical Summary

Ticket

An error was picked up by sentry in which a case made changes to a UCR that does not exist. This PR is simply to check whether the table exists before making the change. If the table does not exist, no work is necessary.

That said, I don't think this addresses the real problem, i.e. why the case change is propagated for this config which is supposedly deleted. This issue is to be investigated further, but this PR change will at least silence the error.

Feature Flag

UCRs

Safety Assurance

Safety story

--

Automated test coverage

No tests, don't think it's necessary for such a simple change.

QA Plan

No QA

Rollback instructions

  • This PR can be reverted after deploy with no further considerations

Labels & Review

  • Risk label is set correctly
  • The set of people pinged as reviewers is appropriate for the level of risk of the change

@Charl1996 Charl1996 added product/feature-flag Change will only affect users who have a specific feature flag enabled product/invisible Change has no end-user visible impact labels May 7, 2024
@Charl1996 Charl1996 marked this pull request as ready for review May 7, 2024 10:59
@Charl1996 Charl1996 requested a review from snopoke May 7, 2024 11:04
Copy link
Contributor

@gherceg gherceg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this was mentioned in slack already, but just want to confirm that while this is the short term solution, there is also an ongoing effort to understand how we are getting to the point where we are trying to update rows on a deleted UCR?

@Charl1996
Copy link
Contributor Author

I think this was mentioned in slack already, but just want to confirm that while this is the short term solution, there is also an ongoing effort to understand how we are getting to the point where we are trying to update rows on a deleted UCR?

Correct.

@gherceg
Copy link
Contributor

gherceg commented May 7, 2024

  FAIL: corehq.apps.userreports.tests.test_save_errors:SaveErrorsTest.test_raise_error_for_missing_table
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/mnt/commcare-hq-ro/corehq/apps/userreports/tests/test_save_errors.py", line 54, in test_raise_error_for_missing_table
      self.adapter.best_effort_save(doc)
  AssertionError: TableNotFoundWarning not raised

Test failure looks related.

@@ -149,8 +149,6 @@ def save_rows(self, rows, use_shard_col=True):
Saves rows to a data source after deleting the old rows
:param use_shard_col: use shard column along with doc id for searching rows to delete/update
"""
if not self.table_exists:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out we expect it to fail if this condition is not met, so removing this piece.

Copy link
Contributor

@mkangia mkangia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Charl1996

4 commits and 1 file changed caught me by surprise, can we please clean up to just keep the relevant commit(s)?

Also, if we are firing a query to check for table each time we delete, should we rather delete happen and catch the exception instead?

@@ -208,6 +208,9 @@ def bulk_save(self, docs):
self.save_rows(rows)

def bulk_delete(self, docs, use_shard_col=True):
if not self.table_exists:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this fire a new query or it already is aware?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question.

The table_exists method invokes the has_table method on the engine object. The engine is created here. Looking manually through the sqlalchemy code it looks like it uses cached info.

I'm just wondering now, if it's using cached info, what's the chance of it getting out of date? I'll try and see if I can find out more about how this works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkangia Ok, I can't find good documentation on how the caching updates work, so the safest course is probably to just catch the exception, since we're making a DB request in any case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a draft PR

@Charl1996
Copy link
Contributor Author

Closed in favour of #34595

@Charl1996 Charl1996 closed this May 10, 2024
@mkangia mkangia deleted the fix-missing-ds-error branch May 10, 2024 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product/feature-flag Change will only affect users who have a specific feature flag enabled product/invisible Change has no end-user visible impact
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants