Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add assertMaxWriteQueries to test max queries against a table #8446

Merged
merged 13 commits into from
May 23, 2018

Conversation

HazAT
Copy link
Member

@HazAT HazAT commented May 15, 2018

This PR adds a context method for unit tests to check against write queries on tables called: assertMaxWriteQueries

Usage:

class JavascriptIntegrationTest(TestCase):
    def test_adds_contexts_without_device(self):
        with self.assertWriteQueries({
            'sentry_projectoptions': 4,
            'sentry_project': 2,
            'sentry_grouptagkey': 12
        }):

This checks for UPDATE INSERT DELETE queries and counts how often they will be executed in a single unit test.
While this this approach is very basic, it would have caught the outlier in queries responsible for the latest outage.

See example:
If we re-add the line that was responsible for the outage again:

def get_option(self, *args, **kwargs):
        from sentry.models import ProjectOption
        self.update_rev_for_option() # <---------------------------------- this line
        return ProjectOption.objects.get_value(self, *args, **kwargs)

The test fails with:

======================================================== FAILURES =========================================================
_______________________________ JavascriptIntegrationTest.test_adds_contexts_without_device _______________________________
....
AssertionError: 22 write queries executed on `sentry_projectoptions`, expected <= 4
================================================ 1 failed in 2.92 seconds =================================================

This means in the future if someone introduces more "write" queries to the event ingestion, we would notice it upfront.

@HazAT HazAT self-assigned this May 15, 2018
real_queries = {}

for query in self.captured_queries:
match = re.search(r"u'([^']*)'", query['sql'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this use ast.literal_eval:

>>> import ast
>>> ast.literal_eval('u"Hello World!"')
u'Hello World!'

@mattrobenolt
Copy link
Contributor

What’s wrong with just the builtin assertNumQueries? I get that this provides a bit more behavior, but seems a little overkill imo. I don’t think we typically care which tables, and read vs write usually. It’s just as easy to introduce a bunch of extra read queries that cause a similar incident.

@mitsuhiko
Copy link
Member

mitsuhiko commented May 15, 2018

@mattrobenolt easier to debug because you can see what the extra query is that was not expected before.

FWIW I did not make good experiences in the past with just query counting in tests as you typically just define a max and then refactoring just invalidates the point of it if they drop or caching changes some assumptions. I would almost like to fully sign off on all the queries but that seemed like a good middleground.

@mattrobenolt
Copy link
Contributor

How does this work with JOINs and subqueries, etc? Which table are they counted against? All of them? One for each?

@mattrobenolt
Copy link
Contributor

The only other comment I have with this is why explicitly only testing for write queries? What about read queries? I think these are all valid since it's more likely that we introduce unintended read queries.

@HazAT
Copy link
Member Author

HazAT commented May 16, 2018

@mattrobenolt The library we use for parsing the query doesn't really work reliable for subqueries/joins to count the "reads".
https://github.com/andialbrecht/sqlparse

So we could invest more time to make it work by parsing the queries manually to correctly get the "read" count but not sure if we should really do this now.

@HazAT HazAT merged commit 9a03fd5 into master May 23, 2018
@HazAT HazAT deleted the feat/assertWriteQueries branch May 23, 2018 08:16
@github-actions github-actions bot locked and limited conversation to collaborators Dec 21, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants