-
Notifications
You must be signed in to change notification settings - Fork 682
bug 1431259: fix headless tests with localhost #4779
bug 1431259: fix headless tests with localhost #4779
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried the test suite on my dev environment and in staging, and it works. Nice update!
I have some nits on the xfail
reasons, mostly because they won't be fixed by adding to the sample database.
tests/headless/test_cdn.py
Outdated
"""Ensure that these requests that should return 200 are cached.""" | ||
if is_local_url: | ||
if any(slug.startswith(p) for p in ('/diagrams/', '/presentations/')): | ||
pytest.xfail('not yet available in the sample database') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are the legacy files, not part of the sample database. We could add SERVE_LEGACY
to the information in _kuma_status.json
, or just assume they aren't available. But they won't be added to the sample DB.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, what was I thinking?!
tests/headless/test_cdn.py
Outdated
"""Ensure that these requests that should return 301 are cached.""" | ||
if is_local_url and any(slug.startswith(p) for p in ('/files/', '/@api/')): | ||
pytest.xfail('not yet available in the sample database') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Files won't be part of the sample database either. Stage has these files because we restored an anonymized backup and synced attachments at one point.
tests/headless/test_cdn.py
Outdated
if any(slug.startswith(p) for p in ('/diagrams/', '/presentations/')): | ||
pytest.xfail('not yet available in the sample database') | ||
elif (not is_searchable) and slug.endswith('/dashboards/macros'): | ||
pytest.xfail('search is not available and populated') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 this gives a good diagnostic message if search isn't populated.
'/en-US/docs/Web/HTML$revision/1293895', | ||
'/en-US/docs/Web/HTML$compare?locale=en-US&to=1299417&from=1293895', | ||
'/en-US/docs/Web/HTML$revision/1252409', | ||
'/en-US/docs/Web/HTML$compare?locale=en-US&to=1287251&from=1252409', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for finding resources already in the sample database!
Thanks for the message updates, @escattone! |
This PR fixes the headless tests such that they now successfully run against the local development instance loaded with a stock sample database.
is_local_url
andis_searchable
fixtures.xfail
from within the test function. This was simpler than trying to create a dynamic fixture./en-US/docs/Web/HTML$revision/1252409
rather than/en-US/docs/Web/HTML$revision/1293895
)I also removed the
sensitive_url
fixture since it was unused.