Skip to content
This repository has been archived by the owner on Aug 26, 2022. It is now read-only.

bug 1431259: fix headless tests with localhost #4779

Merged
merged 1 commit into from
May 15, 2018
Merged

bug 1431259: fix headless tests with localhost #4779

merged 1 commit into from
May 15, 2018

Conversation

escattone
Copy link
Contributor

This PR fixes the headless tests such that they now successfully run against the local development instance loaded with a stock sample database.

  • Add is_local_url and is_searchable fixtures.
  • Use pytest's ability to mark a test with xfail from within the test function. This was simpler than trying to create a dynamic fixture.
  • Modify some URL's to work on both a local instance loaded with the stock sample database as well as the stage and production instances (e.g., /en-US/docs/Web/HTML$revision/1252409 rather than /en-US/docs/Web/HTML$revision/1293895)

I also removed the sensitive_url fixture since it was unused.

@escattone escattone requested a review from jwhitlock May 14, 2018 22:24
Copy link
Contributor

@jwhitlock jwhitlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the test suite on my dev environment and in staging, and it works. Nice update!

I have some nits on the xfail reasons, mostly because they won't be fixed by adding to the sample database.

"""Ensure that these requests that should return 200 are cached."""
if is_local_url:
if any(slug.startswith(p) for p in ('/diagrams/', '/presentations/')):
pytest.xfail('not yet available in the sample database')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the legacy files, not part of the sample database. We could add SERVE_LEGACY to the information in _kuma_status.json, or just assume they aren't available. But they won't be added to the sample DB.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, what was I thinking?!

"""Ensure that these requests that should return 301 are cached."""
if is_local_url and any(slug.startswith(p) for p in ('/files/', '/@api/')):
pytest.xfail('not yet available in the sample database')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Files won't be part of the sample database either. Stage has these files because we restored an anonymized backup and synced attachments at one point.

if any(slug.startswith(p) for p in ('/diagrams/', '/presentations/')):
pytest.xfail('not yet available in the sample database')
elif (not is_searchable) and slug.endswith('/dashboards/macros'):
pytest.xfail('search is not available and populated')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 this gives a good diagnostic message if search isn't populated.

'/en-US/docs/Web/HTML$revision/1293895',
'/en-US/docs/Web/HTML$compare?locale=en-US&to=1299417&from=1293895',
'/en-US/docs/Web/HTML$revision/1252409',
'/en-US/docs/Web/HTML$compare?locale=en-US&to=1287251&from=1252409',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding resources already in the sample database!

@jwhitlock
Copy link
Contributor

Thanks for the message updates, @escattone!

@jwhitlock jwhitlock merged commit b914ad9 into mdn:master May 15, 2018
@escattone escattone deleted the fix-headless-tests-for-local-dev-1431259 branch May 21, 2018 20:12
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants