Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃И Add snapshot testing for cached doctrees #1020

Merged
merged 2 commits into from Sep 13, 2023
Merged

Conversation

chrisjsewell
Copy link
Member

@chrisjsewell chrisjsewell commented Sep 12, 2023

This adds some snapshot testing for cached doctrees of a sphinx build, with the snapshot_doctree fixture.

(only one has been added currently, as proof of principle, more can be added in the future)

This adds some snapshot testing for cached doctrees of a sphinx build, with the `snapshot_doctree` fixture.
Copy link
Member

@danwos danwos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like the idea to test the doctree via snapshots as well.
But I'm not sure how stable a doctree is between different Sphinx/docutils versions, so this may affect the matrix build in future.

I see also two magic names, which should be documented somehow:

  1. The file name of the snapshot, e.g. it contains test_functtion_name and test_app0.
    So the snapshot-file selection is done based on this, we should document this somewhere.

  2. <document source="<source>"> in the snapshot.
    source seems to be set by hand to <source>, right?
    So this is a manual action we need to do for all new doctree-snapshots. This we should document as well, as the current docs tell us with pytest tests/ --snapshot-update all magic is done :)

@chrisjsewell
Copy link
Member Author

But I'm not sure how stable a doctree is between different Sphinx/docutils versions, so this may affect the matrix build in future.

Tests in sphinx-design are based pretty much purely on doctree snapshots 馃槃: https://github.com/executablebooks/sphinx-design/blob/main/tests/test_snippets,
as are a lot of the tests in https://github.com/executablebooks/MyST-Parser,
so I have a lot of experience here

The file name of the snapshot, e.g. it contains test_function_name and test_app0.
So the snapshot-file selection is done based on this, we should document this somewhere.

Yep I can mention it in the documentation, although this is just the standard functioning of https://github.com/tophat/syrupy, and there is already a link to their documentation.

<document source="<source>"> in the snapshot. source seems to be set by hand to <source>, right?
So this is a manual action we need to do for all new doctree-snapshots.

No, its part of the test fixture, so it gets done automatically for snapshot_doctree (see the additions in tests/conftest.py)

Copy link
Member

@danwos danwos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh ok, so it's much easier than I thought. Thanks for the details.

@chrisjsewell chrisjsewell merged commit 6f28dc2 into master Sep 13, 2023
11 checks passed
@chrisjsewell chrisjsewell deleted the snapshot-doctree branch September 13, 2023 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants