Adding documents and images from January 2022 plugin testing workshop.

[Nadalyn-CZI]

Description

The first section of the workshop has been broken down into 5 articles. They are: 1. Python's assert keyword, 2. Pytest testing frameworks, 3. Napari plugins, and 4. Testing coverage. The 5th one on testing widgets is on hold for now. Feedback is encouraged.

Type of change

• This change adds new documentation

Final checklist:

• My PR is the minimum possible work for the desired functionality
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• If I included new strings, I have used trans. to make them localizable.
  For more information see our translations guide.

[Nadalyn-CZI]

These documents should be reviewed by Draga Pop, Chi-Li Chiu, seankmartin, and the members of the Metacell group.

[chili-chiu]

@psobolewskiPhD would love your feedback here too!

[seankmartin]

Hi @Nadalyn-CZI, thank you so much for the amazingly detailed documentation about this workshop! 🚀

There are a few little nitty formatting things that could be improved upon. I have left a few comments and suggestions that might give an idea of these changes. @chili-chiu the changes I have listed below in the review are not complete, they are just to give an idea. I'm happy to make a follow up commit to this PR adjusting formatting in this style if you would like!

As for the overall content, it looks great in general. I might have some further feedback if the formatting is adjusted as I will be able to read the built documentation a bit easier at that point.

[Nadalyn-CZI]

Wow! Sean! Thank you so much.

[Nadalyn-CZI]

I did not intend to have the backticks showing. I will correct it, thanks.

…

On Mon, Nov 14, 2022 at 4:08 AM Sean Martin ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In docs/plugins/testing_workshop_docs/Pythons-assert-keyword.md <#35 (comment)>: > +Article 1: This article +Article 2: [Pytest testing framework](./Pytest-testing-frameworks) +Article 3: [Readers and fixtures](./Readers-and-fixtures) +Article 4: [Test coverage](./Test-Coverage) +Article 5: Testing widgets + +This article covers: +[Test for Pass](#test-for-pass) +[Test for Fail](#test-for-fail) + +Resources +The example plugin and all the tests discussed in this article are available in [this GitHub repository](https://github.com/DragaDoncila/plugin-tests). + +The key to testing in Python is the [assert](https://docs.google.com/presentation/d/1RFja0o6cZ8lAalAve8heuJ-Lrb4nOSUnfdpOSEhqqNo/edit#slide=id.g107356847e4_0_22) keyword. We *assert* a Boolean expression is true and create an error message that appears when that expression is false. + + `assert <Boolean expression>, <error message>` Please keep the back-ticks on all the code looking things! I only mention it in this case because (I think - I may have missed another example) this is the only case of an indented block that also has back ticks. If you want to keep them it's no problem! I just though that perhaps it was by accident. I'll try demonstrate in case that makes it clearer. I thought you intended case 2, but had written case 3. 1. back-ticked code inline with text Here is some inline code. It needs the back ticks 100%. 2. indented code (a code block) Here is some code It does not need back ticks, you could put it in if you want 3. indented code with ticks Here is some `code` And now it is indented and has back ticks — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3CMNTGEY5DVUZN2FTDAI2TWIIFQBANCNFSM6AAAAAARQNHP44> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[psobolewskiPhD]

Love this!
I have a number of small comments.
Also, I worry that this can be silo'd in the workshops and maybe should be surfaced to How-to/Tutorials? (I'm not clear what goes where). Maybe @melissawm has some thoughts?

[melissawm]

I think we're almost there, there are still a few things I noticed:

• We should probably crop both docs/images/coverage_report.png and docs/images/second_coverage_report.png to remove windows decorations
• There is quite a lot of whitespace at the end of lines. Personally I tend to remove it all (and it can be done automatically) but this is a matter of style, it won't cause issues.
• I have changed all curly braces in code to be straight (" instead of “). They are very similar but like @seankmartin mentioned somewhere above, they are not parsed correctly by Markdown so they are best avoided, unless they come from raw output copy-paste (for example, in the documents above there are a few console output sessions that have curly braces. Those are ok.)
• I have also changed a few "bash" blocks to "console" - I found that they were not rendering correctly before but they are now.
• Finally, there is the questions of the "Article" naming. Like I said I don't have a strong opinion but it may be worth deciding on a pattern.

@Nadalyn-CZI I have also removed the Activity Tab folder (I think you meant to add those to another pull request and they came up here by accident)

Hope this is better, the docs are now building cleanly and you can see the artifacts above. Cheers!

EDIT: @Nadalyn-CZI to update your local copy of this pr, you will need to checkout the testing_workshop branch and pull the latest changes that I made. This can be done via the GitHub Desktop app or on the console by doing the following: if your fork is called origin (please confirm by doing git remote -v), then:

git pull origin testing_workshop --rebase

You may be presented with conflicts you will have to resolve, but I hope this is not too bad. Ping me if you need help figuring it out 😄

[Nadalyn-CZI]

Ok, so when you say "remove windows decorations" do you mean this: [image: second_coverage_report-cropped-1.png] or this: [image: second_coverage_report-cropped-2.png] I usually include the URL or path name in the address bar so people will know where to look to find what's in the screenshot. In this case it just says Users\user\plugin etc., etc., etc. so it doesn't need to be cropped to protect anyone's privacy. Thanks, Nadalyn

…

On Sun, Jan 22, 2023 at 12:30 PM Melissa Weber Mendonça < ***@***.***> wrote: I think we're almost there, there are still a few things I noticed: - We should probably crop both docs/images/coverage_report.png and docs/images/second_coverage_report.png to remove windows decorations - There is quite a lot of whitespace at the end of lines. Personally I tend to remove it all (and it can be done automatically) but this is a matter of style, it won't cause issues. - I have changed all curly braces in code to be straight (" instead of “). They are very similar but like @seankmartin <https://github.com/seankmartin> mentioned somewhere above, they are not parsed correctly by Markdown so they are best avoided, unless they come from raw output copy-paste (for example, in the documents above there are a few console output sessions that have curly braces. Those are ok.) - I have also changed a few "bash" blocks to "console" - I found that they were not rendering correctly before but they are now. - Finally, there is the questions of the "Article" naming. Like I said I don't have a strong opinion but it may be worth deciding on a pattern. @Nadalyn-CZI <https://github.com/Nadalyn-CZI> I have also removed the Activity Tab folder (I think you meant to add those to another pull request and they came up here by accident) Hope this is better, the docs are now building cleanly and you can see the artifacts above. Cheers! — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3CMNTFWRQ7MMN4FJD4WRGTWTV4EHANCNFSM6AAAAAARQNHP44> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[melissawm]

It is up to you, what I'm suggesting is removing the Chrome menu bar at least - the purple one at the top of the images. If you prefer keeping the address that is fine, I think. I will defer to the maintainers on this one though 😄

[Nadalyn-CZI]

@DragaDoncila and @psobolewskiPhD:

Once again, I think this is ready to merge. Would you please review and merge it if there are no problems with it?

Thanks,

Nadalyn

[DragaDoncila]

Approving now (just removing the last few mentions of the word article). @melissawm would you mind doing a final pass, as I've noticed you still have changes requested 😊

[melissawm]

Last comments from me, but after those this is good to go! Thanks @Nadalyn-CZI !

[Nadalyn-CZI]

All have been committed. What do we do now?

…

On Thu, Feb 9, 2023 at 1:55 PM Melissa Weber Mendonça < ***@***.***> wrote: ***@***.**** approved this pull request. Last comments from me, but after those this is good to go! Thanks @Nadalyn-CZI <https://github.com/Nadalyn-CZI> ! ------------------------------ In docs/plugins/testing_workshop_docs/2-pytest-testing-frameworks.md <#35 (comment)>: > @@ -0,0 +1,116 @@ +# 2: Pytest testing framework + +This lesson explains how to use the pytest testing framework to make testing easier. Because it is the first time we are mentioning it, I suggest adding a link to the pytest official page here: ⬇️ Suggested change -This lesson explains how to use the pytest testing framework to make testing easier. +This lesson explains how to use the [pytest testing framework](https://docs.pytest.org/en/7.2.x/) to make testing easier. ------------------------------ In docs/plugins/testing_workshop_docs/3-readers-and-fixtures.md <#35 (comment)>: > +* 2: [Pytest testing framework](./2-pytest-testing-frameworks.md) +* 3: This lesson (Readers and fixtures) +* 4: [Test coverage](./4-test-coverage.md) +* Resource links: [Testing resources](./testing-resources.md) + +### This lesson covers: +* [Readers](#reader) +* [Built-in fixtures](#built-in-fixtures) +* [Custom fixtures and round-trip tests](#custom-fixtures-and-round-trip-tests) +* [Enclosed testing](#enclosed-testing) + +### Resources +The example plugin and all the tests discussed in this lesson are available in [this GitHub repository](https://github.com/DragaDoncila/plugin-tests). + +## Introduction +In this lesson, we discuss a plugin called [plugin_tests](https://github.com/DragaDoncila/plugin-tests/tree/main/src/plugin_tests), generated using the [cookiecutter](https://github.com/napari/cookiecutter-napari-plugin), which has a reader and a widget. The reader is the cookiecutter numpy file reader, `napari_get_reader`. It checks whether a path ends in `.npy`. If it doesn't, it returns `None`, and if it does, it returns the `reader_function`, which loads the data. To reduce the chances of someone confusing a pytest plugin with a napari plugin (both have this functionality) I suggest making this explicit. I would also be a little more specific with the plugin to reduce ambiguity: ⬇️ Suggested change -In this lesson, we discuss a plugin called [plugin_tests](https://github.com/DragaDoncila/plugin-tests/tree/main/src/plugin_tests), generated using the [cookiecutter](https://github.com/napari/cookiecutter-napari-plugin), which has a reader and a widget. The reader is the cookiecutter numpy file reader, `napari_get_reader`. It checks whether a path ends in `.npy`. If it doesn't, it returns `None`, and if it does, it returns the `reader_function`, which loads the data. +In this lesson, we discuss a napari plugin called [plugin_tests](https://github.com/DragaDoncila/plugin-tests/tree/main/src/plugin_tests), generated using the [cookiecutter](https://github.com/napari/cookiecutter-napari-plugin), which has a reader and a widget. The reader is the cookiecutter [NumPy `.npy` file](https://numpy.org/doc/stable/reference/generated/numpy.lib.format.html#npy-format) reader, `napari_get_reader`. It checks whether a path ends in `.npy`. If it doesn't, it returns `None`, and if it does, it returns the `reader_function`, which loads the data. ------------------------------ In docs/plugins/testing_workshop_docs/3-readers-and-fixtures.md <#35 (comment)>: > + + reader = napari_get_reader(my_test_file) + assert callable(reader) + + layer_data_list = reader(my_test_file) + assert isinstance(layer_data_list, List) and len(layer_data_list) > 0 + + layer_data_tuple = layer_data_list[0] + layer_data = layer_data_tuple[0] + np.testing.assert_allclose(layer_data, original_data) +``` +We’re going to assert a list length greater than zero. There must be a layer in there; otherwise, we didn't read it correctly. We also assert that it is a list. + +We will test that inside that list is what we expected - layer data tuples. The first item of a layer data tuple is the actual data. We’re going to test that explicitly. + +Then we assert, using `numpy`’s asserting mechanism, `np.testing.assert_allclose` that they are all close, even though they should be exactly the same. This is standard practice when working with floating point precision. Numpy also has [other assertion options](https://numpy.org/doc/stable/reference/routines.testing.html) you may find useful. The layer data we read back with the reader function should be the same as the original data. If that's true, then we made the entire round trip. We saved the file and we used the reader to read the file. ⬇️ Suggested change -Then we assert, using `numpy`’s asserting mechanism, `np.testing.assert_allclose` that they are all close, even though they should be exactly the same. This is standard practice when working with floating point precision. Numpy also has [other assertion options](https://numpy.org/doc/stable/reference/routines.testing.html) you may find useful. The layer data we read back with the reader function should be the same as the original data. If that's true, then we made the entire round trip. We saved the file and we used the reader to read the file. +Then we assert, using `numpy`’s asserting mechanism, `np.testing.assert_allclose` that they are all close, even though they should be exactly the same. This is standard practice when working with floating point precision. NumPy also has [other assertion options](https://numpy.org/doc/stable/reference/routines.testing.html) you may find useful. The layer data we read back with the reader function should be the same as the original data. If that's true, then we made the entire round trip. We saved the file and we used the reader to read the file. — Reply to this email directly, view it on GitHub <#35 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3CMNTBZAZ4MKRRHBQOSEU3WWVDTTANCNFSM6AAAAAARQNHP44> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[melissawm]

Now, we wait for a merge (unless there are any further observations!) 😄

[DragaDoncila]

@Nadalyn-CZI woohoo this is looking great 🎉 ! With @melissawm's approval I'm now starting the 24 hour merge clock on this and will merge tomorrow morning or first thing Monday 😊

[psobolewskiPhD]

Don't want to step on any toes, but I'm going to merge this! It's gotten plenty of time I think.
Thanks everyone involved! 🙏 ❤️

[Nadalyn-CZI]

Thank you, Peter! I thought someone was merging it already. If there are any corrections, I will make them later.

…

On Tue, Feb 14, 2023 at 2:37 PM Peter Sobolewski ***@***.***> wrote: Merged #35 <#35> into main. — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3CMNTAEDFM4JME25SSZIJLWXPUG7ANCNFSM6AAAAAARQNHP44> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[DragaDoncila]

Thanks @psobolewskiPhD ! Turns out I suck at 24 hour merge clocks 😅