From 26865b212b953a2fcee9aef6bad337007931d0ce Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 7 Dec 2019 17:23:57 -0500
Subject: [PATCH 001/152] Begin section on testing code
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 38 +++++++++++++++++++
1 file changed, 38 insertions(+)
create mode 100644 Python/Module5_OddsAndEnds/Testing_Your_Code.md
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
new file mode 100644
index 00000000..a7086d35
--- /dev/null
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -0,0 +1,38 @@
+---
+jupyter:
+ jupytext:
+ formats: ipynb,md
+ text_representation:
+ extension: .md
+ format_name: markdown
+ format_version: '1.2'
+ jupytext_version: 1.3.0
+ kernelspec:
+ display_name: Python 3
+ language: python
+ name: python3
+---
+
+<!-- #raw raw_mimetype="text/restructuredtext" -->
+.. meta::
+ :description: Topic: Writing tests for your code, Difficulty: Medium, Category: Section
+ :keywords: test, pytest, automated, unit, integration, property-based, hypothesis
+<!-- #endraw -->
+
+<!-- #region -->
+# Testing Your Code
+
+As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful projects. That being said, as our code becomes more complex, how will we know that it all works as expected? Obviously, one will naturally try out their project to verify its behavior. But how often will we test our projects this way; will it be every time we touch our code? Furthermore, will we run the same test every time, or are there several conditions under which we will run our code? Here, we will discuss the critically-important process of writing tests for Python code.
+
+With great power comes great responsibility.
+
+```python
+x += 2
+
+x+= 4
+```
+<!-- #endregion -->
+
+```python
+
+```
From ce05d2c9e0467fda74abfe4183200b8d555a1513 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 7 Dec 2019 17:50:34 -0500
Subject: [PATCH 002/152] work on intro
---
Python/Module5_OddsAndEnds/Testing_Your_Code.md | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index a7086d35..22742d31 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -22,7 +22,13 @@ jupyter:
<!-- #region -->
# Testing Your Code
-As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful projects. That being said, as our code becomes more complex, how will we know that it all works as expected? Obviously, one will naturally try out their project to verify its behavior. But how often will we test our projects this way; will it be every time we touch our code? Furthermore, will we run the same test every time, or are there several conditions under which we will run our code? Here, we will discuss the critically-important process of writing tests for Python code.
+This section will introduce us to the critically-important and often-overlooked process of testing code. We will begin by taking time to understand the basic motivations behind writing tests. Next, we will study the basic anatomy of a test-function, including its nucleus: the `assert` statement.
+
+Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/). This will inform how we structure our tests alongside our Python project that we are developing, and will allow us to incisively run our tests with the press of a single button. Furthermore, it will allow us to greatly streamline and even automate some of our tests.
+
+We will take some time to consider some of the unexpected utility of writing tests along with
+
+As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful projects. That being said, as our code becomes more complex, how will we know that it all works as expected? Obviously, one will naturally try out their project to verify its behavior. But how often will we take the time to test our projects this way; will it be every time we touch our code? Furthermore, will we run the same test every time, or are there several conditions under which we will run our code? These questions only scratch the surface of the various considerations at play. Here, we will distill the essentials process of writing tests for Python code.
With great power comes great responsibility.
From 1042780e0813189cd2fc16bd133ad550286dfb67 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 7 Dec 2019 18:12:15 -0500
Subject: [PATCH 003/152] complete first cut of intro
---
Python/Module5_OddsAndEnds/Testing_Your_Code.md | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 22742d31..330b9717 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -22,15 +22,12 @@ jupyter:
<!-- #region -->
# Testing Your Code
-This section will introduce us to the critically-important and often-overlooked process of testing code. We will begin by taking time to understand the basic motivations behind writing tests. Next, we will study the basic anatomy of a test-function, including its nucleus: the `assert` statement.
-
-Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/). This will inform how we structure our tests alongside our Python project that we are developing, and will allow us to incisively run our tests with the press of a single button. Furthermore, it will allow us to greatly streamline and even automate some of our tests.
-
-We will take some time to consider some of the unexpected utility of writing tests along with
+This section will introduce us to the critically-important and often-overlooked process of testing code. We will begin by taking time to understand the basic motivations behind writing tests. Next, we will study the basic anatomy of a test-function, including its nucleus: the `assert` statement. Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/). This will inform how to structure our tests alongside our Python project that we are developing, and will allow us to incisively run our tests with the press of a single button. Furthermore, it will allow us to greatly streamline and even begin to automate some of our tests. Finally, we will take a step back to consider some strategies for writing effective tests. Among these is a methodology that is near and dear to my heart: property-based testing. This will take us down a bit of a rabbit hole, where we will find the powerful property-based testing library [Hypothesis](https://hypothesis.readthedocs.io/) waiting to greet us (adorned with the mad Hatter's cap and all).
+## Why Should We Write Tests?
As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful projects. That being said, as our code becomes more complex, how will we know that it all works as expected? Obviously, one will naturally try out their project to verify its behavior. But how often will we take the time to test our projects this way; will it be every time we touch our code? Furthermore, will we run the same test every time, or are there several conditions under which we will run our code? These questions only scratch the surface of the various considerations at play. Here, we will distill the essentials process of writing tests for Python code.
-With great power comes great responsibility.
+With great power comes great responsibility: tests help us be responsible creators.
```python
x += 2
From ea239ec65b7491fda1468a245a5bca8938095c61 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 7 Dec 2019 18:40:22 -0500
Subject: [PATCH 004/152] working on motivation
---
Python/Module5_OddsAndEnds/Testing_Your_Code.md | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 330b9717..4f0271ec 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -25,9 +25,15 @@ jupyter:
This section will introduce us to the critically-important and often-overlooked process of testing code. We will begin by taking time to understand the basic motivations behind writing tests. Next, we will study the basic anatomy of a test-function, including its nucleus: the `assert` statement. Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/). This will inform how to structure our tests alongside our Python project that we are developing, and will allow us to incisively run our tests with the press of a single button. Furthermore, it will allow us to greatly streamline and even begin to automate some of our tests. Finally, we will take a step back to consider some strategies for writing effective tests. Among these is a methodology that is near and dear to my heart: property-based testing. This will take us down a bit of a rabbit hole, where we will find the powerful property-based testing library [Hypothesis](https://hypothesis.readthedocs.io/) waiting to greet us (adorned with the mad Hatter's cap and all).
## Why Should We Write Tests?
-As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful projects. That being said, as our code becomes more complex, how will we know that it all works as expected? Obviously, one will naturally try out their project to verify its behavior. But how often will we take the time to test our projects this way; will it be every time we touch our code? Furthermore, will we run the same test every time, or are there several conditions under which we will run our code? These questions only scratch the surface of the various considerations at play. Here, we will distill the essentials process of writing tests for Python code.
+With great power comes great responsibility: tests help us be responsible for the code that we create and that others will (hopefully) use.
+
+As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful and increasingly-sophisticated projects. It is only naturally try using your code to verify its behavior. You may even devise several scenarios to exercise your project. Clearly this sort of testing need no justification; it is a ubiquitous practice among coders. Less obvious are the major pitfalls associated with this highly-manual means of testing.
+
+Let's consider some of the pitfalls of casual, manual tests. To do so, consider the following unfortunate scenario: you carefully run your code through several test scenarios and see that
+
+-
+Fortunately, it is exceedingly easy to convert this casual and flawed testing workflow to one that is far more powerful and efficient.
-With great power comes great responsibility: tests help us be responsible creators.
```python
x += 2
From 4b7cf34a42aabe40b440aa0db3cc4c1aa4b3f457 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 10:15:10 -0500
Subject: [PATCH 005/152] work on motivation
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 36 ++++++++++++++++++-
1 file changed, 35 insertions(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 4f0271ec..6de38a02 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -22,11 +22,45 @@ jupyter:
<!-- #region -->
# Testing Your Code
-This section will introduce us to the critically-important and often-overlooked process of testing code. We will begin by taking time to understand the basic motivations behind writing tests. Next, we will study the basic anatomy of a test-function, including its nucleus: the `assert` statement. Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/). This will inform how to structure our tests alongside our Python project that we are developing, and will allow us to incisively run our tests with the press of a single button. Furthermore, it will allow us to greatly streamline and even begin to automate some of our tests. Finally, we will take a step back to consider some strategies for writing effective tests. Among these is a methodology that is near and dear to my heart: property-based testing. This will take us down a bit of a rabbit hole, where we will find the powerful property-based testing library [Hypothesis](https://hypothesis.readthedocs.io/) waiting to greet us (adorned with the mad Hatter's cap and all).
+This section will introduce us to the critically-important and often-overlooked process of testing code.
+We will begin by considering the basic motivations behind writing tests.
+Next, we will study the basic anatomy of a test-function, including its nucleus: the `assert` statement.
+Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/).
+This will inform how to structure our tests alongside our Python project that we are developing, and will allow us to incisively run our tests with the press of a single button.
+Furthermore, it will allow us to greatly streamline and even begin to automate some of our tests.
+Finally, we will take a step back to consider some strategies for writing effective tests.
+Among these is a methodology that is near and dear to my heart: property-based testing.
+This will take us down a bit of a rabbit hole, where we will find the powerful property-based testing library [Hypothesis](https://hypothesis.readthedocs.io/) waiting to greet us (adorned with the mad Hatter's cap and all).
+<!-- #endregion -->
+
+<!-- #region -->
## Why Should We Write Tests?
With great power comes great responsibility: tests help us be responsible for the code that we create and that others will (hopefully) use.
+The fact of the matter is that everyone already tests their code to some extent.
+After coding, say, a new function, it is only natural to feed it a contrived input and to check that it returns the output that you expected.
+To the extent that anyone would want to see evidence that their code works, we need not motivate the importance of testing.
+Less obvious is the massive benefits that we stand to gain from formalizing this testing process.
+And by "formalizing", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
+We will accumulate these functions into a "test suite" that we can run quickly and repeatedly.
+
+There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing tests:
+
+- It saves (lots of) time:
+ > After you have devised a test scenario for your code, it may only take you a second or so to run it - perhaps you need only run a couple of Jupyter notebook cells to check the output.
+ > This quickly becomes unwieldy as you write more code and devise more test scenarios.
+ > Soon you will be dissuaded from running all of your tests except for on rare occasions.
+ > With a proper test suite, you can all of your test scenarios with a single button push.
+ > In a single moment, you view a series of green check-marks (or red x's...)
+- It increases the "shelf life" of your code:
+ > If you've ever dusted off a project that you haven't used for years (or perhaps only months), you might know the tribulations of getting old code to work.
+ > Perhaps, in the interim, new versions of your project's dependencies, like PyTorch or matplotlib, have come out and may be incompatible
+
+<!-- #endregion -->
+
+<!-- #region -->
+## SCRATCH
As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful and increasingly-sophisticated projects. It is only naturally try using your code to verify its behavior. You may even devise several scenarios to exercise your project. Clearly this sort of testing need no justification; it is a ubiquitous practice among coders. Less obvious are the major pitfalls associated with this highly-manual means of testing.
Let's consider some of the pitfalls of casual, manual tests. To do so, consider the following unfortunate scenario: you carefully run your code through several test scenarios and see that
From b9968740cd48a58d9887748e082acef7339cb11d Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 10:53:19 -0500
Subject: [PATCH 006/152] continue developing motivation
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 28 +++++++++++--------
1 file changed, 17 insertions(+), 11 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 6de38a02..6d94e893 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -45,17 +45,23 @@ Less obvious is the massive benefits that we stand to gain from formalizing this
And by "formalizing", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
We will accumulate these functions into a "test suite" that we can run quickly and repeatedly.
-There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing tests:
-
-- It saves (lots of) time:
- > After you have devised a test scenario for your code, it may only take you a second or so to run it - perhaps you need only run a couple of Jupyter notebook cells to check the output.
- > This quickly becomes unwieldy as you write more code and devise more test scenarios.
- > Soon you will be dissuaded from running all of your tests except for on rare occasions.
- > With a proper test suite, you can all of your test scenarios with a single button push.
- > In a single moment, you view a series of green check-marks (or red x's...)
-- It increases the "shelf life" of your code:
- > If you've ever dusted off a project that you haven't used for years (or perhaps only months), you might know the tribulations of getting old code to work.
- > Perhaps, in the interim, new versions of your project's dependencies, like PyTorch or matplotlib, have come out and may be incompatible
+There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing (good) tests:
+
+- It saves us lots of time:
+ > After you have devised a test scenario for your code, it may only take us a second or so to run it - perhaps we need only run a couple of Jupyter notebook cells to check the output.
+ > However, this will quickly become unwieldy as we write more code and devise more test scenarios.
+ > Soon we will be dissuaded from running all of our tests except for on rare occasions.
+ > With a proper test suite, we can run all of our test scenarios with the push of a button a series of green check-marks (or red x's...) will summarize the health of our project (insofar as our tests serve as good diagnostics) in a single moment.
+ > In the long run, our test suite will save us large amounts of time and will afford us the ability to aggressively exercise our tests with little cost.
+- It increases the "shelf life" of our code:
+ > If you've ever dusted off a project that you haven't used for years (or perhaps only months or weeks...), you might know the tribulations of getting old code to work.
+ > Perhaps, in the interim, new versions of your project's dependencies, like PyTorch or matplotlib, were released and have incompatibilities with our project.
+ > And perhaps we can't even _remember_ all of the ways in which our project is supposed to work.
+ > Our test suite provides us with a simple and incisive way to dive back into our work.
+ > It will point us to any of
+- It makes it easier for others to contribute to our project:
+- It will inform the quality and design of our project for the better:
+ > Although it may not be outright obvious, writing testable code leads to writing better code.
<!-- #endregion -->
From 1126909ee753d544ac66e3eded86d03a0832d61b Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 11:52:26 -0500
Subject: [PATCH 007/152] complete draft of motivations
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 28 ++++++++++++-------
1 file changed, 18 insertions(+), 10 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 6d94e893..188609e7 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -39,29 +39,37 @@ This will take us down a bit of a rabbit hole, where we will find the powerful p
With great power comes great responsibility: tests help us be responsible for the code that we create and that others will (hopefully) use.
The fact of the matter is that everyone already tests their code to some extent.
-After coding, say, a new function, it is only natural to feed it a contrived input and to check that it returns the output that you expected.
+After coding, say, a new function, it is only natural to contrive an input to feed it, and to check that it returns the output that you expected.
To the extent that anyone would want to see evidence that their code works, we need not motivate the importance of testing.
+
Less obvious is the massive benefits that we stand to gain from formalizing this testing process.
And by "formalizing", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
We will accumulate these functions into a "test suite" that we can run quickly and repeatedly.
-There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing (good) tests:
+There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing a robust test suite:
- It saves us lots of time:
> After you have devised a test scenario for your code, it may only take us a second or so to run it - perhaps we need only run a couple of Jupyter notebook cells to check the output.
> However, this will quickly become unwieldy as we write more code and devise more test scenarios.
- > Soon we will be dissuaded from running all of our tests except for on rare occasions.
- > With a proper test suite, we can run all of our test scenarios with the push of a button a series of green check-marks (or red x's...) will summarize the health of our project (insofar as our tests serve as good diagnostics) in a single moment.
- > In the long run, our test suite will save us large amounts of time and will afford us the ability to aggressively exercise our tests with little cost.
+ > Soon we will be dissuaded from running our tests except for on rare occasions.
+ > With a proper test suite, we can run all of our test scenarios with the push of a button, and a series of green check-marks (or red x's...) will summarize the health of our project (insofar as our tests serve as good diagnostics).
+ > This, of course, also means that we will find and fix bugs much faster!
+ > In the long run, our test suite will afford us the ability to aggressively exercise (and exorcise) our code at little cost.
- It increases the "shelf life" of our code:
> If you've ever dusted off a project that you haven't used for years (or perhaps only months or weeks...), you might know the tribulations of getting old code to work.
- > Perhaps, in the interim, new versions of your project's dependencies, like PyTorch or matplotlib, were released and have incompatibilities with our project.
+ > Perhaps, in the interim, new versions of your project's dependencies, like PyTorch or Matplotlib, were released and have incompatibilities with our project.
> And perhaps we can't even _remember_ all of the ways in which our project is supposed to work.
> Our test suite provides us with a simple and incisive way to dive back into our work.
- > It will point us to any of
-- It makes it easier for others to contribute to our project:
-- It will inform the quality and design of our project for the better:
- > Although it may not be outright obvious, writing testable code leads to writing better code.
+ > It will point us to any potential incompatibilities that have accumulated over time.
+ > It also provides us with a large collection of detailed use-cases of our code;
+ > we can read through our tests remind ourselves of the inner-workings of our project.
+- It will inform the design and usability of our project for the better:
+ > Although it may not be obvious from the outset, writing testable code leads to writing better code.
+ > This is, in part, because the process of writing tests gives us the opportunity to actually _use_ our code under varied circumstances.
+ > The process of writing tests will help us suss out cumbersome function interfaces, brittle statefulness, and redundant capabilities in our code. If _we_ find it frustrating to use our code within our tests, then surely others will find it frustrating to use in applied settings.
+- It makes it easier for others to contribute to a project:
+ > Having a healthy test suite lowers the barrier to entry for a project.
+ > A contributor can make improvements to the project and quickly check to see if they have broken it or changed any of its behavior.
<!-- #endregion -->
From 9ed1048eb077f368de6f1c89a03c08b538e66673 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 11:59:43 -0500
Subject: [PATCH 008/152] fix inconsistency in count_vowels docstring
---
Python/Module5_OddsAndEnds/Writing_Good_Code.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Writing_Good_Code.md b/Python/Module5_OddsAndEnds/Writing_Good_Code.md
index 520e62b7..164ea0c3 100644
--- a/Python/Module5_OddsAndEnds/Writing_Good_Code.md
+++ b/Python/Module5_OddsAndEnds/Writing_Good_Code.md
@@ -627,7 +627,7 @@ To be more concrete, let's revisit our `count_vowels` function:
```python
def count_vowels(x: str, include_y: bool = False) -> int:
- """Returns the number of vowels contained in `in_string`"""
+ """Returns the number of vowels contained in `x`"""
vowels = set("aeiouAEIOU")
if include_y:
vowels.update("yY")
From 37b55096769545facd130e49abfa9f7b60856407 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 12:08:26 -0500
Subject: [PATCH 009/152] fix inconsistency in count_vowels docstring
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 24 +++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 188609e7..db3eab12 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -71,6 +71,30 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> Having a healthy test suite lowers the barrier to entry for a project.
> A contributor can make improvements to the project and quickly check to see if they have broken it or changed any of its behavior.
+This all sounds great, but where do we even begin to kick off the process of writing a test suite?
+Let's start by seeing what constitutes a basic test function.
+<!-- #endregion -->
+
+<!-- #region -->
+## The Basic Anatomy of a Test Function
+Let's write a function that tests the following `count_values` code:
+
+```python
+def count_vowels(x: str, include_y: bool = False) -> int:
+ """Returns the number of vowels contained in `x`
+
+ Examples
+ --------
+ >>> count_vowels("happy")
+ 1
+ >>> count_vowels("happy", include_y=True)
+ 2
+ """
+ vowels = set("aeiouAEIOU")
+ if include_y:
+ vowels.update("yY")
+ return sum(1 for char in x if char in vowels)
+```
<!-- #endregion -->
<!-- #region -->
From 26625d3f0b31ada48663b3745fb9b4f2c795b8fb Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 12:12:56 -0500
Subject: [PATCH 010/152] fix inconsistency between docstring and type-hint
---
Python/Module5_OddsAndEnds/Writing_Good_Code.md | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Writing_Good_Code.md b/Python/Module5_OddsAndEnds/Writing_Good_Code.md
index 164ea0c3..aaf5d822 100644
--- a/Python/Module5_OddsAndEnds/Writing_Good_Code.md
+++ b/Python/Module5_OddsAndEnds/Writing_Good_Code.md
@@ -765,14 +765,16 @@ def pairwise_dists(x: np.ndarray, y: np.ndarray) -> np.ndarray:
Parameters
----------
x : numpy.ndarray, shape=(M, D)
- An optional description of ``x``
+ An array of M, D-dimensional vectors.
+
y : numpy.ndarray, shape=(N, D)
- An optional description of ``y``
+ An array of N, D-dimensional vectors.
Returns
-------
numpy.ndarray, shape=(M, N)
- The pairwise distances
+ The pairwise distances between the M rows of ``x`` and the N
+ rows of ``y``.
Notes
-----
@@ -820,7 +822,7 @@ def compute_student_stats(grade_book: Dict[str, Iterable[float]],
Parameters
----------
- grade_book : Dict[str, List[float]]
+ grade_book : Dict[str, Iterable[float]]
The dictionary (name -> grades) of all of the students'
grades.
From fd6e029828d998a7398d9d7c68f2b7fc8b11e735 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 12:25:29 -0500
Subject: [PATCH 011/152] add first example
---
Python/Module5_OddsAndEnds/Testing_Your_Code.md | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index db3eab12..1a645d6b 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -95,6 +95,18 @@ def count_vowels(x: str, include_y: bool = False) -> int:
vowels.update("yY")
return sum(1 for char in x if char in vowels)
```
+
+(Note that we will be making use of [type hinting](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Writing_Good_Code.html#Type-Hinting) to help document the interfaces of our functions.
+You may want to briefly review the linked material if this is unfamiliar to you)
+
+For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output:
+
+```python
+def test_count_vowels_basic():
+ assert count_vowels("aA bB yY", include_y=False) == 2
+ assert count_vowels("aA bB yY", include_y=True) == 4
+```
+
<!-- #endregion -->
<!-- #region -->
From f76dd1e476106f8b32aca6dc9ba518ee61c1ec6d Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 8 Dec 2019 14:09:26 -0500
Subject: [PATCH 012/152] discuss asserts
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 40 +++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 1a645d6b..5ebef4b1 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -80,6 +80,8 @@ Let's start by seeing what constitutes a basic test function.
Let's write a function that tests the following `count_values` code:
```python
+# Defining a function that we will be testing
+
def count_vowels(x: str, include_y: bool = False) -> int:
"""Returns the number of vowels contained in `x`
@@ -102,14 +104,52 @@ You may want to briefly review the linked material if this is unfamiliar to you)
For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output:
```python
+# Writing a test function for `count_vowels`
+
def test_count_vowels_basic():
assert count_vowels("aA bB yY", include_y=False) == 2
assert count_vowels("aA bB yY", include_y=True) == 4
```
+Note that this test function doesn't take in any inputs;
+thanks to [Python's scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html), we can reference our `count_vowels` function within our test as long as they exist in the same "namespace".
+I.e. either we can define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels` from wherever it is defined, and into the file containing our test.
+
+To run this test, we simply call the function:
+
+```python
+# running our test function
+>>> test_count_vowels_basic()
+```
+
+and... voilà? Wait, nothing happened.
+Indeed, this is the expected behavior: our test function should either run successfully and complete "silently", or it should raise an error if one of our assertions failed.
+This is exactly the behavior afforded to us by the `assert` statements that we included in our tests.
+Let's take some time to understand how `assert` statements work, and how we should use them.
<!-- #endregion -->
<!-- #region -->
+### Assert Statements
+Similar to `return`, `def`, or `if`, the term `assert` is a reserved term in the Python language.
+It has the following specialized behavior:
+
+```python
+# demonstrating the rudimentary behavior of an `assert`
+
+# asserting an expression that evaluates to `True` does nothing
+>>> assert 1 < 2
+
+# asserting an expression that evaluates to `False` raises an error
+>>> assert 2 < 1
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-5-c82711d5fe4d> in <module>
+----> 1 assert 2 < 1
+
+AssertionError:
+```
+
+<!-- #endregion -->
## SCRATCH
As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful and increasingly-sophisticated projects. It is only naturally try using your code to verify its behavior. You may even devise several scenarios to exercise your project. Clearly this sort of testing need no justification; it is a ubiquitous practice among coders. Less obvious are the major pitfalls associated with this highly-manual means of testing.
From af93863f68d09f7957c6da6b8302fcbdc272e410 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 9 Dec 2019 22:30:48 -0500
Subject: [PATCH 013/152] minor wording changes
---
Python/Module5_OddsAndEnds/Testing_Your_Code.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 5ebef4b1..46ff54a6 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -123,9 +123,9 @@ To run this test, we simply call the function:
```
and... voilà? Wait, nothing happened.
-Indeed, this is the expected behavior: our test function should either run successfully and complete "silently", or it should raise an error if one of our assertions failed.
+Indeed, this is the expected behavior: our test function should either run successfully and complete "silently", or it should raise an error if one of our assertions failed to hold true.
This is exactly the behavior afforded to us by the `assert` statements that we included in our tests.
-Let's take some time to understand how `assert` statements work, and how we should use them.
+Let's take a moment to understand how `assert` statements work, and how we should use them.
<!-- #endregion -->
<!-- #region -->
From f398f88e889bb7f36cb861c4378ff5c279881bb5 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 15 Dec 2019 09:51:19 -0500
Subject: [PATCH 014/152] minor formatting fix: italicized was broken
---
Python/Module2_EssentialsOfPython/Functions.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module2_EssentialsOfPython/Functions.md b/Python/Module2_EssentialsOfPython/Functions.md
index 2ad15dda..2d670f3d 100644
--- a/Python/Module2_EssentialsOfPython/Functions.md
+++ b/Python/Module2_EssentialsOfPython/Functions.md
@@ -137,7 +137,7 @@ Write a function named `count_even`. It should accept one input argument, named
<!-- #region -->
## The `return` Statement
-In general, any Python object can follow a function's `return` statement. Furthermore, an **empty** `return` statement can be specified, or the **return** statement of a function can be omitted altogether. In both of these cases, *the function will return the `None` object*.
+In general, any Python object can follow a function's `return` statement. Furthermore, an **empty** `return` statement can be specified, or the **return** statement of a function can be omitted altogether. In both of these cases, *the function will return the* `None` *object*.
```python
# this function returns `None`
From f89496c2d3ef32cc68924ad675f371cc405c8134 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 15 Dec 2019 10:17:49 -0500
Subject: [PATCH 015/152] finish section on count-vowels test
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 60 ++++++++++++++++---
1 file changed, 51 insertions(+), 9 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 46ff54a6..18c5514b 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -101,7 +101,11 @@ def count_vowels(x: str, include_y: bool = False) -> int:
(Note that we will be making use of [type hinting](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Writing_Good_Code.html#Type-Hinting) to help document the interfaces of our functions.
You may want to briefly review the linked material if this is unfamiliar to you)
-For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output:
+For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output.
+The desired behavior for this test function, upon being run, is to:
+
+- Raise an error if any of our assertions *failed* to hold true.
+- Complete "silently" if all of our assertions hold true (i.e. our test function will simply [return None](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Functions.html#The-return-Statement))
```python
# Writing a test function for `count_vowels`
@@ -111,10 +115,6 @@ def test_count_vowels_basic():
assert count_vowels("aA bB yY", include_y=True) == 4
```
-Note that this test function doesn't take in any inputs;
-thanks to [Python's scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html), we can reference our `count_vowels` function within our test as long as they exist in the same "namespace".
-I.e. either we can define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels` from wherever it is defined, and into the file containing our test.
-
To run this test, we simply call the function:
```python
@@ -122,10 +122,26 @@ To run this test, we simply call the function:
>>> test_count_vowels_basic()
```
-and... voilà? Wait, nothing happened.
-Indeed, this is the expected behavior: our test function should either run successfully and complete "silently", or it should raise an error if one of our assertions failed to hold true.
-This is exactly the behavior afforded to us by the `assert` statements that we included in our tests.
-Let's take a moment to understand how `assert` statements work, and how we should use them.
+As described above, the fact our function runs, simply returning `None` without raising any errors, means that our code has "passed" this test!
+
+Let's look more carefully at the structure of `test_count_vowels_basic`.
+Note that this function doesn't take in any inputs;
+thanks to [Python's scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html), we can reference our `count_vowels` function within our test as long as they exist in the same "namespace".
+I.e. either we can define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material in a notebook) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels` from wherever it is defined, and into the file containing our test.
+More on this later.
+
+<!-- #region -->
+<div class="alert alert-info">
+
+**Reading Comprehension: Writing a Basic Test Assertion**
+
+Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
+Make sure to run your updated test to see if it passes.
+
+</div>
+<!-- #endregion -->
+
+With our first test function under our belt, it is time for us understand how `assert` statements work and how they should be used.
<!-- #endregion -->
<!-- #region -->
@@ -149,6 +165,7 @@ AssertionError Traceback (most recent call last)
AssertionError:
```
+
<!-- #endregion -->
## SCRATCH
As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful and increasingly-sophisticated projects. It is only naturally try using your code to verify its behavior. You may even devise several scenarios to exercise your project. Clearly this sort of testing need no justification; it is a ubiquitous practice among coders. Less obvious are the major pitfalls associated with this highly-manual means of testing.
@@ -166,6 +183,31 @@ x+= 4
```
<!-- #endregion -->
+<!-- #region -->
+## Links to Official Documentation
+
+
+<!-- #endregion -->
+
+<!-- #region -->
+## Reading Comprehension Solutions
+
+**Writing a Basic Test Assertion**
+
+Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
+Make sure to run your updated test to see if it passes.
+
```python
+def test_count_vowels_basic():
+ assert count_vowels("aA bB yY", include_y=False) == 2
+ assert count_vowels("aA bB yY", include_y=True) == 4
+ assert count_vowels("", include_y=True) == 0
+```
+```python
+# running the test in a notebook-cell: the function should simply return
+# `None` if all assertions hold true
+>>> test_count_vowels_basic()
```
+
+<!-- #endregion -->
From 1e1e885c5cbd757be5360643b88789319b038df1 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 15 Dec 2019 10:30:34 -0500
Subject: [PATCH 016/152] touch up working in first couple of sections
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 21 ++++++++++---------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 18c5514b..9fba4f0d 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -23,10 +23,10 @@ jupyter:
# Testing Your Code
This section will introduce us to the critically-important and often-overlooked process of testing code.
-We will begin by considering the basic motivations behind writing tests.
-Next, we will study the basic anatomy of a test-function, including its nucleus: the `assert` statement.
+We will begin by considering some driving motivations for writing tests.
+Next, we will study the basic anatomy of a test-function, including the `assert` statement, which serves as the nucleus of our test functions.
Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/).
-This will inform how to structure our tests alongside our Python project that we are developing, and will allow us to incisively run our tests with the press of a single button.
+This will inform how we structure our tests alongside our Python project that we are developing; with pytest, we can incisively run our tests with the press of a single button.
Furthermore, it will allow us to greatly streamline and even begin to automate some of our tests.
Finally, we will take a step back to consider some strategies for writing effective tests.
Among these is a methodology that is near and dear to my heart: property-based testing.
@@ -57,8 +57,8 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> In the long run, our test suite will afford us the ability to aggressively exercise (and exorcise) our code at little cost.
- It increases the "shelf life" of our code:
> If you've ever dusted off a project that you haven't used for years (or perhaps only months or weeks...), you might know the tribulations of getting old code to work.
- > Perhaps, in the interim, new versions of your project's dependencies, like PyTorch or Matplotlib, were released and have incompatibilities with our project.
- > And perhaps we can't even _remember_ all of the ways in which our project is supposed to work.
+ > Perhaps, in the interim, new versions of our project's dependencies, like PyTorch or Matplotlib, were released and have incompatibilities with our project's code.
+ > And perhaps _we can't even remember_ all of the ways in which our project is supposed to work.
> Our test suite provides us with a simple and incisive way to dive back into our work.
> It will point us to any potential incompatibilities that have accumulated over time.
> It also provides us with a large collection of detailed use-cases of our code;
@@ -71,8 +71,8 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> Having a healthy test suite lowers the barrier to entry for a project.
> A contributor can make improvements to the project and quickly check to see if they have broken it or changed any of its behavior.
-This all sounds great, but where do we even begin to kick off the process of writing a test suite?
-Let's start by seeing what constitutes a basic test function.
+This all sounds great, but where do we even start the process writing a test suite?
+Let's begin by seeing what constitutes a basic test function.
<!-- #endregion -->
<!-- #region -->
@@ -122,12 +122,13 @@ To run this test, we simply call the function:
>>> test_count_vowels_basic()
```
-As described above, the fact our function runs, simply returning `None` without raising any errors, means that our code has "passed" this test!
+As described above, the fact our function runs, simply returning `None` without raising any errors, means that our code has passed this test. We've written and run our very first test!
Let's look more carefully at the structure of `test_count_vowels_basic`.
Note that this function doesn't take in any inputs;
-thanks to [Python's scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html), we can reference our `count_vowels` function within our test as long as they exist in the same "namespace".
-I.e. either we can define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material in a notebook) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels` from wherever it is defined, and into the file containing our test.
+thanks to [Python's scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html), we can reference our `count_vowels` function within our test as long as it is defined in the same "namespace" as `test_count_vowels_basic`.
+That is, we can either define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material in a notebook) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels` from wherever it is defined, and into the file containing our test.
+The latter scenario is by far the most common one, in practice.
More on this later.
<!-- #region -->
From 1932d551334815b610e410ff2f244ee91fa25a1a Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 15 Dec 2019 12:34:08 -0500
Subject: [PATCH 017/152] complete section on assertions
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 142 +++++++++++++++---
1 file changed, 123 insertions(+), 19 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 9fba4f0d..8fdafd41 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -19,7 +19,6 @@ jupyter:
:keywords: test, pytest, automated, unit, integration, property-based, hypothesis
<!-- #endraw -->
-<!-- #region -->
# Testing Your Code
This section will introduce us to the critically-important and often-overlooked process of testing code.
@@ -32,9 +31,8 @@ Finally, we will take a step back to consider some strategies for writing effect
Among these is a methodology that is near and dear to my heart: property-based testing.
This will take us down a bit of a rabbit hole, where we will find the powerful property-based testing library [Hypothesis](https://hypothesis.readthedocs.io/) waiting to greet us (adorned with the mad Hatter's cap and all).
-<!-- #endregion -->
-<!-- #region -->
+
## Why Should We Write Tests?
With great power comes great responsibility: tests help us be responsible for the code that we create and that others will (hopefully) use.
@@ -73,7 +71,6 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
This all sounds great, but where do we even start the process writing a test suite?
Let's begin by seeing what constitutes a basic test function.
-<!-- #endregion -->
<!-- #region -->
## The Basic Anatomy of a Test Function
@@ -128,13 +125,13 @@ Let's look more carefully at the structure of `test_count_vowels_basic`.
Note that this function doesn't take in any inputs;
thanks to [Python's scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html), we can reference our `count_vowels` function within our test as long as it is defined in the same "namespace" as `test_count_vowels_basic`.
That is, we can either define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material in a notebook) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels` from wherever it is defined, and into the file containing our test.
-The latter scenario is by far the most common one, in practice.
+The latter scenario is by far the most common one in practice.
More on this later.
<!-- #region -->
<div class="alert alert-info">
-**Reading Comprehension: Writing a Basic Test Assertion**
+**Reading Comprehension: The Basic Anatomy of a Test**
Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
Make sure to run your updated test to see if it passes.
@@ -151,12 +148,12 @@ Similar to `return`, `def`, or `if`, the term `assert` is a reserved term in the
It has the following specialized behavior:
```python
-# demonstrating the rudimentary behavior of an `assert`
+# demonstrating the rudimentary behavior of the `assert` statement
-# asserting an expression that evaluates to `True` does nothing
+# asserting an expression whose boolean-value is `True` will complete "silently"
>>> assert 1 < 2
-# asserting an expression that evaluates to `False` raises an error
+# asserting an expression whose boolean-value is `False` raises an error
>>> assert 2 < 1
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
@@ -164,36 +161,112 @@ AssertionError Traceback (most recent call last)
----> 1 assert 2 < 1
AssertionError:
+
+# we can include an error message with our assertion
+>>> assert 0 in [1, 2, 3], "0 is not in the list"
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-8-e72fb36dc785> in <module>
+----> 1 assert 0 in [1, 2, 3], "0 is not in the list"
+
+AssertionError: 0 is not in the list
```
+The general form of an assertion statement is:
-<!-- #endregion -->
-## SCRATCH
-As we become capable Python users, we will naturally find ourselves moving away from writing short, trivial programs in favor of creating useful and increasingly-sophisticated projects. It is only naturally try using your code to verify its behavior. You may even devise several scenarios to exercise your project. Clearly this sort of testing need no justification; it is a ubiquitous practice among coders. Less obvious are the major pitfalls associated with this highly-manual means of testing.
+```python
+assert <expression> [, <error-message>]
+```
-Let's consider some of the pitfalls of casual, manual tests. To do so, consider the following unfortunate scenario: you carefully run your code through several test scenarios and see that
+In an assertion statement, `bool` is called on the object that is returned by `<expression>` - if `bool(<expression>)` returns `False`, then an `AssertionError` is raised.
+If you included a string in the assertion statement - separated from `<expression>` by a comma - then this string will be printed as the error message.
--
-Fortunately, it is exceedingly easy to convert this casual and flawed testing workflow to one that is far more powerful and efficient.
+See that the assertion statement:
+```python
+assert expression, error_message
+```
+is effectively shorthand for the following code:
```python
-x += 2
+# long-form equivalent of: `assert expression, error_message`
+if bool(expression) is False:
+ raise AssertionError(error_message)
+```
+
+<!-- #endregion -->
+<!-- #region -->
+<div class="alert alert-info">
+
+**Reading Comprehension: Assertions**
-x+= 4
+Given the following objects:
+
+```python
+a_list = []
+a_number = 22
+a_string = "abcdef"
```
+
+Write two assertion statements, each one with the corresponding behavior:
+
+- asserts that `a_list` is _not_ empty
+- asserts that the number of vowels in `a_string` is less than `a_number`; include and error message that prints the actual number of vowels
+
+</div>
<!-- #endregion -->
<!-- #region -->
-## Links to Official Documentation
+#### What is the Purpose of an Assertion?
+In our code, an assertion should be used as _a statement that is true unless there is a bug our code_.
+It is plain to see that the assertions in `test_count_vowels_basic` fit this description.
+However, it can also be useful to include assertions within our source code itself.
+
+For instance, we know that `count_vowels` should always return a non-negative integer, and that this count should not exceed the number of characters in the input string.
+We can explicitly assert that this is the case:
+
+```python
+# an example of including an assertion within our source code
+
+def count_vowels(x: str, include_y: bool = False) -> int:
+ """Returns the number of vowels contained in `x`
+
+ Examples
+ --------
+ >>> count_vowels("happy")
+ 1
+ >>> count_vowels("happy", include_y=True)
+ 2
+ """
+ vowels = set("aeiouAEIOU")
+ if include_y:
+ vowels.update("yY")
+ count = sum(1 for char in x if char in vowels)
+
+ # This statement should always be true: it is not checking for
+ # bad input from a user, it is asserting that the internal logic
+ # of our function is correct
+ assert isinstance(count, int) and 0 <= count <= len(x)
+ return count
+```
+Note that this assertion *is not meant to check if the user passed bad inputs for* `x` *and* `include_y`.
+Rather, it is meant to assert that our own internal logic holds true.
+
+Admittedly, the `count_vowels` function is simple enough that the inclusion of this assertion is rather pedantic.
+That being said, as we write increasingly sophisticated code, we will find that the inclusion of assertions will help us catch bad internal logic and oversights within our code base.
+<!-- #endregion -->
<!-- #endregion -->
+## Links to Official Documentation
+
+
+
<!-- #region -->
## Reading Comprehension Solutions
-**Writing a Basic Test Assertion**
+**The Basic Anatomy of a Test: Solution**
Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
Make sure to run your updated test to see if it passes.
@@ -211,4 +284,35 @@ def test_count_vowels_basic():
>>> test_count_vowels_basic()
```
+**Assertions: Solution**
+```python
+a_list = []
+a_number = 22
+a_string = "abcdef"
+```
+
+Assert that `a_list` is _not_ empty:
+
+```python
+>>> assert a_list
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-10-2eba8294859e> in <module>
+----> 1 assert a_list
+
+AssertionError:
+```
+
+> You may have written `assert len(a_list) > 0` - this is also correct.
+> However, recall that calling `bool` on any sequence (list, tuple, string, etc.) will return `False` if the sequence is empty.
+> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - that `bool` will be called on whatever the provided expression is.
+
+Assert that the number of vowels in `a_string` is fewer than `a_number`; include and error message that prints the actual number of vowels:
+
+```python
+>>> assert count_vowels(a_string) < a_number, f"Number of vowels, {count_vowels(a_string)}, exceeds {a_number}"
+```
+
+> Note that we make use of an [f-string](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) as a convenient means for writing an informative error message.
+
<!-- #endregion -->
From 94b494a39157925f7054446c7986ba20d2b7a6ca Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 15 Dec 2019 13:09:47 -0500
Subject: [PATCH 018/152] Add section on testing tests
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 76 ++++++++++++++++++-
1 file changed, 72 insertions(+), 4 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 8fdafd41..a51e4af4 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -128,7 +128,7 @@ That is, we can either define `count_vowels` in the same .py file (or Jupyter no
The latter scenario is by far the most common one in practice.
More on this later.
-<!-- #region -->
+<!-- #endregion -->
<div class="alert alert-info">
**Reading Comprehension: The Basic Anatomy of a Test**
@@ -137,10 +137,9 @@ Add an additional assertion to the body of `test_count_vowels_basic`, which test
Make sure to run your updated test to see if it passes.
</div>
-<!-- #endregion -->
+
With our first test function under our belt, it is time for us understand how `assert` statements work and how they should be used.
-<!-- #endregion -->
<!-- #region -->
### Assert Statements
@@ -257,15 +256,35 @@ Admittedly, the `count_vowels` function is simple enough that the inclusion of t
That being said, as we write increasingly sophisticated code, we will find that the inclusion of assertions will help us catch bad internal logic and oversights within our code base.
<!-- #endregion -->
+### Testing Our Tests
+
+It is surprisingly easy to unwittingly write a test that always passes or that fails to test our logic in a useful way.
+This is a particularly treacherous mistake to make as it leads us to falsely believe that our function is working as-expected.
+**Thus a critical step in the test-writing process is to intentionally mutate your function of interest - to corrupt its behavior in such a way that your test ought to raise an error.**
+Once you confirm that your test does indeed raise an error as-expected, restore your function to its original form and re-run the test and see that it passes. Take care that you mutate your function in a way that is trivial to undo - make use of code-comments towards this end.
+
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Testing Your Test via Manual Mutation**
+
+Temporarily change the body of `count_vowels` such that the second assertion in `test_count_vowels_basic` raises an error.
+Run the test to confirm that the second assertion raises,
+and then restore `count_vowels` to its original form.
+Finally, rerun the test to see that `count_vowels` once again passes all of the assertions.
+
+</div>
+
<!-- #endregion -->
## Links to Official Documentation
-<!-- #region -->
## Reading Comprehension Solutions
+<!-- #region -->
**The Basic Anatomy of a Test: Solution**
Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
@@ -314,5 +333,54 @@ Assert that the number of vowels in `a_string` is fewer than `a_number`; include
```
> Note that we make use of an [f-string](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) as a convenient means for writing an informative error message.
+<!-- #endregion -->
+
+<!-- #region -->
+**Testing Your Test via Manual Mutation: Solution**
+
+Temporarily change the body of `count_vowels` such that the _second_ assertion in `test_count_vowels_basic` raises an error.
+> Let's comment out the `if include_y` block in our code - this should prevent us from counting y's, and thus should violate the second assertion in our test.
+```python
+# Breaking the behavior of `include_y=True`
+def count_vowels(x: str, include_y: bool = False) -> int:
+ vowels = set("aeiouAEIOU")
+ # if include_y:
+ # vowels.update("yY")
+ return sum(1 for char in x if char in vowels)
+```
+
+```python
+# the second assertion should raise an error
+>>> test_count_vowels_basic()
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-5-32301ff829e9> in <module>
+----> 1 test_count_vowels_basic()
+
+<ipython-input-4-99ef0ca3d859> in test_count_vowels_basic()
+ 1 def test_count_vowels_basic():
+ 2 assert count_vowels("aA bB yY", include_y=False) == 2
+----> 3 assert count_vowels("aA bB yY", include_y=True) == 4
+
+AssertionError:
+```
+
+> Great! That assertion really does help to ensure that we are counting y's correctly.
+
+Restore `count_vowels` to its original form and rerun the test to see that `count_vowels` once again passes all of the assertions.
+
+```python
+# Restore the behavior of `include_y=True`
+def count_vowels(x: str, include_y: bool = False) -> int:
+ vowels = set("aeiouAEIOU")
+ if include_y:
+ vowels.update("yY")
+ return sum(1 for char in x if char in vowels)
+```
+
+```python
+# confirming that we restored the proper behavior in `count_vowels`
+>>> test_count_vowels_basic()
+```
<!-- #endregion -->
From a3c9ce304b8fe3afe5c9d3ac9ee1beede84704eb Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 15 Dec 2019 13:35:14 -0500
Subject: [PATCH 019/152] add test suite to package structure
---
.../Modules_and_Packages.md | 25 +++++++++++++------
1 file changed, 17 insertions(+), 8 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index 3324669e..f1c7d257 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -5,7 +5,7 @@ jupyter:
extension: .md
format_name: markdown
format_version: '1.2'
- jupytext_version: 1.3.0rc1
+ jupytext_version: 1.3.0
kernelspec:
display_name: Python 3
language: python
@@ -406,15 +406,15 @@ It must be mentioned that we are sweeping some details under the rug here. Insta
### Installing Your Own Python Package
-Suppose that we are happy with the work we have done on our `face_detector` project. We will want to install this package - placing it in our site-packages directory so that we can import it irrespective of our Python interpreter's working directory. Here we will construct a basic setup script that will allow us to accomplish this.
+Suppose that we are happy with the work we have done on our `face_detector` project. We will want to install this package - placing it in our site-packages directory so that we can import it irrespective of our Python interpreter's working directory. Here we will construct a basic setup script that will allow us to accomplish this. For completeness, we will also indicate how one would include a test suite along side the source code in this directory structure.
We note outright that the purpose of this section is strictly to provide you with the minimum set of instructions needed to install a package. We will not be diving into what is going on under the hood at all. Please refer to [An Introduction to Distutils](https://docs.python.org/3/distutils/introduction.html#an-introduction-to-distutils) and [Packaging Your Project](https://packaging.python.org/tutorials/packaging-projects/#packaging-your-project) for a deeper treatment of this topic.
Carrying on, we will want to create a setup-script, `setup.py`, *in the same directory as our package*. That is, our directory structure should look like:
```
-- setup.py
-- face_detection/
+- setup.py # script responsible for installing `face_detection` package
+- face_detection/ # source code of `face_detection` package
|-- __init__.py
|-- utils.py
|-- database.py
@@ -423,8 +423,18 @@ Carrying on, we will want to create a setup-script, `setup.py`, *in the same dir
|-- __init__.py
|-- calibration.py
|-- config.py
+- tests/ # test-suite for `face_detection` package (to be run using pytest)
+ |-- conf.py # optional configuration file for pytest
+ |-- test_utils.py
+ |-- test_database.py
+ |-- test_model.py
+ |-- camera/
+ |-- test_calibration.py
+ |-- test_config.py
```
+A `tests/` directory can be included at the same directory level as `setup.py` and `face_detection/`. This is the recommended structure for using pytest as our test-runner.
+
<!-- #region -->
The bare bones build script for preparing your package for installation, `setup.py`, is as follows:
@@ -434,12 +444,11 @@ import setuptools
setuptools.setup(
name="face_detection",
- version="1.0",
- packages=setuptools.find_packages(),
+ version="1.0.0",
+ packages=find_packages(exclude=["tests", "tests.*"]),
)
```
-
-
+The expression `exclude=["tests", "tests.*"]` is included to ensure that the code in your test-suite is not included in the installation of `face_detection`.
<!-- #endregion -->
If you read through the additional materials linked above, you will see that there are many more fields of optional information that can be provided in this setup script, such as the author name, any installation requirements that the package has, and more.
From 68123666181c7e3413792137b0d9f233314b4831 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 15 Dec 2019 13:35:28 -0500
Subject: [PATCH 020/152] use consistent voice in testing tests
---
Python/Module5_OddsAndEnds/Testing_Your_Code.md | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index a51e4af4..971ff8a1 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -116,10 +116,10 @@ To run this test, we simply call the function:
```python
# running our test function
->>> test_count_vowels_basic()
+>>> test_count_vowels_basic() # passes: returns None | fails: raises error
```
-As described above, the fact our function runs, simply returning `None` without raising any errors, means that our code has passed this test. We've written and run our very first test!
+As described above, the fact our function runs and simply returns `None` means that our code has passed this test. We've written and run our very first test!
Let's look more carefully at the structure of `test_count_vowels_basic`.
Note that this function doesn't take in any inputs;
@@ -258,10 +258,10 @@ That being said, as we write increasingly sophisticated code, we will find that
### Testing Our Tests
-It is surprisingly easy to unwittingly write a test that always passes or that fails to test our logic in a useful way.
+It is surprisingly easy to unwittingly write a test that always passes or that fails to test our fails in its intended way.
This is a particularly treacherous mistake to make as it leads us to falsely believe that our function is working as-expected.
-**Thus a critical step in the test-writing process is to intentionally mutate your function of interest - to corrupt its behavior in such a way that your test ought to raise an error.**
-Once you confirm that your test does indeed raise an error as-expected, restore your function to its original form and re-run the test and see that it passes. Take care that you mutate your function in a way that is trivial to undo - make use of code-comments towards this end.
+**Thus a critical step in the test-writing process is to intentionally mutate your function of interest - to corrupt its behavior in such a way that our test ought to raise an error.**
+Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes. We ought to mutate our function in a way that is trivial to undo - we can use of code-comments towards this end.
From cbdd2b5db5870155d5e9a870b51eb15ec880f9ca Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 16 Dec 2019 12:34:32 -0500
Subject: [PATCH 021/152] rearrange mateirals: testing tests goes before
assertions
---
.../Module5_OddsAndEnds/Testing_Your_Code.md | 145 +++++++++---------
1 file changed, 73 insertions(+), 72 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
index 971ff8a1..6d36a378 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module5_OddsAndEnds/Testing_Your_Code.md
@@ -80,7 +80,9 @@ Let's write a function that tests the following `count_values` code:
# Defining a function that we will be testing
def count_vowels(x: str, include_y: bool = False) -> int:
- """Returns the number of vowels contained in `x`
+ """Returns the number of vowels contained in `x`.
+
+ The vowel 'y' is included optionally.
Examples
--------
@@ -104,6 +106,8 @@ The desired behavior for this test function, upon being run, is to:
- Raise an error if any of our assertions *failed* to hold true.
- Complete "silently" if all of our assertions hold true (i.e. our test function will simply [return None](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Functions.html#The-return-Statement))
+Here, we will be making use of Python's `assert` statements, whose behavior will be easy to deduce from the context of this test alone; we will be formally introduced to them soon.
+
```python
# Writing a test function for `count_vowels`
@@ -119,7 +123,7 @@ To run this test, we simply call the function:
>>> test_count_vowels_basic() # passes: returns None | fails: raises error
```
-As described above, the fact our function runs and simply returns `None` means that our code has passed this test. We've written and run our very first test!
+As described above, the fact our function runs and simply returns `None` means that our code has passed this test. We've written and run our very first test! It certainly isn't the most robust test, but it is a good start.
Let's look more carefully at the structure of `test_count_vowels_basic`.
Note that this function doesn't take in any inputs;
@@ -139,7 +143,34 @@ Make sure to run your updated test to see if it passes.
</div>
-With our first test function under our belt, it is time for us understand how `assert` statements work and how they should be used.
+### Testing Our Tests
+
+It is surprisingly easy to unwittingly write a test that always passes or that fails to test our code as we had intended.
+This is a particularly treacherous mistake to make as it leads us to falsely believe that our function is working as-expected.
+**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior in such a way that our test ought to raise an error.**
+Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
+
+We ought to mutate our function in a way that is trivial to undo; we can use of code-comments towards this end.
+All [IDEs](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) have the ability to "block-comment" selected code.
+In a Jupyter notebook code cell, we can highlight multiple lines of code and press `CTRL + /`: this will comment-out these lines of code.
+The same key-combination will also un-comment a highlighted block of commented code.
+
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Testing Your Test via Manual Mutation**
+
+Temporarily change the body of `count_vowels` such that the second assertion in `test_count_vowels_basic` raises an error.
+Run the test to confirm that the second assertion raises,
+and then restore `count_vowels` to its original form.
+Finally, rerun the test to see that `count_vowels` once again passes all of the assertions.
+
+</div>
+
+
+
+With our first test function under our belt, it is time for us to clearly understand how `assert` statements work and how they should be used.
<!-- #region -->
### Assert Statements
@@ -177,7 +208,7 @@ The general form of an assertion statement is:
assert <expression> [, <error-message>]
```
-In an assertion statement, `bool` is called on the object that is returned by `<expression>` - if `bool(<expression>)` returns `False`, then an `AssertionError` is raised.
+When an assertion statement is executed, the built-in `bool` function is called on the object that is returned by `<expression>`; if `bool(<expression>)` returns `False`, then an `AssertionError` is raised.
If you included a string in the assertion statement - separated from `<expression>` by a comma - then this string will be printed as the error message.
See that the assertion statement:
@@ -220,31 +251,20 @@ Write two assertion statements, each one with the corresponding behavior:
In our code, an assertion should be used as _a statement that is true unless there is a bug our code_.
It is plain to see that the assertions in `test_count_vowels_basic` fit this description.
However, it can also be useful to include assertions within our source code itself.
-
-For instance, we know that `count_vowels` should always return a non-negative integer, and that this count should not exceed the number of characters in the input string.
+For instance, we know that `count_vowels` should always return a non-negative integer for the vowel-count, and that it is illogical for this count to exceed the number of characters in the input string.
We can explicitly assert that this is the case:
```python
# an example of including an assertion within our source code
def count_vowels(x: str, include_y: bool = False) -> int:
- """Returns the number of vowels contained in `x`
-
- Examples
- --------
- >>> count_vowels("happy")
- 1
- >>> count_vowels("happy", include_y=True)
- 2
- """
vowels = set("aeiouAEIOU")
if include_y:
vowels.update("yY")
count = sum(1 for char in x if char in vowels)
- # This statement should always be true: it is not checking for
- # bad input from a user, it is asserting that the internal logic
- # of our function is correct
+ # This assertion should always be true: it is asserting that
+ # the internal logic of our function is correct
assert isinstance(count, int) and 0 <= count <= len(x)
return count
```
@@ -253,33 +273,12 @@ Note that this assertion *is not meant to check if the user passed bad inputs fo
Rather, it is meant to assert that our own internal logic holds true.
Admittedly, the `count_vowels` function is simple enough that the inclusion of this assertion is rather pedantic.
-That being said, as we write increasingly sophisticated code, we will find that the inclusion of assertions will help us catch bad internal logic and oversights within our code base.
-<!-- #endregion -->
-
-### Testing Our Tests
-
-It is surprisingly easy to unwittingly write a test that always passes or that fails to test our fails in its intended way.
-This is a particularly treacherous mistake to make as it leads us to falsely believe that our function is working as-expected.
-**Thus a critical step in the test-writing process is to intentionally mutate your function of interest - to corrupt its behavior in such a way that our test ought to raise an error.**
-Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes. We ought to mutate our function in a way that is trivial to undo - we can use of code-comments towards this end.
-
-
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Testing Your Test via Manual Mutation**
-
-Temporarily change the body of `count_vowels` such that the second assertion in `test_count_vowels_basic` raises an error.
-Run the test to confirm that the second assertion raises,
-and then restore `count_vowels` to its original form.
-Finally, rerun the test to see that `count_vowels` once again passes all of the assertions.
-
-</div>
-
+That being said, as we write increasingly sophisticated code, we will find that this sort of assertion will help us catch bad internal logic and oversights within our code base.
<!-- #endregion -->
## Links to Official Documentation
+- [The assert statement](https://docs.python.org/3/reference/simple_stmts.html?highlight=assert#the-assert-statement)
## Reading Comprehension Solutions
@@ -302,37 +301,6 @@ def test_count_vowels_basic():
# `None` if all assertions hold true
>>> test_count_vowels_basic()
```
-
-**Assertions: Solution**
-```python
-a_list = []
-a_number = 22
-a_string = "abcdef"
-```
-
-Assert that `a_list` is _not_ empty:
-
-```python
->>> assert a_list
----------------------------------------------------------------------------
-AssertionError Traceback (most recent call last)
-<ipython-input-10-2eba8294859e> in <module>
-----> 1 assert a_list
-
-AssertionError:
-```
-
-> You may have written `assert len(a_list) > 0` - this is also correct.
-> However, recall that calling `bool` on any sequence (list, tuple, string, etc.) will return `False` if the sequence is empty.
-> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - that `bool` will be called on whatever the provided expression is.
-
-Assert that the number of vowels in `a_string` is fewer than `a_number`; include and error message that prints the actual number of vowels:
-
-```python
->>> assert count_vowels(a_string) < a_number, f"Number of vowels, {count_vowels(a_string)}, exceeds {a_number}"
-```
-
-> Note that we make use of an [f-string](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) as a convenient means for writing an informative error message.
<!-- #endregion -->
<!-- #region -->
@@ -384,3 +352,36 @@ def count_vowels(x: str, include_y: bool = False) -> int:
>>> test_count_vowels_basic()
```
<!-- #endregion -->
+
+<!-- #region -->
+**Assertions: Solution**
+```python
+a_list = []
+a_number = 22
+a_string = "abcdef"
+```
+
+Assert that `a_list` is _not_ empty:
+
+```python
+>>> assert a_list
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-10-2eba8294859e> in <module>
+----> 1 assert a_list
+
+AssertionError:
+```
+
+> You may have written `assert len(a_list) > 0` - this is also correct.
+> However, recall that calling `bool` on any sequence (list, tuple, string, etc.) will return `False` if the sequence is empty.
+> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - that `bool` will be called on whatever the provided expression is.
+
+Assert that the number of vowels in `a_string` is fewer than `a_number`; include and error message that prints the actual number of vowels:
+
+```python
+>>> assert count_vowels(a_string) < a_number, f"Number of vowels, {count_vowels(a_string)}, exceeds {a_number}"
+```
+
+> Note that we make use of an [f-string](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) as a convenient means for writing an informative error message.
+<!-- #endregion -->
From 8155a1334d892c6aa1023708da0326459beccbec Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Fri, 20 Dec 2019 09:37:37 -0500
Subject: [PATCH 022/152] move testing-code material to its own module
---
.../Intro_to_Testing.md} | 19 ++++---------------
Python/module_6.rst | 19 +++++++++++++++++++
2 files changed, 23 insertions(+), 15 deletions(-)
rename Python/{Module5_OddsAndEnds/Testing_Your_Code.md => Module6_Testing/Intro_to_Testing.md} (92%)
create mode 100644 Python/module_6.rst
diff --git a/Python/Module5_OddsAndEnds/Testing_Your_Code.md b/Python/Module6_Testing/Intro_to_Testing.md
similarity index 92%
rename from Python/Module5_OddsAndEnds/Testing_Your_Code.md
rename to Python/Module6_Testing/Intro_to_Testing.md
index 6d36a378..b677ce53 100644
--- a/Python/Module5_OddsAndEnds/Testing_Your_Code.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -15,22 +15,11 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
- :description: Topic: Writing tests for your code, Difficulty: Medium, Category: Section
- :keywords: test, pytest, automated, unit, integration, property-based, hypothesis
+ :description: Topic: Writing tests for your code, Difficulty: Easy, Category: Section
+ :keywords: test, automated, unit, assert
<!-- #endraw -->
-# Testing Your Code
-
-This section will introduce us to the critically-important and often-overlooked process of testing code.
-We will begin by considering some driving motivations for writing tests.
-Next, we will study the basic anatomy of a test-function, including the `assert` statement, which serves as the nucleus of our test functions.
-Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework [pytest](https://docs.pytest.org/).
-This will inform how we structure our tests alongside our Python project that we are developing; with pytest, we can incisively run our tests with the press of a single button.
-Furthermore, it will allow us to greatly streamline and even begin to automate some of our tests.
-Finally, we will take a step back to consider some strategies for writing effective tests.
-Among these is a methodology that is near and dear to my heart: property-based testing.
-This will take us down a bit of a rabbit hole, where we will find the powerful property-based testing library [Hypothesis](https://hypothesis.readthedocs.io/) waiting to greet us (adorned with the mad Hatter's cap and all).
-
+# Introduction to Testing Code
## Why Should We Write Tests?
@@ -109,7 +98,7 @@ The desired behavior for this test function, upon being run, is to:
Here, we will be making use of Python's `assert` statements, whose behavior will be easy to deduce from the context of this test alone; we will be formally introduced to them soon.
```python
-# Writing a test function for `count_vowels`
+# Writing a rudimentary test function for `count_vowels`
def test_count_vowels_basic():
assert count_vowels("aA bB yY", include_y=False) == 2
diff --git a/Python/module_6.rst b/Python/module_6.rst
new file mode 100644
index 00000000..3aacaf32
--- /dev/null
+++ b/Python/module_6.rst
@@ -0,0 +1,19 @@
+Module 6: Testing Your Code
+===========================
+This module will introduce us to the critically-important and often-overlooked process of testing code.
+We will begin by considering some driving motivations for writing tests.
+Next, we will study the basic anatomy of a test-function, including the :code:`assert` statement, which serves as the nucleus of our test functions.
+Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework `pytest <https://docs.pytest.org/>`_.
+This will inform how we structure our tests alongside our Python project; with pytest, we can incisively run our tests with the press of a single button.
+Furthermore, it will allow us to greatly streamline and even begin to automate some of our tests.
+Finally, we will take a step back to consider some strategies for writing effective tests.
+Among these is a methodology that is near and dear to my heart: property-based testing.
+This will take us down a bit of a rabbit hole, where we will find the powerful property-based testing library `Hypothesis <https://hypothesis.readthedocs.io/>`_ waiting to greet us (adorned with the mad Hatter's cap and all).
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Contents:
+
+ Module6_Testing/Writing_Good_Code.md
+
+
From 5801b68ac71a73ffa0f8ff29272eee069b93af29 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Fri, 20 Dec 2019 09:38:15 -0500
Subject: [PATCH 023/152] fix rst delimiter length mismatch for module 5
---
Python/module_5.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/module_5.rst b/Python/module_5.rst
index 0c0bf09f..63c8036d 100644
--- a/Python/module_5.rst
+++ b/Python/module_5.rst
@@ -1,5 +1,5 @@
Module 5: Odds and Ends
-=====================================
+=======================
This module contains materials that are extraneous to the essentials of Python as a language and of NumPy, but are nonetheless critical to doing day-to-day work using these tools.
The first section introduces some general guidelines for writing "good code". Specifically, it points you, the reader, to a style guide that many people in the Python community abide by. It also introduces a relatively new and increasingly-popular feature of Python, called type-hinting, which permits us to enhance our code with type-documentation annotations. The reader will also be introduced to NumPy's and Google's respective specifications for writing good docstrings.
From 8132f2084dd51ab83a40070b2ae64717210fcd0b Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Fri, 20 Dec 2019 10:28:37 -0500
Subject: [PATCH 024/152] introduce merge-max-mappings
---
Python/Module6_Testing/Intro_to_Testing.md | 70 ++++++++++++++++++----
1 file changed, 57 insertions(+), 13 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index b677ce53..6098ac85 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -19,7 +19,11 @@ jupyter:
:keywords: test, automated, unit, assert
<!-- #endraw -->
-# Introduction to Testing Code
+# Introduction to Testing
+
+This section will show us just how simple it is to write rudimentary tests. We need only recall some of Python's basic scoping rules and introduce ourselves to the `assert` statement to write a genuine test function. That being said, this will simply kick off our testing journey. We will immediately encounter some important questions. How do we know that our tests work? And, how do we know that our tests are effective?
+
+Before we hit the ground running, let's take a moment to consider some motivations for testing out code.
## Why Should We Write Tests?
@@ -63,15 +67,26 @@ Let's begin by seeing what constitutes a basic test function.
<!-- #region -->
## The Basic Anatomy of a Test Function
-Let's write a function that tests the following `count_values` code:
+Let's define a couple of functions that may look familiar from previous modules. We will be writing tests for these.
```python
-# Defining a function that we will be testing
+# Defining functions that we will be testing
-def count_vowels(x: str, include_y: bool = False) -> int:
+def count_vowels(x, include_y=False):
"""Returns the number of vowels contained in `x`.
The vowel 'y' is included optionally.
+
+ Parameters
+ ----------
+ x : str
+ The input string
+ include_y : bool, optional (default=False)
+ If `True` count y's as vowels
+
+ Returns
+ -------
+ vowel_count: int
Examples
--------
@@ -84,10 +99,39 @@ def count_vowels(x: str, include_y: bool = False) -> int:
if include_y:
vowels.update("yY")
return sum(1 for char in x if char in vowels)
-```
-(Note that we will be making use of [type hinting](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Writing_Good_Code.html#Type-Hinting) to help document the interfaces of our functions.
-You may want to briefly review the linked material if this is unfamiliar to you)
+
+def merge_max_mappings(dict1, dict2):
+ """ Merges two dictionaries based on the largest value
+ in a given mapping.
+
+ Parameters
+ ----------
+ dict1 : Dict[str, float]
+ dict2 : Dict[str, float]
+
+ Returns
+ -------
+ merged : Dict[str, float]
+ The dictionary containing all of the keys common
+ between `dict1` and `dict2`, retaining the largest
+ value from common mappings.
+
+ Examples
+ --------
+ >>> x = {"a": 1, "b": 2}
+ >>> y = {"b": 100, "c": -1}
+ >>> merge_max_mappings(x, y)
+ {'a': 1, 'b': 100, 'c': -1}}
+ """
+ # `dict(dict1)` makes a copy of `dict1`. We do this
+ # so that updating `merged` doesn't also update `dict1`
+ merged = dict(dict1)
+ for key in dict2:
+ if key not in merged or dict2[key] > merged[key]:
+ merged[key] = dict2[key]
+ return merged
+```
For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output.
The desired behavior for this test function, upon being run, is to:
@@ -134,9 +178,10 @@ Make sure to run your updated test to see if it passes.
### Testing Our Tests
-It is surprisingly easy to unwittingly write a test that always passes or that fails to test our code as we had intended.
-This is a particularly treacherous mistake to make as it leads us to falsely believe that our function is working as-expected.
-**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior in such a way that our test ought to raise an error.**
+It is surprisingly easy to unwittingly write a broken test: a test that always passes, or a test that simply doesn't exercise our code in the way that we had intended.
+Broken tests are insidious - they are alarms that will not sound when they are supposed to.
+They create misdirection in the bug-finding process and can mask problems with our code.
+**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
We ought to mutate our function in a way that is trivial to undo; we can use of code-comments towards this end.
@@ -158,11 +203,10 @@ Finally, rerun the test to see that `count_vowels` once again passes all of the
</div>
-
-With our first test function under our belt, it is time for us to clearly understand how `assert` statements work and how they should be used.
-
<!-- #region -->
### Assert Statements
+With our first test function under our belt, it is time for us to clearly understand how `assert` statements work and how they should be used.
+
Similar to `return`, `def`, or `if`, the term `assert` is a reserved term in the Python language.
It has the following specialized behavior:
From 2fbd4e604358a89fb4991ab85a5cc13f11d7c7fc Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Fri, 20 Dec 2019 11:09:02 -0500
Subject: [PATCH 025/152] adding summary to basic test structure
---
Python/Module6_Testing/Intro_to_Testing.md | 38 +++++++++++++++++-----
1 file changed, 30 insertions(+), 8 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 6098ac85..33bfe349 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -66,9 +66,12 @@ This all sounds great, but where do we even start the process writing a test sui
Let's begin by seeing what constitutes a basic test function.
<!-- #region -->
-## The Basic Anatomy of a Test Function
+## Writing Our First Tests
+
+### Our "Source Code"
+We need some code to test. For the sake of this introduction, let's borrow a couple of functions that may look familiar from previous modules.
+These will serve as our "source code"; i.e. these are functions that we have written for our project and that need to be tested.
-Let's define a couple of functions that may look familiar from previous modules. We will be writing tests for these.
```python
# Defining functions that we will be testing
@@ -122,18 +125,25 @@ def merge_max_mappings(dict1, dict2):
>>> x = {"a": 1, "b": 2}
>>> y = {"b": 100, "c": -1}
>>> merge_max_mappings(x, y)
- {'a': 1, 'b': 100, 'c': -1}}
+ {'a': 1, 'b': 100, 'c': -1}
"""
# `dict(dict1)` makes a copy of `dict1`. We do this
# so that updating `merged` doesn't also update `dict1`
merged = dict(dict1)
- for key in dict2:
- if key not in merged or dict2[key] > merged[key]:
- merged[key] = dict2[key]
+ for key, value in dict2.items():
+ if key not in merged or value > merged[key]:
+ merged[key] = value
return merged
```
-For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output.
+As always, it is useful for us follow along with this material in a Jupyter notebook.
+We ought to take time to define these functions and run inputs through them to make sure that we understand what they are doing.
+Testing code that we don't understand is a lost cause!
+<!-- #endregion -->
+<!-- #region -->
+### The Basic Anatomy of a Test
+
+Let's start by testing `count_vowels`. For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output.
The desired behavior for this test function, upon being run, is to:
- Raise an error if any of our assertions *failed* to hold true.
@@ -156,7 +166,7 @@ To run this test, we simply call the function:
>>> test_count_vowels_basic() # passes: returns None | fails: raises error
```
-As described above, the fact our function runs and simply returns `None` means that our code has passed this test. We've written and run our very first test! It certainly isn't the most robust test, but it is a good start.
+As described above, the fact our function runs and simply returns `None` (i.e. we see no output when we run this test in a console or notebook cell) means that our code has passed this test. We've written and run our very first test! It certainly isn't the most robust test, but it is a good start.
Let's look more carefully at the structure of `test_count_vowels_basic`.
Note that this function doesn't take in any inputs;
@@ -166,6 +176,18 @@ The latter scenario is by far the most common one in practice.
More on this later.
<!-- #endregion -->
+
+<div class="alert alert-warning">
+
+**Takeaway**:
+
+A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code, and assert that the code behaves as-expected. The basic anatomy of a test function typically is structured such that:
+
+- it typically does not accept any arguments - calling the function should
+
+</div>
+
+
<div class="alert alert-info">
**Reading Comprehension: The Basic Anatomy of a Test**
From 42117d278faf7b356037873eb267048716c9f984 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Fri, 20 Dec 2019 11:39:18 -0500
Subject: [PATCH 026/152] add test example for merge-max-dicts
---
Python/Module6_Testing/Intro_to_Testing.md | 105 ++++++++++++++++-----
1 file changed, 79 insertions(+), 26 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 33bfe349..7e7b6403 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -181,16 +181,18 @@ More on this later.
**Takeaway**:
-A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code, and assert that the code behaves as-expected. The basic anatomy of a test function typically is structured such that:
+A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code, and assert that the code behaves as-expected. The basic anatomy of a test function typically is structured such that it:
-- it typically does not accept any arguments - calling the function should
+- consists of a series of `assert` statements, each of which will raise an error if our source code misbehaves
+- simply returns `None` if all of the assertions held true
+- can be run end-to-end simply by calling the test function without passing it any parameters; we rely on Python's scoping rules to call our source code within the body of the test function without explicitly passing anything to said test function
</div>
<div class="alert alert-info">
-**Reading Comprehension: The Basic Anatomy of a Test**
+**Reading Comprehension: Adding Assertions to a Test**
Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
Make sure to run your updated test to see if it passes.
@@ -198,35 +200,16 @@ Make sure to run your updated test to see if it passes.
</div>
-### Testing Our Tests
-
-It is surprisingly easy to unwittingly write a broken test: a test that always passes, or a test that simply doesn't exercise our code in the way that we had intended.
-Broken tests are insidious - they are alarms that will not sound when they are supposed to.
-They create misdirection in the bug-finding process and can mask problems with our code.
-**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
-Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
-
-We ought to mutate our function in a way that is trivial to undo; we can use of code-comments towards this end.
-All [IDEs](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) have the ability to "block-comment" selected code.
-In a Jupyter notebook code cell, we can highlight multiple lines of code and press `CTRL + /`: this will comment-out these lines of code.
-The same key-combination will also un-comment a highlighted block of commented code.
-
-
-
<div class="alert alert-info">
-**Reading Comprehension: Testing Your Test via Manual Mutation**
+**Reading Comprehension: The Basic Anatomy of a Test**
-Temporarily change the body of `count_vowels` such that the second assertion in `test_count_vowels_basic` raises an error.
-Run the test to confirm that the second assertion raises,
-and then restore `count_vowels` to its original form.
-Finally, rerun the test to see that `count_vowels` once again passes all of the assertions.
+Write a rudimentary test function for `merge_max_mappings`. This should adhere to the basic structure of a test function that we just laid out. See if you can think of some "edge cases" to test, which we may have overlooked when writing `merge_max_mappings`.
</div>
-
<!-- #region -->
-### Assert Statements
+## Assert Statements
With our first test function under our belt, it is time for us to clearly understand how `assert` statements work and how they should be used.
Similar to `return`, `def`, or `if`, the term `assert` is a reserved term in the Python language.
@@ -331,6 +314,34 @@ Admittedly, the `count_vowels` function is simple enough that the inclusion of t
That being said, as we write increasingly sophisticated code, we will find that this sort of assertion will help us catch bad internal logic and oversights within our code base.
<!-- #endregion -->
+## Testing Our Tests
+
+It is surprisingly easy to unwittingly write a broken test: a test that always passes, or a test that simply doesn't exercise our code in the way that we had intended.
+Broken tests are insidious - they are alarms that will not sound when they are supposed to.
+They create misdirection in the bug-finding process and can mask problems with our code.
+**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
+Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
+
+We ought to mutate our function in a way that is trivial to undo; we can use of code-comments towards this end.
+All [IDEs](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) have the ability to "block-comment" selected code.
+In a Jupyter notebook code cell, we can highlight multiple lines of code and press `CTRL + /`: this will comment-out these lines of code.
+The same key-combination will also un-comment a highlighted block of commented code.
+
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Testing Your Test via Manual Mutation**
+
+Temporarily change the body of `count_vowels` such that the second assertion in `test_count_vowels_basic` raises an error.
+Run the test to confirm that the second assertion raises,
+and then restore `count_vowels` to its original form.
+Finally, rerun the test to see that `count_vowels` once again passes all of the assertions.
+
+</div>
+
+
+
## Links to Official Documentation
- [The assert statement](https://docs.python.org/3/reference/simple_stmts.html?highlight=assert#the-assert-statement)
@@ -339,15 +350,19 @@ That being said, as we write increasingly sophisticated code, we will find that
## Reading Comprehension Solutions
<!-- #region -->
-**The Basic Anatomy of a Test: Solution**
+**Adding Assertions to a Test: Solution**
Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
Make sure to run your updated test to see if it passes.
```python
def test_count_vowels_basic():
+ # test basic strings with uppercase and lowercase letters
assert count_vowels("aA bB yY", include_y=False) == 2
assert count_vowels("aA bB yY", include_y=True) == 4
+
+ # test empty strings
+ assert count_vowels("", include_y=False) == 0
assert count_vowels("", include_y=True) == 0
```
@@ -358,6 +373,44 @@ def test_count_vowels_basic():
```
<!-- #endregion -->
+<!-- #region -->
+**The Basic Anatomy of a Test: Solution**
+
+Write a rudimentary test function for `merge_max_mappings`.
+
+> Let's test the use case that is explicitly documented in the Examples section of the function's docstring.
+> We can also test cases where one or both of the inputs were empty dictionaries: can often be problematic edge cases that we didn't consider when writing our code.
+
+```python
+def test_merge_max_mappings():
+ # test documented behavior
+ dict1 = {"a": 1, "b": 2}
+ dict2 = {"b": 20, "c": -1}
+ expected = {'a': 1, 'b': 20, 'c': -1}
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test empty dict1
+ dict1 = {}
+ dict2 = {"a": 10.2, "f": -1.0}
+ expected = dict2
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test empty dict2
+ dict1 = {"a": 10.2, "f": -1.0}
+ dict2 = {}
+ expected = dict1
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test both empty
+ assert merge_max_mappings({}, {}) == {}
+```
+
+```python
+# running the test (seeing no errors means the tests all passed)
+>>> test_merge_max_mappings()
+```
+<!-- #endregion -->
+
<!-- #region -->
**Testing Your Test via Manual Mutation: Solution**
From 8672564fc9fc245c2bb5971435e12f00fef35ecf Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 21 Dec 2019 14:22:09 -0500
Subject: [PATCH 027/152] Finish intro to testing
---
Python/Module6_Testing/Intro_to_Testing.md | 141 ++++++++++++---------
Python/Module6_Testing/Pytest.md | 0
2 files changed, 84 insertions(+), 57 deletions(-)
create mode 100644 Python/Module6_Testing/Pytest.md
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 7e7b6403..5e93a6aa 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -21,28 +21,28 @@ jupyter:
# Introduction to Testing
-This section will show us just how simple it is to write rudimentary tests. We need only recall some of Python's basic scoping rules and introduce ourselves to the `assert` statement to write a genuine test function. That being said, this will simply kick off our testing journey. We will immediately encounter some important questions. How do we know that our tests work? And, how do we know that our tests are effective?
+This section will show us just how simple it is to write rudimentary tests. We need only recall some of Python's basic scoping rules and introduce ourselves to the `assert` statement to write a genuine test function. That being said, we will quickly encounter some important questions to ponder. How do we know that our tests work? And, how do we know that our tests are effective? These questions will drive us deeper into the world of testing.
Before we hit the ground running, let's take a moment to consider some motivations for testing out code.
## Why Should We Write Tests?
-With great power comes great responsibility: tests help us be responsible for the code that we create and that others will (hopefully) use.
The fact of the matter is that everyone already tests their code to some extent.
-After coding, say, a new function, it is only natural to contrive an input to feed it, and to check that it returns the output that you expected.
+After writing, say, a new function, it is only natural to contrive an input to feed it, and to check that the function returns the output that we expected.
To the extent that anyone would want to see evidence that their code works, we need not motivate the importance of testing.
-Less obvious is the massive benefits that we stand to gain from formalizing this testing process.
+Less obvious are the massive benefits that we stand to gain from formalizing this testing process.
And by "formalizing", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
-We will accumulate these functions into a "test suite" that we can run quickly and repeatedly.
+We will accumulate these test functions into a "test suite" that we can run quickly and repeatedly.
There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing a robust test suite:
- It saves us lots of time:
- > After you have devised a test scenario for your code, it may only take us a second or so to run it - perhaps we need only run a couple of Jupyter notebook cells to check the output.
- > However, this will quickly become unwieldy as we write more code and devise more test scenarios.
+ > After you have devised a test scenario for your code, it may only take us a second or so to run it; perhaps we need only run a couple of Jupyter notebook cells to verify the output of our code.
+ > This, however, will quickly become unwieldy as we write more code and devise more test scenarios.
> Soon we will be dissuaded from running our tests except for on rare occasions.
+ >
> With a proper test suite, we can run all of our test scenarios with the push of a button, and a series of green check-marks (or red x's...) will summarize the health of our project (insofar as our tests serve as good diagnostics).
> This, of course, also means that we will find and fix bugs much faster!
> In the long run, our test suite will afford us the ability to aggressively exercise (and exorcise) our code at little cost.
@@ -57,7 +57,7 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
- It will inform the design and usability of our project for the better:
> Although it may not be obvious from the outset, writing testable code leads to writing better code.
> This is, in part, because the process of writing tests gives us the opportunity to actually _use_ our code under varied circumstances.
- > The process of writing tests will help us suss out cumbersome function interfaces, brittle statefulness, and redundant capabilities in our code. If _we_ find it frustrating to use our code within our tests, then surely others will find it frustrating to use in applied settings.
+ > The process of writing tests will help us suss out cumbersome function interfaces, brittle statefulness, and redundant capabilities in our code. Ultimately, if _we_ find it frustrating to use our code within our tests, then surely others will find the code frustrating to use in applied settings.
- It makes it easier for others to contribute to a project:
> Having a healthy test suite lowers the barrier to entry for a project.
> A contributor can make improvements to the project and quickly check to see if they have broken it or changed any of its behavior.
@@ -143,7 +143,7 @@ Testing code that we don't understand is a lost cause!
<!-- #region -->
### The Basic Anatomy of a Test
-Let's start by testing `count_vowels`. For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output.
+Let's write a test for `count_vowels`. For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output.
The desired behavior for this test function, upon being run, is to:
- Raise an error if any of our assertions *failed* to hold true.
@@ -171,7 +171,7 @@ As described above, the fact our function runs and simply returns `None` (i.e. w
Let's look more carefully at the structure of `test_count_vowels_basic`.
Note that this function doesn't take in any inputs;
thanks to [Python's scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html), we can reference our `count_vowels` function within our test as long as it is defined in the same "namespace" as `test_count_vowels_basic`.
-That is, we can either define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material in a notebook) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels` from wherever it is defined, and into the file containing our test.
+That is, we can either define `count_vowels` in the same .py file (or Jupyter notebook, if you are following along with this material in a notebook) as `test_count_vowels_basic`, or we can [import](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Import-Statements) `count_vowels`, from wherever it is defined, into the file containing our test.
The latter scenario is by far the most common one in practice.
More on this later.
@@ -181,10 +181,10 @@ More on this later.
**Takeaway**:
-A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code, and assert that the code behaves as-expected. The basic anatomy of a test function typically is structured such that it:
+A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code and assert that the code behaves as-expected. The basic anatomy of a test function is such that it:
-- consists of a series of `assert` statements, each of which will raise an error if our source code misbehaves
-- simply returns `None` if all of the assertions held true
+- consists one or more `assert` statements, each of which will raise an error if our source code misbehaves
+- simply returns `None` if all of the aforementioned assertions held true
- can be run end-to-end simply by calling the test function without passing it any parameters; we rely on Python's scoping rules to call our source code within the body of the test function without explicitly passing anything to said test function
</div>
@@ -194,7 +194,7 @@ A "test function" is designed to provide an encapsulated "environment" (namespac
**Reading Comprehension: Adding Assertions to a Test**
-Add an additional assertion to the body of `test_count_vowels_basic`, which tests whether `count_vowels` handles the empty-string (`""`) case appropriately.
+Add an additional assertion to the body of `test_count_vowels_basic`, which tests that `count_vowels` handles empty-string (`""`) input appropriately.
Make sure to run your updated test to see if it passes.
</div>
@@ -209,8 +209,8 @@ Write a rudimentary test function for `merge_max_mappings`. This should adhere t
</div>
<!-- #region -->
-## Assert Statements
-With our first test function under our belt, it is time for us to clearly understand how `assert` statements work and how they should be used.
+## The `assert` Statement
+With our first test functions under our belt, it is time for us to clearly understand how `assert` statements work and how they should be used.
Similar to `return`, `def`, or `if`, the term `assert` is a reserved term in the Python language.
It has the following specialized behavior:
@@ -254,7 +254,7 @@ See that the assertion statement:
assert expression, error_message
```
-is effectively shorthand for the following code:
+is effectively shorthand for the following code (barring some additional details):
```python
# long-form equivalent of: `assert expression, error_message`
@@ -312,19 +312,20 @@ Rather, it is meant to assert that our own internal logic holds true.
Admittedly, the `count_vowels` function is simple enough that the inclusion of this assertion is rather pedantic.
That being said, as we write increasingly sophisticated code, we will find that this sort of assertion will help us catch bad internal logic and oversights within our code base.
+We will also see that keen use of assertions can make it much easier for us to write good tests.
<!-- #endregion -->
## Testing Our Tests
It is surprisingly easy to unwittingly write a broken test: a test that always passes, or a test that simply doesn't exercise our code in the way that we had intended.
-Broken tests are insidious - they are alarms that will not sound when they are supposed to.
+Broken tests are insidious; they are alarms that fail to sound when they are supposed to.
They create misdirection in the bug-finding process and can mask problems with our code.
**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
-We ought to mutate our function in a way that is trivial to undo; we can use of code-comments towards this end.
+A practical note: we ought to mutate our function in a way that is trivial to undo. We can use of code-comments towards this end.
All [IDEs](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) have the ability to "block-comment" selected code.
-In a Jupyter notebook code cell, we can highlight multiple lines of code and press `CTRL + /`: this will comment-out these lines of code.
+In order to block-comment code in a Jupyter notebook code cell, highlight the lines of code and press `CTRL + /`.
The same key-combination will also un-comment a highlighted block of commented code.
@@ -337,11 +338,30 @@ Temporarily change the body of `count_vowels` such that the second assertion in
Run the test to confirm that the second assertion raises,
and then restore `count_vowels` to its original form.
Finally, rerun the test to see that `count_vowels` once again passes all of the assertions.
+
+Repeat this process given the test that you wrote for `merge_max_mappings`.
+Try breaking the function such that it always merges in values from `dict2`, even if those values are smaller.
</div>
+## Our Work, Cut Out
+
+We see now that the concept of a "test function" isn't all that fancy.
+Compared to other code that we have written, writing a function that simply runs a hand full of assertions is far from a heavy lift for us.
+Of course, we must be diligent and take care to test our tests, but we can certainly manage this as well.
+With this in hand, we should take stock of the work and challenges that lie in our path ahead.
+
+It is necessary that we evolve beyond manual testing.
+There are multiple facets to this observation.
+First, we must learn how to organize our test functions into a test suite that can be run in one fowl swoop.
+Next, it will become increasingly apparent that a test function often contain large amounts of redundant code, shared across its litany of assertions.
+We will want to "parametrize" our tests to distill them down to their most concise and functional forms.
+Finally, and most importantly, it may already evident that the process of contriving known inputs and outputs to use in our tests is a highly manual and tedious process; furthermore, it is a process that will become increasingly cumbersome as our source code becomes more sophisticated.
+To combat this, we will seek out alternative, powerful testing methodologies, including property-based testing.
+
+
## Links to Official Documentation
- [The assert statement](https://docs.python.org/3/reference/simple_stmts.html?highlight=assert#the-assert-statement)
@@ -379,7 +399,8 @@ def test_count_vowels_basic():
Write a rudimentary test function for `merge_max_mappings`.
> Let's test the use case that is explicitly documented in the Examples section of the function's docstring.
-> We can also test cases where one or both of the inputs were empty dictionaries: can often be problematic edge cases that we didn't consider when writing our code.
+> We can also test cases where one or both of the inputs are empty dictionaries.
+> These can often be problematic edge cases that we didn't consider when writing our code.
```python
def test_merge_max_mappings():
@@ -402,7 +423,10 @@ def test_merge_max_mappings():
assert merge_max_mappings(dict1, dict2) == expected
# test both empty
- assert merge_max_mappings({}, {}) == {}
+ dict1 = {}
+ dict2 = {}
+ expected = {}
+ assert merge_max_mappings(dict1, dict2) == expected
```
```python
@@ -411,11 +435,44 @@ def test_merge_max_mappings():
```
<!-- #endregion -->
+<!-- #region -->
+**Assertions: Solution**
+```python
+a_list = []
+a_number = 22
+a_string = "abcdef"
+```
+
+Assert that `a_list` is _not_ empty:
+
+```python
+>>> assert a_list
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-10-2eba8294859e> in <module>
+----> 1 assert a_list
+
+AssertionError:
+```
+
+> You may have written `assert len(a_list) > 0` - this is also correct.
+> However, recall that calling `bool` on any sequence (list, tuple, string, etc.) will return `False` if the sequence is empty.
+> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - that `bool` will be called on whatever the provided expression is.
+
+Assert that the number of vowels in `a_string` is fewer than `a_number`; include and error message that prints the actual number of vowels:
+
+```python
+>>> assert count_vowels(a_string) < a_number, f"Number of vowels, {count_vowels(a_string)}, exceeds {a_number}"
+```
+
+> Note that we make use of an [f-string](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) as a convenient means for writing an informative error message.
+<!-- #endregion -->
+
<!-- #region -->
**Testing Your Test via Manual Mutation: Solution**
Temporarily change the body of `count_vowels` such that the _second_ assertion in `test_count_vowels_basic` raises an error.
-> Let's comment out the `if include_y` block in our code - this should prevent us from counting y's, and thus should violate the second assertion in our test.
+> Let's comment out the `if include_y` block in our code. This should prevent us from counting y's, and thus should violate the second assertion in our test.
```python
# Breaking the behavior of `include_y=True`
@@ -442,10 +499,13 @@ AssertionError Traceback (most recent call last)
AssertionError:
```
-> Great! That assertion really does help to ensure that we are counting y's correctly.
+> See that the error output, which is called a "stack trace", indicates with an ASCII-arrow that our second assertion is the one that is failing.
+> Thus we can be confident that that assertion really does help to ensure that we are counting y's correctly.
Restore `count_vowels` to its original form and rerun the test to see that `count_vowels` once again passes all of the assertions.
+> We simply un-comment out the block of code and rerun our test.
+
```python
# Restore the behavior of `include_y=True`
def count_vowels(x: str, include_y: bool = False) -> int:
@@ -460,36 +520,3 @@ def count_vowels(x: str, include_y: bool = False) -> int:
>>> test_count_vowels_basic()
```
<!-- #endregion -->
-
-<!-- #region -->
-**Assertions: Solution**
-```python
-a_list = []
-a_number = 22
-a_string = "abcdef"
-```
-
-Assert that `a_list` is _not_ empty:
-
-```python
->>> assert a_list
----------------------------------------------------------------------------
-AssertionError Traceback (most recent call last)
-<ipython-input-10-2eba8294859e> in <module>
-----> 1 assert a_list
-
-AssertionError:
-```
-
-> You may have written `assert len(a_list) > 0` - this is also correct.
-> However, recall that calling `bool` on any sequence (list, tuple, string, etc.) will return `False` if the sequence is empty.
-> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - that `bool` will be called on whatever the provided expression is.
-
-Assert that the number of vowels in `a_string` is fewer than `a_number`; include and error message that prints the actual number of vowels:
-
-```python
->>> assert count_vowels(a_string) < a_number, f"Number of vowels, {count_vowels(a_string)}, exceeds {a_number}"
-```
-
-> Note that we make use of an [f-string](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) as a convenient means for writing an informative error message.
-<!-- #endregion -->
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
new file mode 100644
index 00000000..e69de29b
From 5378e3b8b11fcd309a85916347576332ab607f2e Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 21 Dec 2019 14:49:28 -0500
Subject: [PATCH 028/152] add initial pytest section
---
Python/Module6_Testing/Pytest.md | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index e69de29b..2c86a7de 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -0,0 +1,24 @@
+---
+jupyter:
+ jupytext:
+ formats: ipynb,md
+ text_representation:
+ extension: .md
+ format_name: markdown
+ format_version: '1.2'
+ jupytext_version: 1.3.0
+ kernelspec:
+ display_name: Python 3
+ language: python
+ name: python3
+---
+
+<!-- #raw raw_mimetype="text/restructuredtext" -->
+.. meta::
+ :description: Topic: Writing tests for your code, Difficulty: Easy, Category: Section
+ :keywords: test, automated, pytest, parametrize, fixture, suite
+<!-- #endraw -->
+
+# Introducing the Pytest Framework
+
+Thus far, the process of running tests is a entirely manual one. It is time for us to organize our test functions into a proper "test suite" and to learn to leverage the pytest framework to run them. Looking at the tests that we have written, it is also may be apparent that there
From 5a84be95aa74a9d923f29255807f46e81f9b371c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 21 Dec 2019 14:52:06 -0500
Subject: [PATCH 029/152] remove ipynb pairing
---
Python/Module6_Testing/Intro_to_Testing.md | 1 -
Python/Module6_Testing/Pytest.md | 1 -
2 files changed, 2 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 5e93a6aa..d0c88832 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -1,7 +1,6 @@
---
jupyter:
jupytext:
- formats: ipynb,md
text_representation:
extension: .md
format_name: markdown
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 2c86a7de..bbd647fc 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -1,7 +1,6 @@
---
jupyter:
jupytext:
- formats: ipynb,md
text_representation:
extension: .md
format_name: markdown
From 3acb67cf60987f2d7bf90590329024ae55354ec7 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 21 Dec 2019 14:58:31 -0500
Subject: [PATCH 030/152] fix bad import in setuptools code snippet
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index f1c7d257..4873f804 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -440,9 +440,9 @@ The bare bones build script for preparing your package for installation, `setup.
```python
# contents of setup.py
-import setuptools
+from setuptools import find_packages, setup
-setuptools.setup(
+setup(
name="face_detection",
version="1.0.0",
packages=find_packages(exclude=["tests", "tests.*"]),
From d10ce3c693dbdfe3f7fb3cf09195fe0772e8169e Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 22 Dec 2019 13:12:22 -0500
Subject: [PATCH 031/152] fix docstring formatting in numpy-broadcasting
---
.../Module3_IntroducingNumpy/Broadcasting.md | 74 +++++++++----------
1 file changed, 37 insertions(+), 37 deletions(-)
diff --git a/Python/Module3_IntroducingNumpy/Broadcasting.md b/Python/Module3_IntroducingNumpy/Broadcasting.md
index 5c8a96fc..cd83934a 100644
--- a/Python/Module3_IntroducingNumpy/Broadcasting.md
+++ b/Python/Module3_IntroducingNumpy/Broadcasting.md
@@ -5,7 +5,7 @@ jupyter:
extension: .md
format_name: markdown
format_version: '1.2'
- jupytext_version: 1.3.0rc1
+ jupytext_version: 1.3.0
kernelspec:
display_name: Python 3
language: python
@@ -486,16 +486,16 @@ Performing this computation using for-loops proceeds as follows:
def pairwise_dists_looped(x, y):
""" Computing pairwise distances using for-loops
- Parameters
- ----------
- x : numpy.ndarray, shape=(M, D)
- y : numpy.ndarray, shape=(N, D)
+ Parameters
+ ----------
+ x : numpy.ndarray, shape=(M, D)
+ y : numpy.ndarray, shape=(N, D)
- Returns
- -------
- numpy.ndarray, shape=(M, N)
- The Euclidean distance between each pair of
- rows between `x` and `y`."""
+ Returns
+ -------
+ numpy.ndarray, shape=(M, N)
+ The Euclidean distance between each pair of
+ rows between `x` and `y`."""
# `dists[i, j]` will store the Euclidean
# distance between `x[i]` and `y[j]`
dists = np.empty((5, 6))
@@ -545,18 +545,18 @@ Voilà! We have produced the distances in a vectorized way. Let's write this out
def pairwise_dists_crude(x, y):
""" Computing pairwise distances using vectorization.
- This method uses memory-inefficient broadcasting.
-
- Parameters
- ----------
- x : numpy.ndarray, shape=(M, D)
- y : numpy.ndarray, shape=(N, D)
-
- Returns
- -------
- numpy.ndarray, shape=(M, N)
- The Euclidean distance between each pair of
- rows between `x` and `y`."""
+ This method uses memory-inefficient broadcasting.
+
+ Parameters
+ ----------
+ x : numpy.ndarray, shape=(M, D)
+ y : numpy.ndarray, shape=(N, D)
+
+ Returns
+ -------
+ numpy.ndarray, shape=(M, N)
+ The Euclidean distance between each pair of
+ rows between `x` and `y`."""
# The use of `np.newaxis` here is equivalent to our
# use of the `reshape` function
return np.sqrt(np.sum((x[:, np.newaxis] - y[np.newaxis])**2, axis=2))
@@ -628,23 +628,23 @@ In total, we have successfully used vectorization to compute the all pairs of di
```python
def pairwise_dists(x, y):
- """ Computing pairwise distances using memory-efficient
- vectorization.
-
- Parameters
- ----------
- x : numpy.ndarray, shape=(M, D)
- y : numpy.ndarray, shape=(N, D)
-
- Returns
- -------
- numpy.ndarray, shape=(M, N)
- The Euclidean distance between each pair of
- rows between `x` and `y`."""
+ """ Computing pairwise distances using memory-efficient
+ vectorization.
+
+ Parameters
+ ----------
+ x : numpy.ndarray, shape=(M, D)
+ y : numpy.ndarray, shape=(N, D)
+
+ Returns
+ -------
+ numpy.ndarray, shape=(M, N)
+ The Euclidean distance between each pair of
+ rows between `x` and `y`."""
dists = -2 * np.matmul(x, y.T)
- dists += np.sum(x**2, axis=1)[:, np.newaxis]
+ dists += np.sum(x**2, axis=1)[:, np.newaxis]
dists += np.sum(y**2, axis=1)
- return np.sqrt(dists)
+ return np.sqrt(dists)
```
From fc2c7b199eaf6dc325ab1690afd8b489447d4642 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 22 Dec 2019 14:02:56 -0500
Subject: [PATCH 032/152] progress on pytest section - basic project layout
---
Python/Module6_Testing/Pytest.md | 40 +++++++++++++++++++++++++++++++-
1 file changed, 39 insertions(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index bbd647fc..538679ec 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -20,4 +20,42 @@ jupyter:
# Introducing the Pytest Framework
-Thus far, the process of running tests is a entirely manual one. It is time for us to organize our test functions into a proper "test suite" and to learn to leverage the pytest framework to run them. Looking at the tests that we have written, it is also may be apparent that there
+Thus far, the process of running tests is a entirely manual one. It is time for us to arranging our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
+We will begin by reorganizing our source code to create an installable [package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
+We will then learn how to structure and run a test suite for this Python package, using pytest.
+
+The pytest framework does much more than just run tests;
+for instance, it will enrich the assertions in our tests to produce verbose, informative error messages.
+Furthermore it provides valuable means for enhancing our tests via mechanisms like fixtures and parameterizing decorators.
+Ultimately, all of this functionality helps to eliminate manual aspects from the testing process.
+
+
+## Organizing a Test Suite
+
+It's time to create a proper test suite.
+Before proceeding any further, we should reread the material presented in [Module 5 - Import: Modules and Packages](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html) and recall the essentials of import statements, modules, and Python packages.
+This material serves as the foundation for this section.
+
+Let's create the a Python package, which we will call `plymi_mod6`, with the following directory structure:
+
+```
+- setup.py # script responsible for installing `plymi_mod6` package
+- plymi_mod6/ # directory containing source code of `plymi_mod6` package
+ |-- __init__.py
+ |-- basic_functions.py
+ |-- numpy_functions.py
+- tests/ # test-suite for `plymi_mod6` package (to be run using pytest)
+ |-- conf.py # optional configuration file for pytest
+ |-- test_basic_functions.py
+ |-- test_numpy_functions.py
+```
+
+A reference implementation of this package can be found [in this GitHub repository](https://github.com/rsokl/plymi_mod6).
+
+
+## Links to Official Documentation
+
+- [pytest](https://docs.pytest.org/en/latest/)
+
+
+## Reading Comprehension Solutions
From 38efd62f9d3d8f826772e910fd2df2ef4d3e50fd Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 23 Dec 2019 12:13:03 -0500
Subject: [PATCH 033/152] minor revision to wording in Install Your Own Package
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index 4873f804..caf5eaf7 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -465,7 +465,7 @@ and voilà, your package `face_detection` will have been installed to site-packa
pip uninstall face_detection
```
-One final but important detail. The installed version of your package will no longer "see" the source code. That is, if you go on to make any changes to your code, you will have to uninstall and reinstall your package before your will see the effects system-wide. Instead you can install your package in develop mode, such that a symbolic link to your source code is placed in your site-packages. Thus any changes that you make to your code will immediately be reflected in your system-wide installation. Thus, instead of running `python setup.py install`, execute the following to install a package in develop mode:
+One final but important detail. The installed version of your package will no longer "see" the source code. That is, if you go on to make any changes to your code, you will have to uninstall and reinstall your package before your will see the effects system-wide. Instead you can install your package in "development mode", such that a symbolic link to your source code is placed in your site-packages. Thus any changes that you make to your code will immediately be reflected in your system-wide installation. Thus, instead of running `python setup.py install`, execute the following to install a package in develop mode:
```shell
python setup.py develop
From 10edfd27183f4215fcd9ac8c10d1b02785202d12 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 23 Dec 2019 12:13:18 -0500
Subject: [PATCH 034/152] complete section on installing your own source code
---
Python/Module6_Testing/Pytest.md | 86 +++++++++++++++++++++++++++-----
1 file changed, 74 insertions(+), 12 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 538679ec..420e9f95 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -20,38 +20,100 @@ jupyter:
# Introducing the Pytest Framework
-Thus far, the process of running tests is a entirely manual one. It is time for us to arranging our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
-We will begin by reorganizing our source code to create an installable [package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
+Thus far, our process for running tests has been a entirely manual one. It is time for us to arranging our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
+We will begin by reorganizing our source code to create an installable [Python package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
We will then learn how to structure and run a test suite for this Python package, using pytest.
The pytest framework does much more than just run tests;
for instance, it will enrich the assertions in our tests to produce verbose, informative error messages.
Furthermore it provides valuable means for enhancing our tests via mechanisms like fixtures and parameterizing decorators.
-Ultimately, all of this functionality helps to eliminate manual aspects from the testing process.
+Ultimately, all of this functionality helps to eliminate manual aspects from the testing process.
+You may want to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments), so that you can work through this material starting from a blank slate.
+If you do, be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
+Let's install pytest. In your terminal where you can access your conda environment, run:
+
+```shell
+conda install -c conda-forge pytest
+```
+
+Installing from [the conda-forge channel](https://conda-forge.org/) will install the most up-to-date version of pytest.
+Or you can install pytest via pip:
+
+```shell
+pip install pytest
+```
+
+<!-- #region -->
## Organizing a Test Suite
It's time to create a proper test suite.
Before proceeding any further, we should reread the material presented in [Module 5 - Import: Modules and Packages](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html) and recall the essentials of import statements, modules, and Python packages.
This material serves as the foundation for this section.
+### Organizing our Source Code
Let's create the a Python package, which we will call `plymi_mod6`, with the following directory structure:
```
-- setup.py # script responsible for installing `plymi_mod6` package
-- plymi_mod6/ # directory containing source code of `plymi_mod6` package
- |-- __init__.py
- |-- basic_functions.py
- |-- numpy_functions.py
-- tests/ # test-suite for `plymi_mod6` package (to be run using pytest)
- |-- conf.py # optional configuration file for pytest
- |-- test_basic_functions.py
- |-- test_numpy_functions.py
+project_dir/ # the "parent directory" houses our source code, tests, and all other relevant files
+ - setup.py # script responsible for installing `plymi_mod6` package
+ - plymi_mod6/ # directory containing source code of `plymi_mod6` package
+ |-- __init__.py
+ |-- basic_functions.py
+ |-- numpy_functions.py
+ - tests/ # test-suite for `plymi_mod6` package (to be run using pytest)
+ |-- conf.py # optional configuration file for pytest
+ |-- test_basic_functions.py
+ |-- test_numpy_functions.py
```
A reference implementation of this package can be found [in this GitHub repository](https://github.com/rsokl/plymi_mod6).
+Populate the _basic_functions.py_ file with the two functions that we were using as our source code in the previous section: `count_vowels` and `merge_max_mappings`.
+In in the _numpy_functions.py_ module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
+Don't forget to include `import numpy as np` in your script in accordance with how `pairwise_dists` calls NumPy functions.
+
+We have arranged these functions so that they can be imported from the `basic_functions` module and the `numpy_functions` module, respectively, which reside in our `plymi_mod6` package.
+Let's fill out our _setup.py_ script and install this package so that we can import it regardless of our current working directory. The content of _setup.py_ will be:
+
+```python
+from setuptools import find_packages, setup
+
+setup(
+ name="plymi_mod6",
+ packages=find_packages(exclude=["tests", "tests.*"]),
+ version="1.0.0",
+ author="Your Name",
+ author_email="your.email@email.com",
+ description="A template Python package for learning about testing",
+ install_requires=["numpy >= 1.10.0"],
+ tests_require=["pytest", "hypothesis"],
+ python_requires=">=3.6",
+)
+```
+
+This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6.
+
+Finally, let's install our package locally [in development mode](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Installing-Your-Own-Python-Package).
+Navigate to the directory containing _setup.py_ and run:
+
+```shell
+python setup.py develop
+```
+Now, we should be able to start a python console or Jupyter notebook in and directory and import our package:
+
+```shell
+(plymi_test_env) C:\Users\plymi_person>ipython
+Python 3.7.5 (default, Oct 32 2200, 15:18:51) [MSC v.1916 64 bit (AMD64)]
+IPython 7.10.2 -- An enhanced Interactive Python. Type '?' for help.
+
+In [1]: from plymi_mod6.basic_functions import count_vowels
+
+In [2]: count_vowels("Happy birthday", include_y=True)
+Out[2]: 5
+```
+<!-- #endregion -->
## Links to Official Documentation
From e946e47149ef41d05aae0fc68f9d7701cf562cad Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 23 Dec 2019 13:43:43 -0500
Subject: [PATCH 035/152] add material for populating/running a test suite
---
Python/Module6_Testing/Pytest.md | 89 ++++++++++++++++++++++++++++++--
1 file changed, 86 insertions(+), 3 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 420e9f95..3ea481d3 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -46,7 +46,7 @@ pip install pytest
```
<!-- #region -->
-## Organizing a Test Suite
+## Creating a Python Package with Tests
It's time to create a proper test suite.
Before proceeding any further, we should reread the material presented in [Module 5 - Import: Modules and Packages](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html) and recall the essentials of import statements, modules, and Python packages.
@@ -73,7 +73,7 @@ Populate the _basic_functions.py_ file with the two functions that we were using
In in the _numpy_functions.py_ module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
Don't forget to include `import numpy as np` in your script in accordance with how `pairwise_dists` calls NumPy functions.
-We have arranged these functions so that they can be imported from the `basic_functions` module and the `numpy_functions` module, respectively, which reside in our `plymi_mod6` package.
+We have arranged these functions so that they can be imported from the _basic_functions_ module and the _numpy_functions_ module, respectively, which reside in our `plymi_mod6` package.
Let's fill out our _setup.py_ script and install this package so that we can import it regardless of our current working directory. The content of _setup.py_ will be:
```python
@@ -101,7 +101,7 @@ Navigate to the directory containing _setup.py_ and run:
python setup.py develop
```
-Now, we should be able to start a python console or Jupyter notebook in and directory and import our package:
+Now, we should be able to start a python console, IPython console, or Jupyter notebook in and directory and import our package:
```shell
(plymi_test_env) C:\Users\plymi_person>ipython
@@ -115,9 +115,92 @@ Out[2]: 5
```
<!-- #endregion -->
+## Populating Our Test Suite
+
+pytest's system for "test discovery" is quite simple:
+pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files that have the word "test" in their names and will run them all.
+Thus, let's populate _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module.
+As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our in the same namespace as our tests.
+A reference implementation can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
+
+Without further ado, let's run our test suite! In our terminals, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
+Following output should appear:
+
+
+```
+$ pytest tests/
+============================= test session starts =============================
+platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
+rootdir: C:\Users\plymi_user\Learning_Python\plymi_mod6_src
+collected 2 items
+
+tests\test_basic_functions.py .. [100%]
+
+============================== 2 passed in 0.02s ==============================
+```
+
+
+This output indicates that two test-functions were found, both of them located in `tests/test_basic_functions.py`, and that both tests "passed", i.e. both functions ran without raising any errors.
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Running a Test Suite**
+
+Temporarily add a new "broken" test to `tests/test_basic_functions.py`.
+The name that you give this test should adhere to pytest's simple rules for test-discovery.
+Design the test function so that is sure to fail when it is run.
+Rerun your test suite and compare its output to what you saw before - is it easy to identify which test failed and what caused it to fail?
+Make sure to remove this function from your test suite once you are finished answering this question.
+
+</div>
+
+
+
## Links to Official Documentation
- [pytest](https://docs.pytest.org/en/latest/)
## Reading Comprehension Solutions
+
+<!-- #region -->
+**Running a Test Suite: Solution**
+
+> Let's add the test function `test_broken_function` to our test suite.
+> We must include the word "test" in the function's name so that pytest will identify it as a test to run.
+> There are limitless ways in which we can make this test fail; we'll introduce a trivial false-assertion:
+
+```python
+def test_broken_function():
+ assert [1, 2, 3] == [1, 2]
+```
+
+> After introducing this broken test into _test_basic_functions.py_ , running our tests should result in the following output:
+
+```
+$ pytest tests/
+============================= test session starts =============================
+platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
+rootdir: C:\Users\Ryan Soklaski\Learning_Python\plymi_mod6_src
+collected 3 items
+
+tests\test_basic_functions.py ..F [100%]
+
+================================== FAILURES ===================================
+____________________________ test_broken_function _____________________________
+
+ def test_broken_function():
+> assert [1, 2, 3] == [1, 2]
+E assert [1, 2, 3] == [1, 2]
+E Left contains one more item: 3
+E Use -v to get the full diff
+
+tests\test_basic_functions.py:41: AssertionError
+========================= 1 failed, 2 passed in 0.06s =========================
+
+```
+
+> Three tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests passed and the third test failed.
+> It then indicates which test failed, and specifically that the assertion was false because a length-2 list cannot be equal to a length-3 list.
+<!-- #endregion -->
From 30ebcf052be6c6535a41f357bc6f6139334a0cab Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Tue, 24 Dec 2019 15:57:51 -0500
Subject: [PATCH 036/152] add module 6 to index.rst, and add add relevant md
files to subsection
---
Python/index.rst | 1 +
Python/module_6.rst | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/Python/index.rst b/Python/index.rst
index f8650de9..3274b75e 100644
--- a/Python/index.rst
+++ b/Python/index.rst
@@ -64,6 +64,7 @@ I started learning to use Python in graduate school for my physics research, and
module_3_problems.rst
module_4.rst
module_5.rst
+ module_6.rst
changes.rst
Indices and tables
diff --git a/Python/module_6.rst b/Python/module_6.rst
index 3aacaf32..53fb7d54 100644
--- a/Python/module_6.rst
+++ b/Python/module_6.rst
@@ -14,6 +14,7 @@ This will take us down a bit of a rabbit hole, where we will find the powerful p
:maxdepth: 2
:caption: Contents:
- Module6_Testing/Writing_Good_Code.md
+ Module6_Testing/Intro_to_Testing.md
+ Module6_Testing/Pytest.md
From 3f021e37f7265b29c01a039601ae3f4ea2f2ffa8 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Tue, 24 Dec 2019 15:59:37 -0500
Subject: [PATCH 037/152] fix sub-bullet indentation
---
Python/Module6_Testing/Intro_to_Testing.md | 57 +++++++++++++---------
1 file changed, 33 insertions(+), 24 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index d0c88832..22f16f6e 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -24,7 +24,7 @@ This section will show us just how simple it is to write rudimentary tests. We n
Before we hit the ground running, let's take a moment to consider some motivations for testing out code.
-
+<!-- #region -->
## Why Should We Write Tests?
The fact of the matter is that everyone already tests their code to some extent.
@@ -37,32 +37,41 @@ We will accumulate these test functions into a "test suite" that we can run quic
There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing a robust test suite:
-- It saves us lots of time:
- > After you have devised a test scenario for your code, it may only take us a second or so to run it; perhaps we need only run a couple of Jupyter notebook cells to verify the output of our code.
- > This, however, will quickly become unwieldy as we write more code and devise more test scenarios.
- > Soon we will be dissuaded from running our tests except for on rare occasions.
- >
- > With a proper test suite, we can run all of our test scenarios with the push of a button, and a series of green check-marks (or red x's...) will summarize the health of our project (insofar as our tests serve as good diagnostics).
- > This, of course, also means that we will find and fix bugs much faster!
- > In the long run, our test suite will afford us the ability to aggressively exercise (and exorcise) our code at little cost.
-- It increases the "shelf life" of our code:
- > If you've ever dusted off a project that you haven't used for years (or perhaps only months or weeks...), you might know the tribulations of getting old code to work.
- > Perhaps, in the interim, new versions of our project's dependencies, like PyTorch or Matplotlib, were released and have incompatibilities with our project's code.
- > And perhaps _we can't even remember_ all of the ways in which our project is supposed to work.
- > Our test suite provides us with a simple and incisive way to dive back into our work.
- > It will point us to any potential incompatibilities that have accumulated over time.
- > It also provides us with a large collection of detailed use-cases of our code;
- > we can read through our tests remind ourselves of the inner-workings of our project.
-- It will inform the design and usability of our project for the better:
- > Although it may not be obvious from the outset, writing testable code leads to writing better code.
- > This is, in part, because the process of writing tests gives us the opportunity to actually _use_ our code under varied circumstances.
- > The process of writing tests will help us suss out cumbersome function interfaces, brittle statefulness, and redundant capabilities in our code. Ultimately, if _we_ find it frustrating to use our code within our tests, then surely others will find the code frustrating to use in applied settings.
-- It makes it easier for others to contribute to a project:
- > Having a healthy test suite lowers the barrier to entry for a project.
- > A contributor can make improvements to the project and quickly check to see if they have broken it or changed any of its behavior.
+**It saves us lots of time**:
+
+> After you have devised a test scenario for your code, it may only take us a second or so to run it; perhaps we need only run a couple of Jupyter notebook cells to verify the output of our code.
+> This, however, will quickly become unwieldy as we write more code and devise more test scenarios.
+> Soon we will be dissuaded from running our tests except for on rare occasions.
+>
+> With a proper test suite, we can run all of our test scenarios with the push of a button, and a series of green check-marks (or red x's...) will summarize the health of our project (insofar as our tests serve as good diagnostics).
+> This, of course, also means that we will find and fix bugs much faster!
+> In the long run, our test suite will afford us the ability to aggressively exercise (and exorcise) our code at little cost.
+
+**It increases the "shelf life" of our code:**
+
+> If you've ever dusted off a project that you haven't used for years (or perhaps only months or weeks...), you might know the tribulations of getting old code to work.
+> Perhaps, in the interim, new versions of our project's dependencies, like PyTorch or Matplotlib, were released and have incompatibilities with our project's code.
+> And perhaps _we can't even remember_ all of the ways in which our project is supposed to work.
+> Our test suite provides us with a simple and incisive way to dive back into our work.
+> It will point us to any potential incompatibilities that have accumulated over time.
+> It also provides us with a large collection of detailed use-cases of our code;
+> we can read through our tests remind ourselves of the inner-workings of our project.
+
+
+**It will inform the design and usability of our project for the better:**
+
+> Although it may not be obvious from the outset, writing testable code leads to writing better code.
+> This is, in part, because the process of writing tests gives us the opportunity to actually _use_ our code under varied circumstances.
+> The process of writing tests will help us suss out cumbersome function interfaces, brittle statefulness, and redundant capabilities in our code. Ultimately, if _we_ find it frustrating to use our code within our tests, then surely others will find the code frustrating to use in applied settings.
+
+**It makes it easier for others to contribute to a project:**
+
+> Having a healthy test suite lowers the barrier to entry for a project.
+> A contributor can make improvements to the project and quickly check to see if they have broken it or changed any of its behavior.
This all sounds great, but where do we even start the process writing a test suite?
Let's begin by seeing what constitutes a basic test function.
+<!-- #endregion -->
<!-- #region -->
## Writing Our First Tests
From a8477063bb9a5a26d0cc3e2f8992843785771f1d Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Tue, 24 Dec 2019 15:59:54 -0500
Subject: [PATCH 038/152] add centered image to Pytest; add warning about
nosetest etc
---
Python/Module6_Testing/Pytest.md | 120 +++++++++++++++++++++++--------
1 file changed, 89 insertions(+), 31 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 3ea481d3..6b0a5ad1 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -18,7 +18,7 @@ jupyter:
:keywords: test, automated, pytest, parametrize, fixture, suite
<!-- #endraw -->
-# Introducing the Pytest Framework
+# Introducing the pytest Framework
Thus far, our process for running tests has been a entirely manual one. It is time for us to arranging our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
We will begin by reorganizing our source code to create an installable [Python package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
@@ -29,22 +29,41 @@ for instance, it will enrich the assertions in our tests to produce verbose, inf
Furthermore it provides valuable means for enhancing our tests via mechanisms like fixtures and parameterizing decorators.
Ultimately, all of this functionality helps to eliminate manual aspects from the testing process.
-You may want to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments), so that you can work through this material starting from a blank slate.
-If you do, be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
+It can be useful to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments), so that we can work through this material starting from a blank slate.
+Be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
-Let's install pytest. In your terminal where you can access your conda environment, run:
+Let's install pytest. In a terminal where conda can be accessed, run:
```shell
conda install -c conda-forge pytest
```
Installing from [the conda-forge channel](https://conda-forge.org/) will install the most up-to-date version of pytest.
-Or you can install pytest via pip:
+Or, pytest is installable via pip:
```shell
pip install pytest
```
+
+<div class="alert alert-warning">
+
+**Regarding Alternative Testing Frameworks** (a note from the author of PLYMI):
+
+When sifting through tutorials, blogs, and videos about testing in Python, it is common to see `pytest` presented alongside, and on an equal footing with, the alternative testing frameworks: `nose` and `unittest`.
+This strikes me as... bizarre.
+
+`unittest` is the testing framework that comes with the Python standard library.
+As a test runner, its design is clunky, archaic, and, ironically, un-pythonic.
+While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) provides extremely valuable functionality for advanced testing, all of its functionality can be leverage via pytest.
+
+`nose`, which simply extends the functionality of `unittest`, **is no longer being maintained**.
+There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project by comparison to `pytest` (as of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads).
+
+The takeaway here is that, when it comes to picking a testing framework for Python, pytest is the clear choice.
+Any discussion that you come across to the contrary is likely outdated.
+</div>
+
<!-- #region -->
## Creating a Python Package with Tests
@@ -92,7 +111,7 @@ setup(
)
```
-This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6.
+This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6. Additionally, we will require pytest and hypothesis for running tests; the latter library will be introduced in a later section.
Finally, let's install our package locally [in development mode](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Installing-Your-Own-Python-Package).
Navigate to the directory containing _setup.py_ and run:
@@ -103,27 +122,26 @@ python setup.py develop
Now, we should be able to start a python console, IPython console, or Jupyter notebook in and directory and import our package:
-```shell
-(plymi_test_env) C:\Users\plymi_person>ipython
-Python 3.7.5 (default, Oct 32 2200, 15:18:51) [MSC v.1916 64 bit (AMD64)]
-IPython 7.10.2 -- An enhanced Interactive Python. Type '?' for help.
-
-In [1]: from plymi_mod6.basic_functions import count_vowels
-
-In [2]: count_vowels("Happy birthday", include_y=True)
-Out[2]: 5
+```python
+# checking that we can import our `plymi_mod6` package
+>>> from plymi_mod6.basic_functions import count_vowels
+>>> count_vowels("Happy birthday", include_y=True)
+5
```
<!-- #endregion -->
## Populating Our Test Suite
pytest's system for "test discovery" is quite simple:
-pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files that have the word "test" in their names and will run them all.
-Thus, let's populate _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module.
+pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files _whose names start with the word "test"_ and will run all such functions.
+
+Thus, let's populate the file _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module.
As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our in the same namespace as our tests.
-A reference implementation can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
+A reference implementation of _test_basic_functions.py_ can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
+Finally, add a dummy test - a test function that will always pass - to _test_basic_numpy.py_.
+We will remove this later.
-Without further ado, let's run our test suite! In our terminals, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
+Without further ado, let's run our test suite! In our terminal, with the desired conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
Following output should appear:
@@ -131,16 +149,19 @@ Following output should appear:
$ pytest tests/
============================= test session starts =============================
platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
-rootdir: C:\Users\plymi_user\Learning_Python\plymi_mod6_src
-collected 2 items
+rootdir: C:\Users\plymi_user\plymi_root_dir
+collected 3 items
-tests\test_basic_functions.py .. [100%]
+tests\test_basic_functions.py .. [ 66%]
+tests\test_basic_numpy.py . [100%]
-============================== 2 passed in 0.02s ==============================
+============================== 3 passed in 0.04s ==============================
```
-This output indicates that two test-functions were found, both of them located in `tests/test_basic_functions.py`, and that both tests "passed", i.e. both functions ran without raising any errors.
+This output indicates that three test-functions were found across two files, and all of tests "passed", i.e. the functions ran without raising any errors.
+The first two tests are located in `tests/test_basic_functions.py`; the two dots indicate that two functions were run, and the `[66%]` indicator simply denotes that the test-suite is 66% (two-thirds) complete.
+The proceeding reading comprehension problem will lead us to see what looks like for pytest to report a failing test.
<div class="alert alert-info">
@@ -150,6 +171,7 @@ This output indicates that two test-functions were found, both of them located i
Temporarily add a new "broken" test to `tests/test_basic_functions.py`.
The name that you give this test should adhere to pytest's simple rules for test-discovery.
Design the test function so that is sure to fail when it is run.
+
Rerun your test suite and compare its output to what you saw before - is it easy to identify which test failed and what caused it to fail?
Make sure to remove this function from your test suite once you are finished answering this question.
@@ -157,6 +179,42 @@ Make sure to remove this function from your test suite once you are finished ans
+We can also direct pytest to run the tests in a specific .py file. E.g. executing:
+
+```shell
+pytest tests/test_basic_functions.py
+```
+
+will cue pytest to only run the tests in _test_basic_functions.py_.
+
+A key component to leveraging tests effectively is the ability to exercise ones tests repeatedly and rapidly with little manual overhead.
+Clearly, pytest is instrumental towards this end - this framework made the process of organizing and running our test suite exceedingly simple!
+That being said, there will certainly be occasions when we want to run a _specific_ test function.
+Suppose, for instance, that we are writing a new function, and repeatedly want to run one of our tests that is pointing to a bug in our work-in-progress.
+We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests incisively.
+
+
+### Utilizing pytest within an IDE
+
+Both [PyCharm and VSCode](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) can be configured to make keen use of pytest.
+
+
+
+<div style="text-align: center">
+<img src="../_build/_images/paris.PNG" alt="Paris" width="600">
+</div>
+
+<!-- #raw raw_mimetype="text/html" -->
+<div style="text-align: center">
+<img src="../_images/paris.PNG" alt="Paris" width="500" height="600">
+</div>
+<!-- #endraw -->
+
+<div style="text-align: center">
+<img src="../_images/paris2.PNG" alt="Paris" width="500" height="600">
+</div>
+
+
## Links to Official Documentation
- [pytest](https://docs.pytest.org/en/latest/)
@@ -182,10 +240,11 @@ def test_broken_function():
$ pytest tests/
============================= test session starts =============================
platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
-rootdir: C:\Users\Ryan Soklaski\Learning_Python\plymi_mod6_src
-collected 3 items
+rootdir: C:\Users\plymi_user\plymi_root_dir
+collected 4 items
-tests\test_basic_functions.py ..F [100%]
+tests\test_basic_functions.py ..F [ 75%]
+tests\test_basic_numpy.py . [100%]
================================== FAILURES ===================================
____________________________ test_broken_function _____________________________
@@ -196,11 +255,10 @@ E assert [1, 2, 3] == [1, 2]
E Left contains one more item: 3
E Use -v to get the full diff
-tests\test_basic_functions.py:41: AssertionError
-========================= 1 failed, 2 passed in 0.06s =========================
-
+tests\test_basic_functions.py:40: AssertionError
+========================= 1 failed, 3 passed in 0.07s =========================
```
-> Three tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests passed and the third test failed.
+> Four tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests in _test_basic_functions_ passed and the third test failed.
> It then indicates which test failed, and specifically that the assertion was false because a length-2 list cannot be equal to a length-3 list.
<!-- #endregion -->
From 762fa86d5381bc16f0682d01b47ebd294bc9daa1 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Thu, 26 Dec 2019 11:44:15 -0500
Subject: [PATCH 039/152] general text cleanup / typo fixes
---
Python/Module6_Testing/Intro_to_Testing.md | 16 ++++++++-------
Python/Module6_Testing/Pytest.md | 23 +++++++++++-----------
Python/module_6.rst | 2 +-
3 files changed, 21 insertions(+), 20 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 22f16f6e..befffc08 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -27,9 +27,9 @@ Before we hit the ground running, let's take a moment to consider some motivatio
<!-- #region -->
## Why Should We Write Tests?
-The fact of the matter is that everyone already tests their code to some extent.
+The fact of the matter is that it is intuitive for most people to test their code to some extent.
After writing, say, a new function, it is only natural to contrive an input to feed it, and to check that the function returns the output that we expected.
-To the extent that anyone would want to see evidence that their code works, we need not motivate the importance of testing.
+To the extent that one would want to see evidence that their code works, we need not motivate the importance of testing.
Less obvious are the massive benefits that we stand to gain from formalizing this testing process.
And by "formalizing", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
@@ -67,7 +67,7 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
**It makes it easier for others to contribute to a project:**
> Having a healthy test suite lowers the barrier to entry for a project.
-> A contributor can make improvements to the project and quickly check to see if they have broken it or changed any of its behavior.
+> A contributor can rely on our project's tests to quickly check to see if their changes to our code have broken the project or changed any of its behavior in unexpected ways.
This all sounds great, but where do we even start the process writing a test suite?
Let's begin by seeing what constitutes a basic test function.
@@ -92,9 +92,10 @@ def count_vowels(x, include_y=False):
----------
x : str
The input string
+
include_y : bool, optional (default=False)
If `True` count y's as vowels
-
+
Returns
-------
vowel_count: int
@@ -157,7 +158,8 @@ The desired behavior for this test function, upon being run, is to:
- Raise an error if any of our assertions *failed* to hold true.
- Complete "silently" if all of our assertions hold true (i.e. our test function will simply [return None](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Functions.html#The-return-Statement))
-Here, we will be making use of Python's `assert` statements, whose behavior will be easy to deduce from the context of this test alone; we will be formally introduced to them soon.
+Here, we will be making use of Python's `assert` statements, whose behavior will be easy to deduce from the context of this test alone.
+We will be formally introduced to the `assert` statement soon.
```python
# Writing a rudimentary test function for `count_vowels`
@@ -191,7 +193,7 @@ More on this later.
A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code and assert that the code behaves as-expected. The basic anatomy of a test function is such that it:
-- consists one or more `assert` statements, each of which will raise an error if our source code misbehaves
+- contains one or more `assert` statements, each of which will raise an error if our source code misbehaves
- simply returns `None` if all of the aforementioned assertions held true
- can be run end-to-end simply by calling the test function without passing it any parameters; we rely on Python's scoping rules to call our source code within the body of the test function without explicitly passing anything to said test function
@@ -284,7 +286,7 @@ a_number = 22
a_string = "abcdef"
```
-Write two assertion statements, each one with the corresponding behavior:
+Write two assertion statements with the respective behaviors:
- asserts that `a_list` is _not_ empty
- asserts that the number of vowels in `a_string` is less than `a_number`; include and error message that prints the actual number of vowels
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 6b0a5ad1..c71617a5 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -18,27 +18,26 @@ jupyter:
:keywords: test, automated, pytest, parametrize, fixture, suite
<!-- #endraw -->
-# Introducing the pytest Framework
+# The pytest Framework
-Thus far, our process for running tests has been a entirely manual one. It is time for us to arranging our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
+Thus far, our process for running tests has been a entirely manual one. It is time for us to arrange our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
We will begin by reorganizing our source code to create an installable [Python package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
We will then learn how to structure and run a test suite for this Python package, using pytest.
The pytest framework does much more than just run tests;
for instance, it will enrich the assertions in our tests to produce verbose, informative error messages.
Furthermore it provides valuable means for enhancing our tests via mechanisms like fixtures and parameterizing decorators.
-Ultimately, all of this functionality helps to eliminate manual aspects from the testing process.
+Ultimately, all of this functionality helps to eliminate manual and redundant aspects of the testing process.
-It can be useful to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments), so that we can work through this material starting from a blank slate.
+Note: It can be useful to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments) for the sake of this lesson, so that we can work through this material starting from a blank slate.
Be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
-Let's install pytest. In a terminal where conda can be accessed, run:
+Let's install pytest. Installing from [the conda-forge channel](https://conda-forge.org/) will install the most up-to-date version of pytest. In a terminal where conda can be accessed, run:
```shell
conda install -c conda-forge pytest
```
-Installing from [the conda-forge channel](https://conda-forge.org/) will install the most up-to-date version of pytest.
Or, pytest is installable via pip:
```shell
@@ -60,7 +59,7 @@ While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) prov
`nose`, which simply extends the functionality of `unittest`, **is no longer being maintained**.
There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project by comparison to `pytest` (as of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads).
-The takeaway here is that, when it comes to picking a testing framework for Python, pytest is the clear choice.
+The takeaway here is that, when it comes to picking a testing framework for Python, `pytest` is the clear choice.
Any discussion that you come across to the contrary is likely outdated.
</div>
@@ -120,7 +119,7 @@ Navigate to the directory containing _setup.py_ and run:
python setup.py develop
```
-Now, we should be able to start a python console, IPython console, or Jupyter notebook in and directory and import our package:
+Now, we should be able to start a python console, IPython console, or Jupyter notebook in any directory and import our package:
```python
# checking that we can import our `plymi_mod6` package
@@ -133,7 +132,7 @@ Now, we should be able to start a python console, IPython console, or Jupyter no
## Populating Our Test Suite
pytest's system for "test discovery" is quite simple:
-pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files _whose names start with the word "test"_ and will run all such functions.
+pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files _whose names start with the word "test"_ and it will run all such functions.
Thus, let's populate the file _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module.
As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our in the same namespace as our tests.
@@ -141,7 +140,7 @@ A reference implementation of _test_basic_functions.py_ can be viewed [here](htt
Finally, add a dummy test - a test function that will always pass - to _test_basic_numpy.py_.
We will remove this later.
-Without further ado, let's run our test suite! In our terminal, with the desired conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
+Without further ado, let's run our test suite! In our terminal, with the appropriate conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
Following output should appear:
@@ -159,7 +158,7 @@ tests\test_basic_numpy.py . [100%]
```
-This output indicates that three test-functions were found across two files, and all of tests "passed", i.e. the functions ran without raising any errors.
+This output indicates that three test-functions were found across two files and that all of the tests "passed"; i.e. the functions ran without raising any errors.
The first two tests are located in `tests/test_basic_functions.py`; the two dots indicate that two functions were run, and the `[66%]` indicator simply denotes that the test-suite is 66% (two-thirds) complete.
The proceeding reading comprehension problem will lead us to see what looks like for pytest to report a failing test.
@@ -191,7 +190,7 @@ A key component to leveraging tests effectively is the ability to exercise ones
Clearly, pytest is instrumental towards this end - this framework made the process of organizing and running our test suite exceedingly simple!
That being said, there will certainly be occasions when we want to run a _specific_ test function.
Suppose, for instance, that we are writing a new function, and repeatedly want to run one of our tests that is pointing to a bug in our work-in-progress.
-We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests incisively.
+We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests in such incisive ways.
### Utilizing pytest within an IDE
diff --git a/Python/module_6.rst b/Python/module_6.rst
index 53fb7d54..4e20e2c1 100644
--- a/Python/module_6.rst
+++ b/Python/module_6.rst
@@ -1,7 +1,7 @@
Module 6: Testing Your Code
===========================
This module will introduce us to the critically-important and often-overlooked process of testing code.
-We will begin by considering some driving motivations for writing tests.
+We will begin by considering some general motivations for writing tests.
Next, we will study the basic anatomy of a test-function, including the :code:`assert` statement, which serves as the nucleus of our test functions.
Armed with the ability to write a rudimentary test, we will welcome, with open arms, the powerful testing framework `pytest <https://docs.pytest.org/>`_.
This will inform how we structure our tests alongside our Python project; with pytest, we can incisively run our tests with the press of a single button.
From dd6f4e59f750a1d707a05bd68cb0e97798208c9a Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Thu, 26 Dec 2019 12:59:16 -0500
Subject: [PATCH 040/152] add IDE images to _build/_images
---
Python/_build/_images/individual_test.png | Bin 0 -> 48853 bytes
Python/_build/_images/test_tree_view.png | Bin 0 -> 36942 bytes
2 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 Python/_build/_images/individual_test.png
create mode 100644 Python/_build/_images/test_tree_view.png
diff --git a/Python/_build/_images/individual_test.png b/Python/_build/_images/individual_test.png
new file mode 100644
index 0000000000000000000000000000000000000000..381fba2dbf37da7158a9571f245d445970b9c4fa
GIT binary patch
literal 48853
zcmeEtWmKD8({5V|JcR-U3Y206iWMvFR;;+Y6?ZQwPNBtJgS!`Zm(t+wB)ArWLlOuC
z4(<Cs-&yD1`E&lASu4r9lkDuBnb~t+`<j_>Rb^Qmtk+nN9zDX5`zWRU=+V>FM~@zd
z{e$uFPBta0{lm*+cXinhkE+JW_a82vT1zNNJbF|ci+yW~{&4-w<)fbaqem2>e;$ux
zcg`#yJ?gTPlakQ%G(JRO`;yP4G2Gsl(_PaHCDeK}vo3vQT}s((S8@pAAVI^ZUml`Y
z4553<8Hi#yv~c$_dGgExK+m-_w#`LNNS-jH_~w2m``4$Qr%|L1HMP+c3NoB84C*|b
zNflBop!z2Qb6V;iKYp~eY0Vv^_^;t9HJ7*lm=MsKGlE@ZUNDYmSEx%=wVvL#f>H6b
zvJE~o|CRJqU!(~BztS2O$oKCjG<fe%Wu*Qq+mA6;!~PZFo*uG)N5o^k@}&KD$oV5Q
z=YK~?QGQyF{&&P6+VhToN8o#X{r{8v)!6x0>p(s_tw7FjSCM~Jn4T&e8Ff9a5$A4C
zO~0cu4}9^j-jv9*0dfjM@}MX(?6_7t!Q9{2;s5HF{?W-m&^z(nVF%Vn3}~+ZV*^5o
zWqQ*@N#&i%nKEq7OS=F0U`xlao@&qndkMH_!$)uY=D*YDpDF4!nI)Ht0g}nJ({Dg}
zNE2F|&zA)#75__K#BT)xNbugo991iHFrF6qKW2Ze($8{QjGwdI+D(ylqii2bs7Hgo
zs;Sj`1T5MBkBawWX;J)Y%IfsoRrKZ3EuHbb>K}f|GV%PllW}Bb7M*+ka;pUAS(j<J
z%}>?Hi___EksK<|uB#k`W6neG^6%rxiB}siCXBg+J^E^O;GZtAHzU80la*ie$|s4<
z=d&xZ?0j+RjrOWG8;Q-9b^q;e<#yfN^of<GJLZpm64n+mRzKG*WE`UCeR}EM(Vp&V
z<26c065ApQj7;S8Ui-UHZ&G?{loYMYLN!z;dA^1sU<JJ65@LP#o)&1wc2)9=ku|!}
z3*ON>%f>?r5J0I?TTg}rt0)z_i0+e)Heg^gM=@JF9$tVe)9kRx#?)Fz2|EsZzzl5*
zqVB`hn(s@oHSH31x(6NSpD|L?>zi)|Z|B2W49YN9R>|9QGs<^OjRWrn_&JTCJjSj*
z`0?e$TM^8_u$yn5XKSl-Kz)1f>t~uj-dQ;<M)|s`{4wH|&QA;ePn*GS6L$K}pSx~M
z@`vf;##7Ve^Ix3?i!j~~XM``+f4;V=7$NKq)w)sJU;m<}c2MIko7iF!kx|V$1tfUz
zt0~VU7ksuQs1n>iv2*>8vIv%3oSS71S1DVYGxGL#91mfCLIC&Uw-yVW0%FVmb-f;3
zt%rA$N~2btDlyMwzFi~|OMBtTmT#lfD@>}`-~Rcht_;)(%!~2iB%&yDuGza<)WUkL
ze3hztQcyROl0;s1ucqB0c@Gr==M#c-PR$KnfC17rSJonGB9UkNPZU@n4DU7O=?yY4
z?4qI4i02uCn7=(&b?&jl5;$~)^0{R4g)Qn37+Kr9-!{kQiwHe2wa<b<wRq4L)=Xl2
zQVKDxg&Yol#(Wz%z3>~mn%qTMVY3TTnLv^vae|8-zL|*(WEj-?iPfA{5scz+J!>O@
zxh2la@J;lID7bS}zi)|-32$>;LO$DF^V9qF6pi!8_L@_6D9&yZv4;SFu?F`C(uQ!I
zEQi_n-d+uHZ|?252*>7n?DHFHK|cu1#=O&ZxX<Mb_eOae(A`b=Y%u&#DLk@NtP-Qq
zD5=@fabEB4Ced|Do6@Km4kI0bgMHU9Zc)?gJG*Y*vBt)kf_*wO2FjLf5}1E^ceFEc
zw+7XY9qZfKX>>;4-C)Ou+OWMm>9(?@+zm%$KzU}zCn5oWmbKU>zvb}{(0PO|W7^PV
zu^U~n)U&C#7BvOsBpYz&wct8oX<;X+u^5Ka=ZU<3;tJjO@3<W}nH1s!8IHfpgnqD-
z3Z3D0l>ObHnlFdn7<h;Qpn)8$VNLJR7fgPmrQe)YTdO;uIrn>~Se9O`Z#}-f>kG{D
zT9W_e&i879;x|Zz;!dG6e(EI^Vnb-ooa6-smOpufNJe2t&CSw^83p>v;+fs4bWz76
zRQlO+^;-RN@{i|1$Ietn(d|*N)SyRpwJnPVOl<HfViF&-6>SOJZ)6XK@m3QsZeu3l
z*JNf>%&2X2YB>Z;w_Ve$(L>gEZ|2ueE-b~Ni(y~2*Pj>}ey3`{Qy;9;TuJ7jIknm?
zM1_1KYUKBCyfQokJBPQCLLBn0G6Rx4057=Zl8$&9@C5&%vF)w;VG4;USTdHmp}dD>
zGz~=SeS$n`_TM=T>xnTKu;s6Py1Fd9Dmy~%c|$zhc;tOJS(}H_OYxYpzGysZd|Y{T
zOc)B{An%H9$lWK<E75i!@j$jTxmyU0c=)PR&{~kCc`b?7V#u!`DY1)ixKQwKx*3q~
zD`MBinj))j8-LR;BYNgi!zDe>GQ`}u#SHmnQZ&D%Y@5D>{*AJTw+_T;qz*!`&*mSV
zu52o4`CauRHDB_=H-t_`vK{*v#v}60zIcdrb=FfwUzd4WbiwG8Bm`TtEnoHY8Tg#d
z9IRXX(`b!xHbc?rd1e0TI)drSd+JxZxr{OS-JygRwH`AY9E5&@%_>21OEurP-YwRE
z*4%~VaCN?O1&nk6@6~VSSE@cUyc&U}vN;gB1~$Wj<qZ)~Z9lp5_iWm+_ulNhVsKz?
zq?TOM)V-Gjo)an=C0!1X$XSap0HOP|7+X+vAxM_*JKvF;w7oWP=7iOqEq2)NVt6ly
zMLJyY-kf&!z|+5Y2;9lKx{;GaOb6@W0?^P~n=DgqP&^7Vfsz9bK~4n_WBC=6nu2Gk
z21#Y?;fF{fPDTRX=kisQB~Eg`w)|}0{HFg_wu{m`CJ8)hB<Yr6ySwYfa%f6bae$f&
zsrpY)s+IKAQi8`Np)F!2CU(@l;DO4M+8V1$sj@|nKr8MAdxd<BZajZihR_VQO&5*7
zk&Nl=i8I<e!Bv4^6%(f^?+!Z7C41KN3=KEk-*r`wA*&X{X%e>=<Bz$)ii5&m@}9Sq
zEjCrJ@!r@tdQee1tJhX=6VVac1no)l0V5-?oo<ogDl&$L8AFQRYa*p)eZ|-{lU07(
zFFG(#uKXuWpJWLdtBFrvko>s7)!OJbc;3oAC_YzB*0-Q!d^v2)9S_!1`xT5LZ0wo)
zu|+5x|2lTdVB*U`!1p)^_WokcwAZOk1%=mLs`hR>FXX9ARPNxNm7m1><|g3tC*xdr
zS`E~Q#_Rk+w^wG23$Yr_Zx_$lAo>_>6+va3j}YD6I&^S6%s?(|Hwvnu75cSaj`%l6
zTl0w2qT3JKgQ&Vj{MzpXmVg;v60<n{{E!s!B{WH;=DOxTfKV;yV`+~;fB`!=+g=K>
z!G%l+m&M)cFg7IgQ=O@1uPx3b?Rgvt{bvbs6oL(s`*QJhXW7}R{Qxu1qIff8N@WE2
zavQX`P}Ot-GlvhLHit`H*t+0)eiEf$VmB6)_FJIGSPpFCJwM-z#8~8RcCXd<&6rwO
zArCvmd(7>x9x0XHUrff-^wr-15)X&AE3%@q)ycGf5x)wXul+XG)Jr~Od&-DE=;v@l
zefCK%+p?Oz(pqoJ^>&%&I^O^S{rGKhLUNcobmwj)1*ddbozk!?;kHM0%CA>K^{=Kj
z&)Ax53r(Bq6vYA>4IF@81<)bAIH%SCTY@q_LnNhpzCgdYPW_#$Krme(CP!TO>+I_R
zpVL9U?jYIwJ;}_qKX{li@U6_cQfA`Fr;Ph+`pc|?84`L?0Wkw^KG|X>#uqr#zu&`K
z1`bU2>V4ztJGrx5JzcfL!q7H9l&f|)e|SNHber#T5pnoA?PINrla(kUv5gL?gtWO%
ztn7HgeSih$K%i}NWo$)n_wtu6z~gkqhDSJUr)rMtC%S{;BOO7EV7*E2*~O<ZgR4}{
z^P8{STqnk85n2LNiy{gYXfahz?3DwbFm^M{o<%xklYIP`w~Y?0W~8|Id1TK<^>>#U
zIcYF7d@W4~^pZvfy>RQQzjs}G?U4Lfy;B+N7^zEtK#$1z_@I&HE6TsY>vc3oY~A2S
zVUCmiR0wd`w3u*Zxnl7Sv@E}ID|dw$OQc=B+d^`)kdXXBpIJuTO7Wef^YoGciJQwA
zh8w?16t1ecu@#Ns9Z6j4tdh4GA*AU1yTm9E!mrk@M+iR16%+G^_YXv`AZa6lr?4GY
zr{g;(i7Q7z!VPATbEo<w&A`K{&lR?wrB#QW#bc~r-oMXONtt*>&{q19oJ46Qg%IBE
zm)S<g6pILPBF842O@YYL&w9comLwGqX~mYwG^;@-WxtxU`HPPXl+SMB{@^5=C^Tn6
zfxw{TEebYtDv<YUkkOWG=M}V4B*ytmm)I-Mdruum_|~Sf)hw6EOPLPP`=@fs^i@jp
z_sQ?byS;hdos7)nq&>O>#PkOT??0@3?tXP%kY?!TY0VfhG%AVBxjNTtb$i(jJ&-y;
z|5{P<gQYl|Lu@7gpFlUig+kuwY@GmK4Ux*wzlrNyv><p=8M?(7wRh5Y`^sm37pk=G
zY|$+I7jfD+JaLvw5~n^MFSQ8@`e}<?!>?d^9`Lx5@mpL$Jr)+j;N%n|y7$)i6Qb?p
zjqBtD^H)W@Cfb9ly*4<F?k`2+w%^_~^-@P1uDq8cCLi_Kfx|g>y%I$r4b%Zjm7dc5
z#1_}TI0g7JIXYmTeS^lH=B^LgTpeiN;++nG0Wj%A;~<|3$b)X}iV9jUmtR!FUAKHj
zwcTlN>uy_v<G$$^6qSQgnN}GIX&RzAOXdE++i<31wKNNr+_?1ga<Pc46+w%i-^C&l
zozqOY3DUw$TgPB$Gd$lmz57BT#dQ9048;A?UKKxKSP{V-rX2=-#WDu$3pVbbr3d3|
zY4-3EFFpiYwakK-HY|?aD9ErrrMYt9(~ZJ8<2w^Se0!}w{k?cTERaa287Mjd;mPTI
zW8);C*CkF6=5+;`*tL};Yc_c6;{xUPh=dHZ+hgs?jAF3(h=eF*Z@7fj`gRlBG{(Ce
zZB0MKNTgpWetT&?3n+gFS!Xk<A5MNG!Iche#N<$$i*$iBH?8MU-4m?cl*qb3ip@}@
z4+f%nZGVQ{5$?Wdp_u#WHx$o;G>gxfl3pddLL?^Gx7sjt76SRP5(;i?rlPVaT&EhY
zV2_)#74<du!@_d(MrXo7Ez(ql@>Qzzf&b~{YwfCbPW`@C@o-0U1J{(?j^?%ISA6#|
z_NpdWBO<x~BEraZB5jjn=E`wB7;@T|e;KKLk*T#Yh&>%9iTYen1~jk`-@h}7WF9Ws
z*`%4hqw(I5-rFj|RCr%)Yw?I~a-6eo108-u3%;^8qww^o_5jETh$Xj#HKo5Y#%2+2
z<9{C*Ku-4)j^$xt0#!{2&U>`fq;hFp+1lE<E!tpIVc%CA?>>T_AU?SFtILwBJHvjU
z%s`lbb>$7AszVEu2J)zgT-jSkg1PIaK2F@$v7WbZp{m4ueUad#`M0IW+rRf~4(14T
zzP5_O>)^q7ST#6ySgk>_w=qLCg~YQv|7HH?KC7gGvzw4V`?LZ?H_S;%z?jLu^woXO
zJ1!+9q}6?oMr6g?n3nlm<h;|3Vr6CJYQ*F&UyM)@mk_m=g|cB$>=Ubye}<J_Rvy%7
zS#~w`2Wla<&SDSuEFdU<D#|<EUJ)omxW~1snSAz}1uf38P>6OB`uu{x!`YF3VbbJi
zvorVUgsAsp;&e9txy|RU_&k|eDn6p4(u4B@@b!5eu2Te_@%PG2Grv5hne2{9ZqD1z
z%XPC+$0|@|9!Z5&u>08|JNwV><+lrjw|e?<A27cELSG4sTz&uG+RcNx1C`x&z02*|
z&?EDAkq%U@c}Vt$kY8^?3<QUkyFd}Rp`Br?$VFV=W&yrQ(ap^T!`&Y5VKm3v17ipO
zJcR(Ow`u(A2Gs!bKBHcV)I5({Hy&Q@X1l&uTTE(Az%+rOpRf#thMt893Jk(!uyKN7
zS1$VJ4^LPmcM>Of9(KcIt9ND_2(-0AgbAuezH?bXuu-{IV8qPeT;2GpWmCln%UI$`
z(ee`^jcu<M%R$w6g3AGfrGCpu%6d1?n;&48nY0~d7~QxQ%d^7pN`dXu^}naa!U7h9
z+Tjho&!5%n)Z-rr2>s3PF?^7IkfBoqKR)~5h_sAM8^4~8>-n}^rofiN;9-O9%zGcW
zgCoVc2n+^$MIMzom|Mt*9-xsP^mBA0@@xP2<6GCYM}QnPHT9Gb*<HK=6-&<G6V2G=
z!UJyfW?+$-R~NyoLRGE%i|A;yD?ijDyNU1Jld4)&UA1~03md^<;8~J|w>X?FXU!QS
zwWe}{yJ^FWK3~bj79);s(q$E2QV#o544k^x#^AAr{2t0C6T7i|^UqnOA8Ct~J0<V7
z^-~z7!2Dtm1tezUzQfuCmy7RUhgdHKsx$uQuqoZCc=Tqtj`qV%>JavE_lhuki?SU_
z422yir7|ZBDP<hma9ZqkyET{3(l00_5W_>&*WARr_0h4bNhHeH+0D$KmcP(gGp3w9
z_7KO?Kl)OPa_N1(waRb%4|LOT&QvH_+3iM4ZKs8^&{E{@tLioKDXwwGwHT95)0L4a
z3+{papBeM(OZFkLatuEk({u)W8a_bHleDnyR#=<TS`q0sIVR3OqI4;LASmrAS64Gm
zAYZ#p1eg2sAmf#AS9_ndR+tF!_|6(QsIgUP5tVqd5shIDjq8(izCSZ!1W`L&?@mlQ
zfJ+LBDxr`sww<fnzOFn%**#p4gG=i_43-Ux7qRDsWue*dwMHU<jZdgmfGa+WXwZLh
zaWPCq45jJgu+wodE@^6-2iNV=?h1Vv(~s$~<mY)6sgS3AhZTFbE8<tyd8-t=zB+=n
z5pbb)n<>Hh7rKws=M{f5LTz#Wrf)<-Tn=Vtz~-c?BV&!@mh?GXK~9yGmCxHApldWA
zW|NstBo@o+_mN1Nd!OP_t2Tehz!tHvW|NdM|G&%ZIIkGUSX$m(`pbOfhwuEJDnJ%I
zf&>1boOJ#K=3u%%AUJ`Uo&78L94p(}`45+x@5){!vg*C(E7#8OC)%ITXI<ePi8c)U
zrBIc~56}MJkgRW*o;V*YXY~{+qu3wj{VVyufaHPA6^XA5Yl!UbOw~3e{Fok={8!XA
z<+JmOyCh1l1LmK&q-t0HrE&falUE$#8-NrlKOgsr>{I!dbY#pcdPzKn?7CaznyIgE
zxCuO@1N${JP#5RXe-Y0*!_N)7AK#523CEOjP0-*b<zvqY8WH6pwI^o0YAB|1#6tAy
zmfF4Jb<U2k#|l)5!OP7!`e<%r6JWSbcCiH>sXG;75mgt&wM95OI)eXF)Jnp`bwocj
zQMJ~)#1r$!K7Vu?JHKw7Jx!(ed+d9)Kb?zJN>`6i`9WWsLW!Bj{2=zr#saDwZrE1m
zf#?<C)f0Lb4n9&6ZrFOs4H8zgo%D`Pmr6ID9$ZURlCJ5S{+QekN*;5b&qFVHvz2Bn
z4AVYIr4rKjVk6o~<Nb<Q6O3HRDqzQpZJml=Yj3SrcbU3<X_%T^&YnE7!@>VmWUiC{
z;3ps<q=^W5Hmr?^wbuhq*Kt=ceFQ1aoP1nI6p*mbJ)batZLddj)MtK8(e%j@gO^A@
zAf_C?G(UHR)0Ml=;%{&FLseg>I{htJ0G$iw(dcn$I^kdI1cVbY{B@nX2e<oIA47bz
z+?gD&{ok=CfkT|$y>ylE+z4X2+APwj5r+w9r_IieYE~Etpa*EZzBh8=&Z#uI98;NN
z)7?Qoaz?<`i?T$_Uw>AZZ+h$Y0X~YH-Ysh}2U)QGrjPB+J29jq;=*RwN+(K;Ge?dq
z&7#!D6lV_VY$ei(Ka(L*FD!xGR3xJ+7q`u{(_eHuz-(6t_bb3b)Lqbfw}|PQrA?}?
z(I<hjQ{%y9sjgO!sQBO;=C$3Uqw;+`X4hcFJ5j<1zf-%7`tf7O(@JdaPRrQ1;ZV3_
zt?e|CVy{LAyC*#HM#~n-%#^vTSEkF#7SaE+RMbZwi!GJ@7zA2Y%fSBSDGzQ^RE?O9
z1^zm_odJ%wPD7&2fEt46Un)NKJ%7HwRj_xuO*LTy0bPxw_A0YH)qPL=r;EBJ9=KJ@
zH?O#vRDq}!{iRTUF=MP-|Bkio@spGzQSuET7D~qwbQC-zKy0iz{yxt1DH>Yu!r*|#
zet8cR90S-YG0S4N?FF$3w|1*w&sSkRoBczl;(OWNdT5aL9hS&jivf;K=DgW+-voC)
zBu>oUhDxs1io-{-ZfOS;cu=OH)Hfi+vBC|3k7u`)HWu4UOJSHAyRqFX(oc!qRPE9t
z{dEWjn&}XbQ))gs+FXpNgAlkxj4=C9tEE_c`OD>c^1)+2TXB28oV!@_MAvu(m&V^j
zfo=cERZZ=%zqi$+oVOoU7-{k?3SqbgPR~Q5PBIOr;`(?OswR3C7D}WKC6a|4eKBV9
zZNKRbeTA%|>Ja*Pb~LwL!>x~YGM(IxyC*+Y<UcoCkPQ?dW@j|3{`@BvM8uxO8&W4o
zP$kW~xf)c6P~tfHXFlw;xrWn3BLk(7%CRHbP*Od`DyQqNxSbL!rJvFl3^+K=@g#3g
z0%k*3Oubd5?0?r^${llT)+5j4K04!==Q{`#4%p>G^TIpGCN|2r%JkV&Ts|yNL-aGg
z`}!YDZZB>Of0{I=;!JDuMcq(k5gipg4XG5kJyh+7bIsdZ={e18?{j|r#Ph7fYwc<e
zf!QcH*cwiw#7U{-M^MHp394HotpFnlU0o^IhU5ypZ37I*BVUT`o<!LssPQkc(^5F|
ze01+X?c5cjDBrl5#pMZlT=FZ^5M1Ag-PYneQN>(Rz_I8-pA}_iFcNOZT8iJmf6>VZ
z=_fNyBSV`4G80<gw?6q}|L8SJXR9-E|8hKr4>Mn(B!>c)fzSFU2uU>hEvDQ=Tj!Xk
zU5(<~&t^!G)<f9MJ=QSH1lh0&PL(zz4T@L#LIrO?b<L8tYU^gli6NquvbkyiISky!
zZqV4WhHAS@QldHg<ldKQW5TZI>gyk7QWb?-MoAPEo^=dvScjI_k=k35Hgs#ExIV1@
zFq+*@ESX5PN@~X|i6cmTYCJpSWRP5iHQO6s=%`??j1U@XS;wmcI@q%g$qfoe#vDjL
zqy8;K`&#W7s7B#jVEKCY3Z%ebDv?BZOSX33Hz4zdTC&BQw&R~QQ%Z`Nv$x%ZZk!FW
zrv}N_Q4uPz(6hpFkgm@NYYS}Npb^5_Tp`U-Qt&2yQj>>lhiq&On@lbH=m-&%ZWI+=
ze3G9xv8r3u9mMG65>RZBpsvdey+N8!C{Bz|A7j=5Z4gt_33M%{p9qq;q{W?cg=^JR
zh52ym>G3wV3Q#s&=nLCVCo{x^p6!`FIyr&%h8t_FJE~))j4Jicjz2zpHsdbaFRq}V
z!GV?s#bL=QE6indd~tGe{Bwi~XWiyo%Unbx4JdSC^m3y^(KX)N_t8oL7JNU4N7`_7
z>{So5vxhlk(i*n4f<c5WMPU^-p@2{FOCRO^b(Cg`a^{z$s|1OrwFHLHeAP#gm$moV
z@W$gSA<RZtR(e8y@{Oh=UbfJ3HLDOcTiBD6_{YgirGr?Pg^+V6licSx`OkS!W+g;B
z%-qQmgRB|f72MUIRIgF#=4e};A>H$%96R5&O-S82Fw393Q4U*}`^>0MrYJHKs{=r+
ze=2BQHoGz5A*~1T#wC{A1G?3YAKzY=E^23CrGzyJswAN_)T;6v9S+ILgVMOPG`r@4
zt(k9X_I`^`-VqJU`4QBz%0`%W&xDzC==A3K6)!}3Q<RfV9`M)~6L@q!sfdw|z-*mJ
zVf`pMRruJKrb+4dmk0{k8_Qo?tm#M9_mp8Lw|)3s0YuW$E3QPX!Az-X$r0BV^ydC4
z5$+V5?yOK7<XsMR*S)k}4)z5~%mX8+TjS>}^~2raaG>FWH_nP!>F2E?5zasXq)D>4
zXtQ3Li61f_D%KpF!i=uKOyelj=}7;hG`8<7Yufc=yO_rTJHPApKQUnjdzaFid>79T
zVl&0t2Xa)T!WGoKC(q^9eezcM$Vj#y-F5p0rBI%9ktAgUeDecHNcq+dr3VA4WJPVK
zOr(>@2|NN)9>WyQg<Q9#+PmKJI#q@ed`k?VSMhf%wY{)j7iuHr-x`k^jz}nm86$td
zG@dj=IUAnkvcDkVs>RI<-zF3wXS3+6oI_pd-u<|4c*zo%c3^lu6zEPHBK0oI&r{m)
z(pGO{_(^AYlz{u_$f(<hIvQa_XqxlDbOm?CxO#)BRi0#cg{X7Ls`>s5S8t;2$O6^T
zb1|u5M;Rl_);^rbOTNo>IoCV}L>7-Ft&i@|Dsv8{i`SZ5UDM~bhiBn-1T4z8lci*F
zZ5wKxYIYc26RXPY;LY0FleoqDgceddTE*mpHDX+o!K>J3QmGitF9v2X9R};K{D9>X
z+RMa5uH73a4(!^goj5y;le@al`@}u~X~wfb%45BP{jCCbdyypmeiUidG=~9p3)2`;
zLG~JJQin^T2uVZ6^A;O-MCQfz2jy|G&)lPodS29D8szT#=(Ka_8t3tz)-_p$VT){i
zP-B@13ZPCiw46vrH%l{I%Z@vA18l$Na#V-eMVeam@1+g9$m10oz7LR}m26q2K@)1)
zA&!D$^Kt7Oi=DIz69}e(IS)|EB}dlMwz}<ZoK-VVHb_nP%H_ZNeb3);e9x4!r8G`$
zjY}Zn;@3vJn9?VE(jv+qXF(#KAaQlS=so}PYeo>eQ+7ln@l6p7asTGl2ubC-sZBO`
zy$}Am+;8@Z@lT|V2lhL!SMm{dnFBuvbQ${`@_JD>9K~9nP8bNCC5i#?CRPYJt2NaH
zxxzG%efZ;1nN$P++zn+LVvkZ_J!FcXcx{k_e7|sp(d_-Mh2o#}Jh;ro@3g^eDoBIZ
z$aDkcTN4v8^aUc2GlF-Rm2wF&{cOhHSgx%p%B_3mi4${)4ar@GwqpOmPwq^HoZkmj
z=_SJs6$pGcE0!R+n5?Ni39c23^-*Dybk|63_H=C(kw+y*z&hEzB8{NCGaBmFNer+U
z#-Ui`+OaU-COko!0Mq){m#`V*1i%{K9|lc|?R9`o7Ugs1XendyBX6j~n=5Tb_FV>J
z=hw7DKhdT80tcq+<l+)nA+SgFGsJHAB;GfLwk=tW8$mbY8w^6@I7+GP3>5cB<j~Me
zdzk`^rEk^KK|#PeBlDFB+z{ME`{L;W+v)>|>@1EKVHp64a!L%YhI78~=w-o6H2&q%
zaZQFiO<3%Xg3LdBDS`3C=OhQ)d)&6z*1UUpp3z^7B_xl5fF0c9$J<F)!sPvRwf)54
z?{S06aw$S9+r)L8HM?>JF@Bu@psMVh5H1p{V+c)thr=wqzhM>E>0m(*(wcd|+4Jra
zBCQQkDZt@RiDT|cB3Jue;@SfB;{71;e!&Z$%9IRN-Lyb^EDRZu=H*H2tbLG%Gc2T&
zv9?^+rhWF!7{JJ_C4n55k)aJQ%V#tiIe4B_^%{3Hb>tIbDgODKSfOzrteO2w3p}#H
zSt_Bz(KgJoyIu&FPVYo1m~#k3+A;g7mX+mTXFHYF8OQPDRde8qKk2?_q}2`HQE1KT
zmn`*2`mI1BTd;4G<Qfi#|2?GBx%7_1moJpikt_->G-Si@<6K%0PTRTWn5cXk3JAFT
zt#)Mm&e#WAn}p}nNqzgIv3elLQaUv`t5OWRp?RUedxCAYQFcvqDlU<ieH7O>q`avB
z30qLLMJft(kG2`;n5TH%m#ux<11#(ZJYL!%t98vQJEU?NfDZ#KNPw}WK=5izNQm8G
z6}f+RZzb$|f*!5He5Hh*8gbniY;r-(BVW6r*+WRy4lPZs+RT-dcpNK9Wv)4^l|s_i
zvRs#CSo6C>CyGw5IA$C}Q-?xkKHWfVzLl@Ea;y%tH=f2trYD_d3MoFQDb@LIKJ}t1
z8Upebl-5(ueri?CCTRtOg5fosU;qGO17`S~G2(aD*e4!bY7$@O?`AYN=cB5(=P>QQ
zG;TjU9Mt5su3Jqo(6+b$;&8Dfl_3v3d((|QtQx)7q;sH<$rTfr+l0TyK!FqJC$Nl2
zy~Q=;V)pbxWn0&@v-x&Pc2gC)RPUWpE_n*CN`l(NVvCJ8;1ojVua=l&V3S*Ym9SL0
zP1lWn|J0~{S#1r0iUudvq!cVx_p;`S`t;!n*S$J=)82-MV_p%4H9HUIzGaA2&div6
zU!!zgiR5o@S*sFko&y$F$tVZs4Xz~lqMWn{x|x@mc855_j3t(b(2I?FY1`Di=j)T(
znw6qCn%SV<_ar*|0%iKU9WH76FQ@z!eqCrFF@9Q9y33>tn25a^bP{0Y_sLqZmbGvg
zSY3bae||Bp{4A;6Mt-n~ZzzlJ8OcuJ(`J)jQO)p#Cf3kjb!&Ro>`8%-(umyWw|k(m
z*`%$aLv6<Ku>$AXlV5C2FG({T5hGy(2HX&_wy5l;ONXr9waVHDN9>r`6dHYj;>h}_
zr}VhHwB7ek*wr@_(p6NX;bOY`=p7y+<{U9iO8djsz#s#_ZaERJr=-QWXB1Xsi6!cL
zgVy1*{xv1Kvb>xUwZCURCe8f5vmT4VLD^7tWUyFwPJklPTIqGXlA0Pgl(SY_%fk$r
zua^oa7G-SLR~cal+=Ke1`gNrw;QNtO4bWN6xa{iktU_wZFrz%KP^qHUOl*4Yj|HJ(
zGVS~I%lfft9j|no?+B81E^{l9s*{2TD8E62me&d-WB5n6wU$B|5b@Op2d~aR-nibc
zWA<*724aR?$TI9&tFhqIaF3?%dL{ZD#5#_K<&_1<ZyQHWNkc$BJOyB-`0>lL)wm|!
z!CVlXLP?%ZL!aFBD(Xgd9{TfQ^4c})2-MQ?>PL$m*L<cJ(_}wYm>~H-6?^8jPHRa5
zj0T)|t!5M=cPgiH3bA+NpPU-%Gh`s5{k&AGBsZ!x{y0|I0^dbWaGKTpOhk#Rj3cmp
zVJVKZ=-@s1Ofe99(=~4G7}^OQIn;~MfOejvlMp5%;5uaYT6q<{)-sUAxE-HtB8O&}
z1-|n|Z1RAp#qUXXLMS${YT#!XEu;kX%jKr&$d9<X)8`jp?G@7h#hO;6RaP%_qAMa+
zD9B`D%g>DRWaC$zSZtHAX#hJJe6w|7buJ~k!;b*oJW>gJMR3&OVlSqsJtQ*w`FTjP
zjQWLtEUz9ajJm`cqhW)8BZQzVt!%NwciZCBTsp(v1~mBWw|fM=y~s0`M7rOEu#sPn
zp}Qk;s7i(rzR^@(Hxh2n#&N?qN&qe!sjgm)+xeu_Hh4{XqwWdv7ZMg~`u*l`>isZl
zvtNrG$ns_=`)$e*pZw5j%GA-9z}CzUdqhb)z|O^OixaEb`4L6<(}au7wC2hg%a@($
z2-wDr+wrJE$)`$?udjf#ZHtqoeySt%jzjddKv80&`Flfw=(R@zh$V(iA8!1&>JaIo
zt_kiVNf(N!69R77z%tq*YeYhHe!lXdpAp=FtZOlWp~WK^Qj6*XysYfP+F7DS0KaQD
zVEkzG?kt9V<TCl^U4C|Qd-E2|?h`B-|A2QGHkf$hp;IM;Uk;K+4thL9V^aitufbzf
z-<@opSaxPyPd_$-xi9JDazkX%VC8Gf`}>d8M&GO1KJ2Y7)cSp~oCE+Z(kwcgPRbU5
z-*uXA1cVXGfV0*>kMHyL3-;BK`Xr{XHUhu0-~^C<>_AATH-#d}$GT|fyDwXfojAyZ
z(+<bI#g=)|w!>g-G!b?HX0w`)QLbOz@VxHvh`tK`lxP}GKQCWnEubROqoa!?OZtcG
z<!|}V`V%Xo7#(-us+!I_F!Y_Y!AP4?c=Qe}w75YhKM!qiG`)&Fyut#2Kk340lOZp>
z%5Z#kj4{vbKM+iIW6G%jW|m~HPmT#u<gS5<e@dN19|>g`p5G0;VXQI4kZ1^Bfw4s4
zIiv<%h#t&6c^fW{+wRlNtd;Tgu6jO_Gs5j|&V#f?68Bp|qFFbQ|KN;j0=jEn1-ht1
z!n;>`?;77E!%6c+F2#fAL!8lfqh@y3+mt(Gnvz)tkk1`CQTA9t3mIHuacUEc)rL9o
z{OgHmy*rJ6JPBw3>ECq|vgy{11~zF5Ay1!W9(@6K=pu!kNUC1>qkz0BL@>2l#P31s
z?^CnPNyIJ8|2!Ryj%rW_syQ^Aj5d#(RcF62Z(Iy${91f9{(PCUq5DLagOtQ3Tq(rM
zBPw*u`2a7Sb}Bue;+q@QY>Z?9r5P2l`>~stFr7bUzu=pf3J?Kjv=iH#G5?t_**4}|
zH_A6`_-kC%9aNBx0w=8JM-7z9-bfdoUvwf7yZqgZN>7)TmNvhG{Zk!$b^w}REb<&`
zVZoa=3tXLM(JhO=#bMcf5+twlwgus{KN#Qr5O3-`vA3N6;P@)>Isf+*_7{rJaBDt-
z(c@bFnXp@oT)_{aAO1QXdQ?lLQumx9Yg5ANQ$sj~<HuYhS38yb)ncO#2kUC78PNpr
z9W|U>wDO+lLqqI}tA3rb;Uz#5f(ul)IF&NIZi;nnmy*Ot)+Ui&J&vF`Xp%LY0~(RS
z^Y5f2G3)9w(aCo>l9Hc&{=Q`D)e(s)ylUR>4ccpY9|aIov|2R`_Z-@CD5`pY2NEg#
zk}*B`jRAqMAth-HU%{Sn3DmtiyK}mg^)Ng7B)cXS=_m^V4K;`hSVy2WRn}rwOQiw_
zKGx-;%RhJnW0v1lEE)=IZygbNfQ_y3Sc6ya=(^dvA@W$`W3QqbhF%Qm%r9xvj{*&E
zMrF;6@{Q1-SQXVuhMycEQzB;BIUjRGUI)5i*1g0?Hn4gk?2W7980}ZZ!@O0(ig}jp
zAVkXTUbA*>KOVlt6sI=D$eiviO_tMl>LCAkYMwa>?k_U=MSgk2t0I%FU~ve(+^OQQ
zN3qv{;;X6DfMqQ52_n%3v<8>Wj48LQ!0@X5-Um7lzuCjyFCZ6}vo_RhmRjq^Dek64
z@dZJwzyM8n)6oON=E)JTN7~&Ocp}tV-ExCRRST601vPf0%|uR1QGa`>eZieZQRVcN
zf28@Ci8}%|7`Y0v+@xv915X+1H8POpTCjgtB7Yyo0)bX9nG8m$*73VM*3|mJUZc~B
z5*WG8CLbC8P|F{~I;V~P1ZY2>$1g;-WZD$`JoC{1xV0vbwi>eJ9uJmXm~x@nY{LDL
zS-!Nxm`Kr3POAu{ebw2WAT6D17Sq(O)||gY%Xe!_%JD9pqJhGKy|=M>Imp0CkF3M5
zctS!=8}qgD(*W0V&xJG3*k@3p7r?C#uXp@Y4-*(#gUGeP`fN^-gyo?S&rOSjn~yX~
z%SsQmqiRwQWuMk(WVVV{hHv)(q(|s_PwNoFl|Q%~89ZV&DM>9_-o5AQC`h#2f@S&S
z;9%9X)@GM3<SmR1NsWUGS9lY14P5KS3i6biC^_>w3t?~L9+SJ`XQNx^kfdIkDBp^b
z3%^d^ZaaATA#{+m!(ngAt-vpHcH+3j#mz8{PQfi$x2a(6j!IXDR8Y}_mqGn~LdC~G
z_^&~aQ_q~!0iV)RxHaB^WrtNm_gaCT3JoQRRqb);5Ziz9C)w7_bY7FuqNlrWGu}o!
zyR}#zvrt+64lp45Rpt%HqRl^x3~WA)TJRzD<TA+>#80o~$*k>q0qlHb0ce|(u470U
z4SVGqNRyKzDnjNyUm>HXeV9`c5t+BY6j>|d9|vkWt;;gsvAW@3A~}kQv_Z3#BlL_s
zf0g{i1Nt1YdR`y7P3~!C<aVQJ=urfS(z1?lEX1+IzAT@br;f>+<UIGxX&Gow)`eU1
zL3id)5H|CPVFS`{mEnCBz?Xr|77TmT5g;+=<ZtjT>xRJQJVXIi_27m-hwrt<ITsvF
zz&e>CjV18r8Gf|&^NwKaj@87T3}bC}Z~Z7(u;(WciYAD>+sV&0?{=T1@mjZfzs$qP
zrlUSeXURjb!H+K~R}NGDySze~ow&`FX~9U537df?mIC}-uC24<)jc}MnG*<#5+G;?
zMD^RuZ|%S%U@b1gm2d?q@_M<dAy$PshJQ}X-&;OOd`4mHU8aCd4q5R>s^cLDAM$2z
zf7ry_F$OsZaF_SF1?@~#$d2RDhMEW%%aRXXyMI-bitllK>cv5mC<xFz2{-s8(`2dD
z$SbiBc3nVFnKh8_=g=65Qa{X6F0swjNGcMOcaVgLL|ijisy|JO<~CFvOx;sbKB%%}
zJ+k@A8*Sj2%-h?RNv7-sVVM%nGNtd|K0D5~5L{jP=}kx3j(WeuaY~H25rnJkG$hWN
zu(Wv@<%+G|7*$%WiAfK9NbTi^ODd0(v-y~oeoU;71&HWR2UpOHY~#3=Sa&y!Y}?WM
z7&vz9q^$F%X4A3l&?fdxG>C`dc&V{7)U4?q3wshMe`u_aJleuY(|i+mQ}DsWOv&lw
zZFsva4V^3p*TKBYG|kjkfP;mZvRrUV)X$Gr)LgW7XUt~;M0E9PZ;dS*u_#ZFRK)!|
zXU8<nP|K&CGWHEnxrU>idH>YWSDpoGiD@Sd&%f#S^V4fS)cec7#`oSG#BPuHp>3Y>
zZ4DqG;5KEKRym54GQnk?*`)<rW}P`VK-CTa=G85`jP29@6bb39Y#6+&K?d{0P=#d%
z_FO>t6Q=t4ut@uxNL+#a2J+3+IeY>ee!nN@ITlCWuhE<4l66xlObzx)V0C=o4{a@^
zjTzB7w}MY9lT?omt!`Qh0Sm_57RgJ-Hp2ithNdqK?i|rYJpf?ZGv-4CRi~3%OO@Bh
z)-+*W<p>X=WTJ5AJPb`L%Huh01FcFwKBDCC&c=d1w^wwrh<|RzA5M5i+tQsDvdR`#
zOHIu$_phfa7&e-K3V+}W6H7YG?;&$oY>jzGYW>D19H0^|r-tErX+s5{#m$Fo#8f?0
zC;G_xG(vL7GP@tBF`EvBeg49g=Eg=_ggQJ^I+f?>fq;DK4fqa>-UQXZKT;Hl$vg#!
zQWTY-`>#2$7^#09M+!C|o({idKPU!P^dO@ZI4Ub2JI)mA0j=dJ**e}}^22#o;W@Di
z^M+yjU80}4b7W$TWd&T_KoKxqu6jbBwbfujlzUK1BV``y7egt6b#|V^$D8VwLR>{9
z7M`iyynmhAZHAM=DngYAE<b};6-efo(wziJ5AH`hW-uNxsICc6>{UM;GK#-MoQDab
zH^O`?bn9P^u%$+4hje=S445{g4i_+rloFn>o3yCTGF+k50cZMt23lK-Y2@3BN9Nu_
zDrpIk7bIi24X!Gu$cEe@<Q(IHZI7`U@ejYbNI*>Rk0V;roBVtBt3o=}ocP07jV1dZ
zRb7PW*zCPOeW{Rw>EB|i6C#LOY?Mgp!;kz*{a;(zQX!t{EAGT#I(CQrC1rt9^ZuLv
z2wng;6C!_cfcIqw_TBpiO3zkY!%H&~1&Zse&Cd=UT=P)Wt0Dr+AX$GtrL4|>>g!9w
z*r&P|DZf0x)A=(eh9=a~ig(uWsp)t=Crf%_kaxhvX%q0-({oT8adq!B&&ZRB0G8}#
z-36P4MAd=d$Q#o-GdJ`<EE8U8BJ<$0nuSp;(QwT<z@Z<CXLetbZbxoUOQ(u*>V}Ro
zZKG~v0b}|Gnp4@-Tu@N|+Na#k75O=wcd2#?T8ZjAk|3GMBb>5ln6MvyuV6_2Y8(0b
zb$M<3bQ`5p<$xmgY<Og1ettzRAi48E5aeC1JR334d*t{}^H@cxU1!12n5CEeE{lv_
zyp6qv3Q>VEU?;u;Tv(uCB_dpLq>#76OJB-2NQvJ63*w_}VLraO#5g?qa);E|+Q&!a
zb@`76j^uFDj{E4ac&aHjbKOMwfa=}n93Wv-D4ZM1FFNgTsiC-n#U^8fw`x_R$g(_1
zgrAR%yRmz$&5Q_1sx0*`)ZruoWGMkL@~=j(bHM-$5p1>KDK*iMLTEozHT~T3+ay0(
zJ)Ah68}T>7;rF89SEwZ7o99aRZuKzqtfitS`%g!h@C&vd`<^dfzmH5dfKoqbR{m|~
z*8ff8*DsHl=6Q`iru;tz1^xBQq%(~^<7d0i_x|*g<)8hx6Adp4(UW1MEs(ZU^M~Qu
ze}k6!-)W@&(^BU3DgW|69ccFdck{m0S|8)%Lf_Z1l1Ot9>8<ir?vm+Bd04|=v8i~g
zw9aa2^=_JdLPlQV^>p=U2&bjRRPvDMk-!npdE=(YUwZY=lR35R6KC5ZjS%m)>v4r%
z&7}*+VV4O$(y$rtw(z}%Zjk+s&%eJL*8gGp>FRjrLhxOiF^c_-Wg9sl9cS%j4)NE-
zKz`ca^g|27i2v#JiVogR+j3L|Ej3T%c2~)oyulI^LD#4r|A5q5(tk7l#)Q^{c}?=)
zg>VxmFr3ntiLceJ=6+3g*2nVI>kWU2af|l9TeR|P<NuE=Tl+-z%&~bMW6$;+X=)t}
z?P+NA>`(5(U6l{_L&l#nf30K8fv#@9ec9TTW-49%LT-c){@P!1d!BJtMCRPnxL_Cr
z%Ngk_YmUtgzWJc&sfZuHz*&s3o9<|DKXd%IT4qx&y$6z{MVhL)E36m9J`<XB%YBI|
zl7%wurAYAL0Rv$7q3lNP&)7P*-D#a7iBmR$Y-iS?qA@9K2)1pl?Yzwd_3h6&J$G+$
z2%e)0Vu^yrX)BTHZgw@dR<{#m^6}Znve2nSekM3{-tTz_=-pHd)6vycIlpL?aadA1
zihuVf0y9=1zZUeLW<|Pf)tLrtddwAitwLJXh};ZnItC&Q(F_4IKPNQ<gDJDFvQ2eS
zJ`zik>OLjJZTD4%!2+!bOTjOW)aF;f6(rH^SdkYp;fi$3dgSW~5tLY5%iwIBy?Nav
z$B$A)>`oXr*0hFE50da7i!&M%aGvs7I$BLSo==w{xuz6^0?xx=trFN@J6h21kyq@Y
z&H)I7=hYO$Y2?~MZjqL>@xI{l&76U1@4CTHM#Bc;9OkjP`-o~aKfv9Gq}?!|do@iS
zOycOqLdTJ(I~wJums<g`vUkTo&-&obSO2NX>HoGZUlsS%R~|c^IcLDU`-<qUUMo1k
zAcD-9c_HUd2ihOW52v9<bGF;-Ijvj>UNu9rn-tuVz*xV#j1<e)hXV|1h=Dt`3xDgA
z9#SP!Qu%*s@-D(AA4;FVe$cWRupc*l^=MCFaPLKXmQOXI9~JC(0J=hk7^wyu7NxS_
zBJbBK$Eg4SKiJT~1jWfVerE6PDE*p1+ZFo7Rii8@<x6%?ch_7#J5C%_ziz|5#dGD!
z_P)Evkk^(S0QFc!C!`C&aR>djP%zs~7JI+RTDz&>y5H8wI1+u@)d6ccVYZL=xs3JP
zUV7)ZUy}#hWHi3raHrjUp6G6CRO|1ln?r5}@&3$b>{(0{*$DXOzRTv9I$t}0XVBuQ
zSWDhZyD_R$PpG~xZ1Yr-P;vo(3=rS2um(4AISb0nUFfb?IHXF(I%h{kY)#-=I*6Hf
zTBYnflmPL9l_AEmXXE3|F8mfIJSSvpBoK}H4f@(Ft;T*phKxpzl--DGEu}_+mchay
zIJz02nJ-XyL^G!l@tK7{?R+SU!>C>CU03Me{b445RZ(mUN{r{^(z<8Py|-Y;PzgS1
zr8mhC^hvnW`3kPiucn7bm0P{zbh#VeVN^bmSqU6W!_4r#UQ|u_c3IC5REK3pvSfm)
zzFPipMt)l=;7xGHOVkQ#9b__yk?wkZD70GCt?z&Ag3(Vn!@F6{n__B(P@Bzk_+~<M
zGScX2(r;K{%z;Vj$YWan8n=?fvD9CPU()K$dv|o=9gC+yFBnwvD;?j-%LC0p`Oh;!
zira;;z9tQ3Wd@02=#jt)2`8k0rh!G_?I@D0=Dn=Mn9#(QCxYAJoizw4mB#f_EhoJP
z2?49>=!BZH2Et_8jh^%poOw^l?BA_(st^T?ELd3y-W<BjMk!jTXt-!6QOi1Ek}gjT
zzf^Z>t)#qS*VUsL&T>j>$2k}k7#v}eSvn79y!tFwuGq_W(RH@Jc^{D>&vj{i*jzS)
ze5-E|5&=t1kl-fu?be_FVKA#ePEu!lFiM`3lw>OX5krjumT(v&O(U-?qdntZZ*6vz
zT>l=ybq!|Kv6Yo{?}n3mNov*CUd5p6HWZ56UnE2@{l2@f{Tz8RzYHVNg|+s$d5!$G
z`2gMDz8xQ%IOyZMz^paSe0L(8(ZMxXT87pVpiRZAq~mG}U!QbDX{uF^|0)NT<8DU$
zH1@TX$*q9aW5k1x`f4TaoEU^W((%N=sUh9(lt9tN5S;q2Yg8)7QwqQUd+cLfAbnM2
zOq3V>{h9JuT68P4JLICejr3x{@LR&1PQlgBBGvJSw0s#ZN{f5yt#2&fl8+MHj4QHj
zz+m7}+ueI!k=36tey;hm*5!DSriyv07SC=<%gso2X#C~KUXb^D96#I-t0cP@V^OdW
zcpXQ8G3+Ggp97dVTR`Een({C8=8q85Qe$st`U1>^Df4%RC#AYgk#opxC~#YH*4#7g
zAv3`_i$M*GKWs`)GV$l?dgpT&?C@j5138tBHv8RHR%i&E5~2$ia%^R}cg&H5j>j#;
z))`CQu=sS~N0a4=tj=DX*V<`Ns$S#6DI)Huo@#ku?K)=1ChW&|Q_arosp{y3ZVjwl
zhssu4vgmzHChDanpCxSc^s#P#*cpP&Pl3jn4?I?G`O($s(3TC(^=hV%5lGdbtR$m0
zz&;Vs%Y^0}x%+!WYf2en=W;vqV6o)D=IV$Q#iR3$_@<YB`&TYr5_t2}M6uj-PMsDv
zYs`QFqv$fv0x<zZUacD>M6QLG3(2b?bmO`gbEQb}cF4`?r?{cm4EA*fWp5{t=;!@z
z!PRw>pdV8!tN9(U_LI55DUi=)6IFS7gLczWE4($sciO9AKzf_`mEb(5pVK*0^s_=1
z*|LvPLSUn901ww8EB9~uYfe82D7iaP4))VZySw)^Ho0Rs%}~adbR;CzcsWUVyLi1M
zw+Ypj491H)&3-1_SbP;YPv1(BN}>tqlbS3?6VIViD`(oI?HCGfD|x75Ucs%ILXHxE
z%o@>8N^X10WOkrqfi&z(j?8lOH&-B`%KXEk7Ej5puFt6n8Ic+RVQjyN)e;&lxd)P1
zCNd^O#m-@7URp})u-45saNIvP|KLaCMY!P(m<UE?soas?;O5K($uzQ4P2b^m9#tPo
zjJI94Y`6JZc%J$kHC0K<Ho8#><k?Bo`E`qM)BL7q8nAC%NQ2(PH5zGyoM<xz_&(>&
z_<?U@n?-&L7Ky#hZ}GB{Wxw;b$Lnec5GNn`=6PM|4qgK^4`d{?FD&WWJ@h?09-pO=
z@zRPt^CwJ*7rY<Q_18joSr6FyrepjXt4+>`3WbH4?T==(XB1<Zp}ntQOCvYflaY|d
z>8lXwLbLN-<(dATu0fqk;8mb4NJ!|R=A0T=2Qz_lbddB6#OSk2E>LlgpMN$uCH|ge
z^d_I~8+wza!`%xWe^jkPepSYl@<XoCw7mLIuAxOUr5oeq@E=^jVz^~{Lj%U^yw5AF
zrLPK}zQ%7({?uUNwEr{yMia``VL5eND_`97(9xQ3qbGgsZ3uvlwV+n3*PV}1RrB?%
znly*BQru^sDXKv?5dF!ltk7VEk^==olMw@tQ(4CPLxGkWU(PnmSka7ZJ0t5$%pifq
z`6VZ;j>eow@&<)ZV-&Vm5=P(_=#h-nv3io=;quhmW8mhly~1VDc5&;E;k;}lVa%~;
z9{aQ6E0dp7Xq({6TV~aSKKr0>dM}1=ABnvJhh_~7(A-6bB4Nv<=OgSSIqiG*pgz1i
zF<K|FtJQ!h@Attwh`zd5#d<2I+k#3pnXtx2!B~T6<0Ov@CM|w9%_d7GVy)lUMYq9>
z$yre`+Cr37Bv63u@)>4_>I@cL^Gm*PgvjR;9T%V0vi8IQa%W~B9ETyHinU<*!~*rs
z2^CS=x6-m7;&EkY%k(5;ow3|N_AV?2Ci+2&5O{9OWvB9Fe`jb4s-&I-nAL`dagvxH
zdu@yeI~!3>Lkvzgdg~vOA)~!b$AHL%Y`K`c7AdKMxyy52xc5a^dDXAki?3DYn2w?G
z0DsMJwr%tNTLC0NI^zQ>oA2!WU+kTAS6ltt<=axA1qv-%yu~TS-Jw{4BEg;F?u7zD
ziWGNuE$;4C++Bkc2o{{+WYXUE?|Ej;nt1^;YqA#qo#b$mbG~xzeSNmk?wtNw2{S1T
z#FhUXRA0$`qeE9`_0Bx~lnVIxIlFu0S3(Mr3TWAK`ACTilXQvj?U`1yy(`^%%;jyX
z4wqo(p?HmM?S}98BQ}=Z4N|eTi9I!e3HNe%;87c|n>%;<89#Xhao*`d!cXFh!-}$&
zlivv)-B@1N9VZo$aBTSP;khfr2rEAm><xvCme2aM@dLR<j2NVgZ>CN`^0fJx=O7Dj
zzqW%5*1ik!j#Z7(g>|f$Qp;$r8fZLQwqqBA%tc0=!;5|h$fHDkoM`yq==r|s69ATd
zTZMYk)k1qojjxsZ6|Nx3p+{}JZj;qyx68G_TgV)qzC_$Q=1TanLsKV<K~(GpEa7@U
z?}oh)l9E2YVc}3^>jo_O(EQ`GSH$XhUzx}d1~?)O`%5Ca=YH{ZWpOGiSe4xN18t!`
zl~MX*Zed7H!>V~hXhj|xVgW2GaKDQq)|P79dOF4PXce<kkFB%DT}th(o#J1cdI&Ye
zF!DMjunSLuj3$jUrui2WyrF)6*=jwa+wtV!_<Cdl8iQz_A6o}M&5ah)r71kKC1WRi
z!Ja6r7#J=5nQeGChgt1;$5ANyZ<uAQ(;_Jd?=4M>`O(R_R{oT;=9|Ez2wWw&GGwAm
zxc2a6VwN#%{BGxBePgpd3nYd-M&qSE`*hjX9qy^=XSn!|DU+ToH9=TqB5=5XPZ%}v
zfNp%h-xImZB=(@mZs*C6vMQtIZ?VlP0&sJnkW&O*%e6qwgu7LRlVZYsQBIrK{x^Nm
zjzWbv8=R^3GaAqitn?{3Fm4uof95f39Mj`*;5N{vfCb8{=F+idl?=`@PQhx3&)Ym8
zo;o>BrNvtDqI`UGPsXA)vTs*aiet+WfWI5A`G9P-?KCg<TR3jfJMmBNQOVQ^)@I$#
zCC03k*0)POkKHI2{bq2@l3?$mFZBT4Jg(WpMquvLb^M8#9O`)3BdUAlbSk+Uzm{YV
zJZIeL($F4Xh8Ge`5}@8Zz@%jO-s~y9cB>rhf(^^+6*e2zQdCIw8?nc*&R`qD?pC`Q
z!@|^$R$p@-r$jcNwCQlR_=1*$0X78bAIG!J9)G~w#4*Drs3x^e6Cf82=q@)nJN
zv2vALAWFrCaMsI=)SiqCX2raUm^G+rl6##V@;$&=gm)LuU)nvu=wRygNKPRRgU-|g
zb#E>A;?mH|#Ay<Q-Ii+HMvSZd8oaba)f^(;*F+7uw?0e_c;Cc`eU22s`-}kq@<oXl
zz+>kY#kW}&vFe_`af&X<>;UU(g^{3&eF>}$bsxTpNa`<>I(SN3lU*M%vKBQF_lPUt
zVpz>pk?H^}n0I^q8K87h>k7Xj_Bbgy^ZX#BzZ}TsAbp~<^()(1Xtg}+Bkk*?P;nIJ
zBZ32s%>1tf-i<m-?)O3}Mz6UEF1I$NS9M)lCvnYVj~lcO72MnE=4V3KESwu?kkT3m
zF4hyi1@XOV7{0PnuB;)}(0}`aPLg8&-0gHJwa5J)T68U8vp>;n40_i_euWR)f)(ZR
z&rt%#@-b(%O*`W=$(giEo6^Bvd9d3P*Odih>-a@+@0y`Sp&FO?1xW;wgboI|y1LxT
z7RIEtbal;KkpG#p!Ytdp#DlzMT=^vdjyf!HLyUS7MeoqlAaS!QoV$LzO|L<=0n|+2
z>E;%1I(`Ew3Mbc|v3j8LXrl{p=j0AC(r@)TMT_e%5~LGRHACI;QO4R5f49gh_m2Lj
zL`{2zFl^-F@FTSSuC&IDgN9LBN=7urMK(PeZS7*6JYZ%qmPNCbW?4>%`%@+wHKmDp
zdhXt3J_Qn_LMLML!ghvqA<{^<OFzla&Fy+5pLsnYJZ}jZI7<Eqcil@mJuZG~7IB$%
z5~CCsS1Qs~M5I#pZEyVf0Q&X#vJ2<XgH*s=Mab|u+G&OkHBxWy4UhBY3#~7i>4>G~
z<Nl_ILK+l)Dx`J6>{xgHv6_HUN?H=rG@qomeXBc@6R}x`U!q$@=eE_qnzajc-aocF
zEBJbP#yh~sBq<|jF9=;F2-P?+RZR8N0^%2Bidmj2Ie6p3Z*;ENOvZG5;(G}{3j?3=
zM?Oc*&RJnvTElb7=4g2S>N|jyz^a@Hk3hN0#t2~yD<pKPa_=QE{!^jw?DrEIr~7Uz
zPl$hG>04h#1v)Mv4wX70?&AYl<}gLZ{bL%#S)-zxh@{<~hT~qMwdy5!58A+y)kE2&
zANM<*CTE7}6wGzcnIF*F-TG%mL34X-6hzvD$_3{CG{EGW8qRr~3bOLzQ1^<g)Dw{u
z9f__rb)J^fJ?!$Y{7gu}yp{>IH|SOQ2EXBGmRlnakVa+)#G%%D&jd{0-o4s)Q;t&S
zLl2!Bo!I#V>R5?vXF0GB&AG?|ag$0An115jBL|SNFN)dj`O=>p-#vc?g_-Qz?E6-T
z*NOqK?q!}PsPF(YMS17_=DHaR1&Qv{=|DFZfU0|Y{$`t@l#gTMBqFRH3yXvjUxy0}
zFKb1?10?v+BG<!jXw4{rpq!Gmz_!hAHz|aEpP{ggq&;4C#@z(=Cmq7C;B(Qw2~qRT
zuR^Xe<V2k-LgLS8s};J&YQ}~LmD{FJ94z?kKehH3kDRGD3TV%?DU|Ym(KN287#}~9
zk~d44W7^@P-{i5}dt)<Qwti3e(CBWt-ejLi3v3>|jKj>|JiWy|+`B|hM<1v!+W#{9
zELMG-MBK-<eW?3MV$StCfwMlZT-{23%yPN@5cWK4fq0Xg$%hUn^mh0DON|cJTM)HG
znd{xxBm>uJkH#v+E~z@C!hO=m(%y3=mX7bQJM4l;NcbjM5L-CX9<q)QoHoNmt~#fd
zg{xJ5uAsqlrPCW79<9t|HOxud4q>w1JJY;Pkdx>2x(Jq+`<DcRMu8BzwU_XBddDjE
zM$uU51t=#4EH&tZ>nq$``39P6PsqH=uquKxqbJ;<yh1;)Jm{}Kxe>|m-Bm$GPR<kf
zn(ZI^WP5Jx-s8f_0khzBK_&+)_^GjQnBTITaOck&-0KE7Ge<^<ev7i9VnUh5wp$_4
zw(1_DmUTH}0TqftY(|v5!LN7;bV|WLFYpir8=*voz^MA)9)p>Q{ArVlHf3G>%tC?O
zVgWgXwPX(ZFE8?=+cvc#Gp1-9GFgp$G!(ur@^>t=U3l^OpLGbi3d%f40sN5Za4Ia`
zWID*STJ-sf*yT@Ly#+_jlP@b43nrM9Pp)!^l~NKg9v}WLa<&45<hq^)jw4f3MiHDz
zCiHDw?PWfUkt-^@3o?xn-n5&)nz!Lz<*h@M#GLiG#d^!uW&<u|{*-YBYE+lD6n9*(
zJo+%Ij2@pVbl@fF38z2}R|&$-Lez6gIVMbJJK+2Ry1Y88pEGUxUG2)vbNjtntGP2x
znrxQr%+sad=8JkR6tn%+UyPO!+tx_>a|Z)2&vKrC)kRLEOq_AT?eKg(v29k;(5^M^
zmRg&&AP=eG9O$XE_|fp+l8!sai2|q^b*Wg+ZQymk*#Uo`J-G4Gq4-rvYwlZK+Jh6=
zSoBJ@ab#0Go=;~69&dx@3K#Fk>AUC8qKB*oboG6HPHR<qTY-6(sGL#<$(&sdG77;g
z!_J>yf&2zT4uhS&4TMc$d7Gz_`U`6qj89aTBhz3U0&{PeL^umC6BwZQnb3D7#cJ}W
zZu0DveJaenzu2UD%Owx=v?SX~hjfK%P`UFY9+$JZ{Fx6S+h&&=tJVt%y<a`@XY1`{
ze>D?ts**`F%Jh1dzR<_7E5iETe%vZmK)T3qlri#jIJ;QQc)U(7=*Y8L9^S6g^#|nD
zNAWhl7}107^Z`xysQY`dlwL$)T;2vM!EYB*lw4b^j>SkVRM5eholRG>0P?KDO1PQO
ziOZdm$Qb8Z*Kc&RiQA=lz{4Am2SvzcW)xjq8kH^zC@i)0b7Jfu9CDo5Nf{|@M@SXp
z!N`BQzzh$9mqvHBkbvV8LP8xDtp=Irs?eFWP?GOm4+uJbqaW8=ih;qv+I7ECphx`p
z7D2~ao{7#3jSInXv_YdKV65N8|ArP{_guRJ<6NlYv3<dvP(tZ4tW-OEtC=v23SBF!
zho;DP?#P>`xQ&Zhz22?ddg}e8HGxeMgsF;YQpm>{d5#w8BeS&|1)U<5#O>kai#Y9C
zUGt^~LtxuEs=%Yz9W1o>+G2P&qNvjWV03Hkz(KmgxOC0yAHbYjrcMWaaQ_`Jmx5In
zt6wl2MCUMQkl-T9JveX_ym*taC}{GvOpQFB$Q^Wb2JmoI(rMLgdBQMn2F#e90k~ae
zS3Js?{W*H%L*P%Z>x3HbPHM<fb9_(-A^Y@4So(#igWhg;xr<pMjw4UubQVY>x~ukb
zPbA!rZ_uD)7T?d@Sn+J?P&5k!F;CjrIKriieb#o_N);H>PJKXApFZ1av%?iMf8%dm
zkto;Ax((fy$7-)(i@V0)E+g-(XF^U@%Qt{>P#+3dhp+`@w$BiA?a}~SR_wFQ?{<8G
zM7b^0AKt}{s4TO{>c1APc|4x^%Rd1ZZDfHW3BRRwVpsQT-}ZG*sIXJBBe#dqZTWkC
zK#{*Z@Wu%JZaRHRr*oX9kqm6@T~anHyqcq%6BQQ+)0B|e72dzN!<0DsNfT(I=kDOL
zu<B6V4+=T!v{!8rN9r@nm)Z^93HaQ$TDxjn!(Ry>T&KsOU|qe+cFk0Gnx(Fpym0lC
zu;Us8j8|UZwkjt2%3#$GYh1WGoq4gB=-J3G&3{xn{>06@QXUx!C5k+^C|NboQm7g8
z>K9x!cKpd{La&igku{-WOE_Z{lRL_ei~IGpc0J7CJMLvjUVo}6jrw(}5;+=sV1tDb
zB+CWZ;r_Y>(K?Qy{+A@zTVJ*Nir|CNag(BB;f~xk+P$4nNTyxMbYcvFub-_Edrv&J
z*e8_L-vAnh&m9!#?Sw_Q*g(t3kQ(p(7sRf)lTPdt+mh>}Z!L-VsG?~wc|X=vQp}51
z`>NHF*qYiW1PO5Occwnx9_e~V@Se~}oG(pQ-<Fym*!Fd%3ZkqAgmazXkDe&fE3Gq1
z1v3TF;E?-USvB+$v2Z-&s#0X|{wd{sp7A)(Lfg#-9auFuP%E%}`649+^|U|gM}mi!
zZGtZ+gK`x;WxG#<{ftG9z_ii{H`}SNMApcQ)?}n0c^=Xfel;KN1{gUrOpN)hXv5fp
za3AV1+b-86wcNU#5)FM)?MEbwbK34BU|~lQx!|#LbIhCB!wpn8`NQPy{dyo=);q2>
z%&q)zN7>1;)`y!LxCS_5gZX`4bTO%ZencydixrIRTB98aDs^w`T>{A&u8)!-*>z**
z_oqG*p6+(xqnB_6eYfMbK)l(;tHy(K^A4$k_^vaxYd5dH)Xlh}-5gjc5$e?H7AeY&
z7Vp~eoNGo8PY}pA*>=g{UNUC5KJ8UzMtrM^QGb;Sk42CJ+mOuk?Y;{lu3_L^pGiZx
zl*H-)Gmud>AEQ|!Nr6!hQGc8DWh`rXjO~hhIZ~jqvDa}2WURFL4g?=(K)gPQ8-4JM
zXjJ-60Ysac^gE&&qJ3>!&P{k7>(t_`&Lu_qO_u8H_rn^u0$H<X+rYVZi@dp`9$S@j
zbSC~aP-j<pB0v<(Q>yIde!VK;$i(SQ&cZIpdKGQ&)He9FG!0b77~#|so`9iP79+SK
zbQDQ%&Yb`zvFm_q)Oxnt^|U&jHT5xNWlcQDI;%4Fjp(?o1=X7(Dgz;i%csq&G)Ht|
z^k*bzkS5iX-Ield))`mYWL05#N&#!1=-6#*Ud&!(!NtjE<ju~X^cX@w@Vy~M!oq>>
z6yl0bh?<`7CHk<U{i8q}bNk!GW!Ly?ekt_k*?R+YJwC2ma^4$Fyu>()g1hIiV<#B%
z2AlH1yseJEDudnY?ZF8)UZEQuRi(@coHvu#F|3?X&b4K7<M~3II5j(FUt=1_|5Vm3
z4%R-Zru({OGJLA<cp6v?yvXaO!5a|=kWD-Sh~+bD;LzcZ=|5vypI$RicFetMlY1Dr
zANhhS4;8hs!8)5ui_9s-^k`j)JbLR@z`nCX$z6i_D4ZFw_h1{<J!`8cByb@MJ?#IE
zMJuVeeLwOPw>vOB9ia2;PNg)1W-J+~bmA4Mnp#XZP9`p&sd2T)a&V*!ND};ELN9X-
z5i@T)G`-xoZ^u>P-Ic>fEKHxhM3WD?A|VcI*Bi>hGMh#@Y5E2zc#+9)P$OMRODeN4
zemDHYevbpMXvM{ZkUxF1dZhK?!`MN0NG)#b(x#ZKKxB=D(6UD1dhXIwl}pK+Ce6(@
zr(PF6&LH7Ms>PW8<E1Iwk<4F)@JZ3Mpd7DINbps#=gw7sI_a^`s-;Eaq?taAg}Xc&
z*NDghn`%gXV^vR}bU15mXu4+N*3`Hj9So#Ud3@7$x-y6J#Lw24Iz*o><+Jf|cEe*O
z@oZU{`RE|wP+wB$Mp&GYU}DM$c5)&^jl!ePOFTqhn8mkrp};ifyo5#JyjEFAE>DZm
zOtQfC#CXivpa`%9a^6Dd>)tj>dIU|F=@#!r_R4-#yv+)5dNHi044=<@Ms4)EL7@&F
zczj>vlU>GcGEt_jzx(rr;2c=|PCdvmZgHf|Wg#;prPgaX(L|S)@Y|h-Kksr}O_8hU
zjI-{>lOaa7AR_RsE%R=d82c8tA#LG&>VlA@9XE5pD`39S>+sF;uPM&SCKFKD@RYrc
zY1)2#p7|PHb%R#09EvJo7P>JLDF*+As>N=9g6Y?3_hdrO-;eC0f&cJv9PKSK_32>R
zr*5iuu~yo&K5157Y;m7`{)`iGf~#~w&03bd-_M>N-i?0`03JRRT;F|W)M{S7%TbF{
zQyCmz&_8JGPnkN~P)vrAT9_yxNDneRYJ#gLr>yr))Wo4o8R5(4W7%z6B)1nC=GEWl
z??cC>2Emvf*E<fiS{I5=Q(JP{^N^6JY09~}OJf~iC}G=bhzwhFzyh#EI(TBs=IExV
z(aUr_hb!8>McQa?OHL617dld{Vk;monZ}*Dq~Dzaejj|NE_(Z}e3-CvR`Wx1i}DF5
z0hh$7@s&|g;$f8<?U3gz2A<6dnwIYn+Yf$|$}clbOry<uA)8`(uE#fv4~^fdeQR>$
zcP>?ovDUc&1s6vuty5)&**qg<?hAEka-y6lC+fU@bP9gv<PKgoOtEzR5|CG4r!lJ9
zGgR*9jzbT4%O($Ony$4{J}mJawXOUDT)4(ccvdKP$)PJb0O~^`$1rkMY9nHDR6MSx
zRB*lcel+wR?3lT)ZM};1Y;-Kgnx-jOpF$Rj3(_quqHWY+Q-p;(7ir9K2QeOBXeU6#
zyWa_&<gKR;D~mt8+ox?m2O^Cl$4Fk=&&H^DSU8~N2lG8nuRg7nD%9?s)$jT(fXrC|
zCsCCf1Nj^dW@#_pjGoxRy#b8eamcE=!9TCy3M0~GpQZ;fJ?Z71OULW@Ak=y~&9{Si
z`vuP!7@mpp6djxZ7@K_E^|<gWhNX@!HX#3{M@c;z<#FAAJNJnD^4+ECS21M0!depA
zHEnc;rNZ{qzqG+Uhl_6V6II{cVWA~ZCzf&G<y<%!J1Q)*TY-3UEyIxlUV-95-+y1A
z>`QZ%IEd7q$$vH@eBsPi)zus8>!*}m!K$o`3Wyd8y4*t=U~N-?B3m|ly2>u6Fa{(x
z11c16G#?tNwc{j?qiKdG=OEoqP$7=XSC+xZ2oQ|RL5$#~?cfnsacqZrmwX3S{6qU2
zlLoCs>r4i5w;^%jWZ>Y9MAQA7rG6_((#?3l>{p@u-_btO1uqF|Mdtnhc}Z-wc^?#G
z8y~8g$y<8)z$D>U_!t!o7d%Y#vfY&D-N&eEM8f-|(f6HcZ#VQ0@Ji{=gB|JVIO5By
zoSvXsqMUs3s~NUWet8M2R{1ziaKP#b;S9Ou<icFPx<o2;o|4~}Xng4Zo{bw=xt%J!
zW&WN3*}a9=P61OQ-Sr=e^qGjc+H^2Y%1BA@NMn>96O)pO9d|<wF?Kk0$w(>3BiPRU
z`IVm93wUbeCIrEc$wRCWCVyW3H9t_iP0>Cwc2YuJ;i&P@nvmOEw&lnu+<(F}O7^_j
z6QeoYi7j>S_7(Ex<GWR_CN+H-@zdY`SVNjVI9UiePWUwo)56UIook*I3|BXRG%#4H
z&tybl?__$n*O`)yefsKrt&Sy=bFysu)o4`T5h2|t{B=~!4>axe$=PwIbFJY5TPArZ
zt}I|KsXRl&=vnoxzK}&wxcL`^+37C~CfGD*^ps0)$Ayw;oCg6-1l<1v*#>8ylDH~E
z64s|eduLn1&1+JyqgzU=7gKuqOXv|64D7^CkhtT<jZ@tnrBsCZ)qh!kSXobY*bq)E
zy2U~Xf{k|6qfcXWv>?@4$-~FNs}mGja)dKABsqoe#~u1z@A27g{+3#=Dp_C*_M33T
zDp9BJze(QGBbGA%`PlzEaNqyXU|5^hf1g7vucRVKCjVHEegi_8etTjq|0I!^OnhF*
zSg}6o?I8+hsES)_LgwCS3t<#?CmVgg%~fd0g}(*}(|K@HwdoI3Ad@P&=Kt=Aj6X`~
zrT=#i?|OY1MoTD*|J9@y_klSJF)|$Tf0KuYwz2?V-S?$J9?s2$`iWSPwRyX;v626N
z$5#gSnNRA}XG-X+_VbGU8gnYK2o%ubRPof6T8NZSCE?d_rrhh6du@cx2G-F64i6jm
zLLFAeHIJNC$mrp(IGiTK>Gk#DPGmii+?uo@rZG@96F&0rR@DC#{DoiI>V}i-VH9|a
zErZ|p(uD$Lb;jgsV}qILr`AnL(B0hf!3RY)T`bo$+rGy#xm~oEu7&%&Gd3k%rwVN?
zQHtZkeZ_LU1C@;$(I&vVJktl}<08m*=o$EFJb5p!tMV*;QiI#0B4`Pq5DF_R?LOTa
ze6x=JN?v9^SgW<IMi<T?lCJG5%RDv*M`J;q9l<sczL;`-+4|Gzubj@g@yyP&`r#O_
zg%Zhb67UD1#fvwbi$PVi|D@g;+I>W~mN;_8E$9nmX7GoPN}ZnDINvIrr4*0#{l0_a
z^LiKg!|xp(;J8S393My!w_`nc5MU7Sh)OM>+V+Co=~6gIC#Sf#adU_o@f7RbH+;X$
z!%XKbK<58-G%dK~DT383zlSB5<FORssX*E{Mhh=pS-ph<W-`?3evGwuLw+1uj|eim
zJw5;cV;k49K8R-D;Vb}jI51$}to?r$v<H}YUzQ(C0Lqr7<b$A2&4hPeKy&d4Ii-ks
z%L?{+IJCf0>iLS~+wO;`px;*ag#_Mhb+o%M$AW9j-9z%K`|L=X9pOi=w)^MBz2C1#
zUZqB7K4073lGs_|kMc83bA?B>`<s9JXV3!<a~tVqJX3*6<@sfx{3G`cOd{l7*II<!
z`_TauW(_t*dR&^`Ewz&3rlBlwEq(-FPdT^ZAKO`pes>|mXOhd{t8Z0B_4a2Q&hSGY
zrp?uc?bhrSB~;#Jho<mPo%l7^I?RK@gP!5$Ve)_H2)@f}yjjBbq<526RW}?v^H^v0
z{(Y+E`F@fPan~URfrJ2H3#Z|gi&@*sh-kzStmZcNwb`cdbG8~D;=Nsw1J*_2a>7TK
zcb@VmmA$9pb5*ckg+^|U7w)mRd8Bpmf;l6-32kp6Wy50bKy)Di^D4VNY-eFAiV(@^
zVbH<>V+7m0UiOs>P+$KOhx=b2XEbMSgZ6=F>*RW=S=40iXH4Wnu1I`q;XtN`ocFGc
znOER@bL#C_pV)#xcb>Y8*CwB<H}WUZYR`?qV=wO0G5E0NvdcZ$LgtZa>bPQ^TJg-T
zvTW7AwsY|ZH<Dw&k&p$fc1M43>%cuoJU}1Bu^e;Os1zXAMa>W|6LtfNDQ_A$A}6vo
zeY%hPa&9pCksJ*j4>2J|ye~=jyUucW(gC-BQimBB8Hby<9-yQ@fdGEGgArAoud`>k
zxL(VF$0{IlCfiaeRlHK|^pJydJlvEaVU-GZKn45UB^tLu+0qPm+*?sRs0&aNt(@^i
z8~{qvS?}rT+E+VfPTLkw*j|5l*lU~W*dz_>ENXONXQJ2lB_Vu0QN$Q-Zp9na&>$Rn
z&+kRZ+Vx}PgIjN(9G$KXOPr<>Y~P{;tu%LvV&ENZ=lrubzH{iiv}@>Sp9kFLZ2$1r
z*zNpSaHay=<>k}v7KS_V;LxLW1eYA@_3*OwRP<Wfx!W^?RC9+wBDrjzo5*rfuJPEX
zM9O$St~j%qc-$jvevl%!z?jp={v$}%bfdLMz_qD~Bs)wxZ1i`V9tgG9rMX81U)r`L
zWyog1?L?QOf1*8Cx;(1mkfyZ5k$&hbVEC9|`!v*ObKw2gNrE1i#nbLWS~IRCmXzT1
z@RuiIv~ApGDc@qG*|u7(!T1VNw`qd|=|Tdh?O#aNg(t(7{0b;h^f4rpM>6@Bj8P}i
z9t4u_4OY$00BmN)D-k3_ydBBOUnDDYNZ#Ne4TwKOH~a1{C$x@|t2_Fey6nX$+lNJJ
zE|#TY;CyCaLt=Zxw?NP^9mxEuTbym_Fw@cTcAVfUbS<>=?x8^WguSXnH=6xN^+i|q
zWpP_)ISjYef$Q3-bm06)$f0Pd0=Y;;22?o)u}?iWFl)iO;ce5HPuzFau;v)V16x6>
z!vf~C9b=*>y!W5j4&p;?{(B{sBaa!wU~WUX^u4XflX{Vhb493?+IEzT)>N?owsqIk
z1<BZeC09am^97}3GY-6roU|643>mWtQ0yTQPd%kY{>Bt!hXgjjkki*%999{G&zD9z
z*BpU1IFQ=tvCz@XH)H1G5HX3^^&rDUr%?q(7cc5!Z`kvD!<IpV0`y}zMJ&Ta;XjLo
zO(STrbR(^PFDm))kGXgI{3wB5{w5+v0r5o|0ixD!(?aLb_kh-iY7Xn6r~|aeH!20e
z<RhB&jgOx)VSe<qKkCqun(D~_FuQ?j;saJ>tiXNhO#X?`Oup7aZZoP^X|?anSNf`L
z4AaLEJ=YG^Ri(VF)hR<bk$3Ht&kNo0?CjP8hXOW4M7$;?=++uv9t<J*pti)uB1{vA
z7>Ch?hugU~Ps>D}Y*~U;mQd-9HOX#2JqC5ACcnXDD67VNM%4A;&=v3T)uvOgRpy!y
zN#h_vrc+I6jbB!9?06`ujq7Fu`8Qngn67dlS7$w`R83nL>=J&&knprY(Z>e;50Fsu
z*Ky+0PSEjdn>3Jr3Mr`a+@z}PVt&BWzU_4RxE*qRj)6eF2P`eoMbhsU;N#^Q?wRUT
z8wBIO3*%x)aa(Z#(Z*rWF|MkFrNI)io`glm*YLC;k=yb|%mqEoCA($-O@4qF@jJWF
z4bUJtQXll(zMrd^)f5^0@PwLOA3xWGZYwWw#HRtPw_vDK^J>u4;~Fx(i*55BMC+7M
zsO_egW!q8;8jxNey>7jTbvD4kYwUetyv=h@QeZ<?Xf!^-HC?~K0Rat#Kd=z?#5_#D
zSkD&3^CHyPm+y6gc0bL!FG6e;%tQn-1N!*_{8%;yvc?|H>I_NwL3oVsOa=%tl`{l`
zLNBT0CXMyOi}Z3nVnGVuJTvfFS1O{bP}MZP^AM7%mCY=T#i&M}<HY?sf;n^5)Gi9@
zOs&e8;vl!|abJzGDLrnfwqfGTI4{`TtJxSnz^A?BVqb!v_HD5B`yp)hMU!b)#S%r=
zSKC8>e!kdqL*_9^xKna6k3T_gQ*i8!8a*FkrF{lsM#a3h^Ef#g>_n_(2dFVxo5Fk=
zdRTB@OnbYQ4UjLuHKKiy?#4AAPY5fjUL`eEl4+XWRCo_Jqnw<iK8OiVWP@KKb|{|p
znI`x=PYFBgM{pe|*n=4FZhhVJ2gh|A6w|2YuMj=537>tqDR+0^D_Eyt635KaD8)y0
zH|nB`evrOa^{Q>DaqWyRmQPFV)=Y>!ocK>n3En!KX)5JBU!|gbT9@4!A$Kg!jR2{e
z$4kEWm12<@9BmS_-S*Irl~X(0E+9G1+p_As(Sz%Xa~E*0Zr?J_ribIa&gi3X5$H>6
zMu7?hTM?|W3R{02uo!3M#z0<O0!A@0%`pOj$w*@Og0qK1Wf|PaZoM~j^y5=IpF*V+
zCoTAk4cy7JSnakR>ZYrvHtuLrCZDewV|`paVAQc61-zEUYQA}}a9zs(o`wPT<W!dn
z=}bNKh>)8Apce+iVg<WP-TepOtw2KW6qn4%TVvXG+WANeZ7+qccYXzCP=NrjD0D{U
zMm?}RFSKY|ex0%YT#x&ionHDlF@)Uln^-LbKX-f1C9?mKok-HI0m}>4Ghs)AWCzAS
z(Epr6l~4CU-X#Rpy}^P1tfz1??#9~xR+MA{;#<YObUC6+=hc&K2ogFVw<xM_#){-F
z(ojnY!A9MViSt!~QtcQlBC&wQKh@73?fX=UHi+k>+j|(iG*x?8LP!jp-Y0Ke>iq49
z34b&?IN-PlTaa=@gFT28gl|4nZip_|k7)8oCj237dN--T8G3vk3PqpFB+@8IGVl^s
zemq|a&zw#4&mJdNbE3Qd^<ZmLF*HOoGQuzgucp1KyBEkR%M;aSD_vh|0fY=}Tdcxf
z>lLa1+_y&^l3!>NI}!_2Gbb&0gL_5*!^|_!ew*8f=TO6x;+%X+<EH}xEwoY+D_$|g
zEq)PmCbJF@9$S=S&sXsAPIUeuGY4Im;{%j`{Mf3Z6RoWnM^D$}@^WtNG&u{DH_z0W
zgO>dH)7a0sBew}Twxz0T_POo&Gnwkz?83VK3uLZZmxWp1qs3mt9##-AoRvBZw(o{4
zF)r0=dvF`QHrd1PC}69MWB-x7!p39dEj{VMg?D@r;aTBE`@xvGwp|63;U(jZ_GoAH
z;Xdb8DXuYA&>A|q%MBD5HaJo{{{2-B2XNwiEi~UHvtDjKc)VIUS||%jZ0CG{{w=<v
z2{qE=oPl+v6*DC%eA}%;9dgdrVgY*l@VcdL1;-5zfOr$4Gb>Iwuh%u+n@;s~cM^x&
z5u(j4JtWOyebr9%m1CH$bV>1zv7pp1q4T@mlbvf@;mFmAyq~WkX2R7!8%0Wh|2C~J
z@p>&|`oT6Sk2y#Ldf4!w-teo^!|C1n2205%G2B&Qf!rcF{f#P!<W;gVk)B96&mg_7
zAr6ltKzmyV6zlYt<o`DIsS(1&qD>NrC!1jYv_Bp70{K0HRxB4jN;&i%IJg2+&^~GN
zZf0)us8K@+2qDA3^d0)KjHm%b3_D#<yr*-UcNKnf=2Nz?qV=Qh`*P%2jv<)f=iQ?l
z*7*H6FlQf%|6!_~*R;Ce9rtfUV9+4|)Qti(#vfHnzHyRwS!VMd>Qw&qBqn+Vr{<h7
zR(ODS%2Hp*AK{dB>#<oImO9NYR3EHEuXJp?a&}T#Jx45Bh)>k{j(yCGO=ov3fL6xq
zagqgP+QIiNv8GkWzpzlA$wJ4NXjjcs4jT~#h-ksZxdJg_e4=2CcGasrM8bpUpsa~~
z3CIjoGh*M!k32+mN_M)arxb^4(^#_Xl31WcQjpK}dExlsMc;nw{K5InPC}=ryusN$
zrFC^uOEJlQk?_Xu5zZ|(#dpL)Us`r7+8$KQao_D6`>!p{DP5!f{i}J%0dwtp`0PP4
zA}*v+viFif`E&r$=QKw-e|o7(ul0^G22epejLOTPIZYTyAnAVG^yyScif<N-+5D1C
zgi@Y&LWlgCAD?Z4jh)*=frep8Iznze=#X^v@xxN(^F$r~Gb#Y4QLAy4bLZqNH1G25
zh*bL?xPE;}ZYk<jGWEwokDH%Kgtq#rb=bU{V-9W?dZv1r?}ojR6Mc3VE4Vjf_>cP9
z;9D;RF1GY78!9{x-YmO7<g`SX2nU;%cgppD#Os&7*q?MBB<QU^z&967`wYG}YC3bK
zEyH20opCW=l-r8Gsk!s7xtTLolV2>6!;}qIn=gP&G@sM0x;GIf>4xEP2||{!i{`Zv
zi_NGAOsTdNy3;{Dz$WpFQT(ogUvc>%$KK@km<KA0HV$MobGZ+C7YLNczFN<5?>om*
z-!!I>nS(Nl7X4b$p`cY;X_L}1V4F}Tt-`>nZ7)0S_1fCdsx6oh(eTEG_!G3T4M2n+
z(dyq<a$;>p&iXCb-1@|hkB=%<nwoszGQlt$R=azSu4b&{-}Fa_8}5zF_6y5bODmPa
z8;6^F$|%lJ(o&eH#Eis_^icqIT&H)IwFH{o^22V5)2QfKp6}3GVcs>#@8~+QK~iEJ
zc{L-!7tW`~oOa2&AQ@Xb=5qnN>Z=sihUKHymfAByr+SM6I$@riR!O%J%`fRerg||N
zJFYd9u?)CiJD0RH$~ac;w8QvawdAxYC*U^1iubdo_i}g8W#+Mov@8?TtSWBYY%D2k
z*}1p=JWIMS%}{<IJ1m3dJqxwEeO$b<cN=d_!TXe^y-nW8^jR|r?eFXr>@_qklho>!
z$^9RH>zHq|1MQXF(ew&sPmLe@tCF@x_eji9O6F4`6QPnEPV3>4zZ|up1cU7T+SZWL
z-1M!{;d8e^bGSLginQ!hpg$E+Q&?txWZYSQ<#94I&`FGFD{Nr-TPtfdGv=tf1H?V6
z2o*drwP!dnz7s1GCSWSNHD^8H8hIhJE&k?5<(&@7YTC<r`97+ydg$gEZ0eb%8>n`i
zJ#kOr<_eXXwo<0&wX9ptw#)kH>jN(X%+^KfSo_rINe-o~0cnvw3z{!Rx_7<Jt+xcn
zRcNh?W&Jj4gz{3;?eg{K;(eW<%1Ok>x2}z_{n&md+O1J-iDZ;2#8khh@rqpzq3Z@h
z&SM0mqRuz__?v#anm(Tc3slN~JA$7o_9^F>Q>`*Tjed3GS1$wmcGtKzx%|>L%Bkkz
z(dcw5`M4&Nndw=}<p6}J<%jPlvKTe6hi1;HZvdL4?3fcZZGO0E1_$q;P%#YX2w*iJ
zS>R(E^!4MtTVB=c6R>kUv$WZDJP=hG+W3{@;rO`_(GZVP2tHBs-Ws4H5Zk8LFJhJ^
zt3E^&5hR4Io15Nv-P|sJ3Nj5wN1&U(o2(Q9&0duqh=ZRaIW`vM80Q*iw0DX*3h_!+
z7nr}qZf-m9hcy3DGdnX$@RxNL4>c9p$-u(9gwvk~K=a)r1w7FMZi78@5?)&C#}3&F
zqQu}Rg@@x8L(|=;-@!VTj*a_YLVIwH7?*YI`j%~Nqy-Se<oTJ0-lJchfN*fqm*AWn
zx;w>(1*GfoOi#)~DAd~xyKGb7^8;8KYKO8yr`9@D0FKEsccK-i2uvLtn}Jup^K;ej
zu*H7Nwyq^`IVy$h5jmVCY;@ZgV6%r&Z&4YJ>}d+xup6fg=o#zZ6);k=={)XTigj+v
zH7<7{$LdHG@eEcCH6{mztt(PQJ4O9O@vQyzvJ91yhTh&8xOd(f&)mWLpFjz@M#CGl
zxXsrQ8VH2`P;3jr-mxi)f@b0yRAF;wj@fYQsr4I;7i*QGB3AsZ>A+WIW5-5|K8^=0
zhVy(%RKb(^^jU%h9v3bCrZM_cFD4EXcOm@Iy0t*#hn;w{r?jb3q(4SJVjA;=t3P4>
zR^E`^w1UaEfca7;Lw&&entfmp<h}oHaHuigOfWp|{q#E|{;ScwE~qlj)*U-ZGK+y3
zcP&P4^uTT0;`80dnT%(@G-+AiT4g)65jwR%Ylj?VBV1jCD#qFQmu@10&8-oo>%qLg
zN?a!mz(;_$9kFB7NhRFq5yjdZ^p3BV-C{(t^1+bs+XyUet|iOMCf82!kvlLkRk<U#
zLUcgsud%%B(8S9WyEZPk&NLtYyf>EqODr-TgHG_FMNDxB!46DVL?t{+L=LbWvnC}J
z0SLKbFslt>L~(=iq3Vb-FZA|AVfg5NG~8V5si_v#*^3cMmqacfZvl&!C}$zo2ecCI
zt8#LvSO{9*TePUQ$JWPEPVR=p<-mv_=Xkv)6&)Rt-ymkJD@$}LpQ!QR@Wmzb#cozb
zvae7iHorp8yFs=USQ+E>YC6H5tN=^NXZ|;dZ70?sskYDK14O#Va)fbVK>RvoXq1zC
zfkGz9H9rO;U7eedOg?B{0$r!BA@?$_y)z(s1-GUEnDMms;DQ%i%1}4*NWZPiC(Po#
zLdWhlq~}e&ox>=>AI9|qqM}&Q(A3?w662m!C}h#R;g{m|Fxf*5lzY(kMvDs*T^5=Z
zQj8boWT|yF)kiCCO{*3D6v>%vbAbkkU`uV9$GlSp#Lh+DRhJ%6`PpVhvIM3&a^W!t
zHJ3Axh`9N$r|PUhkp*4NO|82bjNZ%~$XS!;Hh}z(HoPSzaC+R@dWH9&m$NEWE$4>|
z@Mn%QV~ns41Kx^D(+jCX#hIcP#o#xc@8Wzp?)|Q#KfJsVWuTYojpQmYaDzP2vvqXY
zpqh4te?3RIjIF`9V608<9<0Oaa#8C=D{nS%pF6?}9~daCW<P!_?fwwpt+5^17Cm^0
z2weow?O)sbvo)EhUp`4;Q#wHcpgOG>ZwAHWY#dfEos;QR4rTRkDUgbf5s>xFAGN-?
z$?pura719MZPpNikk=`ZgW;K#^$p<qX8nV=6PBK}W(d&hz7W5M9rfe0gC?vi!Pjm~
z%y}Eg0v)+8^YmX}{>CU18#9=5{7l^URzd4E{gvhNIW*M|RQNgJfI`O$LaAjZ0=?mh
zm#dFI<fTRu$0M$od#2CN=pO&m(F!PC#wa&1h?6+}p0-@_rWQhB%vg3mDz;EfNzN^F
zQrJ$8iRxQ-hKszU!Js@`vT$a%dX=PdR{pYfJb!)b)0A(_T044PS>TBwhDr2YEQu{q
z#rt|ho0Vz{U)BhH*U;tSz>AmiCL@ep#r!*zkT2+#3L3{9j;==X^)&|h0gbP?1p$E%
z@tfQSOVpJ6Nvz!Dq2;mXQ_kz%Ho2+OU&^gccMQR`q0TloQ;B_n2v`Al^ZwT{lx8%2
zn>d9fy=^66aD9qd3l9vw*l15i8sJ7wGLDi(Sw?8>-Re%Ly<JHm{lsm_SYz*irGoQE
zVixlDl?;jrUdssAe04i6qhBE&W7a|vnzwtFeD(qTIQBubC^sMzLfO;H0Gc{+Z3#*3
zIY5H4-#2L`E1}xojI?*&-iqH{Zvib#FWg%+tjS?6IH1RoCy$p#c*6;(sH3$`C)rkE
zA~Z_;dTHaR)pxY>bQlywePA>uBgdcD6VEgmlq&_CuAfWb3H!@!!ut2U4ux{fovN0#
zS?<NYe}y+?;g~EUjV3+0{8yH|M97k5HGuoKWQByC?62kmgS8J>W1p6RhLx%W^h^7{
z`7Y7z&oA)&SewVnnlp@OnQ#Y1#~$#>2^r|;i?-^Mr|Hhgvi(r0`u4XHo7TEfUC0<-
zShO6Fj=CmM;}=Rn7}aRFMH9!w^ma(4Xfz$*{MydWn=Brax%7;CB`}pcsQT@}>uqzZ
z#Q;b{M`-@-xMl1Taog%ha$#8Tf!I&e5;mG?&d|6?nfggQ;$zI>R4{?bxjI=-wBMch
z!=aywE&DuoLYR)NH8W-OPxyCz!auPcnbXptP|$*xa9l9iQQOF}H%&Fd`XGOOa^FP}
zXQ$&c)p{@T1OTz)4N94B*bK)qTwB$e3S6cEiBs4rBgCIg=Nm+IRYIR40b%YqVrLt+
zz|j&`t(Cqa7$sy<K!{$7&eqLbP2P>cF)f(nq|q~-p9P}coVlwCLbUJ|lV2E^^x6xC
zIcm&S$&cB6_b@&~E)aacsK$I8GMyn45TtP`A-s&>(zZ4S{>&|gI>^<(qqXLCp9;)S
z=m||5fR_Lw{9&2$FH@HFX|S%njt18DC*}uxPzV%=9<~2KD^mD^LN5NPwC;IYGAm9)
zyR|pxtfNai!`*)lAzE}87jr(kM{;OJ)C*<e$64(Y&tRj!ZCgE@qecpBFup>a9b7B<
zT7W1odaVUjYnRE^kK^SCnB;_yw1Mw)bRJ(ow>Wb#PlpPLh6deAbmtV9$AAd$^wFA*
zIcEK^RoK&etIH=5-+!4y9waLo`Xb#Kap=Q#JXWpgM6odqhPl1_c&`Oa6YYHl@#xh4
zR;duGE|Fb8b*g*ZSY}^=Ia>t{+ZOo(_-Fkg6yK^|B1hafJ~<vc@HEo7>OPnH4Ryc?
zy!8*b!I-BKuSH9xA7RZ4*hz4D&)bufx=p;l^1ErpTp(JolPP`>?~iItO~?E5XIieR
zDi+G9oUHiZ&%fEp)zq;c2g1rjly-S)SmdmeWzXxvs-3;bbRC1lD{a-*za=}hOu8yZ
z2?i-S@HWcQXlD7V*h5#AzERuo*RvHp17L48d_?s2OHjq8wkj6!4JFCqq^5=1E?>Az
z4+ft!R5cKXQ#xjv+BNj3>*+nI+r|?ZU$qsN4tGLS=A8aSIZulJFDmw56&gX!$x-~@
zsMz7EYRU-g03!JLkF5D0l^Gk!j>!lS5*-t+_3h0OaGePp<~XINYSVL+>49Zuf98Nd
z6(kiQI;%S>_3xP?ZTW0@NooJzkgorK95&(}LSVT6e|q>oNRGc@>(UCNOnrUYwp~kx
zoG0H;wU!-+C2<kFDw=<fAwtJ!o#ctiTSNUaJ~VUWhsbbtqipe@v~9aoTduCx(Es%J
z$-+g%vr~5*el%tXc?`_PAuYHJbWmwOO>l5K71TS3H(wZoEeQQRGR_*o1#=Kp1x-$p
zxu&Iaa(S+()V(#jn9LxmxRl|rHUnCd|2?;iBt|P>iaj6!WpDQfSs*3FnF@kkDR6gK
z<7)doz1{yW^MnrZcoQ`&Nnr^82Q(5wj<RZ=_BLahQ3sXh-;2m`RYo+S<n2ceA`@f?
ztG5U{f`eQXRvG>qcdb}JxAgqbBptZLb|^c>Uw1S&o3vf|JsmMV9UoL=m#^Gj7N>;A
zI9n;I;ihqXvaypA6XQhSm2y@PDbI_&+ZsA*{+f%6x7a~f^|>`QPiYmN)u<=Fueo)$
zuD~EmI*@2v#+JSFpDRExoEE05-dd><SR9z!RY+ZuRl$R<iWWEPta_|!dX1{a_V)z~
zONnjCle-+ZJ}Co3oCjg(^K+8PF6sem1S^(ib*_xI+U&9jTZ9e#^|(#N3>@vnm78Pw
ztYLaQnQe>s0#^52o7ecnmH+Y)Ug&V8y;@dSal3-Q6~WC?Nx6<7MUiYH7Z&aE<Mon3
z)~CaHiTnq?<wvuEMvpQcrXC?in67laRNw*}rZT&Eu@{pyt}=U@eLSLAW&2O26H)Nr
z5fz`Dt+8IQqcN=K=-ISfU+*`7*rBK}<*stvtWUtcr3z^|DG^&LV}`c1K*JU1VR=n?
zB!;}Ijx+VppU`DMabprwa?M+l&EHz+e0x5W47M}FjJY=CoGm2kx|;c;Pw8m)D^`hL
zSYjaC=?e!}bDHdyu2QZt`&n33IgEasId(Jcxeb_8ec&rb(BQ6&#FLG+)juYPe&Tu2
z7*U>{>)C;yCCRDh4XrfmuEuc_=0M#wVNtb}Ift<)sD^qLP3>r~zlH8motVB|4J@A2
zR15e5;3lMEU;2FH&BpWbsw&9FDc+no$WsIBoz<`jap(hcWr(G*ZqpLi*PR{M-y19E
z2_|RWFBJq1(}vB-4WwLQDJTctGRs)>b636B?J(QDfn<Q7ZWE)g&T_t|H?lCrbVZrz
zPrFMeBT+!1&I^qDjOCfER--l=+SWMi%$lVLAHkD2ok;ri+}eiRaz}1G5GadT8q>Bu
z68~FlTkA!JcwGxDN3_&!w2yPQG(ohb^9<I!a$Tt_0nq#j`>r9uRykROJojms3e@a)
z<x=*gq@oo87+!tg56qd#E^Zc!EBLqx89>A|J_4Y+6LPU9cLoXgEG=v!Fn&*0ESMTM
zhKVdp3HJM^&yR(pb*`AZOzX_jYR&`@$Lo(c2p^^%0_JN!Avd+8u?EKv@#Hrg?{c_q
zL-bA{n45Em=lU0cP?5#iWlxe`6^aCcDob|vHE1M@&M%zpZCsfass<vG!evwusD!RW
zgU$^q5W`$!N5;co`%1i2Q<kfI&&)9FsEZ)gnlBKOoV=hOR#}PuS=~#0i;W;z=!Bz)
zKc>YjN@!I!7pv*7XN#|5#cQHp>UmQ(WabQrDCa?4oI{uKftUxeDgoF~$_9(cw7py3
zuV;;P!<L<^y4`||VubzvI9pJl-=DF=ycDr4h<jpO5#MW~=~$YY+f;9<z$G%M_hR^3
z#zytsM2#tFNKH*NX?{c4;rqI~CP<87W6}W>$8Te?+;=c5c%@Rmdt)C(s%=uTTe^aq
zw-tctdD+^K8n=Fzz3ifgS?64wyCL4gW^59^jI`~*TJV{SfR+-rAKx!7zIFJ|KSB2f
zQP7Bckw5Np*a&GX$XGp~VrTGRe!HkGh1u`f)-0wYUeVPT-mo#JZ8vL6%x&AVRJXnK
z(|O9i$#a>?k0zq_dlr)B3mjnw)n9sc)S<&YHLo1@pb3Z{)~r7XbwyHwgn;d6XSeb7
zoj%Y%!*c8B-}63YY4ROT(&+;B`Gd0wk;{_U3EV9>VmqX?X~jM|$Hx9r7{@fe2;;D5
z0mFX5s_aWV188fo${Tb^#cFnlO6N>^T|FVIX5BDE5imUv8)?Ap@QdK{T*jKjK6hab
zT6O#G-53Uq`L*)ZNoxfo!75YAa2Md;`4{j6h#1!aOAJ*HO&ee|$1@;h@^5yK@Ql0j
zKp0SL5KpJM5;n7kIZ)IyvhCs+nrG`6^UY=oW_FcRb13s?&he)4>8<kYLn_g;LwK`?
zZ+bTA=&!nsxddeoIEfhk8K!a;5MQjc9Bgx7XbR4<clh?M^AiWSRhpDkL~WA9Gx#b1
zf|MG*`8@NZ=f|af`hJt4u}}DXn=#Z*t{}|uOB$fmVABd;aS67is|g!4@FnEpCC9+l
z@4;)6D!2s5h=b(c)9fNnv-Nv|M2bmQr|<U*%wCyP58)E?nb_muuiz!byxQmZeaDc<
zrfqps?XFRd#z8}yr0ENuPZ&zV;QM7;$i$y(XxQhubRn*|Ge8vcs?(#Qxlu(~Y~VE;
zu>XuRD64c$KM95M5c``wRb2T%60GFppNB69x;8S@5wJ`<Bt@6J5ju0Y7pJxkm0c_K
zy*+cAoUy{k7RLc}s7U_A0O1=cW#$G1!kfv;&Q5C5@{VngcYdgvsPcQFrz8?L$gR;#
ziEt=15ZEHIG*?boaA%Oiix?4#`9mt*)1A@uo8XKi5tS=~3wnPi@g9T%htoGPwWpbz
z+-rVeSyDV1ts1~kQ7lqV|C*APW}Jyh@@6S~!F8C?2hy#xJO^Ekm@i?kx*aO1bg-$?
z=(j({QN=qLz%1P=W>BZBsUfNWJ$E!#C}pr~L$ey*cJ~#QMaiGjts{~kjrJ3rGC$bH
z=LYq2z5)ge{i>@gdaU*|b}Rq5-?KXJ7t8Ly<-(8H=Hi!YzMWsSHQjO${z6qf589ZA
z<-B*|v|ViEqI>cYl&5jzZPFPCx-w7pqI7Wj=VpxN={HqV{?1xr;qlWnq;_cMt-_gM
zgW)L=E*>%#FkUd1kC@2*tmoxSaKFppJRrOnr*6~X`gV|vUJ~|OkhYrD?9>|9#55#y
zcjKMve?bykt+I$O9c62wMzw<<r>#uJKN|_Z^wgtqV5#OBD$TbtMNUYq`4wOqH{*+q
z067lNj1?pN?sVfvG7xF9Wm_}1FV;Fqqz#Ad0pcL6Q4lM(Y8iQS@7&i~GmeqF*kAY+
zy*5^+d)}|mJ<~_xh?bNYFE~wmx2*tbDpX56CuC(;75|yOu|cKzdNzM2?wsI=e<p7I
zK4fb|R>c>e2KnJoQ;G(ElvQk1is!i&hsj7*!L1#CW97oR5nf1X=H)co)>108=i1NC
z9W-g35e(Nquj9YZmV6_8*$EWriw2Dc^{%x{jv;nyNqT?)if>uXA?FTquI^{gjfB4Z
zeMbn^A97>g`($h1wv_?J>P8Gr(%G*Ve-Vgu?SDLpM1lG0r3QA~Q!#)6>xvR~;>xSK
z?!#AK{@ic4W?#zJ*=W??5v?6oYo)N~^~I_te5r9Y<IKB88y&HlW2%41(5?HMs)<NM
zd-}hajQ_Yg|9$=6VKV+d?$qdp)k>8v`@z2G<C!HZ<h+jJe*!+yq!^oDTnX9}IO0m^
zWB)OLOTkZ(A1Zp3yde0?Nmn20?|0oe0^>kg#fusqQml@b5Y;+Tav?X8vd!Z^pfL`!
zYtOP{lHbu=tL3zlf0}hmrX?PlJzRe=O6aF}%*;Gp)?Z+TLDs#eRe=(1DcB`?#e6{w
zXm|FfzeF@u9cWrhSn=#GXdy!4-7^+K_vW4-(8Ss^XU4pin~X%@47|FJWsDl=cgOmU
z3VrtnEM~LY1L<$ftyFD%;hmX^4@W{!JBl3hf21;Vfo7X-D(e{9vSX}$S-NNoS_orD
zUv7t+g8rw$-;D?jtTVOwP)OjuHxJQLi081D0@Ov8$v+@q_)Fr#Y5w%gwAdZf`45Fl
z`2hWM!(8k@pX@qg+Az9%C1%k}0$27}sJ9&#l3i641ZCdze{^@&VNrE&zef>JBm_Yo
z8bLZnx<QnXmTpkGhwfoSPz0o5h@k{Y2|+p~q(fS|8DJQiA%-|xeV*UR_dV}*>W{Pj
z+SfIE_F8+-ti9s?-k-1hg7rSp8#~0~zn-6eGrG!CaM`tbyVr8Y7T8>(W~OA?eY)-a
zB|z?m*okx)@zE1FK?nCwA5p%m1G0JFx2?jA;Y6W;aGqP|_~YYe*TJM2MW!Jsb?5Z5
zJR0NX#;$9muF!j+tC@y#&CtQL(Kl9_C!2IAsVqE`{G(7c0J_WQ_MKncZ-3Ire=qkE
zd%5A&Jy46M8fIn(?Q{RSNm^GGBpqb0^Zl2c7t~-E$2xXqHUW6ZmO&tF8!VoXBT^!X
zOWgXwYepEvtGYSieK+NAMr!x`!GL~JK9&wfWAo*Etx3wupqE5(p4IT1)+j{fs~oRc
zLv==no#*YuTO*w&fNQK<Quf#@L==l_3z@K)Q8tKKxNr;L<meC`j{(2XaQ6cF_iNH9
zI$Qb9gU>PTi>xGaL4US|1ZHPSNE&Lyktl1Rf||^6`nMwPYS5PhCCB$jBJor~V2d(g
z5O*#9pAd8r=&!=V93_vBH0~3F!%6o@<o@IX81VZwRMB-%6At>pC?!W$FC6Aj)9tU(
z?i(YZ2f?8c?+_Dyz#9Zx&X-uBX!TOF*sK)vXwq_hHGBcyzDGY!5yBLBR~>?HsRH^z
z@$0I8Z{)J4thc8cNRd6@(9aO1d8R9lw=c7T6?nabWlu{a&B3nJN|@W6b^MEfKZ7~`
zR<y>>c2P^bP*^Q(JgXmF#Gzj!Mm~w)lrgA#x1|Hx^6acPbyKSbJF0zmJM@R2zawSp
z{$K`JIxkMYc|tipc*AjKGSo%|Nl-&1?&h4tE$&}FlV|K4C+WgbI`=`jH;F+3_m+pf
zGSjpn(J~4(#B?2HCa?0C;j5Bqx$Au;3JL-%SVEKEbkR*@LFH|}?mK%`PeQVeZ=Uy@
zytzs0mQghW4WQ`lpJC_^Ir>-@f1gDf&FAtR+8|!5l5N^x2E7RmxEIK`=8|cn06UY@
zu>O1=`OeVeKCsiZMUo<xyOqDw$xpxGS<$`0ZlFp-+lP)b5CL-#n>zC-uvr|~e|0IF
zM9Y+?^YA9h#myAuxlw!31yBs|R_@xyajt?I?jXyI!z&t>4lr`Q7t6^aZnyQ1aLP@*
zE*@_w)<p_^uzm{mbO!jUB1V;?E-ac(b?jP&@hYF3OITw+2x0Pzh!zjOUoQI)H!eWM
zaHIic8nq}kn=(C(VcGQEZF}`rYMWr`uJ8-GvTnML^P?N98&guubs5!C^j_7Nn9R`#
zPe@@@=y&;eW0~d&o<3XKCT7+<><9Toi!jIG`8=^RfAZ}k%!?_|)+$5Yn+c(8$RvDP
zFV0Ep&!+CSt2wTxy>xRXb->O&=gd1ZYOkh8RqZu6t{`=9oR%fN1@Mq?3lAatfeIdy
z3?iS(#c#aJj{hxfou~Fec!M^T+xSgS0LWhGIvq1(Mf*2+-(Nlv(N&*WmNEj!=>xdb
z18_AKctt$;Skz~o^;5#F{_21UW4_F+<7s3N%mo@y30ZY};$&VFjCO>L6q6Z4KZ@KT
zL)u;xOdfRfRvo{ufIfrG=|^uqaxXAFsKahLEo_hQbC(~9s}Q(Ye0r;7M!#CM_%P)$
zQs9kCU+R6P&xI08jk|Z1Ul-}^qtTE8$G?bftx}kT97MvuluF$5`6ABF-Ev4Wn>_hp
zt_(~|@&o_Y<5GFGTu97Q*xE<XYBZ2pO(|18!hvMSA}MZ*(py?!kkV^JpP|}=-~s!h
z!hK@mw^K#V8hw7eV-3W}D9Ajb+^)C5cARk)nCY`MyWVX!r*yeX{<HcU417f}@NnTF
zAC?b>HapTC{^F|zwO2D&b>bCO9dW5}24?Pr2Ac5@_HE{c5%n4`&gEFPxi%NjD}=a)
zZ-xw8xAT|HIAcOIe+3PIj1fC<>kLQ`sl0Zc_i}#yjBRiSn7u{i2tH-e1m5Wz<~>(-
zxAuNy;I|NPdFC(Ip_i~;T#W_J#_=;`sJC0tw#x*&SiV36o*9-$rK7?h#PF>~46zT<
zb$s`WOSi2k1DF*Wuu%80+d*HRxD;w9OcrrfK%_F6rgPON6mh3+BB}5nTjOnEj2E(;
z76<gz?NW&a%_YG-LPAmA`?I;7?pN9p5TO9yx#dsu*76wXc@bl4kUq|QcF+5r311Tt
zK+8*d);M^o3J-hcqj4cGn&KUw#(vhO{uqx48;sD)KDSZ-HiTwn1ok7{DY3y}uQo#1
z+wNpa7+ge?vx|JWPFuSZy-ziBpOa#OsqZ3kz)pWc@O9|IHoX|{U1n^n=>C(bK$yW|
zhd0&?pH@d$zKGo%tD2!IYv+OcBsUp_!SUrQ|6*p%#ev)PXV8;p5*EW4<+z|Cf2S)!
zee_}-@2-yH+3+EZg2@mIFiD@nU6=&~B{=eF0k8zNAmJXxCr)f%EJnqLBT>xQhPfm+
z3$3SZ%mE)EU%+HNn9ZpQ6g})dSn6#e!izKC(frU~`X>aopIU$oZu}Mz@qb4!`c2K?
zGOwP*C!%8$c)~!vq^H0Wrj2Ww&AE)E<o*K&5C&$avZm!y?Tkw1;n~ZU`93k5tGBH0
zMdtPUGK~>0zm`ghNgpn?*?D?aR>x6|`YI(FZ~qxn>2cAgq@}T(WgMp&Q<Ux@#dV!4
z@~D<&#@_q*$?UtZP9^c-i#F2!;`1t}AFB3Oi@e+RcNT>Buv}M`S+)a5#B3RRe6=$I
z4|HC(2iH2KeD$K^{uRWyb}PBkdykFL2)>yBB+aN;yOG)&tXEM>e+J!omMLAl{^C>-
zGdFFTn5deZqW9_AN`1%S%=d|ra&WVyeruI?4q0s9Xd~(&5Hra>>V>cH-KZ8?^{RdL
z^SSwT?0q){?v@vKZ|78zl$<jvak<qlxuYIn%xgrdAHS~vK(fy$^?ND5^*d9hl}j@_
zO2Nau{kqWfW@$w+O+U^xo&3+)FWFC9A`AB-guZCTNHRou|97&@m8QbEe5X;*<*(b?
zpNMTSoknx#H%->SJFQF$7fLF^4KG8pPhC*CT7UF>rJgQC>)HYcPiMukL$Q>(N~aIa
zF>}+<#%bcP_oU=NnpLwEo4$iLsh0F>Clh?35|#F8JA9?Q$bgb;g5`XW!HL)C2RF$B
zGdXbWi?jqmVv$sOt0a*rEj}&Rb}op_Eqj|5R7*9>+ch}$4dPG(@`ckb|F(>@p6w3>
z)9bd~X(M|#?K^s$Hc8>%Vk4$GPNqy}71e?j!oDTnqKeSVnq3wt)~}GgE3zYz1-iX7
zfws4Iq|+dpQr@YCPQTf`9EG&5!K?TlYahLIxu=c0Iss;LG0tcYC6eiB=)~OlLhSf`
zXX0jY(@J0h1vyg-+w94Zzt1{!{=N>l=0z-8m5SPFPHA;FvU0DBegqFYDA&s-KNu{a
z>NPv#RNS56h#dV$csEiOccBP#B>l8It1p+Yb{zE;Wne8bn|{+O&L{fR5yY+(E;Os-
z+|Z}h4%Tz0?m*TiqK*%Y5PQHp@trfs+i=@Ehh(#Hko!)c>EtYF?GU?42J{0HV`}%W
zT{vb$XRVVC)zwnHqRe&MIULzhL1<SiWd{97cm^sM+TQryF33@8mJ+U@a&5@@TVFYS
zNAFRNsXC0<O}8SCpGckM7;42$w(BmNcERHLLpA0Gm1|Mh`uVqo@|Yl4O5Kt_pTV%2
zv(rrl%iTS+-tar-+STb;3Ou*JR+uA7u%k+yK_y);r4Cdn1aT&vN8W99Z@b8>^?^G2
zT`<$diQOdLQGGcle23?4&(VP~|GvO7iY;NAoEgm^V00@pPBWDw%It3q<i0q^L=NpP
zga|6$K<?T|%-F@zy_pTbSvV>%a@3zbY@lEpvm#~g%59mGfbF~H5)GJVg$6tye;v;D
z_1PSg+PB~Aku8~Rq4b~K2!u7eK9gF)Z{_2}2J3R->S~=kz*PamBH|hFC{;^D45B$Q
zu7}SPvK;l)lD{SzQle-$!}z-3LWqgw;Q@ZDe!aCvia`$_s~fog9P;GbO^p|wOiSqN
z@q3hC7Qy=2w{Lq|*9{QfBYfkVCO_1zc_HR<6Ni$&sv0SrwUVp}_F4>@Z^;TA4tD?-
zrA;yZ$N>GivI?X=)h$_;$=3ABGDY&#y%~_OCRu;VnK=~<AJN)-Y7)+RcoO-Y2CRm7
z!mr;nFA-`gkevc_GseUj`<C|z-~(TXDMcR**sJrrQH`(KqVmuyYQiUTCDK4Otef*s
zNO_ksrHnCs5yzCme{o<@cd^G=veP|QbENA{P|Yj(7aXkNSRr!=zP)voS}#79&}t`&
zq?)7$FZyQkAQ^Qk(B2klHqjs*#&`O?_ql7Z!SD1>?z%4y@TA;|1O~fydFtN;oIWgA
z@GU}%o+82zPUp2h^IWN|#d^QNCxLeb-aLVYrzvaC0=}rUePcZ52XD&EOClII!0r@~
zQEuDY)bc?jLI4plqkot;i;U&7jPByNYsPa|s0yohd5pWy#g&EYNWw1lcq5}1GImJN
z>jutSiAN>fXBDz5IE3UMU}*>?wdXVg5_WEJ5N*`SYPLt4$QMt`I2=58bX`-^2jBVC
zRmAj<wTa+@gmy)_v(=avqQYqeFM5_rt~z5CdEWJEHEWv1Nmx#K@7RV1{`7EXx;Qwl
zdwXvn?r8OGgYWHMV!}}2-(o@#=fUVrX!KnO{xm~qz!p^DMF!m!eZv1hP3W37<5bzX
z4D{q*w|~8)bfqSzQuIs%c~lebq!*NibN=S>SQ_&2ibNK&@>G?X(e1t7S%Pr|nbY8h
z=c`{rrcwa^oz<TC$uf8dl#O~WEW-%tTQ8q>VRPzYlzb<kE6(*mWc{c(qneUW)e=-u
zb~y?oq2#$fI)C_bpT&4*@r_Ywj&|Ctq1CL4th@_bY`19l(_ksXk&uaCl6bP(`lch_
z`QD@j8iATLE;EKZ9W2J>0XRyow-vbJByHDvOtuOn_8G9>^2u~e3IUr8Mwe3DuimEN
zfnz5X86{Yt6stJRGA!w5g~5*~S5h;M1vKBO1QTq}3g_e1sH=Xsl>U!s0O9YACH{2d
z&BMqjuX!UX#j_2)0`Xi%M4QfeTr3|Vi3vMVq#P3hgaz4vbRM=k4}xrsS@NGMww&sN
z&MjQrZdCr4mg-jypG?lZd~ZJJRuTLw&^A~ZsX=vKo=f?&($xA#g|wnpepq}PVf-Fk
zXJ0fVlI$vWWL8`;n4XjJh2Ob%XjTH0vWELBG2bfaF2P*dlo;RZ?0~($OpD_v8Nd^M
zQU<@Wz<<jFU(x)L2d46@58OO_KbNbzNUv`ACa95d=3Fs0!&_^#HTaq0gLgMA%cRSs
z9R>za%qJh2%uRCj0NP`I=pMSSy=nTUBEzb?xka^*A=-r?0Z$+Ex}U**A+SQF2I6qH
z*&88uPD<R*F~alB5+pH$)&Pz0SIAHLYvD3?siGwy74GeLf|utxVan*#^Oh?zHp<{y
zND(B918V=$7TFIVWqd{LpdoY)eE#d?*)&!{?q#&-{A`vI`V%XINk!Z_Df(AL!fTEk
zruRHe^h91C+-?=FYo595xA#NEJCuL=&ekzYb@cEIccGtcgRdz9CM#Xm2Z;9sd*@Fx
zM4B%NJ<_2#N^bJb`~U<)wC`}(b5U^SV}Y#f$7T0%wN=s-tF268f~V}y+oj*f$NxCm
zVFr@6hSvqSc4@x#siIj$aZ{Qv@!+gWNGqmER`rl(R|5{1C!u+tDJI@dc;YMO64(MN
zf8#%)K7tg!$l+0!i2PtCZI~JURHG}DDWQkEp$P9sLhKvBIW-x{!ul5vJJsORJjvZx
zj}>qw=>SctTk{@6vll`RkfKTbk)mNHm}*ZVCg0y5F|+84ZX3eWJA?c0nfGbV9<C@<
zF=U9oB+(y&0LiM|?!N4Vpv5y<Rkq8gF88c$Y2Apv!k$e(?(`XX^Sn-PB?G)bGb=Aw
zVN9_EBp43+VcQRN6*^XwSPrpMG8Fyvsd{iaZ8!XxE5T?++1|<Ssai(uC;YmXyuJC7
z6b~Kj*1&XaFxs=){UC+oOBa!Q#fkZBKROLl3{Ahopoh&}pW?h#vmin^^b~Oq5-!jt
zY)+CP6c%g|vze<dqn0%04np2Aux++G+KZYcSh>YkCnkP06k|=qut4@QPKJllrJ*T{
z*ld9IR=w7~Pmb_mE2U+06=E8~HgJc?3#WG^8Uti*O(bPo-K)yRA*0q>QYRi`_mEsO
zFqh=pVA(pnu)Dl-H_^$?r_toQ>hf_An)1rHL<JDd1X!EBgg~+(7yFV`LVisqT-E9$
z$6ogve+|Of8Iw2_jjVd$3(IT(5U$}G$cI4hW<Rp2Zq{}zOH0&0{SIP8fYg_Qz16pr
z!``R;ogE!u%PR4=6i*LDBl|ZhWQc9rSPDReWG>SDgz{`WRlZWr<=scMOmG5Ycrna|
z<F^(s?H297p~VXt{V%n6JPE(Gc>Ln&pMEy01pBl|iQG*|vcxL<m4d0+d`r}~-AcoV
zN<*uTP@K3q!WxfaoR7ivpOolAO*nt<7Q_7fl>`T^X)*#_Pb>_+XT>}5uy=T(f^Y~7
zrBFT29A~K#_G?PvkD<Ij8HzukX|0m&_<%h=#W9CdXr5MExChj`UMBIL3GXEAd<w5{
z*18nTmGX1;r>Q*SU0{{Xmu-`#A(bDmGgDK38OU~Da@Iek?pnukX}UjcZ|N-fS61uI
zT;H=Gtn9%Bw|jzR;P2G*9nDQupNQ*JtHWeT)6-tgZSdYd_rC}FMTJqDymsk^iZR=|
zK~p3aF?CEW4$sV3?^7E~*qwsefq;Vj76Zo!Ubz$mOEmAKuxa^b%S>oZN68eUK@jyZ
zeq!c$v{8ke!4aKZ1&tF#ikoe6Ot^sg!DGLssS;&eBJlqj6$Y5DasD^p7lz?MWdA$T
zwZ(hY0f#m!J}VzToK3!JE$WLEW%|)sG=1wjh*e`Ci<;#7zocF(124Qrd(D_ts%LHg
zq8RD>E?2aTTKDpLmR<f!8rMm`LhodL*2M|v=){LFzy7j7QkB4~HrI>T5*e0jOIaIz
z<4lV1%GsAi8@Fh)=xv~RH-rXz3JF$?(~JBb+eT>u=%s)^3<2&GOBrhqorI@dDR1xh
z)Efki(hj2JS5$vPTsANM1#y|kE3us^PgDdrS8)W;TmTxv#+Wj@`c`RNqjOgF(}3{z
zd1ryBZb)j=981)}!OdR~5nr?y6Qhz?+RA{{-#$}~opp0S>^ctnZy6hKJZqUaFu^qs
z%;nzy-u(Tq_$q(v1(k&Chvk45S3?YcZ%?XD6-fj1K(ev52N`Dj2EhNgKwq_-n$pgB
z^ty@35itW&8YSm{l!m{<H;i$~;|V}rze_jvzZswSwetSoinI*K6&A8jHx(5vyB497
z|EZCM1&HYI*=bT)lat#=lgqU>`W`;^J+K6#27i}S0OsS~I>xT``16;0c7=r^{%EfH
zhK5)e>|LEZf>Bb^1WNup`uyvsenik1sVVO#7LI=y8@Jq^!f*4!$S9(|UJT``q=c`e
zr1aQnTHtd0pF0uIc)TZsZ!pTIp&ye^$bES<s_wenafg72$X@tNpC^83AcY?fk?x&w
z#q+P`+y-_9hkZxC#w5XWB%Pg{<SZ<5&_7jxlNH~-ecL@SFiv;>@AeFz<^x_wO5^nL
z&qVo$+K;v!QRM`sJsD13mSAB@pZHaWcAQN>u3-Luj4@MNgH-f5Gk~0Kd1$oo(9_x7
zwNy}-x8K6r0c}%Y-1k~Gi<+`cRl{^ZI<y_+Zy~ilnpwp~BKPOVlf#A2<K5}AJG{<n
zs$X(%w+Q!0QXZDIr|BdydB5AA83sC|^vCl#$n*kWRTdOf!?uQ4e>q*Lvd>()yxJ2#
zZS1Vp?K^Khlu<$`nYde6RyO(W6Df1>jF3uO9uO@$lRBBQ%t~LaxuA-^T9<^}VZOU^
zlbZw^yj6kO+oMQ&T5_S&0dd&zBxg%0H9PtjaVEL&AL2|2NOlg0Y*2#$gd`8ZaTEK*
zV!Sw@QhnVtMbIdxtOGgiT-WA`=8O4Mg#ELkinxLG1r`T%6d3p`o}MlQkV!A-=7X3)
zmH}vS{pM_}bWCOko?n(K?(zf2MjA?}FaB<0+afhdBq5QY=X2bCYw~pmnVO)W8{A=U
zMzi3Sk4$5B6J=_bm(5QD90QdC4p!^`b5w|CT#ZNTKNu8JT~TN{MV3g))Fg@P6r^7=
z!K9CdJ1)8!@{%&2R27-8zZ>XIn&a%bzjXluS}-x$EJfxKUoYi%=Z`Z9l%!8e+z0?p
zk(qtC5b<bjBvnZlBZojBB7qkoK+ia~NsHf&-AzP3S65dsFi5!~09MF1jhZQu2V`0H
z(j2r$+M9fr4GE&pf0=+JS7naAi=1|KkPIuXV<MC;P^R{7ntRXoL}TlVyFz@eSvb6m
zdpIc$uI(wbUdj>bx20ZiWxKi?v<$tYKrFyyai-J#YA^nMCl)qXL(FUYFF(Kj>k&HI
zGt)XEKyYjM_}`5o<>n>TluyEw9vHt}ro_M41&>pF7(3Hou743F>VVi#e*b<s6!@zh
zpPV40%@Lhqz-wc<Z38vocMmKef;6x`dLDmR=2_!Fj6MCglLZeIuuOW$NTnQpKo_Om
zgUQuzF*bW%+SA`d-f};5NxBGhTjAne`Z-y5vS}8Gfiaw(MbBm9KO>(M9EP*yrkjkA
z@b^l2XJ4#g#Bbl?!)%DUV#Wt$0)7Azf2hc7!_f9<rEpD`XN4eJ$w+g_1Hk9`i@h)Y
zAAB*2m_m3WrwD(dMD6?W89u)18hw#oQ^6!9((f&NPv2>aTIoYpxK~3H7tZ8^TfGHd
zL@#HWr6ep2Z7Sng`UqT&ywK(~s&%uE{1>&28z_?!Kr>w>QRM@|9II9WyhT3Rko(
zACg}_u}*CKCux9--T$~!2WLXVj>IzN{-Q(BZ5braQT#3r&ZAWchnnkCo`jSdndCd0
zz)XPEG;rH$G5p%g+a*P|$*bk+*8C~M#i#xysi7#Y!-(vD-D7IPstuOelW*fpe4a{w
zfXhFSma)TH6{)YqU?}FVF7w)SjkCs4FFGInv|chTLj{|Qo0xM%JgU)AT@%`tIOR#T
zuw58_q}Ofcb$`OFj?BupeeERT;PgAM>uSvD+}WF-JmP_W9k0&IQ_J)7*8Xu};-0n>
zxnEd4n@~8W_&XHvTpoU@rKWa$wjAi%h&R70o&F}XDq=S>?M6Z_F;R&g3dgG|e=+iF
zYo>)<V6>hku{q=%%k9W9GtOAuD3e%x2GTvW`MthbyQ|$C0Ld_zIV4#LRL;l6hmf5C
zgQIe_Em$l4q6yy%pdoHkc)G<H%Q#REHzQS1oITZro9HEOR|m8RZ?ISu{lXBTl=tnL
zp^)EB9EJzRYt-=3wv18RJ1LJ}i+dAnDBo1-1=vV;btD?AKpPkRk%te?QYNi&uV6C{
za1qup4^xY)4=(clQV%#%ZwZp61B}Qw6^HZR#9RCo%}6u$>xna_#RP2FNHipXaQQtT
zY+&`@fmPI&E&offiaSiAu>K#gid2|pe%SreKVTKFHfGj))&CQ)$`Rv#1y-s58>}+<
z?}Jrl{vU!>e$X7yGGf23S$H@tQgcU8y!sWQ&)jIUEEcoqw^FwN8QMosn%`+8tg6zW
z)B2&#%_0ANWT4ivos|2@UrxG`hxMpxbvD2ihU+(4<w4`6=GSKJQt)qku;p!{fLhe)
z_2m3zSGCE4FAY@OwSE}pRa5I4M-<MY-z%tNXA52nW&1!BIij`_YIqBSeXQTdM%rbE
z=S}*<$Q$}Cs1h#FkvikGean`?Fwc{$Gdft!L3B<j^*x!Oum=@&$zgk44nW~G$4Kp#
zwJda$UtXS5V=$iE=(8BU2Q!!FE4!0X(t&4(kQS>?vaBOn)mq5j&8y8@K5mR0iiEJv
z?1R~`LM*nxiI7<`<rE}*wVXKrX|lP%Se-K1p}PE9wgdkYjjOwK`rT2tx;aZ5mN~sX
zKVxeyaq|>X8GRw+t;Z{#)Db(Ga0}~}4@irV819CWp4YCg(EC!t8EAs}G?a&cVE5~(
zLF)SG^vQ-gK&Di0shx3Fd+HOsjBrJ`p|1h&qvX%6+&+h&<9wC#Bsrv!;bgbKbr<~(
zZOPH1yK(~~tV8gV2hzuUOG9x+%^<}4p+!{0rQ1=cH0%+E{s}or<C*N9t{F@4WB=uK
zc~(Hm=9+WVcx*En7Vl0y^o{E^Y9sl0W+gFSvtmV>g@x7U%%iRD@D1(|rZ+MDTb|{@
zg*-orn)1oVeLK$4qd-u@kAQA5Aq|i3hTJ${wGCLndFHA?(zuyYaFG+;0tXU518HO$
zw-f5!5cpU5&0<Wo&pWUT9@w<?eI`b;inNhkI%wMR%SyQz@B5Y9AYFwp2EioHOid9M
z+TC@9-J~dz-F*>GM5FE`xt8vpbp~~PcpI`gZKhwuXC>HNN1L>96Q9fESeMKAklcLg
zqY>ZKqamN-s8NR9D7lpS+w*Bi3%ZXv<OM00`JC;ux*>MYy$|bFMtFB<O*f0`#HCo>
zJRV)_4GiH+=3+GTy4Ru$e{#HRxhNgOXYt_#&y&jTJ(vjS_r3lxzR3>(7ihoeYN8s-
zbi3Dhqrh{ApDW?h(v(Wz3(45$^7GPmSJ9B^Ac}N-RyQ#Oxd;qK*8q!2QmS#Ee``(3
zMmgo(4T;O27Z>Acz~!%~VB$~?_@FFjm^1OnC$to+AO3!;S<FX3N5JDV^F+uojC>?u
zH9zAcoJ#B>ndUE^X0X8P+Q6wSnt%~W0S&|GwG>t}(z>m{smhUsz*24NQ@N?jHh(nO
z7hw0?uuF`J*ve;0eGosvYvwn=jAoYmdEAbB9!`NG-tsv<Ay#r%Zn%UCLrT9J$;yXD
z&m^hXHSKt9#*C~`PM>6LZw@{0(gPPOJ#<1Q%oO6hdfR`faMyoGzi|LR#Ol#QCbV1u
zw_p8|JF9U<gvELlvz>w1<l62>T;=ivfoMCC6-j2Aa}kb~OAoZ3B&OV0?_;Ebq~5t4
z3)LDTtl-grl*?%SV=L2LLN|uZCUX_>(H*HoSk&9iA7;D3tW?DoekpYzP#+?%7|}`X
zt9pvb{K+BSa#&Ik*oLnfo-*L8tbfRhauyR)hCUA#+c`eCxbfQX+1|U+na20VTY{8g
zusBFQV@04&ys@8`EG*5$XXg1MZt2rRjJ>e(+nHgoUW;pU$}C~7am&X6&qaMh#>1S0
z$nR(zW=l0SpFjsAih3byY~oB4ax)B~M2^*WI$a}b`_Qx3c=eQy@^SoX6gge1#%p;D
zv2S3lR$zOM;UZZ)%!)!ezYpnh7EZ3aSliOo%%J|F%i%Gv_G2Ow=9<k|XWzwG)M!_Y
zj}BUUcz*L?0j`@vnf<Y3Mh)it(TJPV>Dx+wbZ={{Zru^wThRtnM+yd^PY}{;Q!O)M
zIpfY}N|M67;__XLx!2AD<j~8WSq~w7u_dtB>GYyaK$#buyFShdWn8Xyms_sACT9f=
z@)byE)NC&)#~n(vimK_NV&?0*pmtw)6)sCvGHP=C(}b2g(AfP%rm5N4#$lcm9pGpy
zf?3cvx(o=z#VVj_HNrU(7*4rh&^NTO7~Nv=wQZ+4o9Y}=xsoVvHTKep50G?T_z@iF
zaK4Sq?2QWg5CGB9_A))2>OpM??U>Hl;@IVCH>TqdX6y9eO1qu8Bn+8KPFi23gIM{G
zY3nJeek8MhCZO%RG9@B3Or;<<>sjqgedK>jVGoV&1N9UX$8eT%Uc`LN=QS<wmvkXG
z@-g7VT_GJha~boS^Ft=kqxgiK0``UMc+kd1jY{V0(#>5KWqfxr*s#*<A6a#n#1P~C
zff=irg)4o!fFrY#O!3`0cDQpxEB@;1fO?KziD1ia=71t)Rm#J2r<hMeU1QSg2U7Oc
z7ESYAkY?{FxqhT1H*ZZ0|C3zZCT=s4S7;fFR{NTIKBGg}7lZ}YYkpF}`GD+BBh(jr
z{|g=M>dt`<TQ=?%iBwpy=>11PLOKq~*+Ef3ab^;1r)i$>!K|%T$l;)K0@ypHCm}j2
z&-ucU)uI&Y<Fry)Y7m&P)t&1WL6(%GYu{)fSOe41wdpO?Tgz7rVlv8iEZDd_FKk%P
zUr)_+J)4R;@Nckp`)ssc1=jh|#aS~5FEw?y@1u2`GLIZ8oU(Sx`C9W5@Pcz|NIvI{
zuNPVy^Eu2OAU9m0h6p3N5rAosPMA#OowpngrHdL}#j@O$gX`+6@9H8e5Ydqz$9Cb7
zRNz2|EYVr#OrvLD7HduHGSb{m6uPk>lM1{xAj(m)|6n~pMHB<gI~L;fte^DB`ON;>
zP+@QVeyb76MJ&}%wMZN7wT1_d%6h{(6qW%oDX+drWb}bAhiH~l-nwUcqP(}RUVgbs
zmV%xI-Jl0?zHpebnKRo-HjP{mED;SdVtVJ+$*%>B6j5%O6m_j|J^nk|?-reua$Dh<
z(9Ax;ZwJF!uY~GQ*ofpEzAley(=t!m#<wE$TIa&1$W+~iy0%T{d1fexvq4@-NcBtV
z<BJ8YgBc3qsU)l?o@4VWH}|scXOGv9pyuwFRSCbzs+tPiFHH&fPTh$7p14>=+r;Cl
z^r<Q`=@WHHl9ktdr528j64Kc2)k?~k;wU4=ZMEcuR6+)?ANqli!Jfuf(8kKznD6K^
zVxq8*scEDvKR$JTll`M1wHh{uWRqYDo1+`AXvUxpn)&!ySM&98Yf7e&WJ1iR`9KYL
z@|@<$(Ot@Xg4b8GCZFi$Q)H2GDc-ta!d6ge?hgu<+D8fMm=jVl!~<gQN?2+}>GbwU
zvUxT6=i~J;FY1z&Y3Hxp{b$L6OP_W@%<!d`ooXx!{pChH<)A4Fve+=En4#;p-G#*c
z{tvOVDKif=%pS-VPgCb^Z04CIgRhsLK5J|c?vD+H=?aMw+KUenB==l5;!bIQ;%#?y
zWBK@XaA#|C>LGt<DPzN7?9Lmaq_XA_)(XcaW7G<EjdM=BO5B{7^8_O+O%bzUFqiaE
zc`D0?BbcM+y^Z|}WTU6ooJdJ^S`=vJmU&Y#vR+^8llna1qx&+_W>~JdQ;U#_%s%+N
z_++Rykg)ae^Q*TLt(d+qaM=)*1&X5x>d*(^>!4^B%6!2Fq+KCV3K5}Kb>?B1qts`i
zjO*{V?ID*I=AM6X%h3;MM!((}zaPn{x}!>9Z4_7{oL@Z?Si@7Z$6R@qY{FVO;#b8+
zc6O=08hs6q{dx8orHnt&lBKn-*mAMCn7OTtmF0JP{)L&8igMdU{&<c?P+;x!MIuX`
zb6{9lKJD^mO{%HWqM&}CSFI9-w3T2MN;XRU`AJOodDS%KxPL{GXoJj7vwf6AdA%5D
z=|nyCnium`yCFatb<~#|G{UcbqWI~s%xY9=H|Ak8pZ}Ki;qzCoNkuP`C7`n6Pv&h)
z0^H*5%$VS)QE3x(JW8eRf=Z>Y%1qIYwB_8oH?Hj~s0b=6{iURIw$2#{(P%XxgJgnk
z(<vu2Bv1EB?$Af?{%Wyizt}UEY1S6sSN1wa)ptTnZHU&Nw|Ty`@RDFmn$Mw?Z92(}
zvKwuhJNV}0u&w;`OYPh_L;mMIi4%3}$hlZg>(9c12@+2AdsA|ww4rz`+t=RfBRX2r
z>OxAEriMb9YqZ+mhk7%nrzPgu6xz6^&CI{2dQqF$be50hpr>bg3J3&LyU{p1(TdM%
z;L2kz5dk!blI!KB6R@~J(j~Yv*d57w@Y3BkUk_$6T_o%tkT&jYbA*qI=Hp|tHub8$
z^v~F);pH%)!?0DJ={Is4F(v36eLQXNE9)r)mBurdw^_h?isRhtm)(jGlNtHhH=pPi
z9Tw8we(yzmnUql}YY1O`$lqfSC;ZTGgf1~uSV(cQC!z82hNOX;E%oWHa~hL3h<i3e
zt>CIt<%e#DlJW`ES3NKI=V0qXWM7mwQ-vpr2D7iKO5{<o;o%1Hanhddx0uy1SCJ8Q
z5nv8Jq*`lO=UtT&a-u2!1Sx7L_Jq8D74?p;A!94?C-+HTdQC~JkJ{eHhe^23@SQpK
z8Rw=66-Ow?95F>1-sE+Mu5aKe<J9@aN%1|9@Rx39n+C_0d=Kuj?tRG$%(2X7t(D(x
z{yM^vS3<Kk>7Dwjy6-DD94d~Rsq=Spyl5w>>dvdvC+wdf&p9GhPZCShMqb;V37UO<
zQ+CPox%U9avvI6@FRR0F8ei&EWaVoq=Wtc!br@bQ=D5}vI&`v)Skk`d7@05G5M9Dc
z*z#!Z`oUFr&-1>jsmrf{=g6l9A6FqppDaG9XMilY&I3k9RgwE-GsXyR=VY$rLHinZ
zDxOQ*M~vk^YaPLAiW00ZM`Dz*YX(YI+`6(WyY;obnA-~R5I@KrhSF_gqJAEI;S`76
zsYA}X8i>u?dfxN2zzGK?e+X-Ott&4vie!}ShBe*L4)Hdp@lmu;Pw9Qiy4G2#258>S
zB$bu`vz~CYO^vWKTD_uP%fV)RW^MiS%k%&ZCR+30w1LuKN>EbZ>-vj$_YOx^P0*|U
zu8O%SqysrbGVBqQ`Pyln9N~C{XF!U111S_WE!S~dDydQ0hGxXIgPXx-_*mm|?l_sY
zr7w(%860$SJO(mT7p?nc?SEWc9BhQpT(FssA52Y%32JQlB8Nt$Y&Y=*@?Y<)bTJ;R
z<q}%^li2l5S+}X=`Qd6FY`dN$DZZHL$qkxV&(+urhoe>lz9}YmB|ZPNO3|fxkf9ve
zaM6X5i{QsNNi9j{RbK4;Jld&vHG*Y#CP%#%K1fdLkV2(wE%Ffi5@QM7h4ZbhJgdSb
zMV1<XX!{IV#dQp;^rDV>!KU-vmr~NnU6bG>>M}kH*XmcDykd>&UM&}S1au7@9QM1#
zetjs*b+Xc-qy<zzduQ5bZ3Tfj1!QR*WZFVYk-yp!TGlw~-z_1v8^d8+7hRJU<t#%3
z%8!*h*aVXoe{k<a)NE9pNTCDi>~lh$%z8|qI)>#C*v#H5Z*YT$se8FcWyjYW+1#SD
zoY-aCiIYT0(8iKH;s{(x>aZ~}?w>owA%jrqf(P8I&@RNbS%6aD50R4E79rx(F1fc*
zheL3FBq2{G|8YZai)yKg05>qhw?fS(bhuratZT&vmE-!hUl7y@aNb^aU5fxE$G{(?
zmjC1&@u10BA=G6P7Rjilx-0T|zOg#DbxHLuYo27@YCg$Sren-i*t<974Nl4@Lt^|H
zxz_Y+rE$usuFT5YnO{VsKN`~N9G<0k`*>awSj<;+^y@W40-b-Bx_2Z<1PU-iPrnSO
z)`)(dCvCYNtBQTfd{}NAPw4zfy}sq}nf=~d+j$)7RprlRu(QmXXo&TTI7#(hji<s`
zU*9hdu(JBxrD~<d%9>F15=LDs4tVsX-$C3Zz04~4JlBh=zCFfRGgw?eJD9}17O;5>
zs{s1Fh3*y>mdnw0B6d6FLL`s<;PShkaqcm%_Dxy6*CvGn<egNv=j;<;3_qygDJr{_
z)7ij06OVKL=7S^lLM*H?z`WvkJ*~IXUO3HJAEk@YN8w_<{7#L9#ij^{1M9Zatz@H!
z%6nH*x(!PL*+X`^4dcy&g8){kg{{E=IM;?tw-M(9IT#BI>o)c0IV`LK;#=SwOg`9H
zSWjYR>!;RJOG@*wu)a~Z8hyJx51a**tlRT9<@a#2<}IfuyV3sD&--`I^-EP)9^$oJ
zzi#wO)5^d~o8lWzm>m~-bycdXt7g#0?#986g>@Xc$o=xxueyNp!b1?}&(YH-d$@m&
zLS8Qt{yDmR19*<V%e|X#g#R2d-1_IcPj3Iyme==yrT3qYA2R%E^{?{(`31j)T}ZPW
WeO7CP;vM|DT}3%H+0v&k-~1<AtsK_?
literal 0
HcmV?d00001
diff --git a/Python/_build/_images/test_tree_view.png b/Python/_build/_images/test_tree_view.png
new file mode 100644
index 0000000000000000000000000000000000000000..5b352aa58702ec021470a4e16293dee84bc51b30
GIT binary patch
literal 36942
zcmbrm1yEdD(=MC@3lKt(;1XO12*HB{f)DO8xNC49AV`99aCg_iU4uh{yW8LngUcPx
zdCynx_gCGjTlKFh7>3<@@7}%E>ec=9vv#n8oCF3M5!#CvFEFGeMU`H>Kumn`;-%?p
zWcYt5E<Sj}KVCX2Nr=2CA12v_e?c@8mKA>SqB0Ww-T(>y9o0@!)A7X%(x1;iFC(F+
zCNEw@Zb*p=tGMYNK6ttj&wyF)y?^2dzhF@84#Zc?wZ#(-$u$Pn83SW!c;(iq>rAwx
zbP8--7P>2|t6e>B+qP4;r8c(GFEb{N+M1hXy>=HkEXK?Fs{jj?D<I1y6Kz+`Y*F7j
zny6U%b@5=_m%1O#rhoe)etCiT+LyZjc-wue^SAz=Wb`kFCtX_YG&YY1_$KeL{?lZY
zA|540|I;`aQ0w&>mmhuh97RRfY!9D4gCm0eZd;Uf`jr1^*|BUL|DGYrVBj14k4R9E
z5@nfoJxdme3{?IxMo>Z&_ox@rmNoLvoyI`jcryQ<7K>eEv40t$#rCfnSYloOeH1zI
z-)qv-y!js?MQMWn2PJ;J!1?#8c>yo;nT<FBN_5FO*G<E6UOIp4YF@7YUrPUPdD?%U
zQW_QEztsJ2g_I_y532;_H9zoB#tnHYgLCI7lp*_1H$L~|w;<E|rIn{vL5Z=G<2cH=
zggB#{5NKg3C`_y;RG))r3MQC=$^C7qyVh#P{w8LqVa)lfj{LJ#VgUsDAx4T)sFrGF
zBC1o^%vu^@zp^Qv!7GOg4Rem@hliwgHFfKzuJujcj3?;(2i-3w4!Yab(TX)y{r{vK
z`wtjT&vb9YJnSxZJO|CfzUqBy)?ij^SYiPOv)e4x*=V$R@rIzGPeB*J;0CW7_O~QF
znJmwi%0obqQH+g?H`sr;$|-J=*2B^B)c*XIj4y!2ETN6oF@F!;%2=OL_d`~p`AaVA
z1{7f2zA6ahx42011_L9jqy(K#F1?#B!z()>Rd6cq{cURgp;m!lL9mygViZgU4?!lz
zbp?E75W`NhG3$A}CSWpHlk`sC!(oO+GCE+X{Wh+R)>HPvl^+KHYIuh!bdRJGD}Ur8
zvmGwi0-J`6O-$rK2E3hXYp)l#1%F;G`}jydzExIjv3nVUZ!W|}V@TV^`n1`8kNVp*
zrY-l>t>t>R{Xs?VxgVuT9oYi=2L?J3F!_Cj?z>oNNI=-4nw8<RYLg#l5MOmQI38-j
z65O#s*k|YGmBs_u{B=es@vM6J<y3=+r%0-ZVc49ioq0)L5EF`viznNA3evrl^nl7@
zpev!<@q}#TgnMtH`OowAyW?Wc*yn~i)Rkq0Une<<5&eObI}ES7D8KMeEbZ;~lr1mZ
zD!DK+Pc|9;2&sFXRP^tyJ9C-n={u+*`TZ=RLmG_p^9!e2Dc`c&+x7HfbU#%h#hk4A
zV;1X0*e<sOfBDi|{8`O#s#tZ;k(88lV%fv6yiPwEh;1>zG1Zy3>!l+E@d}2NluRUW
zB(T)39=a}1CsfIE)#_uv%wj#dUpF5WDD1KAYjoVUJDlh2cew6@ylzraQ!_lAuPv;d
zyQoc$ih7|tcNcQ|#GxICoZ$EB`wQ>O^NgZ03)3c1ZT3GkS9#Z++aphRJ~Iiw9!7lb
zC2D&<DsmOE$FDXbAv34bF+53t-71$tuf9G)aw$I$pWv&~G@w7$OcTp~8SW}SQ}Ut^
zI={=lK<&?SJ86Vu4y<=r33(xHZ(r}5vOz@B&bW;@g4!im4~}_Pw>;hDQm|gq+EeMa
zdvJQ+HLrv>ucf19PmK+yea=Bnn4g~?9XFJ?t9{FPHQwL<8V{fq5OU@)IX!JKSC!{6
z@R72=#`Rbi*2}wOYU#@mecgUm;B#w6uUI~{y87Yq)aQZqA!)bs#l+$})_x9BKtle$
zbGR#Mw9P4Jj4{wx$|)45EaHvFt8Z)5f3)595A`;B!@Hwh6PPkSV){Iy)9b2~LSEhr
zl0QBqwErGFa;WqJwtL^%@Q}>Km@!6um~d)gb#!ouYrpXvGgP-dfGw&SF6p?I_uRtv
zEA=~HIqT#uFYBiAIir~krz`75VmYVvN2*$tC_c)!2sK5RMid9Cd9}Nxx8Y_mW3JC|
zr_O%n_iQ^r3ZuE^u2vMNmwRHf-F!)BOHH1th#&=}X<+i9KOp)O|3bOq$l05&ZR%4g
zB+sc|#kZ)`fG&L&<gc`^0hXIlY`Kj=*g<<ZU|tIO#$Sz^_wjO{|BT(+tD}h3M6II*
z?P?}5wp~K#ereEH;mCbNhNXi;P2EtK@wC3_pj=#IaofJh$P4`>bM!@GCaDz)ORd|g
z{DhUnnnOtf;Hg|t2q0>=8{9`OAtr_-Eko;dF(z^QN!je~@^Hf&fEV0+)p|p_?DGWX
zNt3G;i*6vqd&xGY)CYA~8_xsw5B*W~v40EqETwfp*r*&6Wupq2g17fc78LJsZ=f8@
z_~$q0+c_5Jru^5d1r@Nzve_2*m|K%y`fMLqSuq<2Ig^iAq}GALh5biw4c*z5n`EZ;
zi9)Z(uYAv3KSX!vHhQS)rGWj)K8w~@C`?QNZ25F&g0&oit*8X$7OE>~evA6_efoZ_
zJ>Pha8XFr1ZEo~_;JZAS%c(n&nLJ#mpBf=Jy}^F-HYlDx6kmPySD>l0dvOpsNw+5>
zkJuw1_zKh#ulfnlK>aGtGE*`m+MOKZ&6{G#SMRp7GOi!HZh9djtKr1l#Y<B?o}x9>
zj|XOg{y%;o!#?rhoi0|H3{`l|IlOuM)?lITuF_^-4Yth8O(*dX6>b`D2<5DX_V17q
z5~jq?Sbjgd%0t%q(L25J_Dzm*qgg9%|9wK?m-Lr<%nY66P|WqE?-x%UjD^BD=ai*`
zV}2q}B@ptyxFRELJ#3TBjb%y%Mh67*)+KI_@JU$DpvRjHl}H~YXF6%|w|k*ziZLn6
zU{yl=Rr%kv)@gKEu9>VxkUkJi&O4?f-Jg~@D1e_VzmnGXX<w;jcpQyXI<_JO24bhW
ze4ZB|Ylu5eMGi%j?-tyIx^ArTg`Qm<`3)U@&0zI9^EfbX8!t#0CgJQb(hdo9e=rwv
zeGA}jv15w)hCM^2p>E$cJB=y-lg?HE*^$Q5+BNgn2ZO=_^fX1k!fZLKL%vc?Llwbl
zn(G!_dLPrlUL-c6KZOd@cMBWGbV?oizsBQAD#k}~oHLR*((dG@YmgKDI%({yX*HPR
z88F#{1C}#xE(*GAy?VmW(px>(RF{KJgEA-8pyJF<3z=tC>|SRBCKs^0D@-q1<U}%`
z`;VpN+GWU7e4imXt*ttT!v>6+9LX`3`6!9`3r@(1-770OaKAlOqF(ZSnH}1CciKl@
zqW7X&(fBV5K1UW=*nYAkw=jjazs%e_eQX{7W8HRLJ*9}#EdXc3X5_d{liW=^gH~cG
zs8Zx6TaV3FrFL!2z^l}YECg$(`$^qEc<dGQ$GE`SwlbgWh5NhT$L|6C95~u6d1r#`
zj6M@l3lhnDSJ3sI(9^jQFSW+r`6_c09p!2?vD^Oo;CCiHT!ssoJ)unYwTH#?gTbZ|
z>EFFu7Piq-v=n_0ks5h64!c*D-+;?~r2T**&Wfn%I&7>tFEM!A-}!*}$<cp!U5kyp
z@QA)PB#?^!Vc!D5zw_`in4wQWkm>xeQr#o2vT5o3u_M-d_bpqQ``O5;LM`HU6!`k4
z{jleIXu>FSzH)*6*-4%~Y$&g!vUU2^%-B;$<P}-3lp>39a>#1*2jU^SzUaXI<Ktg_
zhXMq0Ykd9jPz_}=YZ&S04%|0YlYoz1g~&PYK6OwVQN)+AGjP+9uXSyy)p)kpc{bUV
zEus4uFqDc@{SNdt0Uu1uM}<KL+lDYcUL{FQG#(qvYK!mnuPXiZdOnC%N*VfiS0Au+
zT?|NAb~E}#BE8PjBkNe2yPPejcx6r3<jY((-Gu6WKR|cA((JnNbo1xi$(|LYSjx)6
zR6EnF=-OY%4UH+5wL#cWw-jf@gy|GX12I1TQv=5oHU<k4jdnUM?c*`G&2la)12=b7
zsY_@m9tb;Lg{3!{(`r)cw**z)d*=Wba{RlX%dTE0?Bl%tk)yeB0E;AI0&_k#*kY@}
zbpshnw2y8<y@%_JGxoc{g|w7y&0vGS-(RrBZm2-$>nF`GPNCJotI<2t(<IaHZOdQx
z(@sJdzaJcaNfB}>CaFbVN-I&YVgGXJIW{hMifL1>HU=Iho_^0Fan?0=hQ+_$R^0st
z*#Gc8udfC%(GV#C@#Eyv?nD7B?>Bo~Y-UrFmfraNmTP;8ik;LV2f2_p{{hgqQ2W4$
zYiML>YCDs*8pAt^o|W~h3G{#fW?siw8^6BG3YG$g;T|_yQ@jE?@-|RoOG)r~a(R*Z
z%k0u<b%%^`_>)9T2RO3yQ0={QMhXT8f&Xx13FkLrgF@c(r!uaS8O>EylD{^@s;5n0
z)g#1c3B3_LDiT|&eh~a&Uia>MUOm6gb#=KdNy;1siDcH6!#pVGp0#eH|B(fJ*v-hS
zh3^IVd}?D_H9r4YUqonrf%YvyaEDUL##a)71+;BMj7SISF+&D53ZdKls|`=Jlrt6w
z9HazYcav|u;`*VJqq9MIz`kgw(EQco_NVV{xH_e)<DZXP{3pzY3uOntrX*f1x0P+J
zNG2pri^*-~Xf!$?g`kB_p4(bme=+JJvYxF_Y!&Zh#h-W(6PjbeisI!nl-=D!`#7aj
z1P>y=r^)k|Zw-W)Jhr%Nv^Kq!OCaX{=CW_aK&X7$)QiUQu-$wdVWoH$s`sdLlWnbn
zBZ*?7?8<w}jc=Ln9AyQXAc%k37g~83X+Uiho8cLwQQg_w`}G`X4@+#E-;FBhL0Dhb
z-`V#4@r;$j0h0v?vJS6Uek{51HMO*3n7dLeU5zfdqqV)fD3%4IZAH@Pqz_&}Px^!A
z=WA*>E+b#A3vR-|kPZJ0^1@@nk2;o(xpnM#S6|kBz|8opTqRWGAv2Mt<?A<rmiUcw
zK9~`h;SWn<RA&TB>Yy-)w(RxBdZ3$3k_-LusinKZoXh=wX}tPle!HzMu9~3yQ5TJg
z9_8-!ld4wsk)DQvLT)_P1Tqu|s_>i^g>~cfYx}lqT6ZSk;}$mN=PusNl39_jFXAz5
zn~q;qD)4gm?;Mb$?JLopRo}a-GNfxK{w&uvTAHBUqf9t};rQvyJhsPU*S2^p-gI5}
z<9{)UWpU`VBJ!}siTZI{Den|K87Vx>y=rgdJ>S%iwCQo*d9K$pJk@B<>y9KAzHoA~
zq4x`4$W^FrsU1ms-%?R<^t<Wy^^qp!Usa}eUuSRT`c4A%ZvfIai!U2kKaX)A5ivnC
z?y+42A2q!U_B9?F5en?)2ycmK)#L~Seh8^VP~=*d8sWXlgI!N<lXoR#xJd$7^QTO7
zojC3ki1I=~b9H7XPmei2EBN{%Vo;rwV}_1;x9e20M(uGuS$?=iMHZOPc`WMaymlQj
zfX;S}I95%1iKz}LHm|=e1+^hgsR9*hyWdzDkp1uyI0<}%`;JY;@L|chAt#&QX0Q=>
zgI=k@z*<+?k)m=17-O_(?GK!^N5vQD$PQu0-G)gVQqtYDnzyg@0NT~stvQT}+#Pg!
zMXwqMp|k|=z<N<{fYYnXw<a2((jL~c(4oomaF^kn8HZX20E}IA@To03i>v`=gqa5o
z(F&he1$GSrzjDVq#$hjCiNa*^7;HK@V*G%=#`D(75)Lk}x!X?tfa}OI=8CrSB4S5I
zT=Q4Af>7EEY%egFo>9wpv4w<0z(s}hM~Z2B^HCFeq);8z-iiUWqIm54h0))RkTV)V
zH9Lmspj65otTLL3)~n?b)-hvM2J$W8Y!e{g6G}*vi1tvxkyXBCZL#TeJqWuqPcY+h
zXs0ZF*#h0=Ee1x>()h|POq+m!V00z|>j`;TtsGpVpQ}Na9(oB8nWTZc+EDQ5!G|{A
zdolN-_#Bk4Jcs8D6CZNj&nt#OKga=Z<aHtO5T!xl?JUE7NF4d((Dv@tyRL^@kHc_k
ztub>LW6YxtW0uD{QLjb&tgV6$_Q3DH<FCY=fU`xus7U7?TCWjyuHtQD8qWtrERws0
zGwjxVt&;|n%Gp8^<(pTcE)jU!m7SS7F=+lTRf9P=Qw{SL+J9`5ExaKn*}KZXOF)mF
z3YqU~3xRw?$8;$vb4tHMOE<6TM5|X@KJo+JSb`R65Jm*&G7~0)zD!SyFhOn9G?vz2
z3(Q6UR~!Icp-B$OSNNE$iJB+t!OQd3EaGmEEt2Kk+ODIA1>kIJeyc7#SJNgP#CBhj
z2JA`NT>|kZ^{16bym40CA0d;w3nPO51w`3Tb_$kLI9>%7nLJo*O?PeLlWIx&^gBQM
z_EqVG(DXR~-)#=HCX>F7^kKTX)+>qMy}`|?LaMi?OCOeOH1(L_@qs4_^7aXGIWPC*
z50=uq#aqrjYaAz=PlDrB#)dTo3Yssh82r>N35BSk-{wHo$ougDPZf}O{F&uk68ZLq
zd_!_A_fX-djJS?AoMJOHDOv4$x`&LCQS<JtKeDqv`aZw7$(|$c``wI33yq0mpNAQr
zK=m8{VA?2ntnmZ-R4sJ(35;q9&R%v(DmEDDDHP6nJUV{T%@~p!c>9e^Q!w5MLqb5O
zYtAgC|H`B(?+%>4I|GaOeAGdO=_7p8p8McHLI?`Ea=kA?5e)o%3@RB8&FY7buLW}(
z()}NtI5c;z{8Me&Uir-9Ldv51f9X9~M)Hvl6_9%={As85x?cNvy>#pY8SXpVjA>S=
z6}6(EIYW8`5(#OqC+(bnSYCn3ykVia6_loKz?HU{YsxgAUrxAMbi_Pb!A`$ZbPB}b
z#oBORKc%aHLpNY4$DI>{R33W~z3e2p8n+woQ~uOTr&V(BSUWQk0vqN?jNOBp`s^sI
zbGz-`M`NFgWpgMuL;d+NW@K5Ys+weex?7F@pI0hxVC$c`Jx0lh`ti2)HGQq>Tcp8Q
z`#Gs8co&SFI)e#N{DnB1izY*e{NpXszE;7c{rp5;3)e<H7Hpu3)mrbg+$4K4RmRjv
z+BsnS6<=hw8n5_R-hOR@Zs*PDwCL^V4W+N+W*+`&`C}>(9PvR1p->I(e=nc~y19>`
z?KINvH)LIk-PD>W?2S&-Xt6Y80H*cRb55$pA}B6E+PXf3&8%T7;*$lsKgk)l^V)4I
zBN*cK-A@I^Rg`mdKR0(V-FD_o7Y9}t-wAs8$Y1wYhBvnrZP}pF`02aceI%Lnp(x^)
z)6!pmn;2)qAAxDlkPS>CfUj@iQoQQVIKwZ$kCytDHZ*S@X6$ZdB$U2BhPH>BxN>F}
ze<A)}{)aWNbf2sqVh(@C!OEL?q3+#?ZYu502NmLO-gZc;<9Z)J3#SaiF1lo9-WoL`
z@mY^T4V>*&gk^WvH{G+~$_|qW%++QKUL7yjkuTQCGDHfa{L{S}t)QY&IAUSjM{o)l
zgAdT;?4KcvC|HV3L0H@?Ui&kazUJBSz$iaJlBuB`L)1u|hSD6DYVTi9SK#`XJ5F1J
zQxPyr-6-GJE6$i&#^bTMc?X7Zr#fToFvTW**X2Z>+4p5KHBIzG?^#Cu2nCNDe7{C;
zgbB2IOOEq<Ww{;997#Vs?S`+NkjIGKi293&H0=dTOxDg!kirAk?%Rz=3fLlE-w_61
zr4*<=uSd+%PRbgG2!uUp!{6FK2L06B+7~Ef(;gq+p+Gx(6^IfVK(Z~|;xSnULSRa+
zi>Q?zj7oKL9a%nw#=-u$lzEvq$&c|#1pQ{E9z^jM7s-Z>Hi)4k&s~M1-v(8+3+)lO
z{(ir}ZSkGMBxQ6pI@9)i{v%ILi>(fd?yoEFYi?Aegn(7vpyMQ-Th{BxNZntyP_0An
zH5#w&9<aIMCB>Y}YSE8=aO0X&FrKPb31b0}flkXP7|&qD2Lm&f)`grNR}G-(53H(p
z2H8oScxu@f>$KF%7M4rv0Taiy0Pi+zNYU$LkT_SfkF^nvqET67`>RCD)Imf2FaDBb
z_Ree7Zj3-yQH_M|F*8ppSzJG?U4><Pu?uH8O~tTCAw6fB)tW#SJXSkg4rPDfSWu>7
zt}#5ZGUpxCUB}-MrOn=@^^<A8hWb#y2ccmL(^ekJps^65p()sMcGVPW=*93nLz0Ln
zRl>GMvma20;#s=PgwB&JF#a;}E6X<pu~a14vUlcvz%r&x4(1|!GicWR!JE{|sj|&G
z6)%Ir-92Zis(iN<_ee|sSF8H~xGg|hT_E@!!2gJor9;jU_0cowy%3{zX&tiklyP~+
z>(ZVjaVLD4QmXzi3avvBKZI3ur)6jK6)Wvf1B+ocRNkmdjlaT*F5Tmj(CcCDTXC5y
zWpGP|pUpY`7y+F^w0UE#Gi7icy2M-E(HT*mm26}Okx1TIB)}GzPJ;f7O9q~zzuvIv
zeV^cAAK}85jls<MeR{&l2L7dGC%yCIx6?H8mN50hDm-RS{uVEON;Zn7hT3}f3(gIg
zJO2p-jF6d@0K1f%Y-~P%I`;dE5yV%>t#`?7$Ne7^ilqTGr;xmboN!RVJu9kipP;}*
zLs2m@Ue_kXrfE+qAs~k$J8m9Fp8q2e2rUHvOY&}KUOyf%s;i2Q7=(>&^Il9emPM{W
z`9>jS5hY~SCI4L~O{hiGnL8E36EIu~Dd}%~cp6k?uH`2~iK(MtFxK6EvDD%YypbXF
zTp}VOI?;Q)9@1#xVQ2bDTQfWg{^LX`uTh7X1c#7mNkPfBZS?I*YO(T*OP{rp%$*f4
zv4s-``R!SSFfrAcJKNP%BT^pA=5y=nP@Q3F+`0;mZwOCItCpU0OnZE|BSt=PL^)02
zL=tJRYsa0_cU5l%y3Lvg@TK-t_;3hK8q03oBGqHv<dmCezGGmiC;oU$)U+5xf+B;#
zeCUC^a<>mz5ANeK3O!qrexTiT&g|iQblzQT$AlE$L|MD7Tdjm7h0;ek7_O+9m{IOj
zi80{({)Wp3K8Cm!mc{w`EzqO9p25W!-*V`YJRyDKt;rme9&Qn4uIYTF{^ZxSo~&i5
zK%V}Ck#Q`!uaHX()`d@nT&yH`Hx3KVq=l8U%eXJoj7=RM)w*_2JIL^klY`K7fny>?
zx5J)ePAk3k6AvkBMP)rAu-f`;u|H$RB4t%eTc2k#hf}@K*Z-bQ;gmBAp2VD-CB3!d
znhkaoj7&^ZrYD_hU*<*~b|c^rxZ>$-&^JTej8*a*_$2#9TB2qE9gMq;4yyP(g*dy*
zJ|P~3Q)fNrr=xeD-6UJH+va)P<9XZ;Nk}LA1imOLA`z3LK6o6TkT;shmsQk*^h>pP
zKOi7j5ZVf;u7eW?7?$!qoH``eVd!zejxu!it!QXl`@oJC_Zv$la-a727M|%9d$tB#
zYCjqOuU!NLli0uH;#3E>jZQ@X^jzPTy;AwzmC_>fHndR6L5-zbs%Q~A@bo7$0xuya
zwO;ao^bJXf<9w4d)7g0qs|iaa@t;5VQ#UdxJ7C-!FOJSJSia?Ev)ekDtni9$u=XwK
zdlq6pAeT1A(Tao6*~OK_Uj$34n+Tzt$yp8R3oY)|JWw{n+yQNY`qk6!N7WS5h+y%k
zm?(b_=Nfj@nYeG!WnaJUgne8hAP^n$<yudsMjBnaTH{<{iyt28S$98IFkt@m_A6Nv
z^8L>1utO%dH)G-eAUr_WOuccdFc-u_M7CeO{4!f;H`QZ$?}Ha4IxpZhw);zH>7b_>
zEZ9=w_2{yN7uGCTDIOYH=gmHIjOPHe$&M7K3!5Tt{%xG+iOWlvXWp`c910;VkkcjP
zK1+F5CUbMix)wPTEIKD~I?w7Iq1Wn&NGG4xD+O(i@)TZb7#pQ7!W<4QsBG^E4YB-G
zs>Z}zyI)aTj=OvrYIjMaD<@8&w%Ga>h)%;3cWiq&`1Hv&Y+zzK-;3<CJ+asxR}@uu
zAlfuBM20d~>F|C*z8%MzqhV}6xcic-Txu)?pkc!h&<;LG?xkGb4_yzN$D{7@Rf?+d
zXuwb7_FyYbi0Rp}hoVQ3R|?``1!$UThpgY9L~Z*`j59(tx*elm;+bfM8Pd7iuBwUF
z&t!^9L@A%+lxYODJX%$7*JlF)%L{JMhbG4sDX(}v0C5{v?yWngcWaTHOM9KN&|@7C
zs}#|Xk%>SiGNF)=@OMiS;alt~nqktsP)=0dD~&t6=pr*l>Zy$VGpKyUaX9x!cJ^!t
zAr6tk;E&a4JD6pPU_nt?*#sFKA$+}-)h+A?zmu*1WH!87J99-7o2eIz1xElylW5y^
zB#?dREC+Dma-!UM=|LX(BA#~|C!#gI*g`Ebo4w&I_xCG45su{Eg%B)mv}it&)%bi`
zFvet)Xqs@=-5{Q22j6%7NhkC_c^nh|z;(g>tPyWKOrJG_-x)%9sTkN-cNtIj+f#Q{
zl7?w+Ec33f6cY-SCrJ-=BdS9CuN-A>1hXUBT!un@E}Z((V_5~W2C4&74TK&mhXS9*
zr#a`bcdfPpr6TR(%RMNAr$r1mIXN}!&cwLcF_qa+CXMr49?oJP_NzBGjZUp^&l#(C
z%3>BRvn|GmB*RMcX}cZ=zT3Me`tb&3W9f8Hf<!H6Sj+OpHx!$^dzk37U=1zn&W@el
zC+}N<nqj+N97P?haY{dGztqIq9>tm7ou2nhq7-i>rqeB{tYPaQfQ&iYb#r2o1YdrS
zdgIeRFTypAOU-ot_&6!#DG{R=+9GMjw>qrQMi;d&xyTsA;&8-;!ZN3ABu~0*Kl8Eg
zxF@fge0{VMQ$m|^&Kekf6rEV^p%ka$E0sv!f3aCkS54VK?R5{YiKLqDj$6Whcvrz8
z>PKKn!X7ZWAN-qR-Bd5^1um~xh#Kv+*jm>)d2tR6Qh@-Z#qTJ^D92Rxg2|gNSTDcl
zwQ!VjuDJ=N&(z!ZWi5$Sao3+8SDrkt8gehPx*fB_=z$!n*)-A-?~Oy=gF+U)i$RZ@
zRe}YLj)!TtAg`}qyRILOeInKMJW0VGpPk#!2yWXyRh9RHW$8W^gR&=NzKP*eFH5A!
zU4?yXq;9%|=;}aN#pnjjuT13&GZqY=>zu}xzLMET!Gj7e@7(_q#fLce4oGl_W<nF2
ziN6K2nBBN1@&u6SH0=@Md{+20tqR~A&~4XOU_9aqMA~Sg8FsU8(kokPk|K5SDIHQj
z`0`srz2>^6T<`9pT(7b@q=%8tfisxJp>7KGf>0UJU&+4Sq7-9ajoZQlpJ#syU>=6|
z;`NaP=B%t}n0I}XIC^0tc93LnX3)?QB-P4oeOEHj^i$a2;{&(|OcwG`I|q;4tH%mr
z%(TX~65!MEm-kj`l^nDDBVVZE(Ox{Rxm5k(hCkmf?b(N-GRW`KZ0FNpt4-^%as;%n
z!~;8)-D0z<=(lXBr+zX}sEH=LT->s>WNt-lWkBuCxm;n;mFox0)`rJy4;El4sy)2B
z#DVTVkCjr3C!scedbqYrEqPp>m~zg}&h(nO&H~o9w!Ldq5|cMYY;&v$hQyVJAj+n1
zkA18@MS|iXSD?Dr-ci{HI86Px7PN4r_LO%U7U9f*turX5o+BDOGVOCWH*x|mIBamY
z-z}L}NKI1gp<&;>yUYG9og269gmXopUM_P_<=X(!ZSzW+50Q~E6pZq;QM{6AkId=@
zwK{9dinKN~Fk3!S0e&i?y6j0cOqt}I^*Ao#uUwbeGE%l7$ke58zFBKJ+h&v!U&J|+
zl|Jx{$rB}g0H_Y=3a!loY+@Ti0@wBf;bkWugOGHpJj*QAKl>LsffhkwJ@3qCi?%-8
z-qr<I-33fUOGe6<>f}$-{TRkCi;FnG^cHOV;9w7o#wa0%siI+lwC*<FIy==X7=n$y
zf@2Y<*1i{u4xy&qMHUICj^{(f|3Gu6Ypu*?`>nR$TIO_Xavu|LQ)*|{kjWpvJ}$#n
zsXI){Ew_l63`5xK_vGS#G={@>h*vmC)*!?vvRTf@#U6M5>%|<cFN5}Pj^#!LSPCJn
zUuF3LNood5(Wl1}QO%NnlBhk$xB_`&#wiF0&YLDyrWO=<9<`7ld6kSFk5pq~qlVgn
zmIA2tP1X_dJWC%8M?Q)jc~-%Um6UiBMV4CT$`)6fvbT&wIB@)d!emg{nc3OGd;7?z
zuQId^H5+3RidtNFyLj%OmOb`oc7TqQ(=wY*s9W8yxB3$$&FvVJl$CL&mz)#$SlgH%
zZVz22tv<#fBO^~W?~alXol?UA^Qk5V9X-<mCiM|`{hCFmDfd!`m0=8f!TW50&J9*O
zvzqbU2NI0^o#vFrvgy2y4P|1J=E0T!PiJ0cn;^M0R|+4ipnANTYKf0g7wX#{0e+5L
z?=7c&y*;t|23EmJ9Hd)MDL6#&<S6a9VSIi^pX|n)ST$CbPR23G9VC=cgH68lH1f^c
ze_1yojC(>h)?ex!FW<6y8@VlW@=v0uCv$_BT$>{jtz((LQ%+2jv8J~N;5Ejdz?XpI
z(N<IRbk6ri{uG!nBxGw2mJXj<+$<6I>a<grwW+_#&ABBUrke1FKkY&B^^G1s2oiiz
zf&;`%C^7w~PTC{YDh42)ri{Mdp%t>cKlHBp3#BMfV*WaSDc37TRKG`Z%ltBD+9`6N
z@EEC%|GQ5-@7_npIAC!i+@BZ8fHzSzbMF%k`jnTv81Ku}h<?tv!e9rsIEd}9TSoHX
z`>l5@`|jcEs9Ol^&5>(CS91Ipb9a?Z<+x1hrM(Z4LO;r!Z~LuSA4tfr{bour1qDW;
z^IS^rr~8D$VOwuWco~f>B;V3TEfk&x7gU}*8krF9rBv((J6F)pbhI5Kwd_l`q`JIe
zNv+?`_SShHKV8;04XD-7X(M6UkK0&2BOrTR^q5RGsYw&$^dQY`D={A_>k<H#X*4~0
zLl-R-+YEV$V$Yh7AY!XK#5^_?flxS|Bi)nSQt4cP5W~BsyI=Wz(Uk+rnc7Mo=%-d`
zd%N7{I>2D#A~}D8Plt9w-yRmEtGAE5=d1SDVV&5gMLg#f%9$zjCl~%KFnYB4ZuX*_
z#t>eI2B>^XpbC$rn=+Jes^q9cDfCS6JylLxbI@>5`-HJ0n=JKMs)8^MR9{P_5*N=O
z(O-&}4oJv2Xe86~b?1V5n$OCPUcOcBJQl1SYgJw7hFbaqUJicvsXJr{?5-_sLU1jm
z|Ebv+x4*lmbS@oyUOT_1P@gRg0g`NEPfo>Je6O5#GypUh9xhDf9DEA)$p4j#AR|I4
zoS*c}*m=>f&%yYu;<TMTgU9|=0bRcURa1{Zn~{WuofMDf5eW+@=wpis8aphIbL@k1
z%WAYF8R>6Ahlg9|mbPP9VcBB7NVc9F4k`<3HD?EP(3fJJ8xf{gd6ZH0*CHQvAliIz
zx{mM!p|PPa11IVe>BwJjQ0CRwp?KCVKgvU&AJ@&;H!-5FyQ2Ow!jugcvqq6cFIO{%
ziPLzTcK&MP;3Uvbo)dimua3ELY|q4%sK>I}_D#Oj2*EyBrgivE%H#I>e2a<u+5>$P
zhc2R|>@SNgAT?YK-@apTSO3VL9>04pz+By#Mr@8(=h@Yo5Wnud+!J;Yvdt#{?Ae+4
zkm~04`fVm=yOYQ1-k0mhpO#Bq?z83!gWJJpgQxUYvvjdfo&%FO@-5UW0tkYsYM0-L
zmwW_09$K%TmKuVskTxLlRWE5NlJ@a)NfXY982W15bknukyo_I)**2Z#EO#_euVi&r
z4KHw?&QnWtgyQR?A%86fo1G4KcE6&?LTy>Osb9$k)W4gm<<zjO%v6|Lwh#(gX=}nK
z7p+O%ZFV36EGwxj{H6!g(ChgEb{t8%+Mlj%8sZ_Cz_T*sT;xFo8?`~L+A2ICMJmf@
z@AJ@^(<75IB-w1JpJ>NULo2bnm?^<;yx3wWfQ@L$j(511_lN{HsU&t}Riw6MZu`fB
z>(lqAJA{?L73G*oT3PRy>ejjoIusP#{Zj8B0GLvpuY8X@0!4y?enkR*1S~F6CSp)*
zmnR54G|!w)xU$fafY73Uc-qUlAKz{RRF1X72d@mHsQKh?j*eRGkrI+wyKrb;^G_hi
zhy(M8^rLgVYF$-J8Agu9JpEqM!nAI}E0SaPV_D0U1@Coq?CxqGtY*d`TL~%n@BrWO
z%++-bN227cUF0(>Rt*^myJ3_fgV`vKBNOZ<cH@Ren*B`U?(+IUYU<Pb-2AcyWn0}U
zuN$-jw+k%inhb?c0?+BB(P!5qQRmpz!8j)Mv2h2fo36(1Dcs+Rw>1hED^yikgl)<&
zrlzLg<R+WL!`Wt6$Ze0nQYr0W?+BE1$UassN^kgU4?4@7b?voZI<rbxZ-wX85#ec5
zC@XdkdiIJw9T6xQEy-JFJ7l2E=aWg3o_Z7$Ah!Q|O!)dE=}0ELUVd%(B%ya3eY+v{
z*N*i4^g_Fw8;AH9;Am~c=VcE?V}Yn{zRwes)hOb1JVVrRj*9Hn9oy2)`Fy$ap5p`^
z9i95kiYu$u!P|x<tw)^7qw>!)h7Y$ejfxMqk<^qt&U=##sl1+Yjb>xs8WkF)g|$=k
zPoGxEbtFWqf9{yVF3ux=ZMh+=ti))%b9W-X`q*+3gMbP}q|9z)$itHn9@Ans0oqBu
z(`gW`|6u=B4@8nKsdgXj+DGb2&u!sa+*SK+u|e_hRh<KL(B1?I3*>SBhx=1}GsOge
z(y-Pz<O&KxoFfBoVnJ&Y*x32QzW$6X{Av-p*3?7YW-SZD&(72Ud@xnAlS({_XQ>v3
zyAd{0K+>!qgFyF6C4o0s*W2ya+bG}0yXlMjZ^35z`d=Otv&-w`9@^=H;$^Iehz;12
zCJiieoIhjh?1`4l0<`N^zr!7xji0#qD>s{!eTo6)0=bs<l7YmdD}z$Jq8Lt`SE=iQ
z;)_1l7cDw|8Z&cq!I&c%>c=|S?rS~xWA=y1T3nLw=q`Eh=unh{bF*q{wy6z`RN_*O
zCl;(=j-IcK>hkn#9snTG4-OYElDSSJCWr*D&eqv1*lo|(?uC1^cZVnL;JL1Ld=IMu
zp(PjNVP)w}(^=k7G}%3~PxOf7K`HG|<WFDqlF72*&#z@C)jzqPd=!roWhxz-80-hk
z%CDlq(Pqn2FX{<+1*X6J-YM+Vn9qVaW*nqHmV+roG9AqziI~)F2DK3wRjJW`$!v=L
z=ErF;h1gIY_p*o8yhCs$lCsyQETjq1wo|<)D~%4NcUR_57b8!ltXN|pifBcl=DE-r
z?`yPSx@+)HKw=QRVhoWO3trTu`|+}RHG2bfP{(NVaa*X~{d$7=H>;T$IT?7NFv#W;
zAt+gT9lhb2Qjl~hgq>nG+>{nMfu3^!F#%Cg&HC$H-|l6$kNyWHKcL_1*TF0{l;4!7
zu_TMNlmeTYruL|}a*z_b>#`$u(OAX~K17d5JaaQm+shGH*Y1SoQ|#=5gXjq@;Kj`a
zW=00#KgFtr#ae!1=%G`KK#~7V%-zC$yEX(8sD>NrkKDayUS_f?=WNT3V4{pESk_Ay
zg5vO{H~%i%)h6U(3$oIi^6f!Oj}nbe%Zyd8jYKSPp5Und?$+++%%pb8c@$oBdKu}p
zc7Chm<<*Y$2VmVPDjE!5oQD33_#2|4s}=!eiqENX_=xwORY`&XOLRFUrQUXZzIMOi
ze!ygLXFU2l-C16S-X(+1wKr9^Ih;M6EefaYm|O=syR7f?eLi*0a|*ipTVe?=K}nfC
zAU#Cyz~Rg@KYP$U^DHF2yJ}YxLb0V^qL~L7I~fu*dA0Hl&R6Epe8Xl!=xvae_7Aa|
zL5H&a;v9Y$`=7&v|DVHYetNe(tJwEd@=3i{#Bka%O(;CTeB;>k8_vo$qiZYwn;Ooc
zzPS_m_g~MyVt)NMC;Yz&=oLtRbG*miRQ0jxWl&Kj6Fz!%z5++kbN#o}|Ba^pe`9<H
z*Vr!cMNmpS-<#v>n>TNE;52yAIe++o{7<8&c_x29cgM5HAFtN>Nr~q>W?TO+a{GU?
z#s6RE>q)gQ^NUTGe*F*md@yWo!A~ms7u~;X^ugCIQSkoxyJg23+xS{|e}9?xJ`|QW
zpsCREx@2z;M)R+1@*6n0-bFJ5^ZYtOqQZJt32fm|4r&dVxoj<B&5L}^0$@+Npvm07
z<pkTxOr6ES3EK~6eYdINhzX30319G*wv7qP@L6|8{@x@YE1-7_fZ4;n9hjPua*+0;
zHXA9Q7w-2jU-YEY;XLXnTi(V-dKQMU1P+r>CRk&4IC26aGzyUPugz%bOwnoA;Eu4O
zu2NV}dbsf%|K#Wq;a17xJ;57#7DB5|T-^C+23_?0<S7iJPMSdKA-jM2*96tS5qqfk
zHyhKHs2}s)+VN3G9v<esA~LsBofn_Utp)-zPP{6saIF**>-D{E%^a!S-6v1m-QMS6
z6Z>Z!IWbHe;qhWfuNMy9-X?d$*_CU0K9`@ay}ZG)x3ai!ctl^?%#Pth!aBTA>IaxU
z<z}rKLTY{>cze+9TmELH!9@B?dEdl{09r~<I2CJakZanUI~ly|tED;=f}3a5evPhR
zLiE~a*ke1a|N1bbJG<i8LF#ZEWqso`cu^S<|4gOzHrFQ5=OUc!h^>c_u((%kKL);p
zcl4lX<J3^Rk9r2VnYp<JxU{@ji5v&EjM8Th$M@P|3A?(w2o!0t%+XBPaDUCX5({uQ
zchMT$FcLeQolUw^OvMA)(JI%EoHeux7cUw>b~j!-%5+pArGXVH@Ktw@-A!IM;1jyP
z47>E0TRjF|4yH|k{F=kcZ~mWOECd8$H+?4x7ASJMyAv~V;k%36SXh5#9Uetu-{|#N
z^l>%_=-M>dW3VJvHn>rQon%)_H2=hj24u)3396N~;J~A$9085xX8<#^W-%aeP$0Rv
z8l_9qft-t@5Gh_<TzpdEc~W3=cThVy^D3%ab=}{b!#Gc27Z<B(EB9}!NU9A~=0%Z{
zx_?$KxPMKH+{&NvUl*h0j8x`p>On$~nG#3Qy}QfUm?@Ck$mJvLe8gitE4I<Ftj(-z
zAx2?sx+tsqN!>VFnmW|kRbXaqCq>`B{qX=~CZ?C-a9F_8|0(T`7QQe_5gi47mi}UH
zM~^vOg~S$ik-d~t|ENf<Ettj*F4b~>YIqT?A*7;#I^^e0aOtu(&Gz@Ek0iWL1FnBv
z`+jyf5CB^kW*?|{(_h{_G)x`#^q;(5)TxB03kllVZ2UZlr>Q_*c%|#H*2pg8i?%F)
zsNln@IDCXSCbG_oF$)jG%NJPlsllhFbbr80+Rm|Cta4X5k`^oz(YYdcQSelYMXYJj
za%P*obRYY`7Mu>BHS{serinDDy7dxzAHP!%hITJFyOs-2fQhFp&ReZ2Y^B_i{c?>@
zG`eoq&UdR7gGl7|kCrmiLnZNaNgX2gt>f5sgx-ZXIJ?>g-GT~h^x#e;n;DyXz_k$0
zkms+pqRS1U?HA4lE*=SZj=58vevxPYt;V!YS~wF2!mpWfUCtW`r$g}tlBbU)7^2hO
z*jDkJ%wy_x-UM0Y&<=P%D$c%p^vZN%YCHW+t3mp0_0BVS$~nu>7Qb?Nt^s3#>zhbf
zx-rOQ&VF;ppvAMZhy^9js#ztHN^%q6VYg3fU8y-`qUq8J<@urR0~G<M?h_VSFMM(<
zmnxx(KJOp8KK)+#5ISRdcnL6{hcnk(bvd!_#=x_rP@btX`J*q&;WVZ9Q8}!P+cY#S
z$H99PaNDYW;mRDXLUb;X_C1QUn?IV$2xi$m+fr?4??s$DR!86mvoS6tbZkDomY|#E
zQp%6yG-uYl=1diwz;euO(dTsn`L7uM=sf=3EdN=Ev$CNbp<p3g1CrW4vXI4(0_C*{
z_zY|Ot7*bMEpam?D5z>KfH%?4cBKs?rU((QT2l{acx*o8liA2FaMui8Bi_zF@4Itn
zM6+@a6vPh<3usyhhw7hYLPxJfmmM}9W>w7;S%wu4PzO4x@maclz*OlkW^=Yom13+`
zzWF1q!G4rqZE_iGn^|I=@QyBLExs8r)hNSbF4Otxa7@ARGweD(;&9xa#19YT`K|;I
zsO4LbgFqJf5@@qu;q6Ru&(?*#;UUfl=UGt>K#nG|X|?K^U<`e$r<rAC7zt@wa@3q3
zE*B?evs>`Q8w}vVa}XKfv?UYspndsp<xi;m_$za1cPK%)?N7CW(%5<}1BH08Inm2B
zNdfr4j-hx(wSQdRHCRlep{4o*Ery(1OGpIh3DKnlRYCnlea(_=81S~7yS&#a=RG{_
zOC*3ThiXTyKXCeOI8^(Ha>*RKNZGbwIjxQF(|P*nqXSr3^j;g;f4onSUu&s~T)nwL
zV=~au2<tIL;E%Y1ZF|IRH;HGY9IGEO+$(97$8?bV?cK5Ac&v^b{xlcY2RgbIv$X4X
zUK=yV#uqd3*63`(YJBoO7C67{Hu_7B184^1I6I8vm<zeC?KbhBd<A~8WZh)5r}6p4
zr)oV@?y7n50d$Z|R@sHTeV$3*Aw9vMyg`Sdeb$YP72Z-C$5W0=@*1k+BYC6Nc{Ik#
z7s?su#Z9fA!f8!HGfD9DvR=M;zZVVx$IVSxwq~1*tvv7>W|SO@=-6;N92(_p#(nd7
zg9!jk4jxbKSqbLFkzXJTx3vZQk43k)jh4MoyS#<&2N8=9<Ify4Vi)pKj!tv(X|x3;
z#Rc(HW3zZ5<mqoQy(pmA7|AkIm1vfk+vVcWM^V3e7`9+UG#kGPTSx4S4@QC@D&&qU
zOv7?oi?PS!t)PucvlEE!`#(z#DpVg0o2ol4Vq9A4nado_O!v6PKSe$8qc7u*sO|CK
z)BHu*w{=gwg_kJ{-~V3msI*RV{dJrwLkjC1pFY+EdA*1ynoQVbqLOiG)`%{G7~auN
zGIE2ez+Uw=Bm=zhxB_WJ_4=Wz3ufjCyuwS{ia*b8HOFddh4s*5wE%lLGfR1b$iZjZ
zX&cQGJ<;VJaEC_;y>2zHu3wK9OcV4ZDIn3!%*Eb6(qd;3Qh<*r2WlbEkvFt@9C1O%
zO3{p~o0)h>g|uXT>99>)<^4s@;lW))?C*Xz%Eb4C4QDs2%9l`APa~H~bm=~4xZfRg
zfipA$Ht6K&GwO}VXW<;~Ar8nw(UxWir&wKaa(_>?nW0?prd|lOW*I$(9A(ftUkJYU
znY6ecu*$lx=loVAh@^-RpY&D)H)e??-gqD9C4SVm-TqDjs%w)E47c-5$=7%ME7?e0
zGI+7O+<tNiF7N98=!qKbZhm_^p^AxxrAT?7u2$pB^a;sSBYeKe4I6NLe64_Sa*d9r
zU#{uzd{x%Nm%@<uLb@q(>cbiNQlscl<Ha&>%}q@6)7cdbR#*rH&J29xB7F)qsXh@2
z$uCcGuS{0qH2nSin4P0K?+H9y0&L>E99!7~#1;7T1VA2FC9an-pO9B>I)``;qA4`F
zH|FU2w0eF!XO|>TQE#=_1#<5UB(4gIq2~eg?(o{q@Abk{+OB3kPpYYA@m(EvmAJrx
zqWP16s-^l2FtR*n?A{11iw+DP5z-Vx<ZH4&X0m<H4FAjA9cz{M;n6;oZvpL_&&#lU
zReNn58cLl$ON}aV;IE}eiiq@9E(5C^nZfSAI|EM&B{AR}KY7Zy7hXr2kU~tbB%aR9
z?C|Y{5!1b4FVk&)dyckcbhvc2XEhEybb-bAjQPpbk-8Os=f?f?eZp1(m^wf<?j(Kq
zt8V)r29SO5i>UXtk>K|$-Y#ZJfT;1mZJv#Wj{afD`+_DhnSJieyB$Aam`oy6{e}cs
z0aIk&<xv|0l-o*uCj#*;MR_pgI!SH+qhAaf+d$PbG*BFqn8DJyik<nOq^h9c+v`kh
z=#qc~NI2UbnZS#sB*b9fy$P`X+!iGMX!-sZqSbGHd9@vj&~cHLw=591_9lIyYc!YR
zIbu(~(XjF$J$}d3KeG?NfR-gS`oP>i@)fT6I2Og-;-5i5k`>HF0X=MwpF|P0i-EC8
z-k2Z?;Y8yh33kyrF2y9lX_0C|WT%`mfnijcD3x+}<}+r^R`epquC`1zEgxuXCq=a}
z*!LFk)yu{DLuG@Q%q=yv!s=uh<fK#-`|d2d7m-t{<@sD5Cq<Pmp!zav{5O|DN0xO{
zA1P%BdA>!VFdPZue292hgj^Ptjq6wHSEVifZNoqIfpLYc=eniMj_>!ZZ=JT01yB^Z
z56t*}H3!4}QUF2RAIpWHKYMj?mM;p2>yT?{^SoU9x*H?d-AeZ=HPK>u5sIy>dIu%s
zfY`vSzd~Xm&#E)C?Y@|Bf)^$&8&4=e+<Y+T&TyKS&!C^owm0lY@)r+~I<G)d?Q20!
z8%#_uMCtw0iwGjHg$u8%g4LACQVRm6s1p{j&t1Dr4W7Q(i8lXTBJ!a=<KR4+=k?&<
zbAT6-ro~RR)y0lJCwUCk3FK_~!)sBM8$c&gc$6zJTo*aI1w%lhVM~$OGZqozKP=+`
z_r7!$t)QPf%j(`S7j)&m->U;EXxAdrBGlMu^LrG{=C5mr>C)!ta?Kkv{`T@^nA(3%
z0Es2xOjHSPg!HkkKPi!($#vpR{={&~OPv;)$8iUI7c~)QA+Ay&KjlW3veCd04u#)q
zOu?)79GWC<hd&{!Ua=|ZU%KSHjz1EMQl5;mcf$M*96FC+uLflm%BIk^qjFAcml277
z?PK)A+-`3q`pPh;8unA%T6&+85gtB2ou>K}Yp~zzX#Sa8B%|KpW^vwcQoP6DiUqQ8
zPk|iNSlb@%^ts%K<!z_AHCXbC8Nma=Rg$Zg9hPkex2yxz#zjtg)yt9y_5yOV6V4^d
z2#GS_WM&Hs*I}Zz?h}@a9>M2wjl0Ll2g$<p(FY4EOtLR@gV<X5b9s?)w_TM2xr)E#
zVPne_#Kyuhi7B<}<ly0XaTn-F(U^g4)riYbN)w}`Ue%cwkS0`0b01+|7VBuh>V0;r
zSW`$ftf^>KEgX=d1!s9a)d>ADf{lEDGyXyxmTezebUcbSgN}lpYDTi^A{cAuYd*;J
z)2L+l>|&7AHLv<1<qZn<c$eXEr(Wi`0jOWoHdS(xtL`a9$d>?1<Pn2)!KcuQ)#!It
zE5Xw_@e?><%?AmSnTpx_|H(;BL*TmN8Z!X=%M1V{_#+Y+EV7e+4Ca4s!;{ci6qT?o
z2*sZJSG)2VXE3QQ{oP63kfy_9zo|bCNTB@Kz~LXOsgE6L%Uv^U0~dU5u^3fca8Uoq
zfx}a!v|Q1VuyfxOpY5NVJexUs>Sey$rY{NC*aeQxD%ou9`*19(epC~T&r_Q#mcI`D
z@h@usxz67gLcuY~(2vN|k_rpRkNkjwBf}X#hP8Zfr%RF`36DR6%vto`{&x;ZH%Kq#
z(7BWrN?HA~s>d&H(6|!!*K2N2MD_Jyi6<OvM^1P?%a&+-LMSXOtne9QnG5(2u3`G(
z9|(iX%JF};+K)~C|Jow|{{njauNMb!+AOi`pU0YRQ4%^sI~+>Fu&ESOGC_9z4FVrJ
z7f?Vhz=lawH(TH>-m{fB)Nee#6-6Q-AnZ}W*H<nBUj_s`{Gn^7*0A@}-7JSD2>~m*
zzdK<mw`k8DknH&U-hgcD-WSS5P|1$380=i7e@(?p54_V(y+(A~7J2*1OYC`QY>_-8
zH}okcvc9{gq>=h_<9WB^o1jLI_mD$RWj$$#pVunp5o+Pmni56RpF_bYgs03lzrW8!
z{P)c0WBB4h`R?@&Z~$i^gpXag1|%(EC#7K<bsDRbP#E(p4R7?s#;W*@D0Sbu=6jLB
z47}UJdL7Mom^;8-*o`<Lv3dYX`#e9kOkh>=>}RqSU&I+3{UY9b6CQ8PUl5_q4?fRU
zNbdv%o2Z&5Vjw$f#y_06x~?jPqIg@}LsXvAP8o75_`b-hJ(@x;3n)^?N+?$|f~}9s
zgJIu4@W{V=b~l`Pn=A6Ouo@d1|AUc~)vxr9j#jkMf<;A+C5~gUJlfBLfxp_@GZ6YP
zP@UhiZ$WqEMc;MiWsnVZqGU)GGQX4wyu}IWp}{86!S=X=y?^ec5Z^3pD|L0lwG+0M
z*4C4|hZ7*WLd$4a2#xsl0(=Y0V&Z?&@a+2~tAtK#f*<Y8_yXB2<kB~RYzaw<o9$7m
z!ueUkfC>&c{pmk028|5~L3B!0b<Ze!`EHe`T-RnIVSN7-M#;u8eJfe>ybXi`+LrlE
z;aA`Aj&Q!dT~UcdBd-!yyPJ+kEun_-WIBGm$QI?QwWkBO*!OV{uf6lgwXE^CsV(CL
zkTjx+#&7U}(kL>l7Zkl#!gJVwT;EqB%U#}y^a&52H2_Q3=`!r0F#aES#}b;>FzH2j
z*m;)D0y=$>2c>0Zj$J)E=(PLs%6zx?`85=9*Rk*KeEVDT;^JcG|1|fNQCW8Dx*{bl
zpoAbuN|$t_AR^rz3Mh><NT-B!H@rx9cS{S1bazO1gJ-^=Ykh0&{hhP-i80Rn;cyI*
zr{}z{`-&-!jSXa3u{sP9#Xla+u=UA3n8C`<Rc`4ITAs6gCI00*U|_naTCuRQu!Q+3
zWwlvol+0Uj;uW>y)PA2AX$Pq9xi8Ou#PA0r@~ir!e#py>3Hft2^@}xqen+O$-4gEj
zN>?2`(Bd~Q4}c+U>mWi$Y3;Zes`zYecO%-v*!%5gEZ2JScl1$?9{PWOB40KqH+~mi
zTdBe?sxZ5P*H}&ubZ7^x_s^L5S+PY+mItZhgCfCOixn2B`sHyx@%+bCG-qmdVAtu5
zh)f^w6adN?&~x`vxp)GOimc1rN+-5k!*}kFL@ujEK2uO@g%-CxnI#p!qv>fQE@yBS
z(X5NM`l&y&|0RhfXMU}n=SK!oRePIGu7Xm<mM6w1Zy*}ei3AXST$-($(l<bFn94Gh
zrn_fvs30+fG)@??tNYXSO#?%BL2yX#s!|$Whu+`$3{tLy%q9zD-cwP#0>7*lICY@L
zD-HR>ZNok?#>EcAg!8b@GX?+h$;uQ%E40@~8&7LBiioE_P2;wA!)GriPq}A25?47R
z6V?7&i~GgvgijWVdL!!GOaxQ)k=B(W?@SZVmrdsiU~B$Z?#=z+8i-UfTZmwPuA@-F
z=-%}UGf1k)^b7d-LZU8fv3#zLIh`|fHE?TE{;kVUDkRqjOk{u4X5jQEym@<G?>}BY
z-kM~JeXYYBf5bLVn4iySb({eDgBT-SodM5MG+l^f8u3m$sZ$vj_(_1>B_tY_&@7R&
zNkO6Pjz}1>**L0rs^yENl2*`1Aa~$3Fg9-0&y2F>c`~~Ab2X_wiaWq^a?xixb&AeG
zPbD|MXsDodozPLm!>s)B;I)G|$>;yH3lM>aiKpsn_gG1ZzWGNEi$ct06=-d#O*kH5
z&4<wwli%g-F%d6jDkM0@#vK@$3{dA&TQIHc|1>}E=lUcrM;A>lxOyHv=HPr_W(rw4
zLHB`J3vM){S8)~Ibx-C8t#Q%aODFHhA_ltlx1}&d9T(?T_1#vQY5lz?v?sDI)byib
zI$_{b?S7|9po3MDz>sHZA}A$gf9ItWRpZ`<9n1Abf0W`LtVGLq-!DOMxcE9WaTs~|
zj|6HozJv<Cg~{MwRb<t!gFEryhT{Lff_P{)MP~5~O*VbN_4ewg;njG?r1oo1+ok*l
z6r1wl&hq4zW&W*~Sxy(a_6CNjN-l*;+6UD6mnp;_j*lqHc7&k~DahB(v$Gfu?Uo{L
zy2gDWn#d+OEp%~`4}*LzLuGnBh+Ul>x4yjA!FhMaFB%Y`@N8}+A+*D(VP7LpIP(n(
zC*I~Y`{q0HTEEyd&dKYxCz;-6s~TrN*x))XlLgJ6#NP{!Bw=+-^AP3&q)ZNLBMjfr
ztmGr5uXfu8DXyt?wZX>=>_+NXnME9v;@2Tt7cDj4khsFIf#t%D{C|PTU!7eZIu5)u
zZ=-PLc0T5m;;IuAqZpYk*VAxG0T;0R%B5=Rs{6_vsiJlS(r4nU${q!-puBd{IWUW6
zO>}bn#G-3)G+E5;-a8S50w;c(CGJMYODdaCaOz+pDbZ8{K=1~Y|I+2S4^VkLGM~k^
z`rAMpTx08M6*CG8bJoyu8@3RA+#tfbltRA`4e;~}?>hB_VqN5ytgDAbCs!CD%-&v$
zHyFwna^R>I{wPjhVE;M@WB5*RW6GVCI5ri#S+%^<SUKdBFBd9kq`w9d#mf@JcM@O3
z3se)YT{>sae$Zdw26G*#Ubjh~e*O%{#LDXRuqQgZ=Ti`&q=8qp_5JB7NZ>vWZ56pC
z!?fb#r0{I$;?ixYDRQ`N`Lu8;r8oXmQf{E+$cUr7p<)oTI^y*}mHy)>9Gtg5&o7F%
zuIU#uw$zQ^U^ad~o3L14dU!~EP#?|nT~o^&koj5i6_aYp0sHRgtQ(rM&)X&{LnNT{
z4J%hk^XIlU1k@9QYIn*ogR?h{xQUNP=k`yTU1h2LapPp)z)U6X)#kN@7wEFlsd^sK
z;?Ptw5-(J>#t-em)pn+KWZT-HJUrn_Q{5uv7#e(dtdTS7r@3@f-Z%q7=F<BeTWVR%
z*GmP%pF{7Te??&cAqmiBK{!^rj6%LAuxq-kZLw@}0PB6pWckQTj|p$ne(}@3fQ#=7
zmO>?JvKM2@dt2}AD2K7AZyE!#Cf?rgA^HO63#h%sG=Ih1n^OM~bGN^nrGEVzbC;4f
zk@v8mKjzPodh3#7e@^bSiQMS5J?X~rMOLb8U;zRjTgQ)N##e0nUB9wKDx2}Kuv^FS
z!_jW|i)Q5_$NrhILHBpoBA4A!ize;cxLPfCF1svR<pFm>(SijUOv`9BAvb__Kto?u
z4k_Tx<!Z*g3WaE@XIDg3&pnvBqJs5&OH3hv5dgAL)U}uDjtST?q}dwK#`*Y@joME5
z=^7jH#EiIHM3u$|_u6&+1Dmjx5!M!LkVYVTKw)>uo8nSDqG|R4&;FzC9a=hF4oVjC
z%XMM&k56CSC8E~iV9PE;KcOpcnX~)s$RHaw^ogMN@w%_oFpj_khNhRiz?uC;(1SuE
zuy*?Qs>pttSXL}j`K+Q~;xi19P_#=a$iZ{Cnl4JN{+txjKcv_52`jBckv;Fli)u`f
zHiV|4`{IBHo`}G()^7GD*fUpPaR2~3KTp^q$*TqOmJUllO%g$1iuq-@xH4|FLZSTq
z+D>TM9RH9$CS&kO9V<v1y^&h|P2}G4#a(8=Z8S$oxaLJ@W}9HIf7MUljz4c9SGj@P
ztJk_lSUUi5FJ$}3Id9>yNe>nz+6(TKWH4|Hi5hC^DVmKPj#H7I@Qp(odj`DkCq)eo
z&fPt*Vk9YTSW=0Tj+GMl7u|SBRU%BzWZqh{Cf5?U*i2Gn<*VHdhnc9LC#qL4k+;YT
z({qz$Brs)pNQ0R0CR^*~NeCKE)bQ=A3s6vtR*z!Bq%UI%LAtOs7loxVn#gldZo#gX
z+%m$qpCzpuY%m4RW6n)n+YSSoeapKoxp^G1M!?8A^4Q_0NZm%nP}6tMerI^~h$A&k
z2m-c|soI_F<s;%ZC*H*b^|zom2l_xD)t#fc<?Y0H-$gya%-&5}nh`K%jb-s9^s{3Z
zY04KFbq-b1s<I1VbRWub7Pa+S|HWzjtjbtfH3RoHxqBx+2h-BZv->A&LzVS>noWeV
zMIwru46Iw1IRkDKRGj5H!E5a+1XM4|RkM3?BC(JBo(!0~>~YW36DI9E8EIgwNe~-;
z0i6?Yivyrl!LtgT*JJ}>KT{F#36y+MUw3IL6}S0GyV)^F6f6TI4t|8=7SRE~gFwd+
zwfFO*YHwSFZw~`fV|`4xT;B~<+-(L$X|n9PZq%ONL|E3lQ&-;6Fh7?(rB`F0CJLF`
z8mwv6T!-VWjCBQ%($}Cs7aE#Yg8w@f?b#jZUm|)f4*}J8J0N0z#dLqivj1(a+J6@`
z{+AALOND2u_Gyr(6%eYJMasT~^GA;0xT7}(y+&AuS_%#5MN&AUKrbSWreQ{XeX*&@
z?Hr`yMMwf^-(QfW*81%&^xWXM#W)cbf9|0nPEV-Yr$>4wDkQn|kzphioO?*Secps{
zr|96VA?@b8#fQvItYX1C_He-BHylm5+C7{LghBC8E_?~r^@FtF7J4-DfT`JuAK@wD
z??YhzK1A4#7`OE))m?YDh6r8|Ge(9j+>>q%itc3`d}(SMKkh#{e2}hut%}Rqb6$$(
zos+bpqWjDG>F04vPWGwzxQv=V0A8{RWO)?jg%}MGrlpmegTuK$o)auq!E9JjXI7z%
zuh(3=gP<{P8Z`fo^%*=WWuBRe_r!9KjAl8UBF-~^pO16L7#kGt8d8PwmqGfL@9n(;
zN-6jq*IgI{#7~FmU;RwKR>VHR`e?2~M5J;4m16$=&hp&*4$BX<+S>Tu_ZY4es1K~t
zH#@r7Q@2$MhU#Bwr{GhAqBQ$`qo29!20pJ$cTyoZz8xHDWR>IAW2T39hzgYxFR9$B
z-=U4rGq0tCmy#+RPv^CJ|7yox_|v1`$K2K7dm^UW3_oD`Gn#lw#{h0^J@PF=tVeTt
z0a2FzuNUJh{w1a%uduFT&zEZ95I}>mso?$Yk!iT`&q)6pCLB{T(9Pue)x>MO?`Ri}
zs~@^%a-^R2Cy2&t`@HF*Q6M;NMFAg922j#KKvD=KGgB0bt!OK{;H*+VM~>`12}!@l
z^l%-;-#{qiH>iUQ;l5td#X6!$dwT?BK#$V1*%B|3(7EKE**5N0-q>1erEK4mlR~&!
z-ukG9&6(OX<?|wdI!nWF$A=t0f|SJh7nB4b16LfnM&`OS=6L?&tMItQQEp-W_xI27
zR(npvC1E1yhm@(FdrAy=i0h;1IjN3kWC?0xLMjp(?|`Xc;G3)wwL6{TyyXV%^J>1`
zP8L+}O-dkmi7-TEN_S5?BGcHZOJ!`M?Dj2*En#^%lV~rL0kyK`IfK?erewLBiJR=#
zpmXQ7K_!vhH0iPlc08SswAohQOGzjRy<&vr*OI%V;hAy4o-@d;zPx7p2gr5n;4HVV
zk2x_O2!ivNdICB=eZE4E6e6M*vo(20yjgqEDQG#K)ShgLtOokQ%7!1!@;`Jdr#kMz
z&o7tD0y;F0)e5g)XUW(d6g?Guo~v;XJ83H-2YdnkxuNK<y>F;T%w|0@dfBLk3zI<`
z)y0|^^e<^~zeKrOBE}ify;%+>bjIA#3JIIA;{`Txzo#a0Q>J&(c=qZ$Oh9eCd&Sp#
zKC#qCEmQMw*8UxKUJ%Fbw08B7m1E00#hGWeb;qsB0GPNytZ;<Gb~2M)AFy-2tVCh^
zD8arQllSp)2@1m}SW$GMrZZnNo6sXJg333OE0S4gW*a567Xo>k0>v9*;69rfU8i4c
zyJi1+B$fQr&2{R1GEC35k+7nBXa9p4Z-!Um_qCAz$wv44J{L#&hJaLZOGs<$di*A&
zOZO9m92xW)VGd}f3M0a=T<oWb4ux_%pbYnxR=EW4?KI}b9tTzoS@u6OLi*g_gUOy@
zTu5g>>^d$iF&D+5$W$e;9boNSy0zwGBgXjq<D|UWZ>5iaL#<%er(rwB6G6ZpjA&|V
zTD3Xk@?OCcub(fF-UeqG=Nnq>i)xF5&LSoCH`$t>hPiAL>R68Bzu+9obgLu3vF0h`
z)=#e+M)oYB>!z&P@tgW^BaP2%cPUG#^Jt*rNH?zF_u&qJy9li}7=dN&N($GgLM0ec
zPW`ScYIikg4@6;c%R)V=0aflkNd2lIvi|KCrbTO$TrYx@(cWGyqfgZ<a8c^+#_)#p
zDpHjzq}ggd?yi*%O!B<1mi)95iAO*_z^&ii*5=nc{a-kAihY?=j#MU~TLQ@@ZFT!W
zgN<4*<+Ok=;O|F{M%Gw`h4Vsyw|h03vHN{>LgW2_9^4ic4b)%D@|?3X0M=?%nQM3!
z7GaHex2c%0H!a$0yA~)r6c`^?pSdhQ8)kGtWzqR7A>OfWap8nVlyq8ZX&_T8)T6)+
z4~{GW#C?H(g|SoQbvH29c<2v|^(x(3gq09Ea4%Pc_TOmV;vtda=&W05i5#?ve_+gN
zogVi7km&GO!1>}!jTv2)f<I!i<D-pyHQLn1i?cUJXDh%?`77nI!yjHc^Fy_CEj8`z
zH6D?gT2|z2?(nG7Yg&Od!~|z&q%9uIeNzE{<-!-`ZEn{gy7#bz;u&2skYAoKZ#t`^
z2uiLbyo8k?eWJA7`ZM|(M!<RX9y(FO7X&{nuJ`3hcrcV~b4Sq_Z?j9L_Ro7xSL>T~
zYrMig1*f#B`#Y}Zys3z6CMeM?;?LC+DsTQo)_97z$@Rq)9Dkf4Q>nh;ot4?QJJ1)G
zG7ur=yx!nMm^gLRK#cA;!Tq!O!>ic80OGo}O?;&w3gQ5oZkYrQ7i~YF{=8joB?+lh
zyt2u@iWsJok5D}OF{O7Talg@jRD@w-&2r%I_^GXDL`XaR*xSgVZT(Qq@0R3GU0jD?
z#7ZEB#-0s0ROM<zqJ_2M&Zt<+{XBz{isD1#CU(I*J}LggY9}1gS6DS39HHTynNPZm
zyzOQK7GJuFA0^OM00oSt(?SfyhHJB?2v>k>K`yArCu2+heBy)cz(dA`D((~F7Xh7o
zZ?c7)4|~BnmN9G)lp%b2wOBLOyy_`^mUmEFznC2G<Ym59^+jaAJhq8lzDOdM?aWOf
z><g2@%Hy)&Wy#2Lj1m_Ec;M&Khs4_oPzx?wjRsxi!X}xj@rKb5q2NCp`tVFQ^t6GL
zvo9VLIa?rO1qCC>8W|1XS%y=o(F{-086|xoyXd$p5YCnPWZdQfV&4V|i5%a1%oKk|
z3IDzTbk4}d;)(Tk;%Lht+%!VvpdnQKXW-p6WVOIDSNujkrwhf?OQ$^iU#UOgUVGe)
zF&@#Gb~Y~Vd-pa4tXwPLnfx&M@ZsK$^R9aXoDCH66L)laV&6dPwnv%_N+HxE{9zP#
zS(U;&19jS>3Y}4)K{V}>z}EiQs3%?~@#17&u@=PQ_&kl+YPi~bW}lO);L3jGA@pFN
z=Dkf?gWPv@n&ttPI0?s+Rn}02J}!F+H{60<1ey7osn<y0WH(9G(6Sph9^zqs)4}U=
zdmgIIqPFn`bq|C>oL?<++EhH8xxR}hnOn47K!M5v7OK1sP(vp)e$S(n!2ygg4SnLd
zgs*)wMd<fQPHicg&LK;(D~|V4wU6m7C~Gy1o5`}KTaN3zcIv7y=ov7riy_Cwg$tAY
z+uu|IGOQ4Tv-2HIY;>n5D2=iszHtTa*L2iOc*g`;LCG8(XJ@ra)_PF~PUeaD{c)O?
z*6~LFB(C!qm|Bg7l0knpE*(#&`;Kaq#oDec2%VL08+<yit9&ZH-hgKRV!}(5fSy4u
zq-!I5F6fn!^LT!q{JJ<ICaZ-*TRB)2cOP!flyV$fr#NWLQFM%do@N9mdpWl}Fv17L
z${HsHWotEXY(xsBNEi{__r*$;N2z+u5aV8`!KZ(e!WP>~h$PstcL?jIdyORW`3Ww#
zMH0iKGcdvXss=g8<`iFo)|C_?P7IEzC+z*4o^RH-FTnf#?h!%^%aW0LJhx=|EJwhd
z4i&eQ3A#mb0@%*}Kq3ufWHc7P5^35=I{6B;Fp~Mo-k;93zEjU3Jh7%nOV5M@Qu?a3
zw_4vsTjt@Qunee5ZIZ=osuLPB8CG1axmz^HSSJh$hjFcb*A0$R;Wc1=Q{9tZ2nB2C
zTw%Dq`O_h&;zJ3dHWiBN)+e|9sS7T|Df!(+zk3F%rUb=jQpLYst3)Z#1RKj3%N<Wo
zyR!?pHjx1I9|)77x^hmB)WP^^wkfZu`*h#gJq93tosgfwktMW|rq2_r@`jw{wPWCN
zhjaP^ktn}2n4TLAB|Il}m02uEG|EjKSoS%mqPNW(h;Q3~Z1Jcz>y}Sc=)@;Y@u)3q
z==B?thV!k;H0y?$@PHz8G5992)wm>Q^+8-w0;u=Ym+aLnx(8BMcv%2n?R9N7)}I((
z(i9Yj(LIgkD&$x4>o<7bEvsCx?isQh06O6~BdAW<&v`ICFEnjR)X(k5zVDL(J--4A
z(x>t|Si|tID;Eg*P2|)&I5lnU>$}mCjfrhHUYvg7XRL{;Ce&gZFY8k9lw+Q1?6f)z
zV;63?R{<=P%0o*N{{bw=clsSC=Cy0ie#K_bcUcXM-dn^wBDa!CUCsv;)D_=6S8W*A
z`vIGspR2^)Xz0)~>ScH^3UhGwOF$7jNIb7Q_xQ@@P}k09rVfD8>!g1rm8aJAdm3~5
z2kA%_1WEBv!bAkc4oMWTC_Th$r9p8otQ|_8y+adZ7o9ru?xkTWJg^u|WlE=wXH)pd
zC0E*!6?UR}j(f-jQ0cl4toig<;50hhF!Fzg#SI%moE5~+>3yxKl%J3#4X5?nmXxf<
zU;uG`y75v9zhczr#dY#oFNsD_r|d)Uy&_r8vzVx8dp{3;jL7I?Co8^^pdSI*b0MJH
z1>!yt&KGx|PcU<R*HnNPUt_KY4VyE*<Lk}G8oS=f(%>??*DF){HeXEslxHqP1m!fv
zI|CUMy1>s8`|SY$U_UB`AEcvHW8oWmqT7lbDnwl!t!iSifIXLp5^h`#u$ZuL*Bg`@
z+833n7bYM;->F7R$%>g`p+vmJoN?t#gq2|imS!K#qkJTCSi5G2h5)i0aF~IJ8(0oS
z?pD6luU=|$4NXZ<yv#ys6zR#BgvF|JR<o-TPjp*7-bBlRJ3#i6om;>}D%4$m3`NA<
zzn4<x+st=X*c(;-IGv~~kb~cjxChYH_ICWA=Cy;%yB&X@zy9Ct6Oi7L%fj;A|JDN_
z|4(}Wknx^TM{#FDJUkg<Xu|{aD_EER$u7@)jMl_*ab!juHD9NNUh_MWKhR!So>KHK
zkx@wT8}Gadh1Wh~3tyi$@!(qCpW})8>klGgOZovOx#tAF$U@}FPhd8zz;w@MG)Hb?
zlGX+w;buI$3o8*}0a^Y*JcTV|MW1d|Ez&OkS`#Am`~Jk?|0ZyyBQTSVBL<4dd8>w2
zLPO$Vzx50%f1M);`EV8GCGyhQ`#CurM<1+OEuKE~ze)m9c-==aOS1tLnklHyoMw=P
zX!ff}>+K4`qoJDiz<<aCU?{t(Lcd=O@aVrHDNd2rz`DM=BIGoCm6aQ!ix3Nvo-;cv
zcblJL(TLxXPrdefYvBhc8tpaX;tb!*2~$nvx;v#@e2~h$Gy!zy#IPO!{NTg9pxD-`
zMMXLAncv#>p4~y*$4#lKBgC;E7!Q9e1d?~5?P<{E(jW3Xaw88c*TSNYb~zJ5_lda3
zD9}O@;ED2_l=Tp9o|97!wk%DLxWdqnr|qr~B7HjeejQi}7C`=@z0T;f)AI^B=iZ^%
zK4(C}_A0@Qd!3g@GNlk4pY+X#)hi}SC2<9_H=?$NDQ2;M@VXDJz(IG5rCozcH=T|5
zo*p%_x!(i&&Iceuhy`C?CF1<KpOXZnK~EuX#*3akw}G`h!)kVrMAQhho3C+Ze+`J+
zk9LiSgG(gAdG;*~&O8@(JA`tOD~woqqLxGqA^p`WZAWnPyYC^yQ0e|H18aI%1-a`i
z;yYn^O$y!v*62vJ*T7X!HFfSUV}`+7g8Cv=NLvWJ9<iYJrT?w_$69mpUjh2SCT<8C
zlmD1(BZl)t&OMhyKcycCu&9=YjtHTRhST)~MP`0Nk`J_;S0AxXSIQ=ijMIo#VlS_6
zPu)MNmO&_`;!x^XaU9LEf!;M+=n``2BZa^B8;DQRsu`!tfB;pvep94h2Lexn!lO(b
z5&GFgc9K^9#EARF56k6EH^58A8_VQylt*haj0JDEZlmZfdkp<my}dAvA}4ndlxZ=_
zvF9b}Un`mN=4r!83|fgZaYJLKs&@Po*Y8EjV|Wjt;es~$qnY$rUWh)A*2O$6XQ7F}
z02c~r*U4bN05Sn#+NAd*KFWAY38i}Uaro#~cQQnbIVa)Dn$Gitz{w)h1Ii(UN}KV>
zD`KVN<gp{rW!cPJ(XQD+k2x=#Ka52`?QkB!D*_Sc3J-lRCFV8PJj&qaBLzFyPU+v_
zT@qDYe5NY9v3OkYe5jfBJz}Fgt$W;UC6n}wwjaN2_4zRYQ(?>1Yp(H~PorBx#AP2P
zu7f`eZ7cSCUEHWo=SN31=xVr`OGHD{K06diz0@jbxG(Zo>Xzp3sN2%D8|qf@<8SKr
zxHj|ASNm;4lk8HeN-wjq$K8QnM19I9ki|hpc3<)0Qe^S$tMZgjax-%G(k4@fscDz<
zmGC0;W!VRurI&noaE~@}bl8s5mH`tbQZ1%cdc-`WN<J36DN;(5B6ggx)S^FD)PU?w
z#8b18+>TwL^MejU4PEmq8mokTpT}o8)){k8k=Z_c%Q~a`8;o{y+v(lR_?{h0?j{A#
zv(q1yZ>-DAI2#=jdVN3Vt(F`+i&cYjLNCj`;Gn&z)XBEwU!HmKW)dt)n-qTq+eW^B
z1>4>LY-j$r!S**s7boVw0o#6QH;f1LBY)SEa|%QG#z44b{CE#019H|hL4$v=fB2C3
zGaMEQ(-Bu8<ZOYW3U_FJd;NJ;UtLgggR`6_2aX0!n<r>;94X~TexJR%EIM~OXd4`J
z{T@^|Y?oS4!EMrcwZ8?#B<bF2XZ32%y_seEo-R$}qtV|Rut{ctQ0<6YK^=81m}Rr^
z-h|WMGexGF;5R#yYIlrSR#_hbUw<(Tu~g$B#_3NU(GR=36l|{|I=OO>zTRWXhAC`x
zx-TBN95t%lF0g|}G)R%a>jVc(*HiM!Xr=k6Ga*|ZN2(MkIcWTZXKeyH{UoNfEArj8
zT;=OWZ1GmQ2y>vku*FAXI&vJ?mgV<?FsXqnT|UxoO_%=2L@2lN7ri(mx1&#_Zr3e8
z9XP8@+D!3VJpQd&7=Q4Y(8jjC<`N{30aw!8^(}`9lHrsGy^`3L7S1?4y{P5FiRj}H
zau5T^zGmz$mn-6YSW`RH@kI8z?peKNxo~2y?MapHy&RQ%8&|7w##CB+c^*G2<gh#?
z`;&T3Gq@^+{^}b=dB@WrYPIOr0u;x+a59GFDiebVt&Bpn_qdvSq(QkiN^}i8phOp)
z{S2UG%mAUJ-|j_+O|fS`V)Fj@f}LZB0ItwywN=X#+GK6Yv2C>gZmZ2YX)t$|VuFR0
znQy@Ll|tJidWMlT%i<~j(qO&({9IY9JCC{Sg|xBfodR&0m{eslW|_E;NQ_P58-3fU
zUvct363)&R(e0GXHON)~>8TE3L7T)Vml7)p9(tp)gq($pUayW4MBHe$nW?!Mx#tvu
zI*f7&Ts6A6v_4yEfL?C|0Q&|~v7CODH1Du>nXem$%xNwk6E8%xqPPmdd}A{%+HNRU
zdH;<0J2+7*Xa&Ih_Yisb{|!Vw70x9|_C<k0WV{=Q+|T^Kgvi+X65^Ts*rZw?#K;fX
z`=Y~<S<0tg{py9>3bxHg)pqY0RTl9v&hs#CzX%YA)6C0q-poBcF6DVP0{J=BtZTGE
zq&}@uS>dNfLt!PAR<~VTU`vv*m;h$Yw?8)qYtn!6RkivJ%tZ9FFheF44le4etdBX>
z>bl?S0<}}xH5`~y=-R-l_&HO$rK4cCGUkGP4$Ux)$h54=K!~zX+JlEQ`{)mS{Qc|l
z!|6hx`*DH?4d+0<#|A{W_FWe2x+f^$6b0-Z%53@<gUty>>uKq07=!{5hfP{w(^-Un
zY!0(^mrS~^geVDRCk+bA@w{IS5Uk@B)5l5oB7GSG%6KY}E(1%nT-u@VaOU-^$HVzK
z!e@ks&v&L1Ef8Q`4FIUD4gi%^)_)NOU-<u@!(e}q0uQM!CkT7v7k2=Z+OHG08wxQM
z;lUmZt8R|SE_wH1=yOfU*k`GawUK(lUN2jq8A*0G)be~^S;e3Pbx18J{_$-<Ik_0!
zYM!N}qVIIpAoy)gPKK%Tu;cZ3B;atQG?dmGvKb{jhK+_-+u^VcqS>9<jHF0H1+pwg
zrOJoDuR(R$I6Nb3y@xgf05Q%LtC^}7NlI@&rr)!Hc5Awltg_~q#VQ_y;%%=(VWbGV
zpG|ex3RzM`N?7=Tcu?lUOv1-TePpFwU!>M{e6FUdsZlnw&QhkPcX1=AF0?%+awwSa
z??^k+N>xfY<MM%`E}IgR{z+pT;t;WJ+3gKsK|=@O5Q7roQO?kCY9_z+yXoqCf$!O~
z^L0TiMV;0%!caLz?Gef4i)38Q9u4%uIFkvz9HDHyZS5tlk75L*+HmSbPTRCbmZtd(
zvYtrg?DTF!`{%($7cFCwTf!GoN{a>N3rO)wjZhn##pjP<T$$hZl{yGcuan-{Xv(mx
z($#G9U0tRXHM;}1d$+;91<4BYumO-12aEYCYll=(29H*Cm_qAFG^n?@!`&U`C=^-~
zFDWsqUu!xiQIQD~z9L`LD!E7H$NyElMfigu;0jR#*J#7Yk2Kr%wpoU5ayXjaQhQw-
z8nW_~f0bE3I5t~)82g4vUB(DgpYAqcW1+HeyAAQ<-1D&WxxaGHO+B~V^QL?cykyyq
z-gGB*>#>??=dWjtTwXnxTsa<LTu+4<G<?uDa%I+v7w6h`8H~CTN^c7Tc%m}GntShG
z3BKZ?M63zO(Lv&)tRy&o*4QzVT<?j}uzTRAtSzH@g)<cEY{278vvLIOXJz{!ls5q$
zk=)!Gz@!XJb*6y`O#0YcB-NP`F*PamuQ-@>@K%`p3?as#oBlr`xYJ`_TXW?alu?GD
zPQkC9F$84+PbO=?4Tc^63_}^mej_xNv3eGXmr8kwLmzL3@oM+jaC)A>=5EqtDMuF8
zH(K(S?g66YbwQDKH!u(JA21J!&;Gy0!K|W+^y--G0Q=tzNw5c(RV2ALboIL#;a~rC
zk2F=Izjc#C)}VPM`p!vVyZ;y5<g%2f|0>)@$@n)?d)m^{N>)P-!f4oyqr7hAM}WqF
z)J>)iTCq1nmJXFkGOS~x4bU0qa{JKi25XehpHo)fRW`6v#y*knUH2)T3ww@5AH%@5
zA?}9)oD=KnvtKJ4%drDrKYrFG!N+h5pdlu0cjS?0e$BxGTli0LKWu26MEdMbNB;I+
zbFQ!lltB$*wF{Pq<?Z*s)BnsqB?1Pj2LS_O0-D0ePW%Ev$-LQ&6jCN+6}?|ngeb)g
ztIw${8N8)h_7y8WBSIzj&|z>4^ywm*{Rtw{(!mX^%pjQJ*+WuK2{JOW4?N~;;)|MQ
z<+>{7p^;s4av;TQn(>mp;a6JpZh;kD3e<)G9pip584vtYdP8-0e%|Vc<tgy=9^n(Y
z*V5X4pkA{yM)5&J9Jd6^<;}o?YTY^21X_;z{5&~8Bp}bQa0s<lPx>ne1ps%q_Ywkq
zw6XeJXn%CI&n@x1!}RfQIoPkTd|gcH`MxL5KWZ{KGcf1w)QKk(NTc70=I2cV6mgh?
zz(1xk<i3qj%7LT4cGIy62W+GKiGz}!ATm*m#c32N8!iPwhPH_<_3wJQLWV_kn;utd
z&}eQLO_D<KyemX_z`)OqNRGWN&FO&<MK9uMT}7fC0cnn8)EWQ^V1v4rf9=PBC=$PW
zgj{OsN9lWCtzwZGdS8xgX}4MhoBAN3myK;1SagNtv70z-=Gy0w@J4`EA=GrkJD}(J
z&@8`h8XlJ}%XGs1eX_}SaVfeRws%FAt%|%>!${K>QN~Dq-N7tCPT=+)VTwSRIUt`F
zFmj=wmma)0$C4BdGchQE`J{b`>2lekRlXek3i&#mOpTTfBpp+QlT8>qJhoS){&j_`
z^`m>o0qNIat84R{Xt{!S*hEx`RjEl6B|l>&sUpeC>aHzSG;ch59&rOB3b#*MnAHc+
zJ947FJdRS@kH~IWtIqvRWT&y<22L}t>b|Rv@1%FdZ(W!Vft+(^dpzI(bGhyOT`fVF
zo_D!|Hoj=vv1C>xdFL^(O%I?+4;@{M1~ud3kKMk%q@;j}**f*Z-aWfK#~N}dW*Dq9
z%Zz(ZJrWTfJ|*tFw;3BFPkI(DN77>p4TtvX42`oGNM#fzsp)28H)(^dwOz=5pWHl}
zd_|dL{JKtFJ$o3kqRwr})PC<H2!OWRIQVld{@_{&{?)QrUUzqc!&Jz<8HhRyZ#;|7
zLLaG29H9dLp>`W~uxUF^JR0Sm*J$dB*=w(?)*Mqf<>(s!KQH#zTRsN0vLg+s1-bp^
z<V;&1J$Qiky~<_LODJR5c=U}{vr}IEg_d*gH|1hy#3Y$;OJc<XCg+_`uAan=yI3xD
z?-|jHx_&=k*><4gfGMF{{63!s0%0=6nX2Fns{N3Wk-t}&z4*3m30aoAzI2%IkRvDX
zkbo@9)=OHOmM)ej6XHdG60>#`4?-FVxs;mEdsV}IdwU^VZZH|B)e|EbQiBvMRbN6a
zmUR)qZhG63$Js?<GFG(kL#mQWaItR@o!e{bB_+uBY#rL}*(!}Gb@(!|jpT?m{r-xn
z2w^2kW%B0adb}~_YX-6zMeVkiE>v;!ESr~c+(`(OIM#C?0PWdv*W<CblDto>w^P_(
zC)gaI^U6G$*{73H3(m9~S_9R@&3FKwh3dPRy0a`<GWssWXb&K(9Lbx_V*Ex2uBtTY
zefDv&`TR!z_E&lEd<ptp;e$u*Elz#Kof$B>`aJEAveZr6%9f31TYx3>W&Q?KsI%j2
ziYOsE+cXas?(=(E=>;2h_Of_-!WL<B;)Y#NSA#N9B}6QH16bh57JFHtsqWHRmAU!B
zuk+c5CQRJG2Y)(G`-;7-Mrl;m=8&t)NO7a2Iq(MSZb|d1{GI|?qoHQIQn7ZrfAMX>
zA27hz+sz3aReu1zQVDbh$#gf3;9GgdO=^L|iY|1@X5BxUkt0S#K)6k&03w3Yh{<tp
zY*BQ|X-jG>PjQlNIvFdU+UXBDH(g!uy1mkIGsMB3UZ!%UQ5sEAC{x?78EAtx`EEuU
zkcOo}9bKU#@Z0X1>H~Pp9tkqHRVzsk10fVg{`uLvz*zOugDc920tuQgGMbym)~ui_
ztIhy+baI{z7;|>dhb|UWS23Q@E|IF)QOWY`@T%Vm>wm%3MfJ?*@DKqkCcnK<P=)|p
zFuohq-bEG9jQD51#{}D<{y%81|C<ai4l^ez{~pw&1|3w6fG&9E`On%T0;J{LekR}?
zp&QA{zx~_VBaZra?NKte$V1dzk}eM7ZmKPhXtX9zG8!WI!K>e?_aoY^U}N;Knpv?2
z)C+OPJ;WPUk1#cIvNZ0hS|^9=muc0d2;9ggdHzG~p}GJ*`g?G9|5Oq^UA-a}Vtx%@
zsMR&H`zg#=tH`N>O}c(RaaJp)CRTC#I;{lrhAc)3X%pJ@SCvhj_dc50Zis_z16YRh
z%<xbcsL2OBe#(_L2tQB<;W6^F=adas`H2P!GaR?SCjvwx^^*RZQ0~Xh$)eK9>6{{n
zJTz#gK&<WfJw5eOL4TS*nYcqbX($@&X$9uwJGs0k;pt@>&JVbvpFahWFzOSY;=s$q
zTeN}i<gA|E@NQjVwPDbdJrT&%pd&p9%3>bImahH$IS@@r+3jzd@f>Var>6&J27Ek*
zbv^mEs2lVpKZY%{|0++DsvNgw_<hpc2r=*QG0V7qC!*wyy~}|EB}^;X2fH?O26#;~
z)8^f|C$(+2N5t&a0jK>uXe9VUz16~8AQf4+6dR_V|EMK(n8Ba>Df}#YsLY1vlTw`b
z|GWTc->=a)*LbTm>k!S)kwkch!P)+$wtvA*XbAPkXoefo5mw4s>@%;@UyVJq9mWDR
z7+?#LNEEpqfofPuwy2_g(VzxjJH}UHTaKh}70PZVirb_)!A0D?ujpHx`l<pi>-j^U
z{`}cNERxS=FJVjQu=TKRgYZ`p9^d<TIpv}{wb|PO=G=k8k|CC}UGS%p(_*Y-5%)~f
z$;oG8p%2!{ALh!58$1rq_uAD!+0BamZ-zyb4As|gLBP#({=<HuS^_@$9eE`zy)BQL
zhT51o9Ltcteo5`HA`RP<y(#5g;czsisv{PLS4y--#24#TUq;mKyr4EOZQ-uL$gk^_
z%Xs+u!)zDL^feoAZR?l17>J90a+u|3N$s{uyJ52LMcA|UFH!Q~PUuxl4L+_I+o!Pd
z9zF$C(MzuPxOA_bd3HC$(LRcagsp}Z=-!fgkXZW4kK_A0>j$ho)?At@lVffP3A>z-
z&g#SV!r891Lf!Tjum=fuBXDq0w5w^Q_KT3{wm3;S_vIQi*duMj+f>!o+%I@mxR`04
ztT3u+dz`IbCfGBmVjej(!s&W)(7Jxmiz}`BONa+VmwMMs7~QZ$$s<@9f9tF1q7qrW
zVpl=Ql{MxJz`b^1M28D5ra%PkY#F`mwqrGv^Pp?-PUgo#;!^b6szxF<vK;7IgvLUY
zVCzby2lGR@j5__PgK7@x4U3-$*SVCHV@3P9Lan+wnQ{JPu(XaK!k-wr-|TUjT?tHL
zAdbg?^3kKA?cNL)0ZJf9swn~-ZAQ?2JK(`$eVHfz?n`x7$THm}>1-9edLEYEMT5ok
z$gPSfwW7&zf_ns#Ib<YDT))k5#QZ8kU`L{EL%l>;xs<H%G2T`U<q{{xg71s?Xid?*
z_3K^T^}01T`6%}|HCBJoM9vYrlTF1Sj!QP|jrc9IqlEl<v`Vb7zJEL;s5oK!KP%%N
zjOQ4U;`S%`aY;tFQp*A<P+1Bzuwo?9Tqxk)#~3=QK~G|-=n=+m6=wkk2|vs<wW?Zl
zpXJQ$Rc%JM+~vt(s4P)5fxs*O+dd%e-zp<w4vB&17CL$*z&CE`>G{Hb#}ng_BJ1Go
zv^CcB^oaR{$6Zg*MAe>-8$)Us6oFIDqs+>;sc_m-KkYkkWivaU(cTd>6&Czr^^3nQ
zXT_jo9!bB}T;KvV$tXtHDp{|3`c;E-1j<K~r=OvP;b%*if7FHD>38aa1~kOO!`lCP
z8@T3xou!J82*_dn=a~{L{n=0RgV@Gjl&I!v&i@wbG%wAN;-6L~Pjj`=$5Dbr6LdDQ
z>Iwa=HsX0lflGqRQ*W?$37@0IlSsOy+v$+?Jy;45JO4E(qWmrm6_wy(4H|Q8k6)BH
z*sazO<#i~CVGvH~Tpr#>7{_Svn07pFPeNx6-k6B|r&nd)ARyy55yC73D?fVO6lb|^
z%YN%AV*BzN<dvA;%ccJ(m<+Ck8x2i#=Umv!_Q>GJVC||hpUiBb6za`?AoHFLADzW-
zN)Qr|U}_WFK<Pj#1GeqB_@H7<U#h|cpSUciCz6LJS2Q{I8H@JJPIs$nJK*>A*x>0v
zJ_r0;D1!i*5M{27)BZYu97E(Y>5|%kaiU^9ie=asYnrn|P+d!Y#9KsDFN2FDeziU_
zv644HgY}xE&vYxCXGr#FA-2ToXARtdx-{YY!Q<UKIyaf!Sp&a;u>to#x>{S%`~<cQ
z4#BYJy0MF%;)*>=YgTbVw9!XgDH|UVyJ@37w`E3jl<dE@w$U0?CNCJRyz~3RU$2qN
z(-`pb`GL-={Zl?@!294><LE5iW~109KrX|HOKGIA{YssvwKZ8iSgrJP#UNDoDAxzM
zA1KWmn3!CvU1-eq=yz!ID=c8KZ>lXQz9U)Z3^v52eza5BykOjs9j^QxcfjcP4t>7B
z=EaFGTgW-{&<^M$2Rx%*;La;lM1J8v#Tm+;R59x4)7@B1ZvSY77(^IZtl-c$ZHd=Q
zGQGNxC|(mRd%xpsPY`HJ)MQlZ(V<%pLPMS_s75cYZSE2B`5foasN8RuUMm*P17g28
zKOY3dp^nf{%-cR72*%SNO0i7SYy7}Kej@)Cv<|(?_2kyicZbG0?YlZHh`tJJSv66s
z(Wshcm8-gk@Q$n>FxI=b+Tv;yr3_JakLi?G&6g3k8AH?TjsyeN&_|sVvU?upiRbqJ
zELj}J)o$NLnQ#8=cg&|*gI?#ATxW*C#XvB}>4OXMIp3yxB?nn`RTn99^`)(M+8VLg
zEOSPZiQGgb9rwGT0M;4|)=$ta+C;?+o=0d!{a_h;G}2<R=QK1Y$Fw8&-4XFxJ{hI*
z-Br8TZHwd`%WcA-pfR*EfO;+<)MPb1emIf?<d1aXOoa>5{9hX1?P}8ekhJrN<0DA%
z!0aXZW4!Qt=T%LhlLJ!v1i0t#8Gy{5n@z>>p;cyF`5)sycJIBDnO|=FnGV0c*Pa7u
zDUhHy-8i&C5Ami_eRt=!zi07)>rZ|9-<P1g(mp*Hz47jX1F(X<wPBA!Z<$9Br1ro6
zYeAX(Ktxs6eC+1#f<KBuC-XtzKdb5o*uPyl&>M&ax}s8X(L5elp1|}?xL8L*N4-7r
zryI4x0RFiTvpF4d2HTy08g5}JM2V?Uf@hvvznX(HJv0hz0?^SbiOMvPKq|(9yox3m
zrEjCkD@EB!?jf#*=J72G>I?ycH4~hNl|P;M8bJagHMxadiBQJ6$TUeah2YSTMXs>=
z6aN=c2fMqi-lqYdR?}W$SW@a*U*w+^kr!X-RrZjp_BChZhd172Ita6tg3Gk(n|noM
zh9SYw`GCG^$TqzB#;OFK<Ro}2xTuvv3}xN56L;3>k>kDE+V~7H7`j9p#;omNuE*Gq
zT>Zi3zt-`RtIu!FE+X*M^z%wy=M#CsGpnj`bD7n&Y8PXYmv^hzoodt=RA~>2Qc1xb
z0`B^M`Crf@(c|f1o#~jJ=>nWwa6<RhH=)PO;^wys3zI;9KpHM~p<KhCT<uS1asclq
zitF{GPdllqg;gm6bgpt=lCJN9zFCh1ku$?1B;`6vXMXO0{5!4+*!AMk9rLaITQJUI
zOFrH1`Rv35%FE4pcB|ozK6@4$PiAWPl^9;pyY};%(DkFyx<ZD7bw|^`9P?o^bB~h+
zf9vR;5xQJxdLpCh)}KEZBp-F8JG6oA5JsdbKx{{cN;G(1Dq+2&cyhurYru*@c^x@Q
zd4r(+g8s#ajn&A`YP|)#Wr&lL!z6Ev8S30GZmzt-$Ioh{Q5O?AF6QqwpfiJf;LGZ7
zt@XJp-h}3dCn8I>%^OeQ3RCGYvOAXY*|K^DW>C^rAEzg&j0$TE_Lk3U;_)S3*{=%U
zVVgv)df&WM01q;Xh8&g<|6-*Y^YAm^J`9>m>mzX*O7pH4e)2+7DI?esTIMpPI3fLV
zlH`~8Uip*Ir;G{1E{`Kf+$bL&D`LMp+ChG&r~xDYvKwCQI!pK@kxo7z#<fc7u9@!@
z%|%3nZXWXj<945Cojrd;o#aiSOaHBdue;im1{vcDY`(WbwYn)%Ag2v;y_@7z|An;6
zN^F$54{kx@+Q4~juiZua>s#4A2i7WUL6rTfg1{!Smx>xp4d}+{L}dy|0jQ}6$rq>R
zAq_7b_=Zvjoj|M5Lwmyb8LE}{KDQZmA>-}mf>X^#@M{jZt_JcHLpT~D#j@Pzm#~Bx
zSq##8MV`*Dz?~&H)x&TW1UIRN1hB0T+83hFMS?oG=2(kQ1%jXk+D$TY>fv>#Ga=`7
zi|0%;r5I$&XFps-MbIrH84tHS)iyAUw|GrTmyDLB4Uf1AjFvAD@ARi60C_<)Y&<>2
z%H3s^{ZG3Av*>eIQdf9|Jz1(zD2pCWQ<a>!*GjIP%xLpu)QHCP4k{W?7Cnn_IIwc^
zvHB7Z=eLw(qm^d4E_^@sMZPmUOu*)eMg_BmMxJ8Dvne7!&f_Ph$&_%rp}KfU10!6;
zBTs3TG$hFER|PxSunJ3E_$imwyc!r^p73yb94Nm8`D?Uyjrt4viBTBf)eX^>Qq&^o
z@76?~NbcgbUF;S3IAkR-3WZscOcQyf*=?Ri(<KM<`1|ZNi<uIzfqi$+R3P76t-C)Z
zYx~;&Vu&wns1*5l)uo97#JH=rlO#tLdV0dQNwgq_*Q({4v1^!Hx08uXgIiA`(9hF|
z$=wlEFUR3L844<2WB!>RQRCQZE+OL3v1Q;Ma(G{}S1JK{hlOIAs0ERu%%S@Iot-i-
zV)>$U)NU(TG9sG=mEc4|JQVx2A*(88NK&aGmSNEkhtJQNJ2GE*Jm<FH-zhU1r8;!R
z6f;#VlPc4YD-bjIAVwnA+HIo*_Rvfz_(8RkAK49HCd^S>1f2R;YGJ1A42{^IXid5K
z9_8#9cD{n)``3Y+C7BJbk)zM+D{f{LR$A!f=h0dGz#REk9DgybmQ?~Z-(y#kfTKHt
zJu1;@OrtM%Lr*=lsG1?JvM1~A>5;j}GSAYdYxJtpti@1>f=}9CspyCf)268>2fi$e
z$yQAWmbB~dXjMnq?NCS2T~U86aUN_@<EtbZWPJYIYam6X^Mdr7hl5%H`RBQ>Fs?&{
zD#u`ZT1)+9mHT4W?$6%f1w~<&U!-R9+U2<{b74v#*ZYu;;h$XPYMdz4=-sRm1K&XN
zr`G{et8S`_yRzKA^Afl0+XIudP%hG{x6}{l`Sy$6T-?2EB%*%AakZQVzaD)qQxGLo
zm%n1r-LV?Eb^X>DhDeJS-gSk9UKuOX1e>BM1tDsiL8h<Yx4#hg%yD0qG--{1sJEPz
z+K{*15O+so0lp1qX~7B`E?VK^E(@NpoHn*T5y#89@9JTGHAaUxQX}vmTdehYNKUT<
z9}e9_CPT(Q0mg3-P^E@`%4iJvtd5kZEc6@>p<szr#HnncJmCHKV_jvigQMMp!HA<I
z{OXkW;-WQ-|5w|{nfw&Qa9@~+I`oT%wW_0|E93Qi@8s!Ov&-e>(w!MxceFTOOsDnX
zyV10Z)~N(&d4ZG%N<_H6BU)<89MN*pMY#(GyV-Vd=m@vLKPz<5)fX8d+GAzeP$&b>
zlzS*jyt?`$3Yu7-L5LWf|CIAW&jk~|+0PF3r-*N*kLN8kYrjmk+HdFxTz3#O4B$v6
zs|zrGjLya<$IJVUC5;i(!~EbIvxo&<$RUpQ{wag&y}2t`7s8#;;n1nKf$SVnJ2b>>
zk=_2?8hKwk4N%f`P%<L}L_%I;@OxVa6JUpSP=Bs~R#afcM~ANXV6QOb^-awzN;vbZ
zHPm>xP<64ejt0T^+@0i>up%qXVo~=Vj7g*Bz<3+H=;`Mz_C(j6E_fHRkJr_Blpu_c
z{ezD_EZ>&sTX%zJeqcL6^8hTYkz1#R?q_^G<P}RSA@WcRx-#z{2Bf>s#d>?m``B2E
zotOk4om|ELG`?M6<^93qvI2Y57ud#s7&QML4KcP+XTj2Ap~w4`xn_h<gmzcBC+$nc
zakV2q`%fqkX>*WOqi!>&#xk896y1npo3MQOw6E8;pVvmA-acbf;}C9mfs<pz2T^7T
z!pmF25UIC>jqM)qI1((qWTwhXoLqSSEW~G5=&P@Ns<7Kyub+p+ITwTpgP^CQQl><k
zSP541gkj_uH`aP;1`>$%UX9KCkna)e!_k|{(KkcMthZg@w18ilbz^z?vrc+jzEksv
z@*1h9cshz1Xh*JY72gXSeeX5CrreOeK^q|&-i%Q=n^pKqW7f?pPQRLop|c|g8EJUL
zX^8G#(>x}Z+1>7F`>cLORbC7NV@sJ*CfwnGNXE-t<lUw?hpHE63ocjPOcRK0&9AaF
zwLuLKTR0k-8Psy@UAx-+C1JNC)7hT!OOGl5F${?8>O>NrWSp-0a1FjF#$`)y0=W=f
z`tS1-Di@aY>z`#M5I2biy|Irlr4aLJe4;$vUS_|^yR9ieZ0vuRbATpI&Y^FT>z=4N
zWHoF?S7DT<W1Kz0^8$zVf^}6_R|Gw8L}MsVuE1`Kw=>_f$_y*5z}uYTBi-W2)7O&?
zU+nORUpNpC1!<-QZXkaswB%jyG(|kOu%#;7-WwHsMCj-mOMg)G`Vl(MuQ}slC5J|Q
zCJ&TV#G{i~DLB?h<5FR3^*Uy+N}krj#H?R6UrY)$X1kXUUKGf>TL$h&8ne2goK-It
z(1TqBY-_hd&N*3H5<Ec(?EFq{5FYZG0DTE^m9a*}eyq<P3U<_OQDdQ=%@x>JDLW$<
ztNgTEI@ld;JP@Y$hJQQjzGmp*(}(K^hzK9*tnjp!S~N(yms-no;F9`a>aba`V&XV5
z6|=>cMrAjCtREawX<wgQ?2a7mGVWq;Io4EdU7eoDzaZ&|#F)bS$~d$6=>;RUA$Olh
z4Sx#ZsvCDnVw{8~S6^iJd9c*kZfR7fpkj?-b;OLVuafXY1K5M9^W>{=*-7-s4=Ocr
z%Fu7>*!rwhIw()uIz~U0#on7R#&-yOd1(9~7lrudTh%9nR-0-WjYycioB^m;S!zon
z7rkLw&FMNb=_x3Q56?BKc)o4_Mp(~nh|b*qnbe03orAEZmxTi|46x9cdf7=wL(5r0
z_2yvg3q=UnK8H1LND2$5{sTM7KCt3wiufc-6e}>lrbU{4zys-d1WQCce*`kNW&Q!g
zsQk}am=P{wRthYggP^_tXTN^~`K++4ubbT3?dN@qUjF=@`oA3+aO>y?*ccE9h6}gy
zTGW3{w1XZT&`>vb3j87a`rkN*|NVpi28;iG0A#mc0xqQ=laNwZhwp$PCL$$V@Lb2^
Fe*nwNB1ZrK
literal 0
HcmV?d00001
From af19faad0a5ccdc10b51a6a904cc0cd546d6ffb3 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Thu, 26 Dec 2019 13:31:25 -0500
Subject: [PATCH 041/152] Add IDE images, links, and descritpions for testing
---
Python/Module6_Testing/Pytest.md | 82 ++++++++++++++++++++++++++++----
1 file changed, 72 insertions(+), 10 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index c71617a5..694c60af 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -129,12 +129,57 @@ Now, we should be able to start a python console, IPython console, or Jupyter no
```
<!-- #endregion -->
+<!-- #region -->
## Populating Our Test Suite
pytest's system for "test discovery" is quite simple:
pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files _whose names start with the word "test"_ and it will run all such functions.
-Thus, let's populate the file _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module.
+Thus, let's populate the file _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module.
+E.g. our test
+
+```python
+# we import the functions we are testing
+from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
+
+
+def test_count_vowels_basic():
+ # test basic strings with uppercase and lowercase letters
+ assert count_vowels("aA bB yY", include_y=False) == 2
+ assert count_vowels("aA bB yY", include_y=True) == 4
+
+ # test empty strings
+ assert count_vowels("", include_y=False) == 0
+ assert count_vowels("", include_y=True) == 0
+
+
+def test_merge_max_mappings():
+ # test documented behavior
+ dict1 = {"a": 1, "b": 2}
+ dict2 = {"b": 20, "c": -1}
+ expected = {'a': 1, 'b': 20, 'c': -1}
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test empty dict1
+ dict1 = {}
+ dict2 = {"a": 10.2, "f": -1.0}
+ expected = dict2
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test empty dict2
+ dict1 = {"a": 10.2, "f": -1.0}
+ dict2 = {}
+ expected = dict1
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test both empty
+ dict1 = {}
+ dict2 = {}
+ expected = {}
+ assert merge_max_mappings(dict1, dict2) == expected
+
+```
+
As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our in the same namespace as our tests.
A reference implementation of _test_basic_functions.py_ can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
Finally, add a dummy test - a test function that will always pass - to _test_basic_numpy.py_.
@@ -142,7 +187,7 @@ We will remove this later.
Without further ado, let's run our test suite! In our terminal, with the appropriate conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
Following output should appear:
-
+<!-- #endregion -->
```
$ pytest tests/
@@ -193,30 +238,47 @@ Suppose, for instance, that we are writing a new function, and repeatedly want t
We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests in such incisive ways.
-### Utilizing pytest within an IDE
+## Utilizing pytest within an IDE
Both [PyCharm and VSCode](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) can be configured to make keen use of pytest.
+The following images show a couple of the enhancements afforded to us by PyCharm.
+The IDEs will "discover" tests, and provide us with the ability to run individual tests.
+For example, in the following image, the green "play button" allows us to run `test_count_vowels_basic`.
-
-
+<!-- #raw raw_mimetype="text/html" -->
<div style="text-align: center">
-<img src="../_build/_images/paris.PNG" alt="Paris" width="600">
+<p>
+<img src="../_images/individual_test.PNG" alt="Running an individual test in PyCharm" width="600">
+</p>
</div>
+<!-- #endraw -->
+
+Furthermore, IDEs can provide a rich tree view of all the tests that are being run.
+This is especially useful as our test suite grows to contain a considerable number of tests.
+In the following image, we see that `test_version` is failing - we can click on the failing test in this tree-view, and our IDE will navigate us directly to the failing test.
<!-- #raw raw_mimetype="text/html" -->
<div style="text-align: center">
-<img src="../_images/paris.PNG" alt="Paris" width="500" height="600">
+<p>
+<img src="../_images/test_tree_view.PNG" alt="Viewing an enchanced tree-view of your test suite" width="600">
+</p>
</div>
<!-- #endraw -->
-<div style="text-align: center">
-<img src="../_images/paris2.PNG" alt="Paris" width="500" height="600">
-</div>
+The first step for leveraging these features in your IDE is to enable the pytest framework in the IDE.
+The following links point to detailed instructions for configuring pytest with PyCharm and VSCode, respectively:
+
+- [Here for PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
+- [Here for VSCode](https://code.visualstudio.com/docs/python/testing)
+
+These include advanced details, like running tests in parallel, which are beyond the scope of this material.
## Links to Official Documentation
- [pytest](https://docs.pytest.org/en/latest/)
+- [Testing in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
+- [Testing in VSCode](https://code.visualstudio.com/docs/python/testing)
## Reading Comprehension Solutions
From 3c5fa9f1f2da4d32ac41afc8f0b5427adc198da5 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Thu, 26 Dec 2019 13:37:50 -0500
Subject: [PATCH 042/152] fix some wording
---
Python/Module6_Testing/Pytest.md | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 694c60af..6dfd2485 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -135,10 +135,11 @@ Now, we should be able to start a python console, IPython console, or Jupyter no
pytest's system for "test discovery" is quite simple:
pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files _whose names start with the word "test"_ and it will run all such functions.
-Thus, let's populate the file _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module.
-E.g. our test
+Thus, let's populate the file _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module:
```python
+# The contents of test_basic_functions.py
+
# we import the functions we are testing
from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
@@ -180,7 +181,7 @@ def test_merge_max_mappings():
```
-As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our in the same namespace as our tests.
+As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our functions are in the same namespace as our tests.
A reference implementation of _test_basic_functions.py_ can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
Finally, add a dummy test - a test function that will always pass - to _test_basic_numpy.py_.
We will remove this later.
From 79c1b6b8196a783f9c39d8e041e2efbc0f54b80e Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Thu, 26 Dec 2019 13:43:26 -0500
Subject: [PATCH 043/152] fix some wording
---
Python/Module6_Testing/Pytest.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 6dfd2485..9d3bdf69 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -269,8 +269,8 @@ In the following image, we see that `test_version` is failing - we can click on
The first step for leveraging these features in your IDE is to enable the pytest framework in the IDE.
The following links point to detailed instructions for configuring pytest with PyCharm and VSCode, respectively:
-- [Here for PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
-- [Here for VSCode](https://code.visualstudio.com/docs/python/testing)
+- [Running tests in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
+- [Running tests in VSCode](https://code.visualstudio.com/docs/python/testing)
These include advanced details, like running tests in parallel, which are beyond the scope of this material.
From 0ba2d6d6583d77e4351d25d9f795461f27d528ef Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Fri, 27 Dec 2019 14:05:58 -0500
Subject: [PATCH 044/152] Finish enriched assertions
---
Python/Module6_Testing/Pytest.md | 129 +++++++++++++++++++++++++++----
1 file changed, 115 insertions(+), 14 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 9d3bdf69..61fc1ca9 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -87,12 +87,12 @@ project_dir/ # the "parent directory" houses our source code, tests, and all
```
A reference implementation of this package can be found [in this GitHub repository](https://github.com/rsokl/plymi_mod6).
-Populate the _basic_functions.py_ file with the two functions that we were using as our source code in the previous section: `count_vowels` and `merge_max_mappings`.
-In in the _numpy_functions.py_ module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
+Populate the `basic_functions.py` file with the two functions that we were using as our source code in the previous section: `count_vowels` and `merge_max_mappings`.
+In in the `numpy_functions.py` module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
Don't forget to include `import numpy as np` in your script in accordance with how `pairwise_dists` calls NumPy functions.
We have arranged these functions so that they can be imported from the _basic_functions_ module and the _numpy_functions_ module, respectively, which reside in our `plymi_mod6` package.
-Let's fill out our _setup.py_ script and install this package so that we can import it regardless of our current working directory. The content of _setup.py_ will be:
+Let's fill out our `setup.py` script and install this package so that we can import it regardless of our current working directory. The content of `setup.py` will be:
```python
from setuptools import find_packages, setup
@@ -113,7 +113,7 @@ setup(
This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6. Additionally, we will require pytest and hypothesis for running tests; the latter library will be introduced in a later section.
Finally, let's install our package locally [in development mode](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Installing-Your-Own-Python-Package).
-Navigate to the directory containing _setup.py_ and run:
+Navigate to the directory containing `setup.py` and run:
```shell
python setup.py develop
@@ -130,17 +130,17 @@ Now, we should be able to start a python console, IPython console, or Jupyter no
<!-- #endregion -->
<!-- #region -->
-## Populating Our Test Suite
+## Populating and Running Our Test Suite
-pytest's system for "test discovery" is quite simple:
-pytest need only be pointed to a directory with .py files in it, and it will find all of the functions in these files _whose names start with the word "test"_ and it will run all such functions.
+pytest's [system for "test discovery"](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery) is quite simple:
+pytest need only be pointed to a directory with files named `test_*.py` in it, and it will find all of the functions in these files _whose names start with the word "test"_ and it will run all such functions.
-Thus, let's populate the file _test_basic_functions.py_ with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module:
+Thus, let's populate the file ``test_basic_functions.py`` with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module:
```python
# The contents of test_basic_functions.py
-# we import the functions we are testing
+# we must import the functions we are testing
from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
@@ -182,8 +182,8 @@ def test_merge_max_mappings():
```
As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our functions are in the same namespace as our tests.
-A reference implementation of _test_basic_functions.py_ can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
-Finally, add a dummy test - a test function that will always pass - to _test_basic_numpy.py_.
+A reference implementation of `test_basic_functions.py` can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
+Finally, add a dummy test - a test function that will always pass - to `test_basic_numpy.py`.
We will remove this later.
Without further ado, let's run our test suite! In our terminal, with the appropriate conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
@@ -230,7 +230,7 @@ We can also direct pytest to run the tests in a specific .py file. E.g. executin
pytest tests/test_basic_functions.py
```
-will cue pytest to only run the tests in _test_basic_functions.py_.
+will cue pytest to only run the tests in `test_basic_functions.py`.
A key component to leveraging tests effectively is the ability to exercise ones tests repeatedly and rapidly with little manual overhead.
Clearly, pytest is instrumental towards this end - this framework made the process of organizing and running our test suite exceedingly simple!
@@ -239,7 +239,7 @@ Suppose, for instance, that we are writing a new function, and repeatedly want t
We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests in such incisive ways.
-## Utilizing pytest within an IDE
+### Utilizing pytest within an IDE
Both [PyCharm and VSCode](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) can be configured to make keen use of pytest.
The following images show a couple of the enhancements afforded to us by PyCharm.
@@ -275,9 +275,110 @@ The following links point to detailed instructions for configuring pytest with P
These include advanced details, like running tests in parallel, which are beyond the scope of this material.
+## Enhanced Testing with pytest
+
+In addition to providing us with a simple means for organizing and running our test suite, pytest has powerful features that will both simplify and enhance our tests!
+
+<!-- #region -->
+### Enriched Assertions
+
+A failing "bare" assertion - an `assert` statement without an error message - can be a frustrating thing.
+Suppose, for instance, that one of our test-assertions about `count_vowels` fails:
+
+```python
+# a failing assertion without an error message is not informative
+
+assert count_vowels("aA bB yY", include_y=True) == 4
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-2-f89f8b6a7213> in <module>
+----> 1 assert count_vowels("aA bB yY", include_y=True) == 4
+
+AssertionError:
+```
+
+The problem with this bare assertion is that we don't know what `count_vowels("aA bB yY", include_y=True)` actually returned!
+We now have to go through the trouble of starting a python console, importing this function, and calling it with this specific input in order to see what our function was actually returning.
+
+An obvious remedy to this is for us to write our own error message, e.g.
+
+```python
+# we can write our own error message, but this quickly becomes unwieldy
+
+our_output = count_vowels("aA bB yY", include_y=True)
+assert our_output == 4, f"{our_output} != 4"
+```
+
+but this too is quite cumbersome when we consider the large number of assertions that we are destined to write.
+
+
+Fortunately, pytest comes to the rescue: it will "hijack" any failing bare assertion and will _insert a useful error message for us_.
+This is known as ["assertion introspection"](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details).
+For example, if the aforementioned assertion failed when being run by pytest, we would see the following output:
+
+```python
+# pytest will write informative error messages for us
+
+assert count_vowels("aA bB yY", include_y=True) == 4
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+~\Learning_Python\Python\Module6_Testing\Untitled1.ipynb in <module>
+----> 1 assert count_vowels("aA bB yY", include_y=True) == 4
+
+AssertionError: assert 2 == 4
+ + where 2 = <function count_vowels at 0x000001B91B913708>('aA bB yY', include_y=True
+```
+
+See that the error message that pytest included for us indicates that `count_vowels("aA bB yY", include_y=True)` returned `2`, when we expected it to return `4`.
+From this we might suspect that `count_vowels` in not counting y's correctly.
+
+Here are some more examples of "enriched assertions", as provided by pytest.
+See that these error messages even provide useful "diffs", which specify specifically _how_ two similar objects differ, where possible.
+
+```python
+# comparing unequal lists
+assert [1, 2, 3] == [1, 2]
+E Left contains one more item: 3
+E Full diff:
+E - [1, 2, 3]
+E ? ---
+E + [1, 2]
+```
+
+```python
+# comparing unequal dictionaries
+assert {"a": 1, "b": 2} == {"a": 1, "b": 3}
+E AssertionError: assert {'a': 1, 'b': 2} == {'a': 1, 'b': 3}
+E Omitting 1 identical items, use -vv to show
+E Differing items:
+E {'b': 2} != {'b': 3}
+E Full diff:
+E - {'a': 1, 'b': 2}
+E ? ^
+E + {'a': 1, 'b': 3}...
+```
+
+```python
+# comparing unequal strings
+assert "moo" == "moon"
+E AssertionError: assert 'moo' == 'moon'
+E - moo
+E + moon
+E ? +
+```
+
+<!-- #endregion -->
+
+### Parameterized Tests
+
+
+
+
## Links to Official Documentation
- [pytest](https://docs.pytest.org/en/latest/)
+- [pytest's system for test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery)
+- [Assertion introspection](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details)
- [Testing in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
- [Testing in VSCode](https://code.visualstudio.com/docs/python/testing)
@@ -296,7 +397,7 @@ def test_broken_function():
assert [1, 2, 3] == [1, 2]
```
-> After introducing this broken test into _test_basic_functions.py_ , running our tests should result in the following output:
+> After introducing this broken test into `test_basic_functions.py` , running our tests should result in the following output:
```
$ pytest tests/
From 5c9f55f7e6616159eb5da4d9bfea5d81dc6fc87b Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Fri, 27 Dec 2019 15:35:32 -0500
Subject: [PATCH 045/152] begin parameterization discussion
---
Python/Module6_Testing/Pytest.md | 44 ++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 61fc1ca9..fb248ee9 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -369,8 +369,51 @@ E ? +
<!-- #endregion -->
+<!-- #region -->
### Parameterized Tests
+Looking back to both `test_count_vowels_basic` and `test_merge_max_mappings`, we see that there is a lot of redundancy within the bodies of these test functions.
+The assertions that we make within a given test-function share identical forms - they differ only in the parameters that we feed into our functions and their expected output. pytest provides a useful tool that will allow us to eliminate this redundancy by transforming our test-functions into so-called _parameterized tests_.
+
+Let's parametrize the following test:
+
+```python
+# a simple test with redundant assertions
+
+def test_range_length():
+ assert len(range(0)) == 0
+ assert len(range(1)) == 1
+ assert len(range(2)) == 2
+ assert len(range(3)) == 3
+```
+
+This test is checking the property `len(range(n)) == n`, where `n` is any non-negative integer.
+Thus the parameter to be varied here is the "size" of the range being created:
+
+```python
+# parameterizing a test
+import pytest
+
+# this test must be run by pytest to work properly
+@pytest.mark.parametrize("size", [0, 1, 2, 3])
+def test_range_length(size):
+ assert len(range(size)) == size
+```
+
+This test must be run by pytest; an error will raise if we manually call `test_range_length()`.
+When executed, pytest will treat this parameterized test as _four separate tests_ - one for each parameter value:
+
+```
+test_basic_functions.py::test_range_length[0] PASSED [ 25%]
+test_basic_functions.py::test_range_length[1] PASSED [ 50%]
+test_basic_functions.py::test_range_length[2] PASSED [ 75%]
+test_basic_functions.py::test_range_length[3] PASSED [100%]
+```
+<!-- #endregion -->
+
+This
+
+
@@ -379,6 +422,7 @@ E ? +
- [pytest](https://docs.pytest.org/en/latest/)
- [pytest's system for test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery)
- [Assertion introspection](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details)
+- [Parameterizing tests](https://docs.pytest.org/en/latest/parametrize.html)
- [Testing in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
- [Testing in VSCode](https://code.visualstudio.com/docs/python/testing)
From 36399c5e11b8597a61d864e0cdf739d1c6ea35e1 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Dec 2019 13:09:53 -0500
Subject: [PATCH 046/152] finish discussion of parameterization
---
Python/Module6_Testing/Pytest.md | 201 +++++++++++++++++++++++++++----
1 file changed, 179 insertions(+), 22 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index fb248ee9..467113c9 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -20,7 +20,7 @@ jupyter:
# The pytest Framework
-Thus far, our process for running tests has been a entirely manual one. It is time for us to arrange our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
+Thus far, our process for running tests has been an entirely manual one. It is time for us to arrange our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
We will begin by reorganizing our source code to create an installable [Python package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
We will then learn how to structure and run a test suite for this Python package, using pytest.
@@ -298,19 +298,7 @@ AssertionError:
```
The problem with this bare assertion is that we don't know what `count_vowels("aA bB yY", include_y=True)` actually returned!
-We now have to go through the trouble of starting a python console, importing this function, and calling it with this specific input in order to see what our function was actually returning.
-
-An obvious remedy to this is for us to write our own error message, e.g.
-
-```python
-# we can write our own error message, but this quickly becomes unwieldy
-
-our_output = count_vowels("aA bB yY", include_y=True)
-assert our_output == 4, f"{our_output} != 4"
-```
-
-but this too is quite cumbersome when we consider the large number of assertions that we are destined to write.
-
+We now have to go through the trouble of starting a python console, importing this function, and calling it with this specific input in order to see what our function was actually returning. An obvious remedy to this is for us to write our own error message, but this too is quite cumbersome when we consider the large number of assertions that we are destined to write.
Fortunately, pytest comes to the rescue: it will "hijack" any failing bare assertion and will _insert a useful error message for us_.
This is known as ["assertion introspection"](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details).
@@ -373,14 +361,18 @@ E ? +
### Parameterized Tests
Looking back to both `test_count_vowels_basic` and `test_merge_max_mappings`, we see that there is a lot of redundancy within the bodies of these test functions.
-The assertions that we make within a given test-function share identical forms - they differ only in the parameters that we feed into our functions and their expected output. pytest provides a useful tool that will allow us to eliminate this redundancy by transforming our test-functions into so-called _parameterized tests_.
+The assertions that we make within a given test-function share identical forms - they differ only in the parameters that we feed into our functions and their expected output.
+Another shortcoming of this test-structure is that a failing assertion will block subsequent assertions from being evaluated.
+That is, if the second assertion in a `test_count_vowels_basic` fails, the third and fourth assertions will not be evaluated in that run.
+This precludes us from potentially seeing useful patterns among the failing assertions. Say, for instance, a function fails for all even-valued inputs; it is much easier to see this if assertions fail for inputs `2`, `4`, `6`, and so on, than it is if only input `2` was evaluated.
+pytest provides a useful tool that will allow us to eliminate these structural shortcomings by transforming our test-functions into so-called _parameterized tests_.
Let's parametrize the following test:
```python
# a simple test with redundant assertions
-def test_range_length():
+def test_range_length_unparameterized():
assert len(range(0)) == 0
assert len(range(1)) == 1
assert len(range(2)) == 2
@@ -388,19 +380,20 @@ def test_range_length():
```
This test is checking the property `len(range(n)) == n`, where `n` is any non-negative integer.
-Thus the parameter to be varied here is the "size" of the range being created:
+Thus the parameter to be varied here is the "size" of the range being created.
+Let's treat it as such by using pytest to write a parameterized test:
```python
# parameterizing a test
import pytest
-# this test must be run by pytest to work properly
+# note that this test must be run by pytest to work properly
@pytest.mark.parametrize("size", [0, 1, 2, 3])
def test_range_length(size):
assert len(range(size)) == size
```
-This test must be run by pytest; an error will raise if we manually call `test_range_length()`.
+Make note that a pytest-parameterized test must be run using pytest; an error will raise if we manually call `test_range_length()`.
When executed, pytest will treat this parameterized test as _four separate tests_ - one for each parameter value:
```
@@ -409,22 +402,130 @@ test_basic_functions.py::test_range_length[1] PASSED [ 50%]
test_basic_functions.py::test_range_length[2] PASSED [ 75%]
test_basic_functions.py::test_range_length[3] PASSED [100%]
```
+
+See that we have successfully eliminated the redundancy from `test_range_length`;
+the body of the function now contains only a single assertion, making obvious the property that is being tested.
+Furthermore, the four assertions are now being run independently from one another and thus we can potentially see patterns across multiple fail cases in concert.
+<!-- #endregion -->
+
+<!-- #region -->
+#### Decorators
+
+The the syntax used to parameterize this test may look alien to us - we have yet to encounter this construct thus far.
+`pytest.mark.parameterize(...)` is a _decorator_ - an object that can be used to "wrap" a function and transform its behavior.
+The `pytest.mark.parameterize(...)` decorator wraps our test function so that pytest can call it multiple times, once for each parameter value.
+The `@` character, in this context, denotes the application of a decorator:
+
+```python
+# general syntax for applying a decorator to a function
+
+@the_decorator
+def the_function_being_decorated(<arguments_for_function>):
+ pass
+```
+
+For an in-depth discussion of decorators, please refer to [Real Python's Primer on decorators](https://realpython.com/primer-on-python-decorators/#simple-decorators).
<!-- #endregion -->
-This
+<!-- #region -->
+#### Parameterization Syntax
+The general form for creating a parameterizing decorator with *a single parameter*, as we formed above, is:
+```python
+@pytest.mark.parametrize("<param-name>", [<val-1>, <val-2>, ...])
+def test_function(<param-name>):
+ ...
+```
+We will often have tests that require multiple parameters.
+The general form for creating the the parameterization decorator for $N$ parameters,
+each of which assuming $J$ values, is:
+```python
+@pytest.mark.parametrize("<param-name1>, <param-name2>, [...], <param-nameN>",
+ [(<param1-val1>, <param2-val1>, [...], <paramN-val1>),
+ (<param1-val2>, <param2-val2>, [...], <paramN-val2>),
+ ...
+ (<param1-valJ>, <param2-valJ>, [...], <paramN-valJ>),
+ ])
+def test_function(<param-name1>, <param-name2>, [...], <param-nameN>):
+ ...
+```
+
+For example, let's take the following trivial test:
+
+```python
+def test_inequality_unparameterized():
+ assert 1 < 2 < 3
+ assert 4 < 5 < 6
+ assert 7 < 8 < 9
+ assert 10 < 11 < 12
+```
+
+and rewrite it in parameterized form.
+The decorator will have three distinct parameter, and each parameter will take on four values.
+
+```python
+# the parameterized form of `test_inequality_unparameterized`
+@pytest.mark.parametrize("a, b, c", [(1, 2, 3), (4, 5, 6), (7, 8, 9), (10, 11, 12)])
+def test_inequality(a, b, c):
+ assert a < b < c
+```
+<!-- #endregion -->
+
+<div class="alert alert-warning">
+
+**Note**
+
+The formatting for multi-parameter tests can quickly become unwieldy.
+It isn't always obvious where one should introduce line breaks and indentations to improve readability.
+This is a place where the ["black" auto-formatter](https://black.readthedocs.io/en/stable/) really shines!
+Black will make all of these formatting decisions for us - we can write our parameterized tests as haphazardly as we like and simply run black to format our code.
+</div>
+
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Parameterizing Tests**
+
+Rewrite `test_count_vowels_basic` as a parameterized test with the parameters: `input_string`, `include_y`, and `expected_count`.
+
+Rewrite `test_merge_max_mappings` as a parameterized test with the parameters: `dict_a`, `dict_b`, and `expected_merged`.
+
+Before rerunning `test_basic_functions.py` predict how many distinct test cases will be reported by pytest.
+
+</div>
+
+
+<!-- #region -->
+Finally, you can apply multiple parameterizing decorators to a test so that pytest will run _all combinations of the respective parameter values_.
+
+```python
+# testing all combinations of `x` and `y`
+@pytest.mark.parametrize("x", [0, 1, 2])
+@pytest.mark.parametrize("y", [10, 20])
+def test_all_combinations(x, y):
+ # will run:
+ # x=0 y=10
+ # x=0 y=20
+ # x=1 y=10
+ # x=1 y=20
+ # x=2 y=10
+ # x=2 y=20
+ pass
+```
+<!-- #endregion -->
## Links to Official Documentation
- [pytest](https://docs.pytest.org/en/latest/)
- [pytest's system for test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery)
-- [Assertion introspection](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details)
-- [Parameterizing tests](https://docs.pytest.org/en/latest/parametrize.html)
- [Testing in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
- [Testing in VSCode](https://code.visualstudio.com/docs/python/testing)
+- [Assertion introspection](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details)
+- [Parameterizing tests](https://docs.pytest.org/en/latest/parametrize.html)
## Reading Comprehension Solutions
@@ -469,3 +570,59 @@ tests\test_basic_functions.py:40: AssertionError
> Four tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests in _test_basic_functions_ passed and the third test failed.
> It then indicates which test failed, and specifically that the assertion was false because a length-2 list cannot be equal to a length-3 list.
<!-- #endregion -->
+
+<!-- #region -->
+**Parameterizing Tests: Solution**
+
+A reference implementation for this solution within the `plymi_mod6` project can be found [here](https://github.com/rsokl/plymi_mod6/blob/parameterized/tests/test_basic_functions.py).
+
+The contents of `test_basic_functions.py`, rewritten to use pytest-parameterized tests:
+
+```python
+import pytest
+from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
+
+
+@pytest.mark.parametrize(
+ "input_string, include_y, expected_count",
+ [("aA bB yY", False, 2), ("aA bB yY", True, 4), ("", False, 0), ("", True, 0)],
+)
+def test_count_vowels_basic(input_string, include_y, expected_count):
+ assert count_vowels(input_string, include_y) == expected_count
+
+
+@pytest.mark.parametrize(
+ "dict_a, dict_b, expected_merged",
+ [
+ (dict(a=1, b=2), dict(b=20, c=-1), dict(a=1, b=20, c=-1)),
+ (dict(), dict(b=20, c=-1), dict(b=20, c=-1)),
+ (dict(a=1, b=2), dict(), dict(a=1, b=2)),
+ (dict(), dict(), dict()),
+ ],
+)
+def test_merge_max_mappings(dict_a, dict_b, expected_merged):
+ assert merge_max_mappings(dict_a, dict_b) == expected_merged
+```
+
+Running these tests via pytest should produce eight distinct test-case: four for `test_count_vowels_basic` and four for `test_merge_max_mappings`.
+
+```
+============================= test session starts =============================
+platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
+cachedir: .pytest_cache
+rootdir: C:\Users\plymi_user\Learning_Python\plymi_mod6_src
+collecting ... collected 8 items
+
+test_basic_functions.py::test_count_vowels_basic[aA bB yY-False-2] PASSED [ 12%]
+test_basic_functions.py::test_count_vowels_basic[aA bB yY-True-4] PASSED [ 25%]
+test_basic_functions.py::test_count_vowels_basic[-False-0] PASSED [ 37%]
+test_basic_functions.py::test_count_vowels_basic[-True-0] PASSED [ 50%]
+test_basic_functions.py::test_merge_max_mappings[dict_a0-dict_b0-expected_merged0] PASSED [ 62%]
+test_basic_functions.py::test_merge_max_mappings[dict_a1-dict_b1-expected_merged1] PASSED [ 75%]
+test_basic_functions.py::test_merge_max_mappings[dict_a2-dict_b2-expected_merged2] PASSED [ 87%]
+test_basic_functions.py::test_merge_max_mappings[dict_a3-dict_b3-expected_merged3] PASSED [100%]
+
+============================== 8 passed in 0.07s ==============================
+```
+
+<!-- #endregion -->
From 2be94ad4b599b7432eaaed80bd6d61ae3508a8d5 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Dec 2019 13:39:16 -0500
Subject: [PATCH 047/152] wording / typo fixes
---
Python/Module6_Testing/Pytest.md | 34 ++++++++++++++++----------------
1 file changed, 17 insertions(+), 17 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 467113c9..8bc5235d 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -54,10 +54,11 @@ This strikes me as... bizarre.
`unittest` is the testing framework that comes with the Python standard library.
As a test runner, its design is clunky, archaic, and, ironically, un-pythonic.
-While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) provides extremely valuable functionality for advanced testing, all of its functionality can be leverage via pytest.
+While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) provides extremely valuable functionality for advanced testing, all of its functionality can be leveraged while using pytest as your testing framework.
`nose`, which simply extends the functionality of `unittest`, **is no longer being maintained**.
-There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project by comparison to `pytest` (as of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads).
+There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project by comparison to `pytest`.
+As of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads.
The takeaway here is that, when it comes to picking a testing framework for Python, `pytest` is the clear choice.
Any discussion that you come across to the contrary is likely outdated.
@@ -91,7 +92,7 @@ Populate the `basic_functions.py` file with the two functions that we were using
In in the `numpy_functions.py` module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
Don't forget to include `import numpy as np` in your script in accordance with how `pairwise_dists` calls NumPy functions.
-We have arranged these functions so that they can be imported from the _basic_functions_ module and the _numpy_functions_ module, respectively, which reside in our `plymi_mod6` package.
+We have arranged these functions so that they can be imported from the `basic_functions` module and the `numpy_functions` module, respectively, which reside in our `plymi_mod6` package.
Let's fill out our `setup.py` script and install this package so that we can import it regardless of our current working directory. The content of `setup.py` will be:
```python
@@ -102,7 +103,6 @@ setup(
packages=find_packages(exclude=["tests", "tests.*"]),
version="1.0.0",
author="Your Name",
- author_email="your.email@email.com",
description="A template Python package for learning about testing",
install_requires=["numpy >= 1.10.0"],
tests_require=["pytest", "hypothesis"],
@@ -110,7 +110,7 @@ setup(
)
```
-This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6. Additionally, we will require pytest and hypothesis for running tests; the latter library will be introduced in a later section.
+This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6. Additionally, we will require pytest and hypothesis for running tests; the Hypothesis library will be introduced in a later section.
Finally, let's install our package locally [in development mode](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Installing-Your-Own-Python-Package).
Navigate to the directory containing `setup.py` and run:
@@ -184,7 +184,7 @@ def test_merge_max_mappings():
As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our functions are in the same namespace as our tests.
A reference implementation of `test_basic_functions.py` can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
Finally, add a dummy test - a test function that will always pass - to `test_basic_numpy.py`.
-We will remove this later.
+We will replace this with a useful test later.
Without further ado, let's run our test suite! In our terminal, with the appropriate conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
Following output should appear:
@@ -242,7 +242,7 @@ We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumea
### Utilizing pytest within an IDE
Both [PyCharm and VSCode](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) can be configured to make keen use of pytest.
-The following images show a couple of the enhancements afforded to us by PyCharm.
+The following images show a couple of the enhancements afforded to us by PyCharm; comparable features are available in VSCode.
The IDEs will "discover" tests, and provide us with the ability to run individual tests.
For example, in the following image, the green "play button" allows us to run `test_count_vowels_basic`.
@@ -272,12 +272,13 @@ The following links point to detailed instructions for configuring pytest with P
- [Running tests in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
- [Running tests in VSCode](https://code.visualstudio.com/docs/python/testing)
-These include advanced details, like running tests in parallel, which are beyond the scope of this material.
+These linked materials also include advanced details, like instructions for running tests in parallel, which are beyond the scope of this material but are useful nonetheless.
## Enhanced Testing with pytest
-In addition to providing us with a simple means for organizing and running our test suite, pytest has powerful features that will both simplify and enhance our tests!
+In addition to providing us with a simple means for organizing and running our test suite, pytest has powerful features that will both simplify and enhance our tests.
+We will now leverage these features in our test suite.
<!-- #region -->
### Enriched Assertions
@@ -318,7 +319,7 @@ AssertionError: assert 2 == 4
```
See that the error message that pytest included for us indicates that `count_vowels("aA bB yY", include_y=True)` returned `2`, when we expected it to return `4`.
-From this we might suspect that `count_vowels` in not counting y's correctly.
+From this we might suspect that `count_vowels` is not counting y's correctly.
Here are some more examples of "enriched assertions", as provided by pytest.
See that these error messages even provide useful "diffs", which specify specifically _how_ two similar objects differ, where possible.
@@ -364,10 +365,9 @@ Looking back to both `test_count_vowels_basic` and `test_merge_max_mappings`, we
The assertions that we make within a given test-function share identical forms - they differ only in the parameters that we feed into our functions and their expected output.
Another shortcoming of this test-structure is that a failing assertion will block subsequent assertions from being evaluated.
That is, if the second assertion in a `test_count_vowels_basic` fails, the third and fourth assertions will not be evaluated in that run.
-This precludes us from potentially seeing useful patterns among the failing assertions. Say, for instance, a function fails for all even-valued inputs; it is much easier to see this if assertions fail for inputs `2`, `4`, `6`, and so on, than it is if only input `2` was evaluated.
-pytest provides a useful tool that will allow us to eliminate these structural shortcomings by transforming our test-functions into so-called _parameterized tests_.
+This precludes us from potentially seeing useful patterns among the failing assertions.
-Let's parametrize the following test:
+pytest provides a useful tool that will allow us to eliminate these structural shortcomings by transforming our test-functions into so-called _parameterized tests_. Let's parametrize the following test:
```python
# a simple test with redundant assertions
@@ -380,7 +380,7 @@ def test_range_length_unparameterized():
```
This test is checking the property `len(range(n)) == n`, where `n` is any non-negative integer.
-Thus the parameter to be varied here is the "size" of the range being created.
+Thus the parameter to be varied here is the "size" of the range-object being created.
Let's treat it as such by using pytest to write a parameterized test:
```python
@@ -411,8 +411,8 @@ Furthermore, the four assertions are now being run independently from one anothe
<!-- #region -->
#### Decorators
-The the syntax used to parameterize this test may look alien to us - we have yet to encounter this construct thus far.
-`pytest.mark.parameterize(...)` is a _decorator_ - an object that can be used to "wrap" a function and transform its behavior.
+The the syntax used to parameterize this test may look alien to us, we have yet to encounter this construct thus far.
+`pytest.mark.parameterize(...)` is a _decorator_ - an object that is used to "wrap" a function in order to transform its behavior.
The `pytest.mark.parameterize(...)` decorator wraps our test function so that pytest can call it multiple times, once for each parameter value.
The `@` character, in this context, denotes the application of a decorator:
@@ -440,7 +440,7 @@ def test_function(<param-name>):
We will often have tests that require multiple parameters.
The general form for creating the the parameterization decorator for $N$ parameters,
-each of which assuming $J$ values, is:
+each of which assume $J$ values, is:
```python
@pytest.mark.parametrize("<param-name1>, <param-name2>, [...], <param-nameN>",
From 1c18e0a6bc7764be8cb2a2e68631f146d3997996 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Dec 2019 15:02:50 -0500
Subject: [PATCH 048/152] complete discussion of fixtures
---
Python/Module6_Testing/Pytest.md | 68 +++++++++++++++++++++++++++++++-
1 file changed, 66 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 8bc5235d..db5b498a 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -82,7 +82,7 @@ project_dir/ # the "parent directory" houses our source code, tests, and all
|-- basic_functions.py
|-- numpy_functions.py
- tests/ # test-suite for `plymi_mod6` package (to be run using pytest)
- |-- conf.py # optional configuration file for pytest
+ |-- conftest.py # optional configuration file for pytest
|-- test_basic_functions.py
|-- test_numpy_functions.py
```
@@ -464,7 +464,7 @@ def test_inequality_unparameterized():
```
and rewrite it in parameterized form.
-The decorator will have three distinct parameter, and each parameter will take on four values.
+The decorator will have three distinct parameter, and each parameters, let's simply call them `a`, `b`, and `c`, will take on four values.
```python
# the parameterized form of `test_inequality_unparameterized`
@@ -518,6 +518,69 @@ def test_all_combinations(x, y):
```
<!-- #endregion -->
+### Fixtures
+
+The final major pytest feature that we will discuss are "fixtures".
+A fixture, roughly speaking, is a means by which we can share information and functionality across our tests.
+Fixtures can be defined within our `conftest.py` file, and pytest will automatically "discover" them and make them available for use throughout our test suite in a convenient way.
+
+Exploring fixtures will quickly take us beyond our depths for the purposes of this introductory material, so we will only scratch the surface here.
+We can read about advanced details of fixtures [here](https://docs.pytest.org/en/latest/fixture.html#fixture).
+
+Below are examples of two useful fixtures.
+
+<!-- #region -->
+```python
+# contents of conftest.py
+
+import os
+import tempfile
+
+import pytest
+
+@pytest.fixture()
+def cleandir():
+ """ This fixture will use the stdlib `tempfile` module to
+ change the current working directory to a tmp-dir for the
+ duration of the test.
+
+ Afterwards, the test session returns to its previous working
+ directory, and the temporary directory and its contents
+ will be automatically deleted.
+
+ Yields
+ ------
+ str
+ The name of the temporary directory."""
+ with tempfile.TemporaryDirectory() as tmpdirname:
+ old_dir = os.getcwd()
+ os.chdir(tmpdirname)
+ yield tmpdirname
+ os.chdir(old_dir)
+
+
+@pytest.fixture()
+def dummy_email():
+ """ This fixture will simply have pytest pass the string:
+ 'dummy.email@plymi.com'
+ to any test-function that has the parameter name `dummy_email` in
+ its signature.
+ """
+ return "dummy.email@plymi.com"
+```
+<!-- #endregion -->
+
+The first one, `cleandir`, can be used in conjunction with tests that need to write files.
+We don't want our tests to leave behind files on our machines; the `cleandir` fixture will ensure that our tests will write files to a temporary directory that will be deleted once the test is complete.
+
+Second is a simple fixture called `dummy_email`.
+Suppose that our project needs to interact with a specific email address, suppose it's `dummy.email@plymi.com`, and that we have several tests that need access to this address.
+This fixture will pass this address to any test function that has the parameter name `dummy_email` in its signature.
+
+A reference implementation of `conftest.py` in our project can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/conftest.py).
+Several reference tests that make use of these fixtures can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/test_using_fixtures.py).
+
+
## Links to Official Documentation
- [pytest](https://docs.pytest.org/en/latest/)
@@ -526,6 +589,7 @@ def test_all_combinations(x, y):
- [Testing in VSCode](https://code.visualstudio.com/docs/python/testing)
- [Assertion introspection](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details)
- [Parameterizing tests](https://docs.pytest.org/en/latest/parametrize.html)
+- [Fixtures](https://docs.pytest.org/en/latest/fixture.html#fixture)
## Reading Comprehension Solutions
From ce7b392ed98cfdf49297d6f7a804d6cf0bc8716f Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Dec 2019 15:21:35 -0500
Subject: [PATCH 049/152] fix wording and add fixture tests
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
Python/Module6_Testing/Pytest.md | 64 +++++++++++++++++++++-
2 files changed, 62 insertions(+), 4 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index befffc08..72f4ef26 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -366,7 +366,7 @@ With this in hand, we should take stock of the work and challenges that lie in o
It is necessary that we evolve beyond manual testing.
There are multiple facets to this observation.
First, we must learn how to organize our test functions into a test suite that can be run in one fowl swoop.
-Next, it will become increasingly apparent that a test function often contain large amounts of redundant code, shared across its litany of assertions.
+Next, it will become increasingly apparent that a test function often contain large amounts of redundant code shared across its litany of assertions.
We will want to "parametrize" our tests to distill them down to their most concise and functional forms.
Finally, and most importantly, it may already evident that the process of contriving known inputs and outputs to use in our tests is a highly manual and tedious process; furthermore, it is a process that will become increasingly cumbersome as our source code becomes more sophisticated.
To combat this, we will seek out alternative, powerful testing methodologies, including property-based testing.
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index db5b498a..5400cd95 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -15,7 +15,7 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
:description: Topic: Writing tests for your code, Difficulty: Easy, Category: Section
- :keywords: test, automated, pytest, parametrize, fixture, suite
+ :keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
<!-- #endraw -->
# The pytest Framework
@@ -29,8 +29,17 @@ for instance, it will enrich the assertions in our tests to produce verbose, inf
Furthermore it provides valuable means for enhancing our tests via mechanisms like fixtures and parameterizing decorators.
Ultimately, all of this functionality helps to eliminate manual and redundant aspects of the testing process.
-Note: It can be useful to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments) for the sake of this lesson, so that we can work through this material starting from a blank slate.
-Be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
+
+
+<div class="alert alert-warning">
+
+**Note**
+
+It can be useful to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments) for the sake of this lesson, so that we can work through this material starting from a blank slate.
+If you do create a new conda environment, be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
+</div>
+
+
Let's install pytest. Installing from [the conda-forge channel](https://conda-forge.org/) will install the most up-to-date version of pytest. In a terminal where conda can be accessed, run:
@@ -570,6 +579,7 @@ def dummy_email():
```
<!-- #endregion -->
+<!-- #region -->
The first one, `cleandir`, can be used in conjunction with tests that need to write files.
We don't want our tests to leave behind files on our machines; the `cleandir` fixture will ensure that our tests will write files to a temporary directory that will be deleted once the test is complete.
@@ -580,6 +590,54 @@ This fixture will pass this address to any test function that has the parameter
A reference implementation of `conftest.py` in our project can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/conftest.py).
Several reference tests that make use of these fixtures can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/test_using_fixtures.py).
+Let's create a file `tests/test_using_fixtures.py`, and write some tests that put these fixtures to use:
+
+```python
+# contents of test_using_fixtures.py
+import pytest
+
+# When run, this test will be executed within a
+# temporary directory that will automatically be
+# deleted - along with all of its contents - once
+# the test ends.
+#
+# Thus we can have this test write a file, and we
+# need not worry about having it clean up after itself.
+@pytest.mark.usefixtures("cleandir")
+def test_writing_a_file():
+ with open("a_text_file.txt", mode="w") as f:
+ f.write("hello world")
+
+ with open("a_text_file.txt", mode="r") as f:
+ file_content = f.read()
+
+ assert file_content == "hello world"
+
+
+# We can use the `dummy_email` fixture to provide
+# the same email address to many tests. In this
+# way, if we need to change the email address, we
+# can simply update the fixture and all of the tests
+# will be affected by the update.
+#
+# Note that we don't need to use a decorator here.
+# pytest is smart, and will see that the parameter-name
+# `dummy_email` matches the name of our fixture. It will
+# thus call these tests using the value returned by our
+# fixture
+
+def test_email1(dummy_email):
+ assert "dummy" in dummy_email
+
+
+def test_email2(dummy_email):
+ assert "plymi" in dummy_email
+
+
+def test_email3(dummy_email):
+ assert ".com" in dummy_email
+```
+<!-- #endregion -->
## Links to Official Documentation
From 4e4be515ce42fd1be1da5e20ff5ec14c21a6fb90 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Dec 2019 15:27:11 -0500
Subject: [PATCH 050/152] conf.py -> conftest.py
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index caf5eaf7..d338cf47 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -424,7 +424,7 @@ Carrying on, we will want to create a setup-script, `setup.py`, *in the same dir
|-- calibration.py
|-- config.py
- tests/ # test-suite for `face_detection` package (to be run using pytest)
- |-- conf.py # optional configuration file for pytest
+ |-- conftest.py # optional configuration file for pytest
|-- test_utils.py
|-- test_database.py
|-- test_model.py
From 9f02917c019a02293fc43323d4435e691c083203 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:33:58 -0500
Subject: [PATCH 051/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 72f4ef26..e77ed1ea 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -55,7 +55,7 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> Our test suite provides us with a simple and incisive way to dive back into our work.
> It will point us to any potential incompatibilities that have accumulated over time.
> It also provides us with a large collection of detailed use-cases of our code;
-> we can read through our tests remind ourselves of the inner-workings of our project.
+> we can read through our tests and remind ourselves of the inner-workings of our project.
**It will inform the design and usability of our project for the better:**
From 25f8a82ceb0b2f8941e0baf47357dfc9c2891ceb Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:34:15 -0500
Subject: [PATCH 052/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index e77ed1ea..b5e2f5e7 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -69,7 +69,7 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> Having a healthy test suite lowers the barrier to entry for a project.
> A contributor can rely on our project's tests to quickly check to see if their changes to our code have broken the project or changed any of its behavior in unexpected ways.
-This all sounds great, but where do we even start the process writing a test suite?
+This all sounds great, but where do we even start the process of writing a test suite?
Let's begin by seeing what constitutes a basic test function.
<!-- #endregion -->
From d34a21bb02ece24b1756097645f83d324372dda2 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:34:28 -0500
Subject: [PATCH 053/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index b5e2f5e7..bb1a9884 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -145,7 +145,7 @@ def merge_max_mappings(dict1, dict2):
return merged
```
-As always, it is useful for us follow along with this material in a Jupyter notebook.
+As always, it is useful for us to follow along with this material in a Jupyter notebook.
We ought to take time to define these functions and run inputs through them to make sure that we understand what they are doing.
Testing code that we don't understand is a lost cause!
<!-- #endregion -->
From 4d932e608c6d7857c06ae52caddee30f8880f166 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:34:42 -0500
Subject: [PATCH 054/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index bb1a9884..fbcdcbb6 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -289,7 +289,7 @@ a_string = "abcdef"
Write two assertion statements with the respective behaviors:
- asserts that `a_list` is _not_ empty
-- asserts that the number of vowels in `a_string` is less than `a_number`; include and error message that prints the actual number of vowels
+- asserts that the number of vowels in `a_string` is less than `a_number`; include an error message that prints the actual number of vowels
</div>
<!-- #endregion -->
From f11eddbe72dfe9d1547a28a8b137dcc294daed2e Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:34:52 -0500
Subject: [PATCH 055/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index fbcdcbb6..806283f0 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -296,7 +296,7 @@ Write two assertion statements with the respective behaviors:
<!-- #region -->
#### What is the Purpose of an Assertion?
-In our code, an assertion should be used as _a statement that is true unless there is a bug our code_.
+In our code, an assertion should be used as _a statement that is true unless there is a bug in our code_.
It is plain to see that the assertions in `test_count_vowels_basic` fit this description.
However, it can also be useful to include assertions within our source code itself.
For instance, we know that `count_vowels` should always return a non-negative integer for the vowel-count, and that it is illogical for this count to exceed the number of characters in the input string.
From 9e1704d42f65c0557b75a375da5627f6c7f08246 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:35:03 -0500
Subject: [PATCH 056/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 806283f0..48a13b17 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -368,7 +368,7 @@ There are multiple facets to this observation.
First, we must learn how to organize our test functions into a test suite that can be run in one fowl swoop.
Next, it will become increasingly apparent that a test function often contain large amounts of redundant code shared across its litany of assertions.
We will want to "parametrize" our tests to distill them down to their most concise and functional forms.
-Finally, and most importantly, it may already evident that the process of contriving known inputs and outputs to use in our tests is a highly manual and tedious process; furthermore, it is a process that will become increasingly cumbersome as our source code becomes more sophisticated.
+Finally, and most importantly, it may already be evident that the process of contriving known inputs and outputs to use in our tests is a highly manual and tedious process; furthermore, it is a process that will become increasingly cumbersome as our source code becomes more sophisticated.
To combat this, we will seek out alternative, powerful testing methodologies, including property-based testing.
From e041c1de82e94a515b64ebcfac2846c7f8142e04 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:35:14 -0500
Subject: [PATCH 057/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 5400cd95..cbb9b2dc 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -98,7 +98,7 @@ project_dir/ # the "parent directory" houses our source code, tests, and all
A reference implementation of this package can be found [in this GitHub repository](https://github.com/rsokl/plymi_mod6).
Populate the `basic_functions.py` file with the two functions that we were using as our source code in the previous section: `count_vowels` and `merge_max_mappings`.
-In in the `numpy_functions.py` module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
+In the `numpy_functions.py` module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
Don't forget to include `import numpy as np` in your script in accordance with how `pairwise_dists` calls NumPy functions.
We have arranged these functions so that they can be imported from the `basic_functions` module and the `numpy_functions` module, respectively, which reside in our `plymi_mod6` package.
From 58152480c1de5b5372c3688a076386b7b893443d Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:35:22 -0500
Subject: [PATCH 058/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index cbb9b2dc..120797e3 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -448,7 +448,7 @@ def test_function(<param-name>):
```
We will often have tests that require multiple parameters.
-The general form for creating the the parameterization decorator for $N$ parameters,
+The general form for creating the parameterization decorator for $N$ parameters,
each of which assume $J$ values, is:
```python
From 63ccf3a4be46cefa6904bfcf0110de0f593289a5 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:35:55 -0500
Subject: [PATCH 059/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 48a13b17..d2c89e9d 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -333,7 +333,7 @@ They create misdirection in the bug-finding process and can mask problems with o
**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
-A practical note: we ought to mutate our function in a way that is trivial to undo. We can use of code-comments towards this end.
+A practical note: we ought to mutate our function in a way that is trivial to undo. We can make use of code-comments towards this end.
All [IDEs](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) have the ability to "block-comment" selected code.
In order to block-comment code in a Jupyter notebook code cell, highlight the lines of code and press `CTRL + /`.
The same key-combination will also un-comment a highlighted block of commented code.
From 4b2f2c46fdcb33fd833d2b9e1c1d0209f667e89f Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:36:20 -0500
Subject: [PATCH 060/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index d2c89e9d..19affd64 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -469,7 +469,7 @@ AssertionError:
> However, recall that calling `bool` on any sequence (list, tuple, string, etc.) will return `False` if the sequence is empty.
> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - that `bool` will be called on whatever the provided expression is.
-Assert that the number of vowels in `a_string` is fewer than `a_number`; include and error message that prints the actual number of vowels:
+Assert that the number of vowels in `a_string` is fewer than `a_number`; include an error message that prints the actual number of vowels:
```python
>>> assert count_vowels(a_string) < a_number, f"Number of vowels, {count_vowels(a_string)}, exceeds {a_number}"
From bf8363ba92b7c5e9dc9efb09585c2cf21c558488 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:36:30 -0500
Subject: [PATCH 061/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 120797e3..b7b048d2 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -81,7 +81,7 @@ Before proceeding any further, we should reread the material presented in [Modul
This material serves as the foundation for this section.
### Organizing our Source Code
-Let's create the a Python package, which we will call `plymi_mod6`, with the following directory structure:
+Let's create a Python package, which we will call `plymi_mod6`, with the following directory structure:
```
project_dir/ # the "parent directory" houses our source code, tests, and all other relevant files
From 398f566964f4ed84ec48bf8070724960db52b000 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:37:00 -0500
Subject: [PATCH 062/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index b7b048d2..9e6db881 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -142,7 +142,7 @@ Now, we should be able to start a python console, IPython console, or Jupyter no
## Populating and Running Our Test Suite
pytest's [system for "test discovery"](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery) is quite simple:
-pytest need only be pointed to a directory with files named `test_*.py` in it, and it will find all of the functions in these files _whose names start with the word "test"_ and it will run all such functions.
+pytest need only be pointed to a directory with files named `test_*.py` in it, and it will find all of the functions in these files _whose names start with the word "test"_ and will run all such functions.
Thus, let's populate the file ``test_basic_functions.py`` with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module:
From 9bf072fa66187edd0b688955ad6d93c0b683d37e Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:37:11 -0500
Subject: [PATCH 063/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 9e6db881..73bf04cd 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -215,7 +215,7 @@ tests\test_basic_numpy.py . [100%]
This output indicates that three test-functions were found across two files and that all of the tests "passed"; i.e. the functions ran without raising any errors.
The first two tests are located in `tests/test_basic_functions.py`; the two dots indicate that two functions were run, and the `[66%]` indicator simply denotes that the test-suite is 66% (two-thirds) complete.
-The proceeding reading comprehension problem will lead us to see what looks like for pytest to report a failing test.
+The following reading comprehension problem will lead us to see what looks like for pytest to report a failing test.
<div class="alert alert-info">
From 06882ef3a27f873afcb0605e03e505ba6c89a429 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:37:23 -0500
Subject: [PATCH 064/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 73bf04cd..54c66e81 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -420,7 +420,7 @@ Furthermore, the four assertions are now being run independently from one anothe
<!-- #region -->
#### Decorators
-The the syntax used to parameterize this test may look alien to us, we have yet to encounter this construct thus far.
+The syntax used to parameterize this test may look alien to us, as we have yet to encounter this construct thus far.
`pytest.mark.parameterize(...)` is a _decorator_ - an object that is used to "wrap" a function in order to transform its behavior.
The `pytest.mark.parameterize(...)` decorator wraps our test function so that pytest can call it multiple times, once for each parameter value.
The `@` character, in this context, denotes the application of a decorator:
From 2b6f6a66e19c4f00455b6233aeca9fa5749dcf09 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:37:36 -0500
Subject: [PATCH 065/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 54c66e81..72a59c4c 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -689,7 +689,7 @@ tests\test_basic_functions.py:40: AssertionError
========================= 1 failed, 3 passed in 0.07s =========================
```
-> Four tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests in _test_basic_functions_ passed and the third test failed.
+> Four tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests in `test_basic_functions` passed and the third test failed.
> It then indicates which test failed, and specifically that the assertion was false because a length-2 list cannot be equal to a length-3 list.
<!-- #endregion -->
From 0f3e5a1e68f6f03a348420787e17b77853afda33 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:37:53 -0500
Subject: [PATCH 066/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: Petar Griggs <petargriggs@gmail.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 72a59c4c..b095b034 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -196,7 +196,7 @@ Finally, add a dummy test - a test function that will always pass - to `test_bas
We will replace this with a useful test later.
Without further ado, let's run our test suite! In our terminal, with the appropriate conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
-Following output should appear:
+The following output should appear:
<!-- #endregion -->
```
From 8dfda2d10233f279b469744860e09da7b210a4dc Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 4 Jan 2020 09:41:51 -0500
Subject: [PATCH 067/152] Update
Python/Module5_OddsAndEnds/Modules_and_Packages.md
Co-Authored-By: Eddie Penta <ecp4224@gmail.com>
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index d338cf47..b2f8b420 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -448,7 +448,7 @@ setup(
packages=find_packages(exclude=["tests", "tests.*"]),
)
```
-The expression `exclude=["tests", "tests.*"]` is included to ensure that the code in your test-suite is not included in the installation of `face_detection`.
+The `exclude` expression is used to ensure that specific directories or files are not included in the installation of `face_detection`. We use `exclude=["tests", "tests.*"]` to avoid installing the test-suite along side `face_detection`
<!-- #endregion -->
If you read through the additional materials linked above, you will see that there are many more fields of optional information that can be provided in this setup script, such as the author name, any installation requirements that the package has, and more.
From 871ba15454b251a87e7de89a60f8872e4bee944d Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 09:47:50 -0500
Subject: [PATCH 068/152] add python_requires and change setup.py commands to
pip
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index b2f8b420..e359b39a 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -446,6 +446,7 @@ setup(
name="face_detection",
version="1.0.0",
packages=find_packages(exclude=["tests", "tests.*"]),
+ python_requires=">=3.5",
)
```
The `exclude` expression is used to ensure that specific directories or files are not included in the installation of `face_detection`. We use `exclude=["tests", "tests.*"]` to avoid installing the test-suite along side `face_detection`
@@ -456,7 +457,7 @@ If you read through the additional materials linked above, you will see that the
Armed with this script, we are ready to install our package locally on our machine! In your terminal, navigate to the directory containing this setup script and your package that it being installed. Run
```shell
-python setup.py install
+pip install .
```
and voilà, your package `face_detection` will have been installed to site-packages. You are now free to import this package from any directory on your machine. In order to uninstall this package from your machine execute the following from your terminal:
@@ -468,7 +469,7 @@ pip uninstall face_detection
One final but important detail. The installed version of your package will no longer "see" the source code. That is, if you go on to make any changes to your code, you will have to uninstall and reinstall your package before your will see the effects system-wide. Instead you can install your package in "development mode", such that a symbolic link to your source code is placed in your site-packages. Thus any changes that you make to your code will immediately be reflected in your system-wide installation. Thus, instead of running `python setup.py install`, execute the following to install a package in develop mode:
```shell
-python setup.py develop
+pip install --editable .
```
From a46350e17c8031b1db7c503f637378b12360733f Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 10:10:25 -0500
Subject: [PATCH 069/152] formalizing -> automating
---
Python/Module6_Testing/Intro_to_Testing.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 19affd64..330e8a09 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -31,8 +31,8 @@ The fact of the matter is that it is intuitive for most people to test their cod
After writing, say, a new function, it is only natural to contrive an input to feed it, and to check that the function returns the output that we expected.
To the extent that one would want to see evidence that their code works, we need not motivate the importance of testing.
-Less obvious are the massive benefits that we stand to gain from formalizing this testing process.
-And by "formalizing", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
+Less obvious are the massive benefits that we stand to gain from automating this testing process.
+And by "automating", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
We will accumulate these test functions into a "test suite" that we can run quickly and repeatedly.
There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing a robust test suite:
From a6ff50f3e4190cd1b2509dab6b8b05fe54cac423 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 10:28:47 -0500
Subject: [PATCH 070/152] PNG -> png
---
Python/Module6_Testing/Pytest.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index b095b034..4574a8f7 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -258,7 +258,7 @@ For example, in the following image, the green "play button" allows us to run `t
<!-- #raw raw_mimetype="text/html" -->
<div style="text-align: center">
<p>
-<img src="../_images/individual_test.PNG" alt="Running an individual test in PyCharm" width="600">
+<img src="../_images/individual_test.png" alt="Running an individual test in PyCharm" width="600">
</p>
</div>
<!-- #endraw -->
@@ -270,7 +270,7 @@ In the following image, we see that `test_version` is failing - we can click on
<!-- #raw raw_mimetype="text/html" -->
<div style="text-align: center">
<p>
-<img src="../_images/test_tree_view.PNG" alt="Viewing an enchanced tree-view of your test suite" width="600">
+<img src="../_images/test_tree_view.png" alt="Viewing an enhanced tree-view of your test suite" width="600">
</p>
</div>
<!-- #endraw -->
From fb2f5487241916f6a9f08ced591ea7d3cdf566f1 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 10:54:19 -0500
Subject: [PATCH 071/152] add disable assertions
---
Python/Module6_Testing/Intro_to_Testing.md | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 330e8a09..133268b9 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -323,6 +323,25 @@ Rather, it is meant to assert that our own internal logic holds true.
Admittedly, the `count_vowels` function is simple enough that the inclusion of this assertion is rather pedantic.
That being said, as we write increasingly sophisticated code, we will find that this sort of assertion will help us catch bad internal logic and oversights within our code base.
We will also see that keen use of assertions can make it much easier for us to write good tests.
+
+<div class="alert alert-warning">
+
+**Disabling Assertions**:
+
+Python code can be run in an "optimized" mode such that *all assertions are skipped by the Python interpreter during execution*.
+This can be achieved by specifying the command line option `-O` (the letter "O", not zero), e.g.:
+
+```shell
+python -O my_code.py
+```
+
+or by setting the `PYTHONOPTIMIZE` [environment variable](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE).
+
+The idea here is that we may want assertions within our source code to perform expensive checks to guarantee internal consistency within our code, and that we want the ability to forgo these checks when we are no longer debugging our code.
+Because they can be skipped in this way, *assertions should never be used for practical error handling*.
+
+</div>
+
<!-- #endregion -->
## Testing Our Tests
@@ -375,6 +394,7 @@ To combat this, we will seek out alternative, powerful testing methodologies, in
## Links to Official Documentation
- [The assert statement](https://docs.python.org/3/reference/simple_stmts.html?highlight=assert#the-assert-statement)
+- [PYTHONOPTIMIZE environment variable](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE)
## Reading Comprehension Solutions
From a95409ff55a48ec7ec99d5b125917ebadc1c43dd Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 11:17:14 -0500
Subject: [PATCH 072/152] add discussion of mutation testing
---
Python/Module6_Testing/Intro_to_Testing.md | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 133268b9..f6dd5de7 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -375,6 +375,17 @@ Try breaking the function such that it always merges in values from `dict2`, eve
+<div class="alert alert-warning">
+
+**Mutation Testing**:
+
+There is an entire subfield of automated testing known as ["mutation testing"](https://en.wikipedia.org/wiki/Mutation_testing), where tools like [Cosmic Ray](https://cosmic-ray.readthedocs.io/en/latest/index.html) are used to make temporary, incisive mutations to your source code - like change a `+` to a `-` or change a `1` to a `-1` - and then run your test suite.
+The idea here is that such mutations *ought to cause one or more of your tests to fail*.
+A mutation that fails to trigger at least one test failure is likely an indicator that your tests could stand to be more robust.
+
+</div>
+
+
## Our Work, Cut Out
We see now that the concept of a "test function" isn't all that fancy.
From fe51dbf30067c5b362adb1cafa12bd11da34ced2 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 11:20:54 -0500
Subject: [PATCH 073/152] foul swoop is not a phrase... it is fell swoop
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index f6dd5de7..25b4106a 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -395,7 +395,7 @@ With this in hand, we should take stock of the work and challenges that lie in o
It is necessary that we evolve beyond manual testing.
There are multiple facets to this observation.
-First, we must learn how to organize our test functions into a test suite that can be run in one fowl swoop.
+First, we must learn how to organize our test functions into a test suite that can be run in one fell swoop.
Next, it will become increasingly apparent that a test function often contain large amounts of redundant code shared across its litany of assertions.
We will want to "parametrize" our tests to distill them down to their most concise and functional forms.
Finally, and most importantly, it may already be evident that the process of contriving known inputs and outputs to use in our tests is a highly manual and tedious process; furthermore, it is a process that will become increasingly cumbersome as our source code becomes more sophisticated.
From 9c8611d4715feadc871c632e5be42c7e9f637883 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 11:27:14 -0500
Subject: [PATCH 074/152] add caveat about mutations
---
Python/Module6_Testing/Intro_to_Testing.md | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 25b4106a..befb448c 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -381,7 +381,9 @@ Try breaking the function such that it always merges in values from `dict2`, eve
There is an entire subfield of automated testing known as ["mutation testing"](https://en.wikipedia.org/wiki/Mutation_testing), where tools like [Cosmic Ray](https://cosmic-ray.readthedocs.io/en/latest/index.html) are used to make temporary, incisive mutations to your source code - like change a `+` to a `-` or change a `1` to a `-1` - and then run your test suite.
The idea here is that such mutations *ought to cause one or more of your tests to fail*.
-A mutation that fails to trigger at least one test failure is likely an indicator that your tests could stand to be more robust.
+A mutation that fails to trigger at least one test failure is likely an indicator that your tests could stand to be more robust.
+
+Automated mutation testing tools might be a bit too "heavy duty" at this point in our testing journey, but they are great to keep in mind.
</div>
From 283223221712766497ac1b19a035395153638872 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 11:45:01 -0500
Subject: [PATCH 075/152] add comments to fixture
---
Python/Module6_Testing/Pytest.md | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 4574a8f7..d044b64e 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -114,7 +114,7 @@ setup(
author="Your Name",
description="A template Python package for learning about testing",
install_requires=["numpy >= 1.10.0"],
- tests_require=["pytest", "hypothesis"],
+ tests_require=["pytest>=5.3", "hypothesis?=5.0"],
python_requires=">=3.6",
)
```
@@ -125,7 +125,7 @@ Finally, let's install our package locally [in development mode](https://www.pyt
Navigate to the directory containing `setup.py` and run:
```shell
-python setup.py develop
+pip install --editable .
```
Now, we should be able to start a python console, IPython console, or Jupyter notebook in any directory and import our package:
@@ -562,10 +562,13 @@ def cleandir():
str
The name of the temporary directory."""
with tempfile.TemporaryDirectory() as tmpdirname:
- old_dir = os.getcwd()
- os.chdir(tmpdirname)
- yield tmpdirname
- os.chdir(old_dir)
+ old_dir = os.getcwd() # get current working directory (cwd)
+ os.chdir(tmpdirname) # change cwd to the temp-directory
+ yield tmpdirname # yields control to the test to be run
+ os.chdir(old_dir) # restore the cwd to the original directory
+ # Leaving the context manager will prompt the deletion of the
+ # temporary directory and its contents. This cleanup will be
+ # triggered even if errors were raised during the test.
@pytest.fixture()
From 6355e4aebac1468a105b09e957bb2310e2ca0b50 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 11:52:46 -0500
Subject: [PATCH 076/152] Peter edits
---
Python/Module6_Testing/Intro_to_Testing.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index befb448c..6e3a285a 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -62,7 +62,7 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> Although it may not be obvious from the outset, writing testable code leads to writing better code.
> This is, in part, because the process of writing tests gives us the opportunity to actually _use_ our code under varied circumstances.
-> The process of writing tests will help us suss out cumbersome function interfaces, brittle statefulness, and redundant capabilities in our code. Ultimately, if _we_ find it frustrating to use our code within our tests, then surely others will find the code frustrating to use in applied settings.
+> The process of writing tests will help us suss out bad design decisions and redundant capabilities in our code. Ultimately, if _we_ find it frustrating to use our code within our tests, then surely others will find the code frustrating to use in applied settings.
**It makes it easier for others to contribute to a project:**
@@ -398,7 +398,7 @@ With this in hand, we should take stock of the work and challenges that lie in o
It is necessary that we evolve beyond manual testing.
There are multiple facets to this observation.
First, we must learn how to organize our test functions into a test suite that can be run in one fell swoop.
-Next, it will become increasingly apparent that a test function often contain large amounts of redundant code shared across its litany of assertions.
+Next, it will become increasingly apparent that a test function often contains large amounts of redundant code shared across its litany of assertions.
We will want to "parametrize" our tests to distill them down to their most concise and functional forms.
Finally, and most importantly, it may already be evident that the process of contriving known inputs and outputs to use in our tests is a highly manual and tedious process; furthermore, it is a process that will become increasingly cumbersome as our source code becomes more sophisticated.
To combat this, we will seek out alternative, powerful testing methodologies, including property-based testing.
From 8059abb798bd52833f45571693fb731fffc88850 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 12:03:07 -0500
Subject: [PATCH 077/152] improve wording to not say fail so many times
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 6e3a285a..13647118 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -381,7 +381,7 @@ Try breaking the function such that it always merges in values from `dict2`, eve
There is an entire subfield of automated testing known as ["mutation testing"](https://en.wikipedia.org/wiki/Mutation_testing), where tools like [Cosmic Ray](https://cosmic-ray.readthedocs.io/en/latest/index.html) are used to make temporary, incisive mutations to your source code - like change a `+` to a `-` or change a `1` to a `-1` - and then run your test suite.
The idea here is that such mutations *ought to cause one or more of your tests to fail*.
-A mutation that fails to trigger at least one test failure is likely an indicator that your tests could stand to be more robust.
+A mutation that does not trigger at least one test failure is likely an indicator that your tests could stand to be more robust.
Automated mutation testing tools might be a bit too "heavy duty" at this point in our testing journey, but they are great to keep in mind.
From cb3d9527216bcf812fb73e749ce4516d03ea4362 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Jan 2020 12:14:57 -0500
Subject: [PATCH 078/152] fix type: >? should be >=
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index d044b64e..20ab35c1 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -114,7 +114,7 @@ setup(
author="Your Name",
description="A template Python package for learning about testing",
install_requires=["numpy >= 1.10.0"],
- tests_require=["pytest>=5.3", "hypothesis?=5.0"],
+ tests_require=["pytest>=5.3", "hypothesis>=5.0"],
python_requires=">=3.6",
)
```
From 8b02be7aaafe38847635b335e93d81b9cd76a54c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 5 Jan 2020 10:47:16 -0500
Subject: [PATCH 079/152] Change module title to Testing OUR Code; add
hypothesis section
---
Python/Module6_Testing/Hypothesis.md | 753 +++++++++++++++++++++++++++
Python/module_6.rst | 5 +-
2 files changed, 756 insertions(+), 2 deletions(-)
create mode 100644 Python/Module6_Testing/Hypothesis.md
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
new file mode 100644
index 00000000..20ab35c1
--- /dev/null
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -0,0 +1,753 @@
+---
+jupyter:
+ jupytext:
+ text_representation:
+ extension: .md
+ format_name: markdown
+ format_version: '1.2'
+ jupytext_version: 1.3.0
+ kernelspec:
+ display_name: Python 3
+ language: python
+ name: python3
+---
+
+<!-- #raw raw_mimetype="text/restructuredtext" -->
+.. meta::
+ :description: Topic: Writing tests for your code, Difficulty: Easy, Category: Section
+ :keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
+<!-- #endraw -->
+
+# The pytest Framework
+
+Thus far, our process for running tests has been an entirely manual one. It is time for us to arrange our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
+We will begin by reorganizing our source code to create an installable [Python package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
+We will then learn how to structure and run a test suite for this Python package, using pytest.
+
+The pytest framework does much more than just run tests;
+for instance, it will enrich the assertions in our tests to produce verbose, informative error messages.
+Furthermore it provides valuable means for enhancing our tests via mechanisms like fixtures and parameterizing decorators.
+Ultimately, all of this functionality helps to eliminate manual and redundant aspects of the testing process.
+
+
+
+<div class="alert alert-warning">
+
+**Note**
+
+It can be useful to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments) for the sake of this lesson, so that we can work through this material starting from a blank slate.
+If you do create a new conda environment, be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
+</div>
+
+
+
+Let's install pytest. Installing from [the conda-forge channel](https://conda-forge.org/) will install the most up-to-date version of pytest. In a terminal where conda can be accessed, run:
+
+```shell
+conda install -c conda-forge pytest
+```
+
+Or, pytest is installable via pip:
+
+```shell
+pip install pytest
+```
+
+
+<div class="alert alert-warning">
+
+**Regarding Alternative Testing Frameworks** (a note from the author of PLYMI):
+
+When sifting through tutorials, blogs, and videos about testing in Python, it is common to see `pytest` presented alongside, and on an equal footing with, the alternative testing frameworks: `nose` and `unittest`.
+This strikes me as... bizarre.
+
+`unittest` is the testing framework that comes with the Python standard library.
+As a test runner, its design is clunky, archaic, and, ironically, un-pythonic.
+While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) provides extremely valuable functionality for advanced testing, all of its functionality can be leveraged while using pytest as your testing framework.
+
+`nose`, which simply extends the functionality of `unittest`, **is no longer being maintained**.
+There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project by comparison to `pytest`.
+As of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads.
+
+The takeaway here is that, when it comes to picking a testing framework for Python, `pytest` is the clear choice.
+Any discussion that you come across to the contrary is likely outdated.
+</div>
+
+<!-- #region -->
+## Creating a Python Package with Tests
+
+It's time to create a proper test suite.
+Before proceeding any further, we should reread the material presented in [Module 5 - Import: Modules and Packages](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html) and recall the essentials of import statements, modules, and Python packages.
+This material serves as the foundation for this section.
+
+### Organizing our Source Code
+Let's create a Python package, which we will call `plymi_mod6`, with the following directory structure:
+
+```
+project_dir/ # the "parent directory" houses our source code, tests, and all other relevant files
+ - setup.py # script responsible for installing `plymi_mod6` package
+ - plymi_mod6/ # directory containing source code of `plymi_mod6` package
+ |-- __init__.py
+ |-- basic_functions.py
+ |-- numpy_functions.py
+ - tests/ # test-suite for `plymi_mod6` package (to be run using pytest)
+ |-- conftest.py # optional configuration file for pytest
+ |-- test_basic_functions.py
+ |-- test_numpy_functions.py
+```
+
+A reference implementation of this package can be found [in this GitHub repository](https://github.com/rsokl/plymi_mod6).
+Populate the `basic_functions.py` file with the two functions that we were using as our source code in the previous section: `count_vowels` and `merge_max_mappings`.
+In the `numpy_functions.py` module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
+Don't forget to include `import numpy as np` in your script in accordance with how `pairwise_dists` calls NumPy functions.
+
+We have arranged these functions so that they can be imported from the `basic_functions` module and the `numpy_functions` module, respectively, which reside in our `plymi_mod6` package.
+Let's fill out our `setup.py` script and install this package so that we can import it regardless of our current working directory. The content of `setup.py` will be:
+
+```python
+from setuptools import find_packages, setup
+
+setup(
+ name="plymi_mod6",
+ packages=find_packages(exclude=["tests", "tests.*"]),
+ version="1.0.0",
+ author="Your Name",
+ description="A template Python package for learning about testing",
+ install_requires=["numpy >= 1.10.0"],
+ tests_require=["pytest>=5.3", "hypothesis>=5.0"],
+ python_requires=">=3.6",
+)
+```
+
+This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6. Additionally, we will require pytest and hypothesis for running tests; the Hypothesis library will be introduced in a later section.
+
+Finally, let's install our package locally [in development mode](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Installing-Your-Own-Python-Package).
+Navigate to the directory containing `setup.py` and run:
+
+```shell
+pip install --editable .
+```
+
+Now, we should be able to start a python console, IPython console, or Jupyter notebook in any directory and import our package:
+
+```python
+# checking that we can import our `plymi_mod6` package
+>>> from plymi_mod6.basic_functions import count_vowels
+>>> count_vowels("Happy birthday", include_y=True)
+5
+```
+<!-- #endregion -->
+
+<!-- #region -->
+## Populating and Running Our Test Suite
+
+pytest's [system for "test discovery"](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery) is quite simple:
+pytest need only be pointed to a directory with files named `test_*.py` in it, and it will find all of the functions in these files _whose names start with the word "test"_ and will run all such functions.
+
+Thus, let's populate the file ``test_basic_functions.py`` with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module:
+
+```python
+# The contents of test_basic_functions.py
+
+# we must import the functions we are testing
+from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
+
+
+def test_count_vowels_basic():
+ # test basic strings with uppercase and lowercase letters
+ assert count_vowels("aA bB yY", include_y=False) == 2
+ assert count_vowels("aA bB yY", include_y=True) == 4
+
+ # test empty strings
+ assert count_vowels("", include_y=False) == 0
+ assert count_vowels("", include_y=True) == 0
+
+
+def test_merge_max_mappings():
+ # test documented behavior
+ dict1 = {"a": 1, "b": 2}
+ dict2 = {"b": 20, "c": -1}
+ expected = {'a': 1, 'b': 20, 'c': -1}
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test empty dict1
+ dict1 = {}
+ dict2 = {"a": 10.2, "f": -1.0}
+ expected = dict2
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test empty dict2
+ dict1 = {"a": 10.2, "f": -1.0}
+ dict2 = {}
+ expected = dict1
+ assert merge_max_mappings(dict1, dict2) == expected
+
+ # test both empty
+ dict1 = {}
+ dict2 = {}
+ expected = {}
+ assert merge_max_mappings(dict1, dict2) == expected
+
+```
+
+As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our functions are in the same namespace as our tests.
+A reference implementation of `test_basic_functions.py` can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
+Finally, add a dummy test - a test function that will always pass - to `test_basic_numpy.py`.
+We will replace this with a useful test later.
+
+Without further ado, let's run our test suite! In our terminal, with the appropriate conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
+The following output should appear:
+<!-- #endregion -->
+
+```
+$ pytest tests/
+============================= test session starts =============================
+platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
+rootdir: C:\Users\plymi_user\plymi_root_dir
+collected 3 items
+
+tests\test_basic_functions.py .. [ 66%]
+tests\test_basic_numpy.py . [100%]
+
+============================== 3 passed in 0.04s ==============================
+```
+
+
+This output indicates that three test-functions were found across two files and that all of the tests "passed"; i.e. the functions ran without raising any errors.
+The first two tests are located in `tests/test_basic_functions.py`; the two dots indicate that two functions were run, and the `[66%]` indicator simply denotes that the test-suite is 66% (two-thirds) complete.
+The following reading comprehension problem will lead us to see what looks like for pytest to report a failing test.
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Running a Test Suite**
+
+Temporarily add a new "broken" test to `tests/test_basic_functions.py`.
+The name that you give this test should adhere to pytest's simple rules for test-discovery.
+Design the test function so that is sure to fail when it is run.
+
+Rerun your test suite and compare its output to what you saw before - is it easy to identify which test failed and what caused it to fail?
+Make sure to remove this function from your test suite once you are finished answering this question.
+
+</div>
+
+
+
+We can also direct pytest to run the tests in a specific .py file. E.g. executing:
+
+```shell
+pytest tests/test_basic_functions.py
+```
+
+will cue pytest to only run the tests in `test_basic_functions.py`.
+
+A key component to leveraging tests effectively is the ability to exercise ones tests repeatedly and rapidly with little manual overhead.
+Clearly, pytest is instrumental towards this end - this framework made the process of organizing and running our test suite exceedingly simple!
+That being said, there will certainly be occasions when we want to run a _specific_ test function.
+Suppose, for instance, that we are writing a new function, and repeatedly want to run one of our tests that is pointing to a bug in our work-in-progress.
+We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests in such incisive ways.
+
+
+### Utilizing pytest within an IDE
+
+Both [PyCharm and VSCode](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) can be configured to make keen use of pytest.
+The following images show a couple of the enhancements afforded to us by PyCharm; comparable features are available in VSCode.
+The IDEs will "discover" tests, and provide us with the ability to run individual tests.
+For example, in the following image, the green "play button" allows us to run `test_count_vowels_basic`.
+
+<!-- #raw raw_mimetype="text/html" -->
+<div style="text-align: center">
+<p>
+<img src="../_images/individual_test.png" alt="Running an individual test in PyCharm" width="600">
+</p>
+</div>
+<!-- #endraw -->
+
+Furthermore, IDEs can provide a rich tree view of all the tests that are being run.
+This is especially useful as our test suite grows to contain a considerable number of tests.
+In the following image, we see that `test_version` is failing - we can click on the failing test in this tree-view, and our IDE will navigate us directly to the failing test.
+
+<!-- #raw raw_mimetype="text/html" -->
+<div style="text-align: center">
+<p>
+<img src="../_images/test_tree_view.png" alt="Viewing an enhanced tree-view of your test suite" width="600">
+</p>
+</div>
+<!-- #endraw -->
+
+The first step for leveraging these features in your IDE is to enable the pytest framework in the IDE.
+The following links point to detailed instructions for configuring pytest with PyCharm and VSCode, respectively:
+
+- [Running tests in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
+- [Running tests in VSCode](https://code.visualstudio.com/docs/python/testing)
+
+These linked materials also include advanced details, like instructions for running tests in parallel, which are beyond the scope of this material but are useful nonetheless.
+
+
+## Enhanced Testing with pytest
+
+In addition to providing us with a simple means for organizing and running our test suite, pytest has powerful features that will both simplify and enhance our tests.
+We will now leverage these features in our test suite.
+
+<!-- #region -->
+### Enriched Assertions
+
+A failing "bare" assertion - an `assert` statement without an error message - can be a frustrating thing.
+Suppose, for instance, that one of our test-assertions about `count_vowels` fails:
+
+```python
+# a failing assertion without an error message is not informative
+
+assert count_vowels("aA bB yY", include_y=True) == 4
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-2-f89f8b6a7213> in <module>
+----> 1 assert count_vowels("aA bB yY", include_y=True) == 4
+
+AssertionError:
+```
+
+The problem with this bare assertion is that we don't know what `count_vowels("aA bB yY", include_y=True)` actually returned!
+We now have to go through the trouble of starting a python console, importing this function, and calling it with this specific input in order to see what our function was actually returning. An obvious remedy to this is for us to write our own error message, but this too is quite cumbersome when we consider the large number of assertions that we are destined to write.
+
+Fortunately, pytest comes to the rescue: it will "hijack" any failing bare assertion and will _insert a useful error message for us_.
+This is known as ["assertion introspection"](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details).
+For example, if the aforementioned assertion failed when being run by pytest, we would see the following output:
+
+```python
+# pytest will write informative error messages for us
+
+assert count_vowels("aA bB yY", include_y=True) == 4
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+~\Learning_Python\Python\Module6_Testing\Untitled1.ipynb in <module>
+----> 1 assert count_vowels("aA bB yY", include_y=True) == 4
+
+AssertionError: assert 2 == 4
+ + where 2 = <function count_vowels at 0x000001B91B913708>('aA bB yY', include_y=True
+```
+
+See that the error message that pytest included for us indicates that `count_vowels("aA bB yY", include_y=True)` returned `2`, when we expected it to return `4`.
+From this we might suspect that `count_vowels` is not counting y's correctly.
+
+Here are some more examples of "enriched assertions", as provided by pytest.
+See that these error messages even provide useful "diffs", which specify specifically _how_ two similar objects differ, where possible.
+
+```python
+# comparing unequal lists
+assert [1, 2, 3] == [1, 2]
+E Left contains one more item: 3
+E Full diff:
+E - [1, 2, 3]
+E ? ---
+E + [1, 2]
+```
+
+```python
+# comparing unequal dictionaries
+assert {"a": 1, "b": 2} == {"a": 1, "b": 3}
+E AssertionError: assert {'a': 1, 'b': 2} == {'a': 1, 'b': 3}
+E Omitting 1 identical items, use -vv to show
+E Differing items:
+E {'b': 2} != {'b': 3}
+E Full diff:
+E - {'a': 1, 'b': 2}
+E ? ^
+E + {'a': 1, 'b': 3}...
+```
+
+```python
+# comparing unequal strings
+assert "moo" == "moon"
+E AssertionError: assert 'moo' == 'moon'
+E - moo
+E + moon
+E ? +
+```
+
+<!-- #endregion -->
+
+<!-- #region -->
+### Parameterized Tests
+
+Looking back to both `test_count_vowels_basic` and `test_merge_max_mappings`, we see that there is a lot of redundancy within the bodies of these test functions.
+The assertions that we make within a given test-function share identical forms - they differ only in the parameters that we feed into our functions and their expected output.
+Another shortcoming of this test-structure is that a failing assertion will block subsequent assertions from being evaluated.
+That is, if the second assertion in a `test_count_vowels_basic` fails, the third and fourth assertions will not be evaluated in that run.
+This precludes us from potentially seeing useful patterns among the failing assertions.
+
+pytest provides a useful tool that will allow us to eliminate these structural shortcomings by transforming our test-functions into so-called _parameterized tests_. Let's parametrize the following test:
+
+```python
+# a simple test with redundant assertions
+
+def test_range_length_unparameterized():
+ assert len(range(0)) == 0
+ assert len(range(1)) == 1
+ assert len(range(2)) == 2
+ assert len(range(3)) == 3
+```
+
+This test is checking the property `len(range(n)) == n`, where `n` is any non-negative integer.
+Thus the parameter to be varied here is the "size" of the range-object being created.
+Let's treat it as such by using pytest to write a parameterized test:
+
+```python
+# parameterizing a test
+import pytest
+
+# note that this test must be run by pytest to work properly
+@pytest.mark.parametrize("size", [0, 1, 2, 3])
+def test_range_length(size):
+ assert len(range(size)) == size
+```
+
+Make note that a pytest-parameterized test must be run using pytest; an error will raise if we manually call `test_range_length()`.
+When executed, pytest will treat this parameterized test as _four separate tests_ - one for each parameter value:
+
+```
+test_basic_functions.py::test_range_length[0] PASSED [ 25%]
+test_basic_functions.py::test_range_length[1] PASSED [ 50%]
+test_basic_functions.py::test_range_length[2] PASSED [ 75%]
+test_basic_functions.py::test_range_length[3] PASSED [100%]
+```
+
+See that we have successfully eliminated the redundancy from `test_range_length`;
+the body of the function now contains only a single assertion, making obvious the property that is being tested.
+Furthermore, the four assertions are now being run independently from one another and thus we can potentially see patterns across multiple fail cases in concert.
+<!-- #endregion -->
+
+<!-- #region -->
+#### Decorators
+
+The syntax used to parameterize this test may look alien to us, as we have yet to encounter this construct thus far.
+`pytest.mark.parameterize(...)` is a _decorator_ - an object that is used to "wrap" a function in order to transform its behavior.
+The `pytest.mark.parameterize(...)` decorator wraps our test function so that pytest can call it multiple times, once for each parameter value.
+The `@` character, in this context, denotes the application of a decorator:
+
+```python
+# general syntax for applying a decorator to a function
+
+@the_decorator
+def the_function_being_decorated(<arguments_for_function>):
+ pass
+```
+
+For an in-depth discussion of decorators, please refer to [Real Python's Primer on decorators](https://realpython.com/primer-on-python-decorators/#simple-decorators).
+<!-- #endregion -->
+
+<!-- #region -->
+#### Parameterization Syntax
+
+The general form for creating a parameterizing decorator with *a single parameter*, as we formed above, is:
+
+```python
+@pytest.mark.parametrize("<param-name>", [<val-1>, <val-2>, ...])
+def test_function(<param-name>):
+ ...
+```
+
+We will often have tests that require multiple parameters.
+The general form for creating the parameterization decorator for $N$ parameters,
+each of which assume $J$ values, is:
+
+```python
+@pytest.mark.parametrize("<param-name1>, <param-name2>, [...], <param-nameN>",
+ [(<param1-val1>, <param2-val1>, [...], <paramN-val1>),
+ (<param1-val2>, <param2-val2>, [...], <paramN-val2>),
+ ...
+ (<param1-valJ>, <param2-valJ>, [...], <paramN-valJ>),
+ ])
+def test_function(<param-name1>, <param-name2>, [...], <param-nameN>):
+ ...
+```
+
+For example, let's take the following trivial test:
+
+```python
+def test_inequality_unparameterized():
+ assert 1 < 2 < 3
+ assert 4 < 5 < 6
+ assert 7 < 8 < 9
+ assert 10 < 11 < 12
+```
+
+and rewrite it in parameterized form.
+The decorator will have three distinct parameter, and each parameters, let's simply call them `a`, `b`, and `c`, will take on four values.
+
+```python
+# the parameterized form of `test_inequality_unparameterized`
+@pytest.mark.parametrize("a, b, c", [(1, 2, 3), (4, 5, 6), (7, 8, 9), (10, 11, 12)])
+def test_inequality(a, b, c):
+ assert a < b < c
+```
+<!-- #endregion -->
+
+<div class="alert alert-warning">
+
+**Note**
+
+The formatting for multi-parameter tests can quickly become unwieldy.
+It isn't always obvious where one should introduce line breaks and indentations to improve readability.
+This is a place where the ["black" auto-formatter](https://black.readthedocs.io/en/stable/) really shines!
+Black will make all of these formatting decisions for us - we can write our parameterized tests as haphazardly as we like and simply run black to format our code.
+</div>
+
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Parameterizing Tests**
+
+Rewrite `test_count_vowels_basic` as a parameterized test with the parameters: `input_string`, `include_y`, and `expected_count`.
+
+Rewrite `test_merge_max_mappings` as a parameterized test with the parameters: `dict_a`, `dict_b`, and `expected_merged`.
+
+Before rerunning `test_basic_functions.py` predict how many distinct test cases will be reported by pytest.
+
+</div>
+
+
+<!-- #region -->
+Finally, you can apply multiple parameterizing decorators to a test so that pytest will run _all combinations of the respective parameter values_.
+
+```python
+# testing all combinations of `x` and `y`
+@pytest.mark.parametrize("x", [0, 1, 2])
+@pytest.mark.parametrize("y", [10, 20])
+def test_all_combinations(x, y):
+ # will run:
+ # x=0 y=10
+ # x=0 y=20
+ # x=1 y=10
+ # x=1 y=20
+ # x=2 y=10
+ # x=2 y=20
+ pass
+```
+<!-- #endregion -->
+
+### Fixtures
+
+The final major pytest feature that we will discuss are "fixtures".
+A fixture, roughly speaking, is a means by which we can share information and functionality across our tests.
+Fixtures can be defined within our `conftest.py` file, and pytest will automatically "discover" them and make them available for use throughout our test suite in a convenient way.
+
+Exploring fixtures will quickly take us beyond our depths for the purposes of this introductory material, so we will only scratch the surface here.
+We can read about advanced details of fixtures [here](https://docs.pytest.org/en/latest/fixture.html#fixture).
+
+Below are examples of two useful fixtures.
+
+<!-- #region -->
+```python
+# contents of conftest.py
+
+import os
+import tempfile
+
+import pytest
+
+@pytest.fixture()
+def cleandir():
+ """ This fixture will use the stdlib `tempfile` module to
+ change the current working directory to a tmp-dir for the
+ duration of the test.
+
+ Afterwards, the test session returns to its previous working
+ directory, and the temporary directory and its contents
+ will be automatically deleted.
+
+ Yields
+ ------
+ str
+ The name of the temporary directory."""
+ with tempfile.TemporaryDirectory() as tmpdirname:
+ old_dir = os.getcwd() # get current working directory (cwd)
+ os.chdir(tmpdirname) # change cwd to the temp-directory
+ yield tmpdirname # yields control to the test to be run
+ os.chdir(old_dir) # restore the cwd to the original directory
+ # Leaving the context manager will prompt the deletion of the
+ # temporary directory and its contents. This cleanup will be
+ # triggered even if errors were raised during the test.
+
+
+@pytest.fixture()
+def dummy_email():
+ """ This fixture will simply have pytest pass the string:
+ 'dummy.email@plymi.com'
+ to any test-function that has the parameter name `dummy_email` in
+ its signature.
+ """
+ return "dummy.email@plymi.com"
+```
+<!-- #endregion -->
+
+<!-- #region -->
+The first one, `cleandir`, can be used in conjunction with tests that need to write files.
+We don't want our tests to leave behind files on our machines; the `cleandir` fixture will ensure that our tests will write files to a temporary directory that will be deleted once the test is complete.
+
+Second is a simple fixture called `dummy_email`.
+Suppose that our project needs to interact with a specific email address, suppose it's `dummy.email@plymi.com`, and that we have several tests that need access to this address.
+This fixture will pass this address to any test function that has the parameter name `dummy_email` in its signature.
+
+A reference implementation of `conftest.py` in our project can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/conftest.py).
+Several reference tests that make use of these fixtures can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/test_using_fixtures.py).
+
+Let's create a file `tests/test_using_fixtures.py`, and write some tests that put these fixtures to use:
+
+```python
+# contents of test_using_fixtures.py
+import pytest
+
+# When run, this test will be executed within a
+# temporary directory that will automatically be
+# deleted - along with all of its contents - once
+# the test ends.
+#
+# Thus we can have this test write a file, and we
+# need not worry about having it clean up after itself.
+@pytest.mark.usefixtures("cleandir")
+def test_writing_a_file():
+ with open("a_text_file.txt", mode="w") as f:
+ f.write("hello world")
+
+ with open("a_text_file.txt", mode="r") as f:
+ file_content = f.read()
+
+ assert file_content == "hello world"
+
+
+# We can use the `dummy_email` fixture to provide
+# the same email address to many tests. In this
+# way, if we need to change the email address, we
+# can simply update the fixture and all of the tests
+# will be affected by the update.
+#
+# Note that we don't need to use a decorator here.
+# pytest is smart, and will see that the parameter-name
+# `dummy_email` matches the name of our fixture. It will
+# thus call these tests using the value returned by our
+# fixture
+
+def test_email1(dummy_email):
+ assert "dummy" in dummy_email
+
+
+def test_email2(dummy_email):
+ assert "plymi" in dummy_email
+
+
+def test_email3(dummy_email):
+ assert ".com" in dummy_email
+```
+<!-- #endregion -->
+
+## Links to Official Documentation
+
+- [pytest](https://docs.pytest.org/en/latest/)
+- [pytest's system for test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery)
+- [Testing in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
+- [Testing in VSCode](https://code.visualstudio.com/docs/python/testing)
+- [Assertion introspection](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details)
+- [Parameterizing tests](https://docs.pytest.org/en/latest/parametrize.html)
+- [Fixtures](https://docs.pytest.org/en/latest/fixture.html#fixture)
+
+
+## Reading Comprehension Solutions
+
+<!-- #region -->
+**Running a Test Suite: Solution**
+
+> Let's add the test function `test_broken_function` to our test suite.
+> We must include the word "test" in the function's name so that pytest will identify it as a test to run.
+> There are limitless ways in which we can make this test fail; we'll introduce a trivial false-assertion:
+
+```python
+def test_broken_function():
+ assert [1, 2, 3] == [1, 2]
+```
+
+> After introducing this broken test into `test_basic_functions.py` , running our tests should result in the following output:
+
+```
+$ pytest tests/
+============================= test session starts =============================
+platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
+rootdir: C:\Users\plymi_user\plymi_root_dir
+collected 4 items
+
+tests\test_basic_functions.py ..F [ 75%]
+tests\test_basic_numpy.py . [100%]
+
+================================== FAILURES ===================================
+____________________________ test_broken_function _____________________________
+
+ def test_broken_function():
+> assert [1, 2, 3] == [1, 2]
+E assert [1, 2, 3] == [1, 2]
+E Left contains one more item: 3
+E Use -v to get the full diff
+
+tests\test_basic_functions.py:40: AssertionError
+========================= 1 failed, 3 passed in 0.07s =========================
+```
+
+> Four tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests in `test_basic_functions` passed and the third test failed.
+> It then indicates which test failed, and specifically that the assertion was false because a length-2 list cannot be equal to a length-3 list.
+<!-- #endregion -->
+
+<!-- #region -->
+**Parameterizing Tests: Solution**
+
+A reference implementation for this solution within the `plymi_mod6` project can be found [here](https://github.com/rsokl/plymi_mod6/blob/parameterized/tests/test_basic_functions.py).
+
+The contents of `test_basic_functions.py`, rewritten to use pytest-parameterized tests:
+
+```python
+import pytest
+from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
+
+
+@pytest.mark.parametrize(
+ "input_string, include_y, expected_count",
+ [("aA bB yY", False, 2), ("aA bB yY", True, 4), ("", False, 0), ("", True, 0)],
+)
+def test_count_vowels_basic(input_string, include_y, expected_count):
+ assert count_vowels(input_string, include_y) == expected_count
+
+
+@pytest.mark.parametrize(
+ "dict_a, dict_b, expected_merged",
+ [
+ (dict(a=1, b=2), dict(b=20, c=-1), dict(a=1, b=20, c=-1)),
+ (dict(), dict(b=20, c=-1), dict(b=20, c=-1)),
+ (dict(a=1, b=2), dict(), dict(a=1, b=2)),
+ (dict(), dict(), dict()),
+ ],
+)
+def test_merge_max_mappings(dict_a, dict_b, expected_merged):
+ assert merge_max_mappings(dict_a, dict_b) == expected_merged
+```
+
+Running these tests via pytest should produce eight distinct test-case: four for `test_count_vowels_basic` and four for `test_merge_max_mappings`.
+
+```
+============================= test session starts =============================
+platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
+cachedir: .pytest_cache
+rootdir: C:\Users\plymi_user\Learning_Python\plymi_mod6_src
+collecting ... collected 8 items
+
+test_basic_functions.py::test_count_vowels_basic[aA bB yY-False-2] PASSED [ 12%]
+test_basic_functions.py::test_count_vowels_basic[aA bB yY-True-4] PASSED [ 25%]
+test_basic_functions.py::test_count_vowels_basic[-False-0] PASSED [ 37%]
+test_basic_functions.py::test_count_vowels_basic[-True-0] PASSED [ 50%]
+test_basic_functions.py::test_merge_max_mappings[dict_a0-dict_b0-expected_merged0] PASSED [ 62%]
+test_basic_functions.py::test_merge_max_mappings[dict_a1-dict_b1-expected_merged1] PASSED [ 75%]
+test_basic_functions.py::test_merge_max_mappings[dict_a2-dict_b2-expected_merged2] PASSED [ 87%]
+test_basic_functions.py::test_merge_max_mappings[dict_a3-dict_b3-expected_merged3] PASSED [100%]
+
+============================== 8 passed in 0.07s ==============================
+```
+
+<!-- #endregion -->
diff --git a/Python/module_6.rst b/Python/module_6.rst
index 4e20e2c1..2b113105 100644
--- a/Python/module_6.rst
+++ b/Python/module_6.rst
@@ -1,5 +1,5 @@
-Module 6: Testing Your Code
-===========================
+Module 6: Testing Our Code
+==========================
This module will introduce us to the critically-important and often-overlooked process of testing code.
We will begin by considering some general motivations for writing tests.
Next, we will study the basic anatomy of a test-function, including the :code:`assert` statement, which serves as the nucleus of our test functions.
@@ -16,5 +16,6 @@ This will take us down a bit of a rabbit hole, where we will find the powerful p
Module6_Testing/Intro_to_Testing.md
Module6_Testing/Pytest.md
+ Module6_Testing/Hypothesis.md
From 34013dde6c0a98af913aaee8738a1cbc5ea62ff6 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 5 Jan 2020 10:47:46 -0500
Subject: [PATCH 080/152] remove redundant content
---
Python/Module6_Testing/Hypothesis.md | 724 +--------------------------
1 file changed, 2 insertions(+), 722 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 20ab35c1..4069e651 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -18,736 +18,16 @@ jupyter:
:keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
<!-- #endraw -->
-# The pytest Framework
+# Hypothesis
-Thus far, our process for running tests has been an entirely manual one. It is time for us to arrange our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
-We will begin by reorganizing our source code to create an installable [Python package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
-We will then learn how to structure and run a test suite for this Python package, using pytest.
-
-The pytest framework does much more than just run tests;
-for instance, it will enrich the assertions in our tests to produce verbose, informative error messages.
-Furthermore it provides valuable means for enhancing our tests via mechanisms like fixtures and parameterizing decorators.
-Ultimately, all of this functionality helps to eliminate manual and redundant aspects of the testing process.
-
-
-
-<div class="alert alert-warning">
-
-**Note**
-
-It can be useful to [create a separate conda environment](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Installing_Python.html#A-Brief-Introduction-to-Conda-Environments) for the sake of this lesson, so that we can work through this material starting from a blank slate.
-If you do create a new conda environment, be sure to activate that environment and install NumPy and Jupyter notebook: `conda install numpy notebook`
-</div>
-
-
-
-Let's install pytest. Installing from [the conda-forge channel](https://conda-forge.org/) will install the most up-to-date version of pytest. In a terminal where conda can be accessed, run:
-
-```shell
-conda install -c conda-forge pytest
-```
-
-Or, pytest is installable via pip:
-
-```shell
-pip install pytest
-```
-
-
-<div class="alert alert-warning">
-
-**Regarding Alternative Testing Frameworks** (a note from the author of PLYMI):
-
-When sifting through tutorials, blogs, and videos about testing in Python, it is common to see `pytest` presented alongside, and on an equal footing with, the alternative testing frameworks: `nose` and `unittest`.
-This strikes me as... bizarre.
-
-`unittest` is the testing framework that comes with the Python standard library.
-As a test runner, its design is clunky, archaic, and, ironically, un-pythonic.
-While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) provides extremely valuable functionality for advanced testing, all of its functionality can be leveraged while using pytest as your testing framework.
-
-`nose`, which simply extends the functionality of `unittest`, **is no longer being maintained**.
-There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project by comparison to `pytest`.
-As of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads.
-
-The takeaway here is that, when it comes to picking a testing framework for Python, `pytest` is the clear choice.
-Any discussion that you come across to the contrary is likely outdated.
-</div>
-
-<!-- #region -->
-## Creating a Python Package with Tests
-
-It's time to create a proper test suite.
-Before proceeding any further, we should reread the material presented in [Module 5 - Import: Modules and Packages](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html) and recall the essentials of import statements, modules, and Python packages.
-This material serves as the foundation for this section.
-
-### Organizing our Source Code
-Let's create a Python package, which we will call `plymi_mod6`, with the following directory structure:
-
-```
-project_dir/ # the "parent directory" houses our source code, tests, and all other relevant files
- - setup.py # script responsible for installing `plymi_mod6` package
- - plymi_mod6/ # directory containing source code of `plymi_mod6` package
- |-- __init__.py
- |-- basic_functions.py
- |-- numpy_functions.py
- - tests/ # test-suite for `plymi_mod6` package (to be run using pytest)
- |-- conftest.py # optional configuration file for pytest
- |-- test_basic_functions.py
- |-- test_numpy_functions.py
-```
-
-A reference implementation of this package can be found [in this GitHub repository](https://github.com/rsokl/plymi_mod6).
-Populate the `basic_functions.py` file with the two functions that we were using as our source code in the previous section: `count_vowels` and `merge_max_mappings`.
-In the `numpy_functions.py` module, add the `pairwise_dists` function that appears in [Module 3's discussion of optimized pairwise distances](https://www.pythonlikeyoumeanit.com/Module3_IntroducingNumpy/Broadcasting.html#Optimized-Pairwise-Distances).
-Don't forget to include `import numpy as np` in your script in accordance with how `pairwise_dists` calls NumPy functions.
-
-We have arranged these functions so that they can be imported from the `basic_functions` module and the `numpy_functions` module, respectively, which reside in our `plymi_mod6` package.
-Let's fill out our `setup.py` script and install this package so that we can import it regardless of our current working directory. The content of `setup.py` will be:
-
-```python
-from setuptools import find_packages, setup
-
-setup(
- name="plymi_mod6",
- packages=find_packages(exclude=["tests", "tests.*"]),
- version="1.0.0",
- author="Your Name",
- description="A template Python package for learning about testing",
- install_requires=["numpy >= 1.10.0"],
- tests_require=["pytest>=5.3", "hypothesis>=5.0"],
- python_requires=">=3.6",
-)
-```
-
-This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6. Additionally, we will require pytest and hypothesis for running tests; the Hypothesis library will be introduced in a later section.
-
-Finally, let's install our package locally [in development mode](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Installing-Your-Own-Python-Package).
-Navigate to the directory containing `setup.py` and run:
-
-```shell
-pip install --editable .
-```
-
-Now, we should be able to start a python console, IPython console, or Jupyter notebook in any directory and import our package:
-
-```python
-# checking that we can import our `plymi_mod6` package
->>> from plymi_mod6.basic_functions import count_vowels
->>> count_vowels("Happy birthday", include_y=True)
-5
-```
-<!-- #endregion -->
-
-<!-- #region -->
-## Populating and Running Our Test Suite
-
-pytest's [system for "test discovery"](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery) is quite simple:
-pytest need only be pointed to a directory with files named `test_*.py` in it, and it will find all of the functions in these files _whose names start with the word "test"_ and will run all such functions.
-
-Thus, let's populate the file ``test_basic_functions.py`` with the functions `test_count_vowels_basic` and `test_merge_max_mappings`, which we wrote in the previous section of this module:
-
-```python
-# The contents of test_basic_functions.py
-
-# we must import the functions we are testing
-from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
-
-
-def test_count_vowels_basic():
- # test basic strings with uppercase and lowercase letters
- assert count_vowels("aA bB yY", include_y=False) == 2
- assert count_vowels("aA bB yY", include_y=True) == 4
-
- # test empty strings
- assert count_vowels("", include_y=False) == 0
- assert count_vowels("", include_y=True) == 0
-
-
-def test_merge_max_mappings():
- # test documented behavior
- dict1 = {"a": 1, "b": 2}
- dict2 = {"b": 20, "c": -1}
- expected = {'a': 1, 'b': 20, 'c': -1}
- assert merge_max_mappings(dict1, dict2) == expected
-
- # test empty dict1
- dict1 = {}
- dict2 = {"a": 10.2, "f": -1.0}
- expected = dict2
- assert merge_max_mappings(dict1, dict2) == expected
-
- # test empty dict2
- dict1 = {"a": 10.2, "f": -1.0}
- dict2 = {}
- expected = dict1
- assert merge_max_mappings(dict1, dict2) == expected
-
- # test both empty
- dict1 = {}
- dict2 = {}
- expected = {}
- assert merge_max_mappings(dict1, dict2) == expected
-
-```
-
-As described before, `count_vowels` and `merge_max_mappings` must both be imported from our `plymi_mod6` package, so that our functions are in the same namespace as our tests.
-A reference implementation of `test_basic_functions.py` can be viewed [here](https://github.com/rsokl/plymi_mod6/blob/master/tests/test_basic_functions.py).
-Finally, add a dummy test - a test function that will always pass - to `test_basic_numpy.py`.
-We will replace this with a useful test later.
-
-Without further ado, let's run our test suite! In our terminal, with the appropriate conda environment active, we navigate to the root directory of the project, which contains the `tests/` directory, and run `pytest tests/`.
-The following output should appear:
-<!-- #endregion -->
-
-```
-$ pytest tests/
-============================= test session starts =============================
-platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
-rootdir: C:\Users\plymi_user\plymi_root_dir
-collected 3 items
-
-tests\test_basic_functions.py .. [ 66%]
-tests\test_basic_numpy.py . [100%]
-
-============================== 3 passed in 0.04s ==============================
-```
-
-
-This output indicates that three test-functions were found across two files and that all of the tests "passed"; i.e. the functions ran without raising any errors.
-The first two tests are located in `tests/test_basic_functions.py`; the two dots indicate that two functions were run, and the `[66%]` indicator simply denotes that the test-suite is 66% (two-thirds) complete.
-The following reading comprehension problem will lead us to see what looks like for pytest to report a failing test.
-
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Running a Test Suite**
-
-Temporarily add a new "broken" test to `tests/test_basic_functions.py`.
-The name that you give this test should adhere to pytest's simple rules for test-discovery.
-Design the test function so that is sure to fail when it is run.
-
-Rerun your test suite and compare its output to what you saw before - is it easy to identify which test failed and what caused it to fail?
-Make sure to remove this function from your test suite once you are finished answering this question.
-
-</div>
-
-
-
-We can also direct pytest to run the tests in a specific .py file. E.g. executing:
-
-```shell
-pytest tests/test_basic_functions.py
-```
-
-will cue pytest to only run the tests in `test_basic_functions.py`.
-
-A key component to leveraging tests effectively is the ability to exercise ones tests repeatedly and rapidly with little manual overhead.
-Clearly, pytest is instrumental towards this end - this framework made the process of organizing and running our test suite exceedingly simple!
-That being said, there will certainly be occasions when we want to run a _specific_ test function.
-Suppose, for instance, that we are writing a new function, and repeatedly want to run one of our tests that is pointing to a bug in our work-in-progress.
-We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests in such incisive ways.
-
-
-### Utilizing pytest within an IDE
-
-Both [PyCharm and VSCode](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) can be configured to make keen use of pytest.
-The following images show a couple of the enhancements afforded to us by PyCharm; comparable features are available in VSCode.
-The IDEs will "discover" tests, and provide us with the ability to run individual tests.
-For example, in the following image, the green "play button" allows us to run `test_count_vowels_basic`.
-
-<!-- #raw raw_mimetype="text/html" -->
-<div style="text-align: center">
-<p>
-<img src="../_images/individual_test.png" alt="Running an individual test in PyCharm" width="600">
-</p>
-</div>
-<!-- #endraw -->
-
-Furthermore, IDEs can provide a rich tree view of all the tests that are being run.
-This is especially useful as our test suite grows to contain a considerable number of tests.
-In the following image, we see that `test_version` is failing - we can click on the failing test in this tree-view, and our IDE will navigate us directly to the failing test.
-
-<!-- #raw raw_mimetype="text/html" -->
-<div style="text-align: center">
-<p>
-<img src="../_images/test_tree_view.png" alt="Viewing an enhanced tree-view of your test suite" width="600">
-</p>
-</div>
-<!-- #endraw -->
-
-The first step for leveraging these features in your IDE is to enable the pytest framework in the IDE.
-The following links point to detailed instructions for configuring pytest with PyCharm and VSCode, respectively:
-
-- [Running tests in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
-- [Running tests in VSCode](https://code.visualstudio.com/docs/python/testing)
-
-These linked materials also include advanced details, like instructions for running tests in parallel, which are beyond the scope of this material but are useful nonetheless.
-
-
-## Enhanced Testing with pytest
-
-In addition to providing us with a simple means for organizing and running our test suite, pytest has powerful features that will both simplify and enhance our tests.
-We will now leverage these features in our test suite.
-
-<!-- #region -->
-### Enriched Assertions
-
-A failing "bare" assertion - an `assert` statement without an error message - can be a frustrating thing.
-Suppose, for instance, that one of our test-assertions about `count_vowels` fails:
-
-```python
-# a failing assertion without an error message is not informative
-
-assert count_vowels("aA bB yY", include_y=True) == 4
----------------------------------------------------------------------------
-AssertionError Traceback (most recent call last)
-<ipython-input-2-f89f8b6a7213> in <module>
-----> 1 assert count_vowels("aA bB yY", include_y=True) == 4
-
-AssertionError:
-```
-
-The problem with this bare assertion is that we don't know what `count_vowels("aA bB yY", include_y=True)` actually returned!
-We now have to go through the trouble of starting a python console, importing this function, and calling it with this specific input in order to see what our function was actually returning. An obvious remedy to this is for us to write our own error message, but this too is quite cumbersome when we consider the large number of assertions that we are destined to write.
-
-Fortunately, pytest comes to the rescue: it will "hijack" any failing bare assertion and will _insert a useful error message for us_.
-This is known as ["assertion introspection"](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details).
-For example, if the aforementioned assertion failed when being run by pytest, we would see the following output:
-
-```python
-# pytest will write informative error messages for us
-
-assert count_vowels("aA bB yY", include_y=True) == 4
----------------------------------------------------------------------------
-AssertionError Traceback (most recent call last)
-~\Learning_Python\Python\Module6_Testing\Untitled1.ipynb in <module>
-----> 1 assert count_vowels("aA bB yY", include_y=True) == 4
-
-AssertionError: assert 2 == 4
- + where 2 = <function count_vowels at 0x000001B91B913708>('aA bB yY', include_y=True
-```
-
-See that the error message that pytest included for us indicates that `count_vowels("aA bB yY", include_y=True)` returned `2`, when we expected it to return `4`.
-From this we might suspect that `count_vowels` is not counting y's correctly.
-
-Here are some more examples of "enriched assertions", as provided by pytest.
-See that these error messages even provide useful "diffs", which specify specifically _how_ two similar objects differ, where possible.
-
-```python
-# comparing unequal lists
-assert [1, 2, 3] == [1, 2]
-E Left contains one more item: 3
-E Full diff:
-E - [1, 2, 3]
-E ? ---
-E + [1, 2]
-```
-
-```python
-# comparing unequal dictionaries
-assert {"a": 1, "b": 2} == {"a": 1, "b": 3}
-E AssertionError: assert {'a': 1, 'b': 2} == {'a': 1, 'b': 3}
-E Omitting 1 identical items, use -vv to show
-E Differing items:
-E {'b': 2} != {'b': 3}
-E Full diff:
-E - {'a': 1, 'b': 2}
-E ? ^
-E + {'a': 1, 'b': 3}...
-```
-
-```python
-# comparing unequal strings
-assert "moo" == "moon"
-E AssertionError: assert 'moo' == 'moon'
-E - moo
-E + moon
-E ? +
-```
-
-<!-- #endregion -->
-
-<!-- #region -->
-### Parameterized Tests
-
-Looking back to both `test_count_vowels_basic` and `test_merge_max_mappings`, we see that there is a lot of redundancy within the bodies of these test functions.
-The assertions that we make within a given test-function share identical forms - they differ only in the parameters that we feed into our functions and their expected output.
-Another shortcoming of this test-structure is that a failing assertion will block subsequent assertions from being evaluated.
-That is, if the second assertion in a `test_count_vowels_basic` fails, the third and fourth assertions will not be evaluated in that run.
-This precludes us from potentially seeing useful patterns among the failing assertions.
-
-pytest provides a useful tool that will allow us to eliminate these structural shortcomings by transforming our test-functions into so-called _parameterized tests_. Let's parametrize the following test:
-
-```python
-# a simple test with redundant assertions
-
-def test_range_length_unparameterized():
- assert len(range(0)) == 0
- assert len(range(1)) == 1
- assert len(range(2)) == 2
- assert len(range(3)) == 3
-```
-
-This test is checking the property `len(range(n)) == n`, where `n` is any non-negative integer.
-Thus the parameter to be varied here is the "size" of the range-object being created.
-Let's treat it as such by using pytest to write a parameterized test:
-
-```python
-# parameterizing a test
-import pytest
-
-# note that this test must be run by pytest to work properly
-@pytest.mark.parametrize("size", [0, 1, 2, 3])
-def test_range_length(size):
- assert len(range(size)) == size
-```
-
-Make note that a pytest-parameterized test must be run using pytest; an error will raise if we manually call `test_range_length()`.
-When executed, pytest will treat this parameterized test as _four separate tests_ - one for each parameter value:
-
-```
-test_basic_functions.py::test_range_length[0] PASSED [ 25%]
-test_basic_functions.py::test_range_length[1] PASSED [ 50%]
-test_basic_functions.py::test_range_length[2] PASSED [ 75%]
-test_basic_functions.py::test_range_length[3] PASSED [100%]
-```
-
-See that we have successfully eliminated the redundancy from `test_range_length`;
-the body of the function now contains only a single assertion, making obvious the property that is being tested.
-Furthermore, the four assertions are now being run independently from one another and thus we can potentially see patterns across multiple fail cases in concert.
-<!-- #endregion -->
-
-<!-- #region -->
-#### Decorators
-
-The syntax used to parameterize this test may look alien to us, as we have yet to encounter this construct thus far.
-`pytest.mark.parameterize(...)` is a _decorator_ - an object that is used to "wrap" a function in order to transform its behavior.
-The `pytest.mark.parameterize(...)` decorator wraps our test function so that pytest can call it multiple times, once for each parameter value.
-The `@` character, in this context, denotes the application of a decorator:
-
-```python
-# general syntax for applying a decorator to a function
-
-@the_decorator
-def the_function_being_decorated(<arguments_for_function>):
- pass
-```
-
-For an in-depth discussion of decorators, please refer to [Real Python's Primer on decorators](https://realpython.com/primer-on-python-decorators/#simple-decorators).
-<!-- #endregion -->
-
-<!-- #region -->
-#### Parameterization Syntax
-
-The general form for creating a parameterizing decorator with *a single parameter*, as we formed above, is:
-
-```python
-@pytest.mark.parametrize("<param-name>", [<val-1>, <val-2>, ...])
-def test_function(<param-name>):
- ...
-```
-
-We will often have tests that require multiple parameters.
-The general form for creating the parameterization decorator for $N$ parameters,
-each of which assume $J$ values, is:
-
-```python
-@pytest.mark.parametrize("<param-name1>, <param-name2>, [...], <param-nameN>",
- [(<param1-val1>, <param2-val1>, [...], <paramN-val1>),
- (<param1-val2>, <param2-val2>, [...], <paramN-val2>),
- ...
- (<param1-valJ>, <param2-valJ>, [...], <paramN-valJ>),
- ])
-def test_function(<param-name1>, <param-name2>, [...], <param-nameN>):
- ...
-```
-
-For example, let's take the following trivial test:
-
-```python
-def test_inequality_unparameterized():
- assert 1 < 2 < 3
- assert 4 < 5 < 6
- assert 7 < 8 < 9
- assert 10 < 11 < 12
-```
-
-and rewrite it in parameterized form.
-The decorator will have three distinct parameter, and each parameters, let's simply call them `a`, `b`, and `c`, will take on four values.
-
-```python
-# the parameterized form of `test_inequality_unparameterized`
-@pytest.mark.parametrize("a, b, c", [(1, 2, 3), (4, 5, 6), (7, 8, 9), (10, 11, 12)])
-def test_inequality(a, b, c):
- assert a < b < c
-```
-<!-- #endregion -->
-
-<div class="alert alert-warning">
-
-**Note**
-
-The formatting for multi-parameter tests can quickly become unwieldy.
-It isn't always obvious where one should introduce line breaks and indentations to improve readability.
-This is a place where the ["black" auto-formatter](https://black.readthedocs.io/en/stable/) really shines!
-Black will make all of these formatting decisions for us - we can write our parameterized tests as haphazardly as we like and simply run black to format our code.
-</div>
-
-
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Parameterizing Tests**
-
-Rewrite `test_count_vowels_basic` as a parameterized test with the parameters: `input_string`, `include_y`, and `expected_count`.
-
-Rewrite `test_merge_max_mappings` as a parameterized test with the parameters: `dict_a`, `dict_b`, and `expected_merged`.
-
-Before rerunning `test_basic_functions.py` predict how many distinct test cases will be reported by pytest.
-
-</div>
-
-
-<!-- #region -->
-Finally, you can apply multiple parameterizing decorators to a test so that pytest will run _all combinations of the respective parameter values_.
-
-```python
-# testing all combinations of `x` and `y`
-@pytest.mark.parametrize("x", [0, 1, 2])
-@pytest.mark.parametrize("y", [10, 20])
-def test_all_combinations(x, y):
- # will run:
- # x=0 y=10
- # x=0 y=20
- # x=1 y=10
- # x=1 y=20
- # x=2 y=10
- # x=2 y=20
- pass
-```
-<!-- #endregion -->
-
-### Fixtures
-
-The final major pytest feature that we will discuss are "fixtures".
-A fixture, roughly speaking, is a means by which we can share information and functionality across our tests.
-Fixtures can be defined within our `conftest.py` file, and pytest will automatically "discover" them and make them available for use throughout our test suite in a convenient way.
-
-Exploring fixtures will quickly take us beyond our depths for the purposes of this introductory material, so we will only scratch the surface here.
-We can read about advanced details of fixtures [here](https://docs.pytest.org/en/latest/fixture.html#fixture).
-
-Below are examples of two useful fixtures.
-
-<!-- #region -->
-```python
-# contents of conftest.py
-
-import os
-import tempfile
-
-import pytest
-
-@pytest.fixture()
-def cleandir():
- """ This fixture will use the stdlib `tempfile` module to
- change the current working directory to a tmp-dir for the
- duration of the test.
-
- Afterwards, the test session returns to its previous working
- directory, and the temporary directory and its contents
- will be automatically deleted.
-
- Yields
- ------
- str
- The name of the temporary directory."""
- with tempfile.TemporaryDirectory() as tmpdirname:
- old_dir = os.getcwd() # get current working directory (cwd)
- os.chdir(tmpdirname) # change cwd to the temp-directory
- yield tmpdirname # yields control to the test to be run
- os.chdir(old_dir) # restore the cwd to the original directory
- # Leaving the context manager will prompt the deletion of the
- # temporary directory and its contents. This cleanup will be
- # triggered even if errors were raised during the test.
-
-
-@pytest.fixture()
-def dummy_email():
- """ This fixture will simply have pytest pass the string:
- 'dummy.email@plymi.com'
- to any test-function that has the parameter name `dummy_email` in
- its signature.
- """
- return "dummy.email@plymi.com"
-```
-<!-- #endregion -->
-
-<!-- #region -->
-The first one, `cleandir`, can be used in conjunction with tests that need to write files.
-We don't want our tests to leave behind files on our machines; the `cleandir` fixture will ensure that our tests will write files to a temporary directory that will be deleted once the test is complete.
-
-Second is a simple fixture called `dummy_email`.
-Suppose that our project needs to interact with a specific email address, suppose it's `dummy.email@plymi.com`, and that we have several tests that need access to this address.
-This fixture will pass this address to any test function that has the parameter name `dummy_email` in its signature.
-
-A reference implementation of `conftest.py` in our project can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/conftest.py).
-Several reference tests that make use of these fixtures can be found [here](https://github.com/rsokl/plymi_mod6/blob/fixtures/tests/test_using_fixtures.py).
-
-Let's create a file `tests/test_using_fixtures.py`, and write some tests that put these fixtures to use:
-
-```python
-# contents of test_using_fixtures.py
-import pytest
-
-# When run, this test will be executed within a
-# temporary directory that will automatically be
-# deleted - along with all of its contents - once
-# the test ends.
-#
-# Thus we can have this test write a file, and we
-# need not worry about having it clean up after itself.
-@pytest.mark.usefixtures("cleandir")
-def test_writing_a_file():
- with open("a_text_file.txt", mode="w") as f:
- f.write("hello world")
-
- with open("a_text_file.txt", mode="r") as f:
- file_content = f.read()
-
- assert file_content == "hello world"
-
-
-# We can use the `dummy_email` fixture to provide
-# the same email address to many tests. In this
-# way, if we need to change the email address, we
-# can simply update the fixture and all of the tests
-# will be affected by the update.
-#
-# Note that we don't need to use a decorator here.
-# pytest is smart, and will see that the parameter-name
-# `dummy_email` matches the name of our fixture. It will
-# thus call these tests using the value returned by our
-# fixture
-
-def test_email1(dummy_email):
- assert "dummy" in dummy_email
-
-
-def test_email2(dummy_email):
- assert "plymi" in dummy_email
-
-
-def test_email3(dummy_email):
- assert ".com" in dummy_email
-```
-<!-- #endregion -->
## Links to Official Documentation
-- [pytest](https://docs.pytest.org/en/latest/)
-- [pytest's system for test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery)
-- [Testing in PyCharm](https://www.jetbrains.com/help/pycharm/pytest.html)
-- [Testing in VSCode](https://code.visualstudio.com/docs/python/testing)
-- [Assertion introspection](https://docs.pytest.org/en/latest/assert.html#assertion-introspection-details)
-- [Parameterizing tests](https://docs.pytest.org/en/latest/parametrize.html)
-- [Fixtures](https://docs.pytest.org/en/latest/fixture.html#fixture)
+- [Hypothesis](https://hypothesis.readthedocs.io/)
## Reading Comprehension Solutions
<!-- #region -->
-**Running a Test Suite: Solution**
-
-> Let's add the test function `test_broken_function` to our test suite.
-> We must include the word "test" in the function's name so that pytest will identify it as a test to run.
-> There are limitless ways in which we can make this test fail; we'll introduce a trivial false-assertion:
-
-```python
-def test_broken_function():
- assert [1, 2, 3] == [1, 2]
-```
-
-> After introducing this broken test into `test_basic_functions.py` , running our tests should result in the following output:
-
-```
-$ pytest tests/
-============================= test session starts =============================
-platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
-rootdir: C:\Users\plymi_user\plymi_root_dir
-collected 4 items
-
-tests\test_basic_functions.py ..F [ 75%]
-tests\test_basic_numpy.py . [100%]
-
-================================== FAILURES ===================================
-____________________________ test_broken_function _____________________________
-
- def test_broken_function():
-> assert [1, 2, 3] == [1, 2]
-E assert [1, 2, 3] == [1, 2]
-E Left contains one more item: 3
-E Use -v to get the full diff
-
-tests\test_basic_functions.py:40: AssertionError
-========================= 1 failed, 3 passed in 0.07s =========================
-```
-
-> Four tests were "discovered" and run by pytest. The pattern `..F` indicates that the first two tests in `test_basic_functions` passed and the third test failed.
-> It then indicates which test failed, and specifically that the assertion was false because a length-2 list cannot be equal to a length-3 list.
-<!-- #endregion -->
-
-<!-- #region -->
-**Parameterizing Tests: Solution**
-
-A reference implementation for this solution within the `plymi_mod6` project can be found [here](https://github.com/rsokl/plymi_mod6/blob/parameterized/tests/test_basic_functions.py).
-
-The contents of `test_basic_functions.py`, rewritten to use pytest-parameterized tests:
-
-```python
-import pytest
-from plymi_mod6.basic_functions import count_vowels, merge_max_mappings
-
-
-@pytest.mark.parametrize(
- "input_string, include_y, expected_count",
- [("aA bB yY", False, 2), ("aA bB yY", True, 4), ("", False, 0), ("", True, 0)],
-)
-def test_count_vowels_basic(input_string, include_y, expected_count):
- assert count_vowels(input_string, include_y) == expected_count
-
-
-@pytest.mark.parametrize(
- "dict_a, dict_b, expected_merged",
- [
- (dict(a=1, b=2), dict(b=20, c=-1), dict(a=1, b=20, c=-1)),
- (dict(), dict(b=20, c=-1), dict(b=20, c=-1)),
- (dict(a=1, b=2), dict(), dict(a=1, b=2)),
- (dict(), dict(), dict()),
- ],
-)
-def test_merge_max_mappings(dict_a, dict_b, expected_merged):
- assert merge_max_mappings(dict_a, dict_b) == expected_merged
-```
-
-Running these tests via pytest should produce eight distinct test-case: four for `test_count_vowels_basic` and four for `test_merge_max_mappings`.
-
-```
-============================= test session starts =============================
-platform win32 -- Python 3.7.5, pytest-5.3.2, py-1.8.0, pluggy-0.12.0
-cachedir: .pytest_cache
-rootdir: C:\Users\plymi_user\Learning_Python\plymi_mod6_src
-collecting ... collected 8 items
-
-test_basic_functions.py::test_count_vowels_basic[aA bB yY-False-2] PASSED [ 12%]
-test_basic_functions.py::test_count_vowels_basic[aA bB yY-True-4] PASSED [ 25%]
-test_basic_functions.py::test_count_vowels_basic[-False-0] PASSED [ 37%]
-test_basic_functions.py::test_count_vowels_basic[-True-0] PASSED [ 50%]
-test_basic_functions.py::test_merge_max_mappings[dict_a0-dict_b0-expected_merged0] PASSED [ 62%]
-test_basic_functions.py::test_merge_max_mappings[dict_a1-dict_b1-expected_merged1] PASSED [ 75%]
-test_basic_functions.py::test_merge_max_mappings[dict_a2-dict_b2-expected_merged2] PASSED [ 87%]
-test_basic_functions.py::test_merge_max_mappings[dict_a3-dict_b3-expected_merged3] PASSED [100%]
-
-============================== 8 passed in 0.07s ==============================
-```
<!-- #endregion -->
From 22995ce541081b8edb85ea0fb7aa1fb4ec6888b8 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 5 Jan 2020 10:48:07 -0500
Subject: [PATCH 081/152] update meta description for pytest section
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 20ab35c1..90f9cbd8 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -14,7 +14,7 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
- :description: Topic: Writing tests for your code, Difficulty: Easy, Category: Section
+ :description: Topic: Introducing the pytest framework, Difficulty: Easy, Category: Section
:keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
<!-- #endraw -->
From df038b99db19ec6f356b885c412f65f0cb6f94a3 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 5 Jan 2020 13:07:08 -0500
Subject: [PATCH 082/152] working on hypothesis intro
---
Python/Module6_Testing/Hypothesis.md | 110 +++++++++++++++++++++++++--
1 file changed, 105 insertions(+), 5 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 4069e651..879efd76 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -7,9 +7,9 @@ jupyter:
format_version: '1.2'
jupytext_version: 1.3.0
kernelspec:
- display_name: Python 3
+ display_name: Python [conda env:.conda-jupy] *
language: python
- name: python3
+ name: conda-env-.conda-jupy-py
---
<!-- #raw raw_mimetype="text/restructuredtext" -->
@@ -18,8 +18,109 @@ jupyter:
:keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
<!-- #endraw -->
-# Hypothesis
+<!-- #region -->
+# Describing Data with Hypothesis
+
+It is often the case that the process of *describing our data* is by far the heaviest burden that we must bear when writing tests. This process of assessing "what variety of values should I test?", "have I thought of all the important edge-cases?", and "how much is 'enough'?" will crop up with nearly every test that we write.
+Indeed, these are questions that you may have been asking yourself when writing `test_count_vowels_basic` and `test_merge_max_mappings` in the previous sections of this module.
+
+[Hypothesis](https://hypothesis.readthedocs.io/) is a powerful Python library that empowers us to write a _description_ (specifications, to be more precise) of the data that we want to use to exercise our test.
+It will then *generate* test cases that satisfy this description and will run our test on these cases.
+
+Let's look at a simple example of Hypothesis in action.
+In the preceding section, we learned to use pytest's parameterization mechanism to test properties of code over a set of values.
+E.g. we wrote the following trivial test:
+
+```python
+import pytest
+
+# A simple parameterized test that only tests a few, conservative inputs.
+# Note that this test must be run by pytest to work properly
+@pytest.mark.parametrize("size", [0, 1, 2, 3])
+def test_range_length(size):
+ assert len(range(size)) == size
+```
+
+which tests the property that `range(n)` has a length of `n` for any non-negative integer value of `n`.
+Well, it isn't *really* testing this property for all non-negative integers; clearly it is only testing the values 0-3.
+We should probably also check much larger numbers and perhaps traverse various orders of magnitude (i.e. factors of ten) in our parameterization scheme.
+No matter what set of values we land on, it seems like we will have to eventually throw are hands up and say "okay, that seems good enough".
+
+Instead of manually specifying the data to pass to `test_range_length`, let's use Hypothesis to simply describe the data:
+<!-- #endregion -->
+
+<!-- #region -->
+```python
+from hypothesis import given
+
+# Hypothesis provides so-called "strategies" for us
+# to describe our data
+import hypothesis.strategies as st
+
+# Using hypothesis to test any integer value in [0, 10 ^ 10]
+@given(size=st.integers(min_value=0, max_value=1E10))
+def test_range_length(size):
+ assert len(range(size)) == size
+```
+<!-- #endregion -->
+
+<!-- #region -->
+```python
+# Running this test once will trigger Hypothesis to
+# generate 100 values based on the description of our data,
+# and it will execute the test using each one of those values
+>>> test_range_length()
+```
+<!-- #endregion -->
+Here we have specified that the `size` value in our test should take on any integer value within $[0, 10^{10}]$.
+We did this by using the `integers` "strategy" that is provided by Hypothesi: `st.integers(min_value=0, max_value=1E10)`.
+When we execute the resulting test (which simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
+by default it will generate 100 test cases - an amount that we can configure - and will evaluate our test for each one of them.
+
+With great ease, we were able to replace our pytest-parameterized test, which only very sparsely tested the property at hand, with a much more robust, hypothesis-driven test.
+This will be a recurring trend - we will generally produce much more robust tests by _describing_ our data with Hypothesis, rather than manually specifying test values.
+
+<!-- #region -->
+<div class="alert alert-warning">
+
+**Hypothesis is very effective...**:
+
+You may be wondering why, in the preceding example, we arbitrarily picked $10^{10}$ as the upper bound to the integer-values to feed to our test.
+I actually didn't write the test that way initially.
+Instead, I wrote the more general test:
+
+```python
+@given(size=st.integers(min_value=0))
+def test_range_length(size):
+ assert len(range(size)) == size
+```
+
+which places no formal upper bound on the integers that Hypothesis will generate.
+But... this found a bug:
+
+```python
+Falsifying example: test_range_length(
+ size=9223372036854775808,
+)
+
+----> 3 assert len(range(size)) == size
+
+OverflowError: Python int too large to convert to C ssize_t
+```
+
+</div>
+<!-- #endregion -->
+
+```python
+@given(size=st.integers(min_value=0))
+def test_range_length(size):
+ assert len(range(size)) == size
+```
+
+```python
+test_range_length()
+```
## Links to Official Documentation
@@ -28,6 +129,5 @@ jupyter:
## Reading Comprehension Solutions
-<!-- #region -->
-<!-- #endregion -->
+
From c20c37af7db319d4b57533341b51a5a750975d96 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 5 Jan 2020 18:40:04 -0500
Subject: [PATCH 083/152] add description of len size limit
---
Python/Module6_Testing/Hypothesis.md | 45 ++++++++++++++--------------
1 file changed, 23 insertions(+), 22 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 879efd76..07dbc331 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -24,7 +24,7 @@ jupyter:
It is often the case that the process of *describing our data* is by far the heaviest burden that we must bear when writing tests. This process of assessing "what variety of values should I test?", "have I thought of all the important edge-cases?", and "how much is 'enough'?" will crop up with nearly every test that we write.
Indeed, these are questions that you may have been asking yourself when writing `test_count_vowels_basic` and `test_merge_max_mappings` in the previous sections of this module.
-[Hypothesis](https://hypothesis.readthedocs.io/) is a powerful Python library that empowers us to write a _description_ (specifications, to be more precise) of the data that we want to use to exercise our test.
+[Hypothesis](https://hypothesis.readthedocs.io/) is a powerful Python library that empowers us to write a _description_ (specification, to be more precise) of the data that we want to use to exercise our test.
It will then *generate* test cases that satisfy this description and will run our test on these cases.
Let's look at a simple example of Hypothesis in action.
@@ -57,7 +57,7 @@ from hypothesis import given
# to describe our data
import hypothesis.strategies as st
-# Using hypothesis to test any integer value in [0, 10 ^ 10]
+# Using hypothesis to test any integer value in [0, 10 ** 10]
@given(size=st.integers(min_value=0, max_value=1E10))
def test_range_length(size):
assert len(range(size)) == size
@@ -65,28 +65,30 @@ def test_range_length(size):
<!-- #endregion -->
<!-- #region -->
+Here we have specified that the `size` value in our test should take on any integer value within $[0, 10^{10}]$.
+We did this by using the `integers` "strategy" that is provided by Hypothesis: `st.integers(min_value=0, max_value=1E10)`.
+When we execute the resulting test (which simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
+by default it will generate 100 test cases - an amount that we can configure - and will evaluate our test for each one of them.
+
```python
# Running this test once will trigger Hypothesis to
# generate 100 values based on the description of our data,
# and it will execute the test using each one of those values
>>> test_range_length()
```
-<!-- #endregion -->
-
-Here we have specified that the `size` value in our test should take on any integer value within $[0, 10^{10}]$.
-We did this by using the `integers` "strategy" that is provided by Hypothesi: `st.integers(min_value=0, max_value=1E10)`.
-When we execute the resulting test (which simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
-by default it will generate 100 test cases - an amount that we can configure - and will evaluate our test for each one of them.
With great ease, we were able to replace our pytest-parameterized test, which only very sparsely tested the property at hand, with a much more robust, hypothesis-driven test.
-This will be a recurring trend - we will generally produce much more robust tests by _describing_ our data with Hypothesis, rather than manually specifying test values.
+This will be a recurring trend: we will generally produce much more robust tests by _describing_ our data with Hypothesis, rather than manually specifying test values.
+
+The rest of this section will be dedicated to learning about the Hypothesis library and how we can leverage it to write powerful tests.
+<!-- #endregion -->
<!-- #region -->
<div class="alert alert-warning">
-**Hypothesis is very effective...**:
+**Hypothesis is _very_ effective...**:
-You may be wondering why, in the preceding example, we arbitrarily picked $10^{10}$ as the upper bound to the integer-values to feed to our test.
+You may be wondering why, in the preceding example, I arbitrarily picked $10^{10}$ as the upper bound to the integer-values to feed to the test.
I actually didn't write the test that way initially.
Instead, I wrote the more general test:
@@ -97,7 +99,7 @@ def test_range_length(size):
```
which places no formal upper bound on the integers that Hypothesis will generate.
-But... this found a bug:
+However, this test immediately found an issue (I hesitate to call it an outright bug):
```python
Falsifying example: test_range_length(
@@ -109,18 +111,17 @@ Falsifying example: test_range_length(
OverflowError: Python int too large to convert to C ssize_t
```
-</div>
-<!-- #endregion -->
+This reveals that the implementation of the built-in `len` function is such that it can only handle non-negative integers smaller than $2^{63}$ (i.e. it will only allocate 64 bits to represent a signed integer - one bit is used to store the sign of the number).
+Hypothesis revealed this by generating the failing test case `size=9223372036854775808`. which is exactly $2^{63}$.
+I did not want this error to distract from what is otherwise merely a simple example, but it is very important to point out.
-```python
-@given(size=st.integers(min_value=0))
-def test_range_length(size):
- assert len(range(size)) == size
-```
+Hypothesis has a knack for catching these sorts of unexpected edge cases.
+Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-negative integers!
+(I wonder how many of the Python core developers know about this 😄).
-```python
-test_range_length()
-```
+
+</div>
+<!-- #endregion -->
## Links to Official Documentation
From 10f572ba523a8d65cbe08cb0f181693c0c3589ca Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 5 Jan 2020 18:42:49 -0500
Subject: [PATCH 084/152] fix punctuation
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 07dbc331..ea270f72 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -112,7 +112,7 @@ OverflowError: Python int too large to convert to C ssize_t
```
This reveals that the implementation of the built-in `len` function is such that it can only handle non-negative integers smaller than $2^{63}$ (i.e. it will only allocate 64 bits to represent a signed integer - one bit is used to store the sign of the number).
-Hypothesis revealed this by generating the failing test case `size=9223372036854775808`. which is exactly $2^{63}$.
+Hypothesis revealed this by generating the failing test case `size=9223372036854775808`, which is exactly $2^{63}$.
I did not want this error to distract from what is otherwise merely a simple example, but it is very important to point out.
Hypothesis has a knack for catching these sorts of unexpected edge cases.
From 61fab2fe31a5c8b94158dfabc6c2139b2718c67b Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:40:06 -0500
Subject: [PATCH 085/152] Update
Python/Module5_OddsAndEnds/Modules_and_Packages.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index e359b39a..91d6a16a 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -406,7 +406,7 @@ It must be mentioned that we are sweeping some details under the rug here. Insta
### Installing Your Own Python Package
-Suppose that we are happy with the work we have done on our `face_detector` project. We will want to install this package - placing it in our site-packages directory so that we can import it irrespective of our Python interpreter's working directory. Here we will construct a basic setup script that will allow us to accomplish this. For completeness, we will also indicate how one would include a test suite along side the source code in this directory structure.
+Suppose that we are happy with the work we have done on our `face_detector` project. We will want to install this package - placing it in our site-packages directory so that we can import it irrespective of our Python interpreter's working directory. Here we will construct a basic setup script that will allow us to accomplish this. For completeness, we will also indicate how one would include a test suite alongside the source code in this directory structure.
We note outright that the purpose of this section is strictly to provide you with the minimum set of instructions needed to install a package. We will not be diving into what is going on under the hood at all. Please refer to [An Introduction to Distutils](https://docs.python.org/3/distutils/introduction.html#an-introduction-to-distutils) and [Packaging Your Project](https://packaging.python.org/tutorials/packaging-projects/#packaging-your-project) for a deeper treatment of this topic.
From feccc6c3085a7f4a8f5790b5a68fc82853c31f4b Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 11 Jan 2020 09:43:44 -0500
Subject: [PATCH 086/152] link to pytest site
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index 91d6a16a..28a81e6f 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -433,7 +433,8 @@ Carrying on, we will want to create a setup-script, `setup.py`, *in the same dir
|-- test_config.py
```
-A `tests/` directory can be included at the same directory level as `setup.py` and `face_detection/`. This is the recommended structure for using pytest as our test-runner.
+A `tests/` directory can be included at the same directory level as `setup.py` and `face_detection/`.
+This is the recommended structure for using [pytest](https://docs.pytest.org/en/latest/) as our test-runner.
<!-- #region -->
The bare bones build script for preparing your package for installation, `setup.py`, is as follows:
From 7c043cf39ab4c4bcf437b21d1e7d847fc6caf3de Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:44:49 -0500
Subject: [PATCH 087/152] Update
Python/Module5_OddsAndEnds/Modules_and_Packages.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index 28a81e6f..f9083600 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -450,7 +450,7 @@ setup(
python_requires=">=3.5",
)
```
-The `exclude` expression is used to ensure that specific directories or files are not included in the installation of `face_detection`. We use `exclude=["tests", "tests.*"]` to avoid installing the test-suite along side `face_detection`
+The `exclude` expression is used to ensure that specific directories or files are not included in the installation of `face_detection`. We use `exclude=["tests", "tests.*"]` to avoid installing the test-suite alongside `face_detection`.
<!-- #endregion -->
If you read through the additional materials linked above, you will see that there are many more fields of optional information that can be provided in this setup script, such as the author name, any installation requirements that the package has, and more.
From 0b7d5596ca584f37d8c54053e2f834a36019e4df Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:45:08 -0500
Subject: [PATCH 088/152] Update
Python/Module5_OddsAndEnds/Modules_and_Packages.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index f9083600..471e1a4e 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -467,7 +467,7 @@ and voilà, your package `face_detection` will have been installed to site-packa
pip uninstall face_detection
```
-One final but important detail. The installed version of your package will no longer "see" the source code. That is, if you go on to make any changes to your code, you will have to uninstall and reinstall your package before your will see the effects system-wide. Instead you can install your package in "development mode", such that a symbolic link to your source code is placed in your site-packages. Thus any changes that you make to your code will immediately be reflected in your system-wide installation. Thus, instead of running `python setup.py install`, execute the following to install a package in develop mode:
+One final but important detail: the installed version of your package will no longer "see" the source code. That is, if you go on to make any changes to your code, you will have to uninstall and reinstall your package before your will see the effects system-wide. Instead, you can install your package in "development mode", such that a symbolic link to your source code is placed in your site-packages. Thus, any changes that you make to your code will immediately be reflected in your system-wide installation. You can add the `--editable` flag to pip to install a package in development mode:
```shell
pip install --editable .
From c0ebc837023059fa0cf97a7066b68e34026719da Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:46:58 -0500
Subject: [PATCH 089/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 13647118..39f51ce7 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -32,7 +32,7 @@ After writing, say, a new function, it is only natural to contrive an input to f
To the extent that one would want to see evidence that their code works, we need not motivate the importance of testing.
Less obvious are the massive benefits that we stand to gain from automating this testing process.
-And by "automating", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
+By "automating", we mean taking the test scenarios that we were running our code through, and encapsulating them in their own functions that can be run from end-to-end.
We will accumulate these test functions into a "test suite" that we can run quickly and repeatedly.
There are plenty of practical details ahead for us to learn, so let's expedite this discussion and simply list some of the benefits that we can expect to reap from writing a robust test suite:
From 9adde9e1adbb293943b32c5d5d9583e567293557 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:47:13 -0500
Subject: [PATCH 090/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 39f51ce7..1734bef7 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -41,7 +41,7 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> After you have devised a test scenario for your code, it may only take us a second or so to run it; perhaps we need only run a couple of Jupyter notebook cells to verify the output of our code.
> This, however, will quickly become unwieldy as we write more code and devise more test scenarios.
-> Soon we will be dissuaded from running our tests except for on rare occasions.
+> Soon we will be dissuaded from running our tests, except for on rare occasions.
>
> With a proper test suite, we can run all of our test scenarios with the push of a button, and a series of green check-marks (or red x's...) will summarize the health of our project (insofar as our tests serve as good diagnostics).
> This, of course, also means that we will find and fix bugs much faster!
From 3649d3a5b882a0dfa8812b8e3f16e75f938ed967 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:47:35 -0500
Subject: [PATCH 091/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 1734bef7..e4b7fa20 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -62,7 +62,7 @@ There are plenty of practical details ahead for us to learn, so let's expedite t
> Although it may not be obvious from the outset, writing testable code leads to writing better code.
> This is, in part, because the process of writing tests gives us the opportunity to actually _use_ our code under varied circumstances.
-> The process of writing tests will help us suss out bad design decisions and redundant capabilities in our code. Ultimately, if _we_ find it frustrating to use our code within our tests, then surely others will find the code frustrating to use in applied settings.
+> The process of writing tests will help us suss out bad design decisions and redundancies in our code. Ultimately, if _we_ find it frustrating to use our code within our tests, then surely others will find the code frustrating to use in applied settings.
**It makes it easier for others to contribute to a project:**
From 7f9c0fbad95b6f9b32a2ff00f277e13d1477f3cd Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:48:08 -0500
Subject: [PATCH 092/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index e4b7fa20..8926b50f 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -152,7 +152,7 @@ Testing code that we don't understand is a lost cause!
<!-- #region -->
### The Basic Anatomy of a Test
-Let's write a test for `count_vowels`. For our most basic test, we can simply call `count_values` under various contrived inputs and *assert* that it returns the expected output.
+Let's write a test for `count_vowels`. For our most basic test, we can simply call `count_vowels` under various contrived inputs and *assert* that it returns the expected output.
The desired behavior for this test function, upon being run, is to:
- Raise an error if any of our assertions *failed* to hold true.
From 125b2dff8d7fa73841cdf2c94d027dbb581aa9eb Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:48:31 -0500
Subject: [PATCH 093/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 8926b50f..934ce27b 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -191,7 +191,7 @@ More on this later.
**Takeaway**:
-A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code and assert that the code behaves as-expected. The basic anatomy of a test function is such that it:
+A "test function" is designed to provide an encapsulated "environment" (namespace to be more precise) in which we can exercise parts of our source code and assert that the code behaves as expected. The basic anatomy of a test function is such that it:
- contains one or more `assert` statements, each of which will raise an error if our source code misbehaves
- simply returns `None` if all of the aforementioned assertions held true
From 3b1725090ebfa6b2005b47ff3ed6fdfe68a79785 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:48:42 -0500
Subject: [PATCH 094/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 934ce27b..11352257 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -349,7 +349,7 @@ Because they can be skipped in this way, *assertions should never be used for pr
It is surprisingly easy to unwittingly write a broken test: a test that always passes, or a test that simply doesn't exercise our code in the way that we had intended.
Broken tests are insidious; they are alarms that fail to sound when they are supposed to.
They create misdirection in the bug-finding process and can mask problems with our code.
-**Thus a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
+**Thus, a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
A practical note: we ought to mutate our function in a way that is trivial to undo. We can make use of code-comments towards this end.
From d7c8fcac11e0c2a0781f591fc0ccff524f409562 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:48:55 -0500
Subject: [PATCH 095/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 11352257..a25ddb26 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -350,7 +350,7 @@ It is surprisingly easy to unwittingly write a broken test: a test that always p
Broken tests are insidious; they are alarms that fail to sound when they are supposed to.
They create misdirection in the bug-finding process and can mask problems with our code.
**Thus, a critical step in the test-writing process is to intentionally mutate the function of interest - to corrupt its behavior so that we can verify that our test works.**
-Once we confirm that our test does indeed raise an error as-expected, we restore the function to its original form and re-run the test and see that it passes.
+Once we confirm that our test does indeed raise an error as expected, we restore the function to its original form and re-run the test to see that it passes.
A practical note: we ought to mutate our function in a way that is trivial to undo. We can make use of code-comments towards this end.
All [IDEs](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) have the ability to "block-comment" selected code.
From 7f717bb61072a1326f063fb23d495b80936f04d0 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:55:41 -0500
Subject: [PATCH 096/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index a25ddb26..fbd02f26 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -391,7 +391,7 @@ Automated mutation testing tools might be a bit too "heavy duty" at this point i
## Our Work, Cut Out
We see now that the concept of a "test function" isn't all that fancy.
-Compared to other code that we have written, writing a function that simply runs a hand full of assertions is far from a heavy lift for us.
+Compared to other code that we have written, writing a function that simply runs a handful of assertions is far from a heavy lift for us.
Of course, we must be diligent and take care to test our tests, but we can certainly manage this as well.
With this in hand, we should take stock of the work and challenges that lie in our path ahead.
From 446fe1daa1506d577010954e279981321a35edff Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:55:58 -0500
Subject: [PATCH 097/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index fbd02f26..103ec735 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -500,7 +500,7 @@ AssertionError:
> You may have written `assert len(a_list) > 0` - this is also correct.
> However, recall that calling `bool` on any sequence (list, tuple, string, etc.) will return `False` if the sequence is empty.
-> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - that `bool` will be called on whatever the provided expression is.
+> This is a reminder that an assertion statement need not include an explicit logical statement, such as an inequality - `bool` will be called on whatever the provided expression is.
Assert that the number of vowels in `a_string` is fewer than `a_number`; include an error message that prints the actual number of vowels:
From 96b3b04589192213c71514f31a5886247c379b7c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:56:30 -0500
Subject: [PATCH 098/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 103ec735..7a8c7ef0 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -543,7 +543,7 @@ AssertionError:
```
> See that the error output, which is called a "stack trace", indicates with an ASCII-arrow that our second assertion is the one that is failing.
-> Thus we can be confident that that assertion really does help to ensure that we are counting y's correctly.
+> Thus, we can be confident that that assertion really does help to ensure that we are counting y's correctly.
Restore `count_vowels` to its original form and rerun the test to see that `count_vowels` once again passes all of the assertions.
From 79f418cfa6567188a5b86d0841f649f06685025b Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:56:48 -0500
Subject: [PATCH 099/152] Update Python/Module6_Testing/Intro_to_Testing.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 7a8c7ef0..35231320 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -547,7 +547,7 @@ AssertionError:
Restore `count_vowels` to its original form and rerun the test to see that `count_vowels` once again passes all of the assertions.
-> We simply un-comment out the block of code and rerun our test.
+> We simply un-comment the block of code and rerun our test.
```python
# Restore the behavior of `include_y=True`
From 3d66b48c61a943acc1e9f81237b6f8a37dbb3482 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:57:03 -0500
Subject: [PATCH 100/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 90f9cbd8..e31a0e6c 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -66,7 +66,7 @@ As a test runner, its design is clunky, archaic, and, ironically, un-pythonic.
While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) provides extremely valuable functionality for advanced testing, all of its functionality can be leveraged while using pytest as your testing framework.
`nose`, which simply extends the functionality of `unittest`, **is no longer being maintained**.
-There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project by comparison to `pytest`.
+There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project in comparison with `pytest`.
As of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads.
The takeaway here is that, when it comes to picking a testing framework for Python, `pytest` is the clear choice.
From b32ab37506bf5297effd1ac96784c2be843a0f86 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:57:17 -0500
Subject: [PATCH 101/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index e31a0e6c..ebcfec2f 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -215,7 +215,7 @@ tests\test_basic_numpy.py . [100%]
This output indicates that three test-functions were found across two files and that all of the tests "passed"; i.e. the functions ran without raising any errors.
The first two tests are located in `tests/test_basic_functions.py`; the two dots indicate that two functions were run, and the `[66%]` indicator simply denotes that the test-suite is 66% (two-thirds) complete.
-The following reading comprehension problem will lead us to see what looks like for pytest to report a failing test.
+The following reading comprehension problem will lead us to see what it looks like for pytest to report a failing test.
<div class="alert alert-info">
From 1841a84b2c6d71182ef7b1b404687375a2aebc9d Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:57:30 -0500
Subject: [PATCH 102/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index ebcfec2f..b970bf19 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -233,7 +233,7 @@ Make sure to remove this function from your test suite once you are finished ans
-We can also direct pytest to run the tests in a specific .py file. E.g. executing:
+We can also direct pytest to run the tests in a specific .py file. For example, executing:
```shell
pytest tests/test_basic_functions.py
From cd78e8cdf9174fdac2a038ce3b9fba7a4a32f4c4 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 09:57:47 -0500
Subject: [PATCH 103/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index b970bf19..eaf5d19d 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -241,7 +241,7 @@ pytest tests/test_basic_functions.py
will cue pytest to only run the tests in `test_basic_functions.py`.
-A key component to leveraging tests effectively is the ability to exercise ones tests repeatedly and rapidly with little manual overhead.
+A key component to leveraging tests effectively is the ability to exercise one's tests repeatedly and rapidly with little manual overhead.
Clearly, pytest is instrumental towards this end - this framework made the process of organizing and running our test suite exceedingly simple!
That being said, there will certainly be occasions when we want to run a _specific_ test function.
Suppose, for instance, that we are writing a new function, and repeatedly want to run one of our tests that is pointing to a bug in our work-in-progress.
From f7107ed06e1a3b8095ff0fc089a9c83234beb0c2 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:01:54 -0500
Subject: [PATCH 104/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index eaf5d19d..e5a05007 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -242,7 +242,7 @@ pytest tests/test_basic_functions.py
will cue pytest to only run the tests in `test_basic_functions.py`.
A key component to leveraging tests effectively is the ability to exercise one's tests repeatedly and rapidly with little manual overhead.
-Clearly, pytest is instrumental towards this end - this framework made the process of organizing and running our test suite exceedingly simple!
+Clearly, pytest is instrumental toward this end - this framework makes the process of organizing and running our test suite exceedingly simple!
That being said, there will certainly be occasions when we want to run a _specific_ test function.
Suppose, for instance, that we are writing a new function, and repeatedly want to run one of our tests that is pointing to a bug in our work-in-progress.
We can leverage pytest in conjunction with [an IDE](https://www.pythonlikeyoumeanit.com/Module1_GettingStartedWithPython/Getting_Started_With_IDEs_and_Notebooks.html) to run our tests in such incisive ways.
From 5de88300609aa6b9430e1ce04fe3077617c74d56 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:02:07 -0500
Subject: [PATCH 105/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index e5a05007..82fc6bf2 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -373,7 +373,7 @@ E ? +
Looking back to both `test_count_vowels_basic` and `test_merge_max_mappings`, we see that there is a lot of redundancy within the bodies of these test functions.
The assertions that we make within a given test-function share identical forms - they differ only in the parameters that we feed into our functions and their expected output.
Another shortcoming of this test-structure is that a failing assertion will block subsequent assertions from being evaluated.
-That is, if the second assertion in a `test_count_vowels_basic` fails, the third and fourth assertions will not be evaluated in that run.
+That is, if the second assertion in `test_count_vowels_basic` fails, the third and fourth assertions will not be evaluated in that run.
This precludes us from potentially seeing useful patterns among the failing assertions.
pytest provides a useful tool that will allow us to eliminate these structural shortcomings by transforming our test-functions into so-called _parameterized tests_. Let's parametrize the following test:
From da824f013105609676a9c36db298f651c898ec56 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:02:21 -0500
Subject: [PATCH 106/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 82fc6bf2..b682450d 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -389,7 +389,7 @@ def test_range_length_unparameterized():
```
This test is checking the property `len(range(n)) == n`, where `n` is any non-negative integer.
-Thus the parameter to be varied here is the "size" of the range-object being created.
+Thus, the parameter to be varied here is the "size" of the range-object being created.
Let's treat it as such by using pytest to write a parameterized test:
```python
From 35bd28e545eec0cb7307acd1b35285b5af8d7a86 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:02:47 -0500
Subject: [PATCH 107/152] Update Python/Module6_Testing/Pytest.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index b682450d..cf71bd70 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -473,7 +473,7 @@ def test_inequality_unparameterized():
```
and rewrite it in parameterized form.
-The decorator will have three distinct parameter, and each parameters, let's simply call them `a`, `b`, and `c`, will take on four values.
+The decorator will have three distinct parameters, and each parameter, let's simply call them `a`, `b`, and `c`, will take on four values.
```python
# the parameterized form of `test_inequality_unparameterized`
From 6efb684c5b413d8aefcc4f7ef8b12d6eb3cddbc7 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:03:52 -0500
Subject: [PATCH 108/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index ea270f72..a7779ab9 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -29,7 +29,7 @@ It will then *generate* test cases that satisfy this description and will run ou
Let's look at a simple example of Hypothesis in action.
In the preceding section, we learned to use pytest's parameterization mechanism to test properties of code over a set of values.
-E.g. we wrote the following trivial test:
+For example, we wrote the following trivial test:
```python
import pytest
@@ -131,4 +131,3 @@ Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-
## Reading Comprehension Solutions
-
From 149aa788bcaee7c09306691fa0ab5faf2758efa5 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:04:24 -0500
Subject: [PATCH 109/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index a7779ab9..41f15e36 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -44,7 +44,7 @@ def test_range_length(size):
which tests the property that `range(n)` has a length of `n` for any non-negative integer value of `n`.
Well, it isn't *really* testing this property for all non-negative integers; clearly it is only testing the values 0-3.
We should probably also check much larger numbers and perhaps traverse various orders of magnitude (i.e. factors of ten) in our parameterization scheme.
-No matter what set of values we land on, it seems like we will have to eventually throw are hands up and say "okay, that seems good enough".
+No matter what set of values we land on, it seems like we will have to eventually throw our hands up and say "okay, that seems good enough."
Instead of manually specifying the data to pass to `test_range_length`, let's use Hypothesis to simply describe the data:
<!-- #endregion -->
@@ -130,4 +130,3 @@ Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-
## Reading Comprehension Solutions
-
From 72c0fc19e35c10e8043d49b0721cf7b5b3493919 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:04:47 -0500
Subject: [PATCH 110/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 41f15e36..d04cfa18 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -67,7 +67,7 @@ def test_range_length(size):
<!-- #region -->
Here we have specified that the `size` value in our test should take on any integer value within $[0, 10^{10}]$.
We did this by using the `integers` "strategy" that is provided by Hypothesis: `st.integers(min_value=0, max_value=1E10)`.
-When we execute the resulting test (which simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
+When we execute the resulting test (which can simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
by default it will generate 100 test cases - an amount that we can configure - and will evaluate our test for each one of them.
```python
@@ -129,4 +129,3 @@ Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-
## Reading Comprehension Solutions
-
From 3c8f49df0475d5aaa7f774b52239ab9ea3057f7c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Sat, 11 Jan 2020 10:05:27 -0500
Subject: [PATCH 111/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index d04cfa18..119f6b31 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -68,7 +68,7 @@ def test_range_length(size):
Here we have specified that the `size` value in our test should take on any integer value within $[0, 10^{10}]$.
We did this by using the `integers` "strategy" that is provided by Hypothesis: `st.integers(min_value=0, max_value=1E10)`.
When we execute the resulting test (which can simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
-by default it will generate 100 test cases - an amount that we can configure - and will evaluate our test for each one of them.
+by default, Hypothesis will generate 100 test cases - an amount that we can configure - and will evaluate our test for each one of them.
```python
# Running this test once will trigger Hypothesis to
From a5a027f670a1637b3869f606a4c833f7e68a37a2 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 16 Feb 2020 10:43:54 -0500
Subject: [PATCH 112/152] clean up sentence
---
Python/Module6_Testing/Pytest.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index cf71bd70..1ba87276 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -503,7 +503,7 @@ Rewrite `test_count_vowels_basic` as a parameterized test with the parameters: `
Rewrite `test_merge_max_mappings` as a parameterized test with the parameters: `dict_a`, `dict_b`, and `expected_merged`.
-Before rerunning `test_basic_functions.py` predict how many distinct test cases will be reported by pytest.
+Before rerunning the tests in `test_basic_functions.py` predict how many distinct test cases will be reported by pytest.
</div>
From ebeeddc8604d20fd494ee0d2cb9b7c5777799788 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Mar 2020 13:52:06 -0400
Subject: [PATCH 113/152] Adds discussion of given decorator
---
Python/Module6_Testing/Hypothesis.md | 225 ++++++++++++++++++++++++++-
1 file changed, 218 insertions(+), 7 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 119f6b31..a5c57cc9 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -19,13 +19,26 @@ jupyter:
<!-- #endraw -->
<!-- #region -->
-# Describing Data with Hypothesis
+# Introduction to Testing with Hypothesis
It is often the case that the process of *describing our data* is by far the heaviest burden that we must bear when writing tests. This process of assessing "what variety of values should I test?", "have I thought of all the important edge-cases?", and "how much is 'enough'?" will crop up with nearly every test that we write.
Indeed, these are questions that you may have been asking yourself when writing `test_count_vowels_basic` and `test_merge_max_mappings` in the previous sections of this module.
[Hypothesis](https://hypothesis.readthedocs.io/) is a powerful Python library that empowers us to write a _description_ (specification, to be more precise) of the data that we want to use to exercise our test.
It will then *generate* test cases that satisfy this description and will run our test on these cases.
+Ultimately this an extremely powerful tool for enabling us to write high-quality automated tests for our code.
+
+Hypothesis can be installed via conda:
+
+```shell
+conda install -c conda-forge hypothesis
+```
+
+or pip:
+
+```shell
+pip install hypothesis
+```
Let's look at a simple example of Hypothesis in action.
In the preceding section, we learned to use pytest's parameterization mechanism to test properties of code over a set of values.
@@ -42,7 +55,7 @@ def test_range_length(size):
```
which tests the property that `range(n)` has a length of `n` for any non-negative integer value of `n`.
-Well, it isn't *really* testing this property for all non-negative integers; clearly it is only testing the values 0-3.
+Well, it isn't *really* testing this property for all non-negative integers; clearly it is only testing the values 0-3, as indicated by the `parametrize` decorator.
We should probably also check much larger numbers and perhaps traverse various orders of magnitude (i.e. factors of ten) in our parameterization scheme.
No matter what set of values we land on, it seems like we will have to eventually throw our hands up and say "okay, that seems good enough."
@@ -51,6 +64,8 @@ Instead of manually specifying the data to pass to `test_range_length`, let's us
<!-- #region -->
```python
+# A basic introduction to Hypothesis
+
from hypothesis import given
# Hypothesis provides so-called "strategies" for us
@@ -65,10 +80,10 @@ def test_range_length(size):
<!-- #endregion -->
<!-- #region -->
-Here we have specified that the `size` value in our test should take on any integer value within $[0, 10^{10}]$.
+Here we have specified that the value of `size` in our test *should be able to take on any integer value within* $[0, 10^{10}]$.
We did this by using the `integers` "strategy" that is provided by Hypothesis: `st.integers(min_value=0, max_value=1E10)`.
When we execute the resulting test (which can simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
-by default, Hypothesis will generate 100 test cases - an amount that we can configure - and will evaluate our test for each one of them.
+by default, Hypothesis will generate 100 test cases - an amount that we can configure - and will execute our test function for each one of them.
```python
# Running this test once will trigger Hypothesis to
@@ -77,7 +92,7 @@ by default, Hypothesis will generate 100 test cases - an amount that we can conf
>>> test_range_length()
```
-With great ease, we were able to replace our pytest-parameterized test, which only very sparsely tested the property at hand, with a much more robust, hypothesis-driven test.
+With great ease, we were able to replace our pytest-parameterized test, which only very narrowly tested the property at hand, with a much more robust, Hypothesis-driven test.
This will be a recurring trend: we will generally produce much more robust tests by _describing_ our data with Hypothesis, rather than manually specifying test values.
The rest of this section will be dedicated to learning about the Hypothesis library and how we can leverage it to write powerful tests.
@@ -89,10 +104,11 @@ The rest of this section will be dedicated to learning about the Hypothesis libr
**Hypothesis is _very_ effective...**:
You may be wondering why, in the preceding example, I arbitrarily picked $10^{10}$ as the upper bound to the integer-values to feed to the test.
-I actually didn't write the test that way initially.
+Initially, I didn't write the test that way.
Instead, I wrote the more general test:
```python
+# `size` can be any non-negative integer
@given(size=st.integers(min_value=0))
def test_range_length(size):
assert len(range(size)) == size
@@ -111,7 +127,7 @@ Falsifying example: test_range_length(
OverflowError: Python int too large to convert to C ssize_t
```
-This reveals that the implementation of the built-in `len` function is such that it can only handle non-negative integers smaller than $2^{63}$ (i.e. it will only allocate 64 bits to represent a signed integer - one bit is used to store the sign of the number).
+This reveals that the CPython implementation of the built-in `len` function is such that it can only handle non-negative integers smaller than $2^{63}$ (i.e. it will only allocate 64 bits to represent a signed integer - one bit is used to store the sign of the number).
Hypothesis revealed this by generating the failing test case `size=9223372036854775808`, which is exactly $2^{63}$.
I did not want this error to distract from what is otherwise merely a simple example, but it is very important to point out.
@@ -123,9 +139,204 @@ Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-
</div>
<!-- #endregion -->
+<!-- #region -->
+## The `given` Decorator
+
+Hypothesis' [given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters) is responsible for:
+
+ - drawing values from Hypothesis' so-called "strategies" for describing input data for our test function
+ - running the test function many times (up to 100 times, by default) given different input values drawn from the strategy
+ - "shrinking" the drawn inputs to identify simple fail cases: if an error is raised by the test function during one of the many execution, the `given` decorator will attempt to "shrink" (i.e. simplify) the inputs that produce that same error before reporting them to the user
+ - reporting the input values that caused the test function to raise an error
+
+
+Let's see the `given` decorator in action by writing a simple "test" for which `x` should be integers between 0 and 10, and `y` should be integers between 20 and 30.
+To do this we will make use of the `integers` Hypothesis strategy.
+Let's include a bad assertion statement – that `y` can't be larger than 25 – to see how Hypothesis reports this fail case.
+Note that we aren't really testing any code here, we are simply exercising some of the tools that Hypothesis provides us with.
+
+```python
+from hypothesis import given
+import hypothesis.strategies as st
+
+# using `given` with multiple parameters
+# `x` is an integer drawn from [0, 10]
+# `y` is an integer drawn from [20, 30]
+@given(x=st.integers(0, 10), y=st.integers(20, 30))
+def test_demonstrating_the_given_decorator(x, y):
+ assert 0 <= x <= 10
+
+ # `y` can be any value in [20, 30]
+ # this is a bad assertion: it should fail!
+ assert 20 <= y <= 25
+```
+
+See that the names of the parameters specified in `given` — `x` and `y` in this instance — must match those in the signature of the test function.
+
+To run this test function, we simply call `test_demonstrating_the_given_decorator()`.
+Note that, unlike with a typical function, we do not pass values for `x` and `y` to this function – *the* `given` *decorator will pass these values to the function for us*.
+Executing this `given`-decorated function will prompt Hypothesis to draw 100 pairs of values for `x` and `y`, according to their respective strategies, and the body of the test will be executed for each such pair.
+
+```python
+# running the test
+>>> test_demonstrating_the_given_decorator()
+Falsifying example: test_demonstrating_the_given_decorator(
+ x=0, y=26,
+)
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-29-ea20353bbef5> in test_demonstrating_the_given_decorator(x, y)
+ 10 # `y` can be any value in [20, 30]
+ 11 # this is a bad assertion: it should fail!
+---> 12 assert 20 <= y <= 25
+
+AssertionError:
+
+```
+
+(Note: some of the "Traceback" error message has been removed to improve legibility)
+
+See that the error message here indicates that Hypothesis identified a "falsifying example", or a set of input values, `x=0` and `y=26`, which caused our test function to raise an error. The proceeding "Traceback" message indicates that it is indeed the second assertion statement that is responsible for raising the error.
+
+### Shrinking: Simplifying Falsifying Inputs
+
+The `given` decorator strives to report the "simplest" set of input values that produce a given error.
+It does this through the process of "shrinking".
+Each of Hypothesis' strategies has its own prescribed shrinking behavior.
+For the `integers` strategy, this means identifying the integer closest to 0 that produces the error at hand.
+For instance, `x=12` and `y=29` may have been the first drawn values to trigger the assertion error.
+These values were then incrementally reduced by the `given` decorator until `x=0` and `y=26` were identified as the smallest set of values to reproduce this fail case.
+
+We can print out the examples that Hypothesis generated:
+
+```
+x=0 y=20 - PASSED
+x=0 y=20 - PASSED
+x=0 y=20 - PASSED
+x=9 y=20 - PASSED
+x=9 y=21 - PASSED
+x=3 y=20 - PASSED
+x=3 y=20 - PASSED
+x=9 y=26 - FAILED
+x=3 y=26 - FAILED
+x=6 y=26 - FAILED
+x=10 y=27 - FAILED
+x=7 y=27 - FAILED
+x=3 y=30 - FAILED
+x=3 y=23 - PASSED
+x=10 y=30 - FAILED
+x=3 y=27 - FAILED
+x=3 y=27 - FAILED
+x=2 y=27 - FAILED
+x=0 y=27 - FAILED
+x=0 y=26 - FAILED
+x=0 y=21 - PASSED
+x=0 y=25 - PASSED
+x=0 y=22 - PASSED
+x=0 y=23 - PASSED
+x=0 y=24 - PASSED
+x=0 y=26 - FAILED
+```
+
+See that Hypothesis has to do a semi-random search to identify the boundaries of the fail case; it doesn't know if `x` is causing the error, or if `y` is the culprit, or if it is specific *combinations* of `x` and `y` that causes the failure!
+Despite this complexity, the pairs of variables are successfully shrunk to the simplest fail case.
+<!-- #endregion -->
+
+<div class="alert alert-warning">
+
+**Hypothesis will Save Falsifying Examples**:
+
+Albeit an advanced detail, it is important to note that Hypothesis does not have to search for falsifying examples from scratch every time we run a test function.
+Instead, Hypothesis will save a database of falsifying examples associated with each of your project's test functions.
+The database is saved under `.hypothesis` in whatever directory your test functions were run from.
+
+This ensures that, once Hypothesis finds a falsifying example for a test, the falsifying example will be passed to your test function each time you run it, until it no longer raises an error in your code (e.g. you update your code to fix the bug that was causing the test failure).
+</div>
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Understanding How Hypothesis Works**
+
+Define the `test_demonstrating_the_given_decorator` function as above, complete with the failing assertion, and add a print statement to the body of the function, which prints out the value for `x` and `y`.
+
+Run the test once and make note of the output that is printed. Consider copying and pasting the output to a notepad for reference. Next, rerun the test multiple times and make careful note of the printed output. What do you see? Is the output different from the first run? Does it differ between subsequent runs? Try to explain this behavior.
+
+In your file browser, navigate to the directory from which you are running this test; if you are following along in a Jupyter notebook, this is simply the directory containing said notebook. You should see a `.hypothesis` directory. As noted above, this is the database that contains the falsifying examples that Hypothesis has identified. Delete the `.hypothesis` directory and try re-running your test? What do you notice about the output now? You should also see that the `.hypothesis` directory has reappeared. Explain what is going on here.
+
+</div>
+
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Fixing the Failing Test**
+
+Update the body of `test_demonstrating_the_given_decorator` so that it no longer fails. Run the fixed test function. How many times is the test function actually be executed when you run it?
+
+</div>
+
+
+
+## Describing Data: Hypothesis Strategies
+
+
+
+
## Links to Official Documentation
- [Hypothesis](https://hypothesis.readthedocs.io/)
+- [The given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters)
+- [The Hypothesis example database](https://hypothesis.readthedocs.io/en/latest/database.html)
## Reading Comprehension Solutions
+
+<!-- #region -->
+**Understanding How Hypothesis Works: Solution**
+
+Define the `test_demonstrating_the_given_decorator` function as above, complete with the failing assertion, and add a print statement to the body of the function, which prints out the value for `x` and `y`.
+
+```python
+@given(x=st.integers(0, 10), y=st.integers(20, 30))
+def test_demonstrating_the_given_decorator(x, y):
+ print(x, y)
+ assert 0 <= x <= 10
+
+ # `y` can be any value in [20, 30]
+ # this is a bad assertion: it should fail!
+ assert 20 <= y <= 25
+```
+
+Run the test once and make note of the output that is printed. Consider copying and pasting the output to a notepad for reference. Next, rerun the test multiple times and make careful note of the printed output. What do you see? Is the output different from the first run? Does it differ between subsequent runs? Try to explain this behavior.
+
+> The printed outputs between the first and second run differ. The first set out outputs is typically longer than that of the second run. After the second run, the printed outputs are always the exact same. What is happening here is that Hypothesis has to search for the falsifying example during the first run. Once it is identified, the example is recorded in the `.hypothesis` database. All of the subsequent runs are simply re-running this saved case, which is why their inputs are not changing.
+
+In your file browser, navigate to the directory from which you are running this test; if you are following along in a Jupyter notebook, this is simply the directory containing said notebook. You should see a `.hypothesis` directory. As noted above, this is the database that contains the falsifying examples that Hypothesis has identified. Delete the `.hypothesis` directory and try re-running your test? What do you notice about the output now? You should also see that the `.hypothesis` directory has reappeared. Explain what is going on here.
+
+> Deleting `.hypothesis` removes all of the falsifying examples that Hypothesis found for tests that were run from this particular directory. Thus running the test again means that Hypothesis has to find the falsifying example again from scratch. Once it does this, it creates a new database in `.hypothesis`, which is why this directory "reappears".
+<!-- #endregion -->
+
+<!-- #region -->
+**Fixing the Failing Test: Solution**
+
+Update the body of `test_demonstrating_the_given_decorator` so that it no longer fails.
+
+> We simply need to fix the second assertion statement, specifying the bounds on `y`, so that it agrees with what is being drawn from the `integers` strategy.
+
+```python
+@given(x=st.integers(0, 10), y=st.integers(20, 30))
+def test_demonstrating_the_given_decorator(x, y):
+ assert 0 <= x <= 10
+ assert 20 <= y <= 30
+```
+
+Run the fixed test function. How many times is the test function actually be executed when you run it?
+
+> The `given` decorator, by default, will draw 100 sets of example values from the strategies that are passed to it and will thus execute the decorated test function 100 times.
+
+```python
+# no output (the function returns `None`) means that the test passed
+>>> test_demonstrating_the_given_decorator()
+```
+<!-- #endregion -->
From 3e3a154df73cdb64d7b684b4f3c432df89e8a99c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Mar 2020 15:37:32 -0400
Subject: [PATCH 114/152] Finish discussion of core strats; add initial sub
section
---
Python/Module6_Testing/Hypothesis.md | 193 +++++++++++++++++++++++++++
1 file changed, 193 insertions(+)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index a5c57cc9..01c16454 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -40,6 +40,8 @@ or pip:
pip install hypothesis
```
+## A Simple Example Using Hypothesis
+
Let's look at a simple example of Hypothesis in action.
In the preceding section, we learned to use pytest's parameterization mechanism to test properties of code over a set of values.
For example, we wrote the following trivial test:
@@ -280,6 +282,171 @@ Update the body of `test_demonstrating_the_given_decorator` so that it no longer
## Describing Data: Hypothesis Strategies
+Hypothesis provides us with so-called "strategies" for describing our data.
+We are already familiar with the `integers` strategy;
+Hypothesis' core strategies are all located in the `hypothesis.strategies` module.
+The official documentation for the core strategies can be found [here](https://hypothesis.readthedocs.io/en/latest/data.html).
+
+Here, we will familiarize ourselves with these core strategies and will explore some of the powerful methods that can be used to customize their behaviors.
+
+<!-- #region -->
+### Drawing examples from strategies
+
+Hypothesis provides a useful mechanism for developing an intuition for the data produced by a strategy: a strategy, once initialized, has a `.example()` method that will randomly draw a representative value from the strategy. For example:
+
+```python
+# demonstrating usage of `<strategy>.example()`
+>>> st.integers(-1, 1).example()
+-1
+
+>>> st.integers(-1, 1).example()
+1
+```
+
+**Note: the `.example()` mechanism is only meant to be used for pedagogical purposes. You should never use this in your test suite**
+because (among other reasons) `.example()` biases towards smaller and simpler examples than `@given`, and lacks the features to ensure any test failures are reproducible.
+
+We will be leveraging the `.example()` method throughout the rest of this section to help provide an intuition for the data that Hypothesis' various strategies generate.
+<!-- #endregion -->
+
+<!-- #region -->
+### Exploring Strategies
+
+There are a number critical Hypothesis strategies for us to become familiar with. It is worthwhile to peruse through all of Hypothesis' [core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies), but we will take time to highlight a few here.
+
+#### `st.booleans ()`
+
+[st.booleans()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.booleans) generates either `True` or `False`. This strategy will shrink towards `False`
+
+```python
+>>> st.booleans().example()
+False
+```
+<!-- #endregion -->
+
+<!-- #region -->
+
+#### `st.lists ()`
+
+[st.lists](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.lists) accepts *another* strategy, which describes the elements of the lists being generated. You can also specify:
+ - bounds on the length of the list
+ - if we want the elements to be unique
+ - a mechanism for defining "uniqueness"
+
+For example, the following strategy describes lists whose length varies from 2 to 10, and whose entries are integers on the domain $[-10, 20]$:
+
+```python
+>>> st.lists(st.integers(-10, 20), min_size=2, max_size=10).example()
+[-10, 0, 5]
+```
+
+**`st.lists(...)` is the strategy of choice anytime we want to generate sequences of varying lengths with elements that are, themselves, described by strategies**.
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.floats()`
+
+[st.floats](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.floats) is a powerful strategy that generates all variety of floats, including `math.inf` and `math.nan`. You can also specify:
+ - whether `math.inf` and `math.nan`, respectively, should be included in the data description
+ - bounds (either inclusive or exclusive) on the floats being generated; this will naturally preclude `math.nan` from being generated
+ - the "width" of the floats; e.g. if you want to generate 16-bit or 32-bit floats vs 64-bit
+ (while Python's `float` is (usually) 64-bit, `width=32` ensures that the generated values can
+ always be losslessly represented in 32 bits. This is mostly useful for Numpy arrays.)
+
+For example, the following strategy 64-bit floats that reside in the domain $[-100, 1]$:
+
+```python
+>>> st.floats(-100, 1).example()
+0.3670816313319896
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.tuples()`
+
+The [st.tuples](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.tuples) strategy accepts $N$ Hypothesis strategies, and will generate length-$N$ tuples whose elements are drawn from the respective strategies that were specified as inputs.
+
+For example, the following strategy will generate length-3 tuples whose entries are: integers, booleans, and floats:
+
+```python
+>>> st.tuples(st.integers(), st.booleans(), st.floats()).example()
+(4628907038081558014, False, -inf)
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.just()`
+
+[st.just](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.just) is a strategy that "just" returns the value that you fed it. This is a convenient strategy that helps us to avoid having to manipulate our data before using it.
+
+Write suppose that we want a strategy that describes the shape of an array (i.e. a tuple of integers) that contains 1-20 two-dimensional vectors. E.g. `(5, 2)` is the shape of the array containing five two-dimensional vectors. We can leverage `st.just`, in conjunction with `st.integers` and `st.tuples`, towards this end:
+
+```python
+>>> st.tuples(st.integers(1, 20), st.just(2)).example()
+(7, 2)
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.one_of()`
+
+The [st.one_of](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.one_of) allows us to specify a collection of strategies and any given datum will be drawn from "one of" them. E.g.
+
+```python
+# demonstrating st.one_of()
+st.one_of(st.integers(), st.lists(st.integers()))
+```
+
+will draw values that are *either* integers or list of integers:
+
+```python
+>>> st.one_of(st.integers(), st.lists(st.integers())).example()
+144
+
+>>> st.one_of(st.integers(), st.lists(st.integers())).example()
+[0, -22]
+```
+
+The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` calls:
+
+```python
+# Using the pipe operation, | , in place of `st.one_of`
+# This strategy generates integers or floats
+# or lists that contain just the word "hello"
+
+>>> (st.integers() | st.floats() | st.lists(st.just("hello"))).example()
+['hello', 'hello']
+
+>>> (st.integers() | st.floats() | st.lists(st.just("hello"))).example()
+0
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.sampled_from`
+
+[`st.sampled_from`](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and numpy arrays). The strategy will return values that are sampled from this collections.
+
+For example, the following strategy will sample from a list either `0`, `"a"`, or `(2, 2)`
+
+```python
+>>> st.sampled_from([0, "a", (2, 2)]).example()
+'a'
+```
+<!-- #endregion -->
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Exploring other Core Strategies**
+
+Review the [rest of Hypothesis' core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies).
+Write down a strategy, and print out a representative example, that describes the the data according to each of the following conditions:
+
+ 1. Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False.
+ 2. Length-4 strings whose elements are only lowercase vowels
+ 3. Permutations of the list `[1, 2, 3, 4]`
+
+</div>
@@ -288,6 +455,7 @@ Update the body of `test_demonstrating_the_given_decorator` so that it no longer
- [Hypothesis](https://hypothesis.readthedocs.io/)
- [The given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters)
- [The Hypothesis example database](https://hypothesis.readthedocs.io/en/latest/database.html)
+- [Core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies)
## Reading Comprehension Solutions
@@ -340,3 +508,28 @@ Run the fixed test function. How many times is the test function actually be exe
>>> test_demonstrating_the_given_decorator()
```
<!-- #endregion -->
+
+<!-- #region -->
+**Exploring other Core Strategies: Solution**
+
+Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False.
+
+```python
+>>> st.dictionaries(st.integers(min_value=1), st.booleans()).example()
+{110: True, 19091: True, 136348032: False, 78: False, 9877: False}
+```
+
+Length-4 strings whose elements are only lowercase vowels
+
+```python
+>>> st.text(alphabet="aeiou", min_size=4, max_size=4).example()
+'uiai'
+```
+
+Permutations of the list `[1, 2, 3, 4]`
+
+```python
+>>> st.permutations([1, 2, 3, 4]).example()
+[2, 3, 1, 4]
+```
+<!-- #endregion -->
From 265cfff3f0ae2d7a19601f27b9ea6f9c516155f8 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 28 Mar 2020 15:55:18 -0400
Subject: [PATCH 115/152] fix formatting
---
Python/Module6_Testing/Hypothesis.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 01c16454..5e85fa6b 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -303,7 +303,7 @@ Hypothesis provides a useful mechanism for developing an intuition for the data
1
```
-**Note: the `.example()` mechanism is only meant to be used for pedagogical purposes. You should never use this in your test suite**
+**Note: the** `.example()` **mechanism is only meant to be used for pedagogical purposes. You should never use this in your test suite**
because (among other reasons) `.example()` biases towards smaller and simpler examples than `@given`, and lacks the features to ensure any test failures are reproducible.
We will be leveraging the `.example()` method throughout the rest of this section to help provide an intuition for the data that Hypothesis' various strategies generate.
@@ -425,7 +425,7 @@ The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` ca
<!-- #region -->
#### `st.sampled_from`
-[`st.sampled_from`](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and numpy arrays). The strategy will return values that are sampled from this collections.
+[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and numpy arrays). The strategy will return values that are sampled from this collections.
For example, the following strategy will sample from a list either `0`, `"a"`, or `(2, 2)`
From a8ff23d990abaddb8966574c4e2432dc3bc505be Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 29 Mar 2020 10:36:23 -0400
Subject: [PATCH 116/152] Zac edits: default conda channel and cpython docs
---
Python/Module6_Testing/Hypothesis.md | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 5e85fa6b..f7a338ea 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -31,7 +31,7 @@ Ultimately this an extremely powerful tool for enabling us to write high-quality
Hypothesis can be installed via conda:
```shell
-conda install -c conda-forge hypothesis
+conda install hypothesis
```
or pip:
@@ -40,6 +40,7 @@ or pip:
pip install hypothesis
```
+
## A Simple Example Using Hypothesis
Let's look at a simple example of Hypothesis in action.
@@ -135,7 +136,7 @@ I did not want this error to distract from what is otherwise merely a simple exa
Hypothesis has a knack for catching these sorts of unexpected edge cases.
Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-negative integers!
-(I wonder how many of the Python core developers know about this 😄).
+(This overflow behavior actually documented in the [CPython source code](https://github.com/python/cpython) 😄).
</div>
From 4d687557682f7617c83dfeedf4e9d9b5b361cd99 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 29 Mar 2020 11:55:40 -0400
Subject: [PATCH 117/152] add reading comprehension: improving tests with
hypothesis
---
Python/Module6_Testing/Hypothesis.md | 169 ++++++++++++++++++++++++++-
1 file changed, 168 insertions(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index f7a338ea..9040323f 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -136,7 +136,7 @@ I did not want this error to distract from what is otherwise merely a simple exa
Hypothesis has a knack for catching these sorts of unexpected edge cases.
Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-negative integers!
-(This overflow behavior actually documented in the [CPython source code](https://github.com/python/cpython) 😄).
+(This overflow behavior is now documented in the [CPython source code](https://github.com/python/cpython) because it was discovered while writing this material 😄).
</div>
@@ -375,6 +375,19 @@ For example, the following strategy will generate length-3 tuples whose entries
```
<!-- #endregion -->
+<!-- #region -->
+#### `st.text()`
+
+The [st.text](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) accepts an "alphabet" – a collection of string-characters – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
+
+For example, the following strategy will strings of lowercase vowels from length 2 to length 10:
+
+```python
+>>> st.text("aeiouy", min_size=2, max_size=10).example()
+'oouoyoye'
+```
+<!-- #endregion -->
+
<!-- #region -->
#### `st.just()`
@@ -450,6 +463,45 @@ Write down a strategy, and print out a representative example, that describes th
</div>
+<!-- #region -->
+<div class="alert alert-info">
+
+**Reading Comprehension: Improving our tests using Hypothesis**
+
+We will be writing improved tests for the basic functions – `count_vowels` and `merge_max_mappings` – by leveraging Hypothesis.
+This reading comprehension question will require more substantial work than usual.
+That being said, the experience that we will gain from this will be well worth the work.
+Keep in mind that solutions are included at the end of this page, and that these can provide guidance if we get stuck.
+
+Part 1: Testing correctness by construction
+
+Write a hypothesis-driven test for the `count_vowels`; include this test in `test/test_basic_functions`.
+This is a test function where we can explicit construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
+And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
+We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
+The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
+
+We should ask ourselves: How general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
+
+
+Part 2: Property-based testing
+
+Write a hypothesis-driven test for `merge_max_mappings` ; include this test in `test/test_basic_functions`.
+Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
+Instead, we should *test the expected properties* of the merged dictionary.
+For example, one such property is that the merged dictionary should only contain maximum values.
+Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
+Take some time to think of other such properties that we should test for.
+Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
+
+We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
+Although dictionary keys can be any hashable object, suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
+
+
+**We must remember to temporarily mutate our original functions to verify that these tests can actually catch bugs!**
+</div>
+
+<!-- #endregion -->
## Links to Official Documentation
@@ -534,3 +586,118 @@ Permutations of the list `[1, 2, 3, 4]`
[2, 3, 1, 4]
```
<!-- #endregion -->
+
+**Improving our tests using Hypothesis: Solution**
+
+Part 1: Testing correctness by construction
+
+Write a hypothesis-driven test for the `count_vowels`; include this test in `test/test_basic_functions`.
+This is a test function where we can explicit construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
+And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
+We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
+The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
+
+We should ask ourselves: How general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
+
+<!-- #region -->
+```python
+from string import printable
+from random import shuffle
+
+import hypothesis.strategies as st
+from hypothesis import given, note
+
+# a list of all printable non-vowel characters
+_not_vowels = "".join([l for l in printable if l.lower() not in set("aeiouy")])
+
+
+@given(
+ not_vowels=st.text(alphabet=_not_vowels),
+ vowels_but_not_ys=st.text(alphabet="aeiouAEIOU"),
+ ys=st.text(alphabet="yY"),
+)
+def test_count_vowels_hypothesis(not_vowels, vowels_but_not_ys, ys):
+ """
+ Constructs an input string with a known number of:
+ - non-vowel characters
+ - non-y vowel characters
+ - y characters
+
+ and thus, by constructions, we can test that the output
+ of `count_vowels` agrees with the known number of vowels
+ """
+ # list of characters
+ letters = list(not_vowels) + list(vowels_but_not_ys) + list(ys)
+
+ # We need to shuffle the ordering of our characters so that
+ # our input string isn't unnaturally patterned; e.g. always
+ # have its vowels at the end
+ shuffle(letters)
+ in_string = "".join(letters)
+
+ # Hypothesis provides a `note` function that will print out
+ # whatever input you give it, but only in the case that the
+ # test fails.
+ # This way we can see the exact string that we fed to `count_vowels`,
+ # if it caused our test to fail
+ note("in_string: " + in_string)
+
+ # testing that `count_vowels` produces the expected output
+ # both including and excluding y's in the count
+ assert count_vowels(in_string, include_y=False) == len(vowels_but_not_ys)
+ assert count_vowels(in_string, include_y=True) == len(vowels_but_not_ys) + len(ys)
+```
+<!-- #endregion -->
+
+Part 2: Property-based testing
+
+Write a hypothesis-driven test for `merge_max_mappings` ; include this test in `test/test_basic_functions`.
+Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
+Instead, we should *test the expected properties* of the merged dictionary.
+For example, one such property is that the merged dictionary should only contain maximum values.
+Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
+Take some time to think of other such properties that we should test for.
+Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
+
+We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
+Although dictionary keys can be any hashable object, suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
+
+<!-- #region -->
+```python
+@given(
+ dict1=st.dictionaries(
+ keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
+ ),
+ dict2=st.dictionaries(
+ keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
+ ),
+)
+def test_merge_max_mappings_hypothesis(dict1, dict2):
+ merged_dict = merge_max_mappings(dict1, dict2)
+
+ # property: `merged_dict` contains all of the keys among
+ # `dict1` and `dict2`
+ assert set(merged_dict) == set(dict1).union(dict2), \
+ "novel keys were introduced or lost"
+
+ # property: `merged_dict` only contains values that appear
+ # among `dict1` and `dict2`
+ assert set(merged_dict.values()) <= set(dict1.values()).union(
+ dict2.values()
+ ), "novel values were introduced"
+
+ # property: `merged_dict` only contains key-value pairs with
+ # the largest value represented among the pairs in `dict1`
+ # and `dict2`
+ assert all(dict1[k] <= merged_dict[k] for k in dict1) and \
+ all(dict2[k] <= merged_dict[k] for k in dict2), \
+ "`merged_dict` contains a non-max value"
+
+ # property: `merged_dict` only contains key-value pairs that
+ # appear among `dict1` and `dict2`
+ for k, v in merged_dict.items():
+ assert (k, v) in dict1.items() or \
+ (k, v) in dict2.items(), \
+ "`merged_dict` did not preserve the key-value pairings"
+```
+<!-- #endregion -->
From 273d34b294806a525ca0fffa8353aecdf9b16042 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 29 Mar 2020 13:26:17 -0400
Subject: [PATCH 118/152] Add discussion of: .map(), .filter(), st.data(), and
@example()
---
Python/Module6_Testing/Hypothesis.md | 341 +++++++++++++++++++++++++++
1 file changed, 341 insertions(+)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 9040323f..46bf2e7a 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -499,16 +499,282 @@ Although dictionary keys can be any hashable object, suffice it to use both inte
**We must remember to temporarily mutate our original functions to verify that these tests can actually catch bugs!**
+
+Once we have added these tests to our test suite, we should re-run the entire test suite using `pytest tests` and check that our new Hypothesis-based tests are among the tests being run.
+</div>
+
+<!-- #endregion -->
+
+<!-- #region -->
+## Extending the Functionality of Strategies
+
+Hypothesis strategies can be enriched through the use of two methods: `.map()` and `.filter()`.
+These will permit us to leverage Hypothesis' core strategies to describe much more distinctive and diverse varieties of data.
+We also will see that there is a `st.data()` strategy, which will enable us to draw from strategies interactively from within our tests.
+
+
+### The `.map` method
+
+Hypothesis strategies have the `.map` method, which permits us to [perform a mapping on the data](https://hypothesis.readthedocs.io/en/latest/data.html#mapping) being produced by a strategy.
+This is achieved by passing the `.map` method a function (or any "callable");
+upon drawing a value from a strategy, Hypothesis will feed that value to the function held by `.map()`, and the strategy
+will return the value that was returned by the function.
+In this way the strategy's output is automatically "mapped" to a transformed value via the function that we provided.
+
+For example, if we want to draw only even-valued integers, we can simply use the following mapped strategy:
+
+```python
+# demonstrating the `.map()` method
+
+def make_even(x):
+ return 2 * x
+
+# `even_integers` is now a strategy that will only return even
+# valued integers. This is achieved by ensuring that any integer
+# drawn by `st.integers()` is "mapped" to an even value
+# via the function x -> 2 * x
+even_integers = st.integers().map(make_even)
+```
+
+```python
+>>> even_integers.example()
+-15414
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### A Brief Aside: Lambda Expressions
+
+Python has a syntax, which we have yet to discuss, that permits us to conveniently define functions "on the fly".
+A "lambda expression" is a syntax for defining a simple one-line function, making that function available for use in-place.
+The key here is that, whereas standard functions first must be formally defined before they can be referenced in code, *a lambda expression can be written wherever a function is expected*.
+
+For example, we can simplify the above mapping example by defining our mapping-function _within_ the `.map()` method:
+
+```python
+# Using a lambda expression to define a function
+# "on the fly"
+even_integers = st.integers().map(lambda x: 2 * x)
+```
+
+```python
+>>> even_integers.example()
+220
+```
+
+In general, the syntax for defining a lambda expression is:
+
+```
+lambda <comma-separated variable names>: <expression using variables>
+```
+
+Note that lambda expressions are restricted compared to typical function definitions: they do not permit default values or keyword arguments in their signatures, and a lambda's "body" must fit on one line.
+
+Here are some examples of lambda expressions:
+
+```python
+# function definition
+def add(x, y):
+ return x + y
+
+>>> add(2, 3)
+5
+
+# equivalent lambda expression
+>>> (lambda x, y: x + y)(2, 3)
+5
+
+# function definition
+def get_first_and_last_items(x):
+ return x[0], x[-1]
+
+>>> get_first_and_last_items(range(11))
+(0, 10)
+
+# equivalent lambda expression
+>>> (lambda x: x[0], x[-1])(range(11))
+(0, 10)
+```
+
+We will make keen use of lambdas in order to enrich our Hypothesis strategies.
+<!-- #endregion -->
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Using the `.map` method to create a sorted list**
+
+Using the `.map()` method, construct a Hypothesis strategy that produces a sorted list of integers.
+Generate some examples from your strategy and check that they are sorted (we may have to generate quite a few examples to see a diverse set of values)
+
+</div>
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Getting creative with the `.map` method**
+
+Construct a Hypothesis strategy that produces either the string `"cat"` or the string `"dog"`.
+Then, write a test that uses this strategy;
+it should simply test that either `"cat"` or `"dog"` was indeed produced by the strategy.
+Run the test.
+
+</div>
+
+
+<!-- #region -->
+### The `.filter` method
+
+Hypothesis strategies can also [have their data filtered](https://hypothesis.readthedocs.io/en/latest/data.html#filtering) via the `.filter` method.
+`.filter` takes a function (or any "callable") that accepts as input the data generated by the strategy, and returns:
+
+ - `True` if the data should pass through the filter
+ - `False` if the data should be rejected by the filter
+
+Consider, for instance, that you want to generate all integers other than `0`.
+You can write the filtered strategy:
+
+```python
+# Demonstrating the `.filter()` method
+non_zero_integers = st.integers().filter(lambda x: x != 0)
+```
+
+The `.filter` method is not magic – it is not able to "just know" how to avoid generating all barred values.
+A strategy that filters our too much data will prompt Hypothesis to raise an error.
+For example, let's try to filter `st.integers()` so that it only produces values on $[10, 20]$.
+
+```python
+# Using `.filter()` to filter out a large proportion of generated
+# values will result in an error
+@given(st.integers().filter(lambda x: 10 <= x <= 20))
+def test_aggressive_filter(x):
+ pass
+```
+
+```python
+>>> test_aggressive_filter()
+---------------------------------------------------------------------------
+FailedHealthCheck
+
+FailedHealthCheck: It looks like your strategy is filtering out a lot of data. Health check found 50 filtered examples but only 2 good ones. This will make your tests much slower, and also will probably distort the data generation quite a lot. You should adapt your strategy to filter less. This can also be caused by a low max_leaves parameter in recursive() calls
+See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.filter_too_much to the suppress_health_check settings for this test
+```
+
+Clearly, in this instance, we should have simply used the strategy `st.integers(min_value=10, max_value=20)`.
+<!-- #endregion -->
+
+<!-- #region -->
+### Drawing From Strategies Within a Test
+
+We will often need to draw from a Hypothesis strategy in a context-dependent manner within our test.
+Suppose, for example, that we want to describe two lists of integers, but we want to be sure that the second list is longer than the first.
+[We can use the `st.data()` strategy to use strategies "interactively"](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests) in this sort of way.
+
+Let's see it in action.
+Suppose that we want to generate two non-empty lists of integers, `x` and `y`, but we want to ensure that the values stored in `y` values are *larger than all of the values in* `x`.
+The following test shows how we can leverage Hypothesis to describe these lists
+
+```python
+# We want all of `y`'s entries to be larger than `max(x)`
+from typing import List
+
+# Defining our test function:
+# - `x` is a non-empty list of integers.
+# `- `data` is an object provided by Hypothesis that permits us
+# to draw interactively from other strategies within our test
+@given(x=st.lists(st.integers(), min_size=1), data=st.data())
+def test_two_constrained_lists(x, data):
+ y = data.draw(st.lists(st.integers(min_value=max(x) + 1), min_size=1), label="y")
+
+ largest_x = max(x)
+ assert all(largest_x < item for item in y)
+```
+```python
+# Running the test
+>>> test_two_constrained_lists()
+```
+<!-- #endregion -->
+
+The `given` operator is told to pass two values to our test:
+
+ - `x`, which is a list of integers drawn from strategies
+ - `data`, which is an instance of the [st.DataObject](https://hypothesis.readthedocs.io/en/latest/_modules/hypothesis/strategies/_internal/core.html#DataObject) class; this instance is what gets drawn from the `st.data()` strategy
+
+The only thing that you need to know about `st.DataObject` is that it's `draw` method expects a hypothesis search strategy, and that it will immediately draw a value from said strategy during the test.
+You can also, optionally, pass a string to `label` argument to the `draw` method.
+This simply permits you to provide a name for the item that was drawn, so that any stack-trace that your test produces is easy to interpret.
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Drawing from a strategy interactively**
+
+Write a test that is fed a list (of varying length) of non-negative integers.
+Then, draw a [set](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/DataStructures_III_Sets_and_More.html#The-%E2%80%9CSet%E2%80%9D-Data-Structure) of non-negative integers whose sum is at least as large as the sum of the list.
+Assert that the expected inequality between the sums hold.
+Run the test.
+
+Hint: use the `.filter()` method.
+
</div>
+
+
+## The `example` Decorator
+
+As mentioned before, Hypothesis strategies will draw values (pseudo)*randomly*.
+Thus our test will potentially encounter different values every time it is run.
+There are times where we want to be sure that, in addition the values produced by a strategy, specific values will tested.
+These might be known edge cases, critical use cases, or regression cases (i.e. values that were representative of passed bugs).
+Hypothesis provides [the example decorator](https://hypothesis.readthedocs.io/en/latest/reproducing.html#providing-explicit-examples), which is to be used in conjunction with the `given` decorator, towards this end.
+
+Let's suppose, for example, that we want to write a test whose data are pairs of perfect-squares (e.g. 4, 16, 25, ...), and that we want to be sure that the pairs `(100, 144)`, `(16, 25)`, and `(36, 36)` are tested *every* time the test is run.
+Let's use `example` to guarantee this.
+
+<!-- #region -->
+```python
+# Using the `example` decorator to ensure that specific examples
+# will always be passed as inputs to our test function
+
+from hypothesis import example
+
+# A hypothesis strategy that generates integers that are
+# perfect squares
+perfect_squares = st.integers().map(lambda x: x ** 2)
+
+
+def is_square(x):
+ """Returns True if `x` is a perfect square"""
+ return int(x ** 0.5) == x ** 0.5
+
+
+@example(a=36, b=36)
+@example(a=16, b=25)
+@example(a=100, b=144)
+@given(a=perfect_squares, b=perfect_squares)
+def test_pairs_of_squares(a, b):
+ assert is_square(a)
+ assert is_square(b)
+```
+```python
+# running the test
+>>> test_pairs_of_squares()
+```
<!-- #endregion -->
+Executing this test runs 103 cases: the three specified examples and one hundred pairs of values drawn via `given`.
+
+
## Links to Official Documentation
- [Hypothesis](https://hypothesis.readthedocs.io/)
- [The given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters)
- [The Hypothesis example database](https://hypothesis.readthedocs.io/en/latest/database.html)
- [Core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies)
+- [The .map method](https://hypothesis.readthedocs.io/en/latest/data.html#mapping)
+- [The .filter method](https://hypothesis.readthedocs.io/en/latest/data.html#filtering)
+- [Using data() to draw interactively in tests](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests)
+- [The example decorator](https://hypothesis.readthedocs.io/en/latest/reproducing.html#providing-explicit-examples)
+
## Reading Comprehension Solutions
@@ -701,3 +967,78 @@ def test_merge_max_mappings_hypothesis(dict1, dict2):
"`merged_dict` did not preserve the key-value pairings"
```
<!-- #endregion -->
+
+**Using the `.map` method to create a sorted list: Solution**
+
+Using the `.map()` method, construct a Hypothesis strategy that produces a sorted list of integers.
+Generate some examples from your strategy and check that they are sorted (we may have to generate quite a few examples to see a diverse set of values)
+
+<!-- #region -->
+```python
+# Note that the built-in `sorted` function can be supplied
+# directly to the `.map()` method - there is no need to define
+# a function or use a lambda expression here
+sorted_list_of_ints = st.lists(st.integers()).map(sorted)
+```
+```python
+>>> sorted_list_of_ints.example()
+[-27120, 97, 12805]
+```
+<!-- #endregion -->
+
+
+**Getting creative with the `.map` method: Solution**
+
+Construct a Hypothesis strategy that produces either the string `"cat"` or the string `"dog"`.
+Then, write a test that uses this strategy;
+it should simply test that either `"cat"` or `"dog"` was indeed produced by the strategy.
+Run the test
+
+<!-- #region -->
+```python
+# We "hijack" the `st.booleans()` strategy, which only generates
+# `True` or `False`, and use the `.map` method to transform these
+# two outputs to `"cat"` or `"dog"`.
+#
+# This is only one of many ways that you could have created this
+# strategy
+cat_or_dog = st.booleans().map(lambda x: "cat" if x else "dog")
+
+@given(cat_or_dog)
+def test_cat_dog(x):
+ assert x in {"cat", "dog"}
+```
+```python
+# running the test
+>>> test_cat_dog()
+```
+<!-- #endregion -->
+
+<!-- #region -->
+**Drawing from a strategy interactively: Solution**
+
+Write a test that is fed a list (of varying length) of non-negative integers.
+Then, draw a [set](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/DataStructures_III_Sets_and_More.html#The-%E2%80%9CSet%E2%80%9D-Data-Structure) of non-negative integers whose sum is at least as large as the sum of the list.
+Assert that the expected inequality between the sums hold.
+Run the test.
+
+
+Hint: use the `.filter()` method.
+<!-- #endregion -->
+
+<!-- #region -->
+```python
+@given(the_list=st.lists(st.integers(min_value=0)), data=st.data())
+def test_interactive_draw_skills(the_list, data):
+ the_set = data.draw(
+ st.sets(elements=st.integers(min_value=0)).filter(
+ lambda x: sum(x) >= sum(the_list)
+ )
+ )
+ assert sum(the_list) <= sum(the_set)
+```
+```python
+# running the test
+>>> test_interactive_draw_skills()
+```
+<!-- #endregion -->
From 0d1177f2c3c12d30c21243b50ea5be1a7bdcacf9 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 29 Mar 2020 13:32:55 -0400
Subject: [PATCH 119/152] fix some formatting
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 46bf2e7a..43482811 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -667,7 +667,7 @@ Clearly, in this instance, we should have simply used the strategy `st.integers(
We will often need to draw from a Hypothesis strategy in a context-dependent manner within our test.
Suppose, for example, that we want to describe two lists of integers, but we want to be sure that the second list is longer than the first.
-[We can use the `st.data()` strategy to use strategies "interactively"](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests) in this sort of way.
+[We can use the st.data() strategy to use strategies "interactively"](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests) in this sort of way.
Let's see it in action.
Suppose that we want to generate two non-empty lists of integers, `x` and `y`, but we want to ensure that the values stored in `y` values are *larger than all of the values in* `x`.
From c4267ed3828735321cec86d46eb615b2f7facbf0 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 29 Mar 2020 13:42:25 -0400
Subject: [PATCH 120/152] add some docs to example
---
Python/Module6_Testing/Hypothesis.md | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 43482811..07133158 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -679,10 +679,13 @@ from typing import List
# Defining our test function:
# - `x` is a non-empty list of integers.
-# `- `data` is an object provided by Hypothesis that permits us
+# - `data` is an object provided by Hypothesis that permits us
# to draw interactively from other strategies within our test
@given(x=st.lists(st.integers(), min_size=1), data=st.data())
def test_two_constrained_lists(x, data):
+ # We pass `data.draw(...)` a hypothesis strategy - it will draw a value from it.
+ # Thus `y` is a non-empty list of integers, whose values are guaranteed to be
+ # larger than `max(x)`
y = data.draw(st.lists(st.integers(min_value=max(x) + 1), min_size=1), label="y")
largest_x = max(x)
From 6d4a94a2f552bddc63d97b5f204a97a67787ec3d Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:44:27 -0400
Subject: [PATCH 121/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 07133158..3a014da4 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -26,7 +26,7 @@ Indeed, these are questions that you may have been asking yourself when writing
[Hypothesis](https://hypothesis.readthedocs.io/) is a powerful Python library that empowers us to write a _description_ (specification, to be more precise) of the data that we want to use to exercise our test.
It will then *generate* test cases that satisfy this description and will run our test on these cases.
-Ultimately this an extremely powerful tool for enabling us to write high-quality automated tests for our code.
+Ultimately, this an extremely powerful tool for enabling us to write high-quality automated tests for our code.
Hypothesis can be installed via conda:
From 0b3e4c03c947ec77bef8045bcd5a13b3a465f2e7 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:44:47 -0400
Subject: [PATCH 122/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 3a014da4..daabf61f 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -149,7 +149,7 @@ Hypothesis' [given decorator](https://hypothesis.readthedocs.io/en/latest/detail
- drawing values from Hypothesis' so-called "strategies" for describing input data for our test function
- running the test function many times (up to 100 times, by default) given different input values drawn from the strategy
- - "shrinking" the drawn inputs to identify simple fail cases: if an error is raised by the test function during one of the many execution, the `given` decorator will attempt to "shrink" (i.e. simplify) the inputs that produce that same error before reporting them to the user
+ - "shrinking" the drawn inputs to identify simple fail cases: if an error is raised by the test function during one of the many executions, the `given` decorator will attempt to "shrink" (i.e. simplify) the inputs that produce that same error before reporting them to the user
- reporting the input values that caused the test function to raise an error
From 6821619faebd37a10cd272a340fa7eb1c2339085 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:44:59 -0400
Subject: [PATCH 123/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index daabf61f..f1379f6b 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -352,7 +352,7 @@ For example, the following strategy describes lists whose length varies from 2 t
- bounds (either inclusive or exclusive) on the floats being generated; this will naturally preclude `math.nan` from being generated
- the "width" of the floats; e.g. if you want to generate 16-bit or 32-bit floats vs 64-bit
(while Python's `float` is (usually) 64-bit, `width=32` ensures that the generated values can
- always be losslessly represented in 32 bits. This is mostly useful for Numpy arrays.)
+ always be losslessly represented in 32 bits. This is mostly useful for NumPy arrays.)
For example, the following strategy 64-bit floats that reside in the domain $[-100, 1]$:
From 5588f16e59ec8f570fec64a0d46f611f6b6fb777 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:45:11 -0400
Subject: [PATCH 124/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index f1379f6b..e4ec22fc 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -393,7 +393,7 @@ For example, the following strategy will strings of lowercase vowels from length
[st.just](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.just) is a strategy that "just" returns the value that you fed it. This is a convenient strategy that helps us to avoid having to manipulate our data before using it.
-Write suppose that we want a strategy that describes the shape of an array (i.e. a tuple of integers) that contains 1-20 two-dimensional vectors. E.g. `(5, 2)` is the shape of the array containing five two-dimensional vectors. We can leverage `st.just`, in conjunction with `st.integers` and `st.tuples`, towards this end:
+Suppose that we want a strategy that describes the shape of an array (i.e. a tuple of integers) that contains 1-20 two-dimensional vectors. E.g. `(5, 2)` is the shape of the array containing five two-dimensional vectors. We can leverage `st.just`, in conjunction with `st.integers` and `st.tuples`, towards this end:
```python
>>> st.tuples(st.integers(1, 20), st.just(2)).example()
From ea4207e8df60eb352203265ef4c1db5788cf1a06 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:45:29 -0400
Subject: [PATCH 125/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index e4ec22fc..c0e08ebf 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -404,7 +404,7 @@ Suppose that we want a strategy that describes the shape of an array (i.e. a tup
<!-- #region -->
#### `st.one_of()`
-The [st.one_of](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.one_of) allows us to specify a collection of strategies and any given datum will be drawn from "one of" them. E.g.
+The [st.one_of](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.one_of) strategy allows us to specify a collection of strategies and any given datum will be drawn from "one of" them. For example:
```python
# demonstrating st.one_of()
From bd88886f313e4a735703c0c9563cbdf7d2163ec1 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:46:25 -0400
Subject: [PATCH 126/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index c0e08ebf..7344264a 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -411,7 +411,7 @@ The [st.one_of](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis
st.one_of(st.integers(), st.lists(st.integers()))
```
-will draw values that are *either* integers or list of integers:
+will draw values that are *either* integers or lists of integers:
```python
>>> st.one_of(st.integers(), st.lists(st.integers())).example()
From 40c7161ef8963e60eae870324e9b9710018969ef Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:46:50 -0400
Subject: [PATCH 127/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 7344264a..45eec04c 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -424,7 +424,7 @@ will draw values that are *either* integers or lists of integers:
The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` calls:
```python
-# Using the pipe operation, | , in place of `st.one_of`
+# Using the pipe operator, | , in place of `st.one_of`
# This strategy generates integers or floats
# or lists that contain just the word "hello"
From de0a696aebd3284131caa8fc411a402f5ee6f5b7 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:47:09 -0400
Subject: [PATCH 128/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 45eec04c..ccb70c0f 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -439,7 +439,7 @@ The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` ca
<!-- #region -->
#### `st.sampled_from`
-[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and numpy arrays). The strategy will return values that are sampled from this collections.
+[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and NumPy arrays). The strategy will return values that are sampled from this collection.
For example, the following strategy will sample from a list either `0`, `"a"`, or `(2, 2)`
From 97d6e73fff366b58ecd9f01f3eeeb3c1639648a5 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:47:27 -0400
Subject: [PATCH 129/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index ccb70c0f..8957f23a 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -441,7 +441,7 @@ The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` ca
[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and NumPy arrays). The strategy will return values that are sampled from this collection.
-For example, the following strategy will sample from a list either `0`, `"a"`, or `(2, 2)`
+For example, the following strategy will sample a value `0`, `"a"`, or `(2, 2)` from a list:
```python
>>> st.sampled_from([0, "a", (2, 2)]).example()
From ea9d6abbed3f74c55ff83b0fecafe99af7926df0 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:47:40 -0400
Subject: [PATCH 130/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 8957f23a..dd7a7e75 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -456,7 +456,7 @@ For example, the following strategy will sample a value `0`, `"a"`, or `(2, 2)`
Review the [rest of Hypothesis' core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies).
Write down a strategy, and print out a representative example, that describes the the data according to each of the following conditions:
- 1. Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False.
+ 1. Dictionaries of arbitrary size whose keys are positive-valued integers and whose values are `True` or `False.
2. Length-4 strings whose elements are only lowercase vowels
3. Permutations of the list `[1, 2, 3, 4]`
From 88eacf138c24555e3c0c5bf58112508e55c245aa Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:48:02 -0400
Subject: [PATCH 131/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index dd7a7e75..9414c74c 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -475,7 +475,7 @@ Keep in mind that solutions are included at the end of this page, and that these
Part 1: Testing correctness by construction
-Write a hypothesis-driven test for the `count_vowels`; include this test in `test/test_basic_functions`.
+Write a Hypothesis-driven test for the `count_vowels`; include this test in `tests/test_basic_functions`.
This is a test function where we can explicit construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
From 3667ffc50dc117ea09062e48eff613c4835fefb7 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:48:19 -0400
Subject: [PATCH 132/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 9414c74c..115fb2ad 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -476,7 +476,7 @@ Keep in mind that solutions are included at the end of this page, and that these
Part 1: Testing correctness by construction
Write a Hypothesis-driven test for the `count_vowels`; include this test in `tests/test_basic_functions`.
-This is a test function where we can explicit construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
+This is a test function where we can explicitly construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
From 3936c71fcb44476b8f61e7507e08b9b06f08de0b Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:48:34 -0400
Subject: [PATCH 133/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 115fb2ad..e026543d 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -481,7 +481,7 @@ And thus, by constructing a string with a known number of vowel and non-vowel ch
We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
-We should ask ourselves: How general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
+We should ask ourselves: how general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
Part 2: Property-based testing
From bf61b1b83821d5fbc244bf5a9c7179f97369227f Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Mon, 30 Mar 2020 14:48:48 -0400
Subject: [PATCH 134/152] Update Python/Module6_Testing/Hypothesis.md
Co-Authored-By: David Mascharka <davidmascharka@users.noreply.github.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index e026543d..28ab7517 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -486,7 +486,7 @@ We should ask ourselves: how general are input strings that we are constructing?
Part 2: Property-based testing
-Write a hypothesis-driven test for `merge_max_mappings` ; include this test in `test/test_basic_functions`.
+Write a Hypothesis-driven test for `merge_max_mappings` ; include this test in `tests/test_basic_functions`.
Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
Instead, we should *test the expected properties* of the merged dictionary.
For example, one such property is that the merged dictionary should only contain maximum values.
From 6633bb89dd424f95eb3ed3df53f4afaadfe2eaff Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sat, 4 Apr 2020 16:29:36 -0400
Subject: [PATCH 135/152] add hypothesis practice exercises
---
.../Hypothesis_Practice_Exercises.md | 232 ++++++++++++++++++
1 file changed, 232 insertions(+)
create mode 100644 Python/Module6_Testing/Hypothesis_Practice_Exercises.md
diff --git a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
new file mode 100644
index 00000000..67c6ca95
--- /dev/null
+++ b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
@@ -0,0 +1,232 @@
+---
+jupyter:
+ jupytext:
+ text_representation:
+ extension: .md
+ format_name: markdown
+ format_version: '1.2'
+ jupytext_version: 1.3.0
+ kernelspec:
+ display_name: Python [conda env:scicomp]
+ language: python
+ name: conda-env-scicomp-py
+---
+
+<!-- #raw raw_mimetype="text/restructuredtext" -->
+.. meta::
+ :description: Topic: Practice exercises using Hypothesis, Difficulty: Easy, Category: Section
+ :keywords: hypotehsis, practice, exercise, test
+<!-- #endraw -->
+
+# Practice Exercises Using Hypothesis
+
+Hypothesis will not only improve the quality of our tests, but it should also save us time and cognitive load as it simplifies the process for describing the data that we want to pass to our code.
+That being said, it can take some practice to learn one's way around [Hypothesis' core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies);
+thus this section is dedicated to providing some useful exercises towards this end.
+
+
+<div class="alert alert-info">
+
+**Exercise: Describing data with `st.lists`**
+
+Write a strategy that generates lists of even-valued integers, ranging from length-0 to length-10.
+
+Write a test that checks these properties and run the test.
+
+</div>
+
+
+
+<div class="alert alert-info">
+
+**Exercise: Using Hypothesis to learn about floats.. Part 1**
+
+Use the `st.floats` strategy to identify which float(s) violate the identity: `x == x`.
+That is, write a hypothesis-driven test for which `assert x == x` *fails*, run the test, and identify the input that causes the failure.
+
+Then, revise your usage of `st.floats` such that it only describes values that satisfy the identity.
+Run the test to ensure that your assumptions are correct.
+</div>
+
+
+<!-- #region -->
+<div class="alert alert-info">
+
+**Exercise: Using Hypothesis to learn about floats.. Part 2**
+
+Use the `st.floats` strategy to identify which **positive** float(s) violate the inequality: `x < x + 1`.
+
+(To interpret your findings, it is useful to know that a double-precision (64-bit) binary floating-point number, which is representative of Python's `float`, has a coefficient of 53 bits (which actually only takes 52 bits to represent), an exponent of 11 bits, and 1 sign bit.)
+
+
+Then, revise your usage of `st.floats` such that it only describes values that satisfy the identity. **Use the `example` decorator to ensure that the identified boundary case is tested every time**.
+</div>
+
+<!-- #endregion -->
+
+<div class="alert alert-info">
+
+**Exercise: Stitching together strategies for rich behavior**
+
+Write a strategy that draws a tuple of two perfect squares (integers) or three perfect cubes (integers)
+
+Use view some examples to examine the behavior of your strategy.
+</div>
+
+
+
+<div class="alert alert-info">
+
+**Exercise: Describing objects that evaluate to `False`**
+
+Write a strategy that can return the boolean, integer, float, string, list, tuple, or dictionary that evaluates to `False` (when called on by `bool`)
+
+</div>
+
+
+
+## Solutions
+
+<!-- #region -->
+**Describing data with `st.lists`**
+
+```python
+
+# generates "any" integer and then multiplies that value
+# by two, ensuring that it is an even number
+even_integers = st.integers().map(lambda x: 2 * x)
+
+# Recall that `st.lists(...)` can take any strategy. Thus
+# we feed it our `even_integers` strategy
+@given(x=st.lists(even_integers, min_size=0, max_size=10))
+def test_even_lists(x):
+ assert isinstance(x, list) and 0 <= len(x) <= 10
+ assert all(isinstance(i, int) for i in x)
+ assert all(i % 2 == 0 for i in x)
+
+```
+
+```python
+# running the test
+>>> test_even_lists()
+```
+<!-- #endregion -->
+
+<!-- #region -->
+**Exercise: Using Hypothesis to learn about floats.. Part 1**
+
+```python
+# using `st.floats` to find value(s) that violate `x == x`
+
+@given(x=st.floats())
+def test_broken_identity(x):
+ assert x == x
+```
+
+Running this test reveals..
+
+```python
+>>> test_broken_identity()
+Falsifying example: test_broken_identity(
+ x=nan,
+)
+```
+
+that "NaN" (which stands for [Not a Number](https://en.wikipedia.org/wiki/NaN)) is not equal to itself.
+This is, in fact, [the designed behavior of NaN](https://en.wikipedia.org/wiki/NaN#Comparison_with_NaN) in the specification for floating point numbers.
+
+Now let's updated our strategy for describing floating point numbers to exclude this.
+
+```python
+@given(x=st.floats(allow_nan=False))
+def test_fixed_identity(x):
+ assert x == x
+```
+
+Assuming that NaN is the only floating point that violates the self-identity, this test should now pass.
+
+```python
+>>> test_fixed_identity()
+```
+<!-- #endregion -->
+
+<!-- #region -->
+**Exercise: Using Hypothesis to learn about floats.. Part 2**
+
+```python
+# using `st.floats` to find value(s) that violate `x < x + 1`
+
+@given(x=st.floats(min_value=0))
+def test_broken_inequality(x):
+ assert x < x + 1
+```
+```python
+>>> test_broken_inequality()
+Falsifying example: test_broken_inequality(
+ x=9007199254740992.0,
+)
+```
+We can check that:
+
+```python
+>>> import math
+>>> math.log2(9007199254740992)
+53
+```
+
+Recall that `2 ** 53` is the maximum size that the coefficient of a floating point double can take on. Thus Hypothesis is pointing out that `2 ** 53 + 1` can't be represented as a 64-bit floating point number.
+Let's see what happens when we do try to add one to `2 ** 53`:
+
+```python
+>>> x = 2.0 ** 53; x
+9007199254740992.0
+
+>>> x + 1
+9007199254740992.0
+```
+
+The value doesn't change at all because we would have to exceed 64-bits allotted to represent a floating point number.
+Thus `x < x + 1` fails to hold for `x = 2.0 ** 53`.
+
+
+Now let's update our test to test everything up to this maximum value.
+Note that we want to ensure that we are testing the maximum permitted value every time we run the test as hypothesis does not guarantee that it will do so.
+We leverage the `example` decorator to accomplish this.
+
+```python
+# updating our usage of `st.floats` to generate only values that satisfy `x < x + 1`
+from hypothesis import example
+
+
+@example(x=2.0 ** 53 - 1) # ensures that maximum permissible value is tested
+@given(x=st.floats(min_value=0, max_value=2.0 ** 53, exclude_max=True))
+def test_fixed_inequality(x):
+ assert x < x + 1
+```
+
+```python
+# running our test
+>>> test_fixed_inequality()
+```
+<!-- #endregion -->
+
+<!-- #region -->
+**Stitching together strategies for rich behavior**
+
+```python
+squares = st.integers().map(lambda x: x ** 2)
+cubes = st.integers().map(lambda x: x ** 3)
+
+# Recall that the pipe operator, `|`, is a convenient way for
+# calling `st.one_of(...)` on strategies
+squares_or_cubes = st.tuples(squares, squares) | st.tuples(cubes, cubes, cubes)
+```
+<!-- #endregion -->
+
+<!-- #region -->
+**Describing objects that evaluate to `False`**
+
+```python
+falsies = st.sampled_from([False, 0, 0.0, "", [], tuple(), {}])
+```
+<!-- #endregion -->
From fd703e1a773941b71e1ac246e139eb7d48580a78 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Sun, 5 Apr 2020 15:22:28 -0400
Subject: [PATCH 136/152] add beginning of Writing Your Own Strategies
---
.../Writing_Your_Own_Strategy.md | 1192 +++++++++++++++++
1 file changed, 1192 insertions(+)
create mode 100644 Python/Module6_Testing/Writing_Your_Own_Strategy.md
diff --git a/Python/Module6_Testing/Writing_Your_Own_Strategy.md b/Python/Module6_Testing/Writing_Your_Own_Strategy.md
new file mode 100644
index 00000000..85f318ca
--- /dev/null
+++ b/Python/Module6_Testing/Writing_Your_Own_Strategy.md
@@ -0,0 +1,1192 @@
+---
+jupyter:
+ jupytext:
+ text_representation:
+ extension: .md
+ format_name: markdown
+ format_version: '1.2'
+ jupytext_version: 1.3.0
+ kernelspec:
+ display_name: Python [conda env:scicomp]
+ language: python
+ name: conda-env-scicomp-py
+---
+
+<!-- #raw raw_mimetype="text/restructuredtext" -->
+.. meta::
+ :description: Topic: Writing Your Own Hypothesis Strategy, Difficulty: Medium, Category: Section
+ :keywords: test, python, hypotheis, composite, decorator, strategy, property-based
+<!-- #endraw -->
+
+<!-- #region -->
+# Writing Your Own Hypothesis Strategy
+
+Hypothesis' key role in our test-writing practice is that it permits us to describe the data, and the relationships between the data, that we want to pass to our code, and to generate data according to these descriptions.
+It is often the case that a Python project will use specialized data structures and patterns of data, and thus testing this project's code requires that we can describe this specialized data.
+Towards this end, Hypothesis provides us with the [composite](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.composite) decorator, which permits us to form our own strategies for describing data by composing Hypothesis' built-in strategies.
+
+The general syntax for defining a custom Hypothesis strategy using the `composite` decorator is as follows:
+
+```python
+from hypothesis import composite
+
+@composite
+def custom_strategy(draw, <user-facing parameters>):
+ # Use `draw(...)` to draw values from other hypothesis
+ # strategies. We can leverage the values passed in via
+ # <user-facing parameters>
+ # E.g.
+ # `x = draw(st.integers(1, 4))`
+ # will result in `x` being assigned an integer value
+ # between 1 and 4.
+
+ # A composite strategy should be written to
+ # return standard Python objects (e.g. ints, lists, etc.);
+ # it is *not* meant to return other Hypothesis strategies
+ return <value(s) to be returned by strategy>
+
+```
+<!-- #endregion -->
+
+Let's see this in action by writing a Hypothesis strategy that will produce the left and right bounds for an interval on the number line, and that permits the user to constrain various aspects of the intervals that can be generated by the strategy.
+
+<!-- #region -->
+```python
+from hypothesis.strategies import composite
+import hypothesis.strategies as st
+
+
+@composite
+def interval_bounds(draw, left_bnd, right_bnd, min_size=0.0, max_size=None):
+ """A Hypothesis data strategy for generating ordered bounds on the real number line.
+
+ Note: The `draw` parameter it reserved by Hypothesis and is not exposed by
+ the function signature.
+
+ Parameters
+ ----------
+ left_bnd : float
+ The smallest value that the left-bound can take on.
+
+ right_bnd : float
+ The largest value that the right-bound can take on.
+
+ min_size : float, optional (default=0.0)
+ The guaranteed minimum separation of the bounds.
+
+ max_size : Optional[float]
+ The guaranteed maximum separation of the bounds. Defaults to `right_bnd - left_bnd`
+
+ Returns
+ -------
+ st.SearchStrategy[Tuple[float, float]]
+ """
+ # Let `a` represent left-bound and `b` represent right-bound
+ # let `d` represent the smalled permitted interval
+
+ # `drawn_left` is a float
+ # Is a number in [a, b - d]
+ drawn_left = draw(st.floats(min_value=left_bnd, max_value=right_bnd - min_size))
+
+ # `drawn_right` is a float
+ # Is a number in [a + d, b]
+ drawn_right = draw(
+ st.floats(
+ min_value=drawn_left + min_size,
+ max_value=min(right_bnd, drawn_left + max_size),
+ )
+ )
+
+ # Note that a composite strategy definition should return the drawn values,
+ # and *not* Hypothesis strategies.
+ #
+ # E.g. here we return a tuple of floating point numbers, and
+ # *not* `st.tuples(st.floats(), st.floats())`
+ return (drawn_left, drawn_right)
+```
+
+<!-- #endregion -->
+
+The first argument, `draw`, is required to be present in our function definition by the `@composite` decorator.
+It is a function that is used by Hypothesis to draw values from strategies in order to generate data from our composite strategy. Each `draw(...)` call simply produces a value from that strategy.
+Thus `drawn_left` and `drawn_right` are floating point numbers.
+We then return a tuple of these floats, as expected from this strategy.
+That being said, *the* `draw` *parameter is not exposed to users*.
+See that it is not present in the function signature for our strategy when we query it via `help()`.
+
+
+
+<!-- #region -->
+```python
+# The `draw` is not exposed to users. It is only meant
+# to be leveraged when defining the custom strategy
+>>> help(interval_bounds)
+Help on function interval_bounds in module __main__:
+
+interval_bounds(left_bnd, right_bnd, min_size=0.0, max_size=None)
+[...]
+```
+<!-- #endregion -->
+
+Note that, even though the resulting function `interval_bounds(...)` is a Hypothesis search strategy, the return statement in the body of the definition *specifies what values are to be returned by the strategy*.
+That is, it does *not* return strategy-instances.
+This is at odds with the "Returns" section of the docstring, which informs the user that calling `interval_bounds(...)` in face returns a search strategy.
+
+Let's consider some usages of this strategy:
+
+<!-- #region -->
+```python
+# The following pieces of code generate a Hypothesis search
+# strategy that...
+
+# generates any pair of bounds on [-10, 10]
+>>> interval_bounds(left_bnd=-10, right_bnd=10)
+
+# generates any pair of bounds on [0, 1] whose difference is
+# at least 0.5
+>>> interval_bounds(left_bnd=0, right_bnd=1, min_size=.5)
+
+# generates any pair of bounds on [-100, 100] whose difference is
+# at least 1 and is no larger than 5
+>>> interval_bounds(left_bnd=-100, right_bnd=100, min_size=1, max_size=5)
+```
+<!-- #endregion -->
+
+Now that we have created our own strategy, we can draw values from it in a test as we would any other Hypothesis strategy:
+
+<!-- #region -->
+```python
+# Using our custom strategy in a test
+
+from hypothesis import given
+
+@given(bnd=interval_bounds(left_bnd=0, right_bnd=1000))
+def test_a_function(bnd):
+ left_bnd, right_bnd = bnd
+ # ...
+ # conduct a test using the bounds drawn from our
+ # custom strategy
+```
+<!-- #endregion -->
+
+Note the power of the `composite` operator; we can return *any* sort of value from our custom strategy, generated via arbitrarily-complicated logic.
+To this end, we can leverage this decorator to generate instances of custom classes, patterned NumPy arrays, specialized rest API calls, etc.
+That being said, we must keep in mind that Hypothesis will, by default, make this function call one hundred times for each test evaluation that uses our custom strategy.
+Thus we should be conscientious about the computational cost associated with our strategy's data generation process.
+This is a place where writing efficient code really counts.
+
+<!-- #region -->
+<div class="alert alert-warning">
+
+**With Great Power Comes Great Responsibility**:
+
+Hypothesis' `composite` enables us to create rich strategies for describing the data that we want to use to test our code.
+This is empowering indeed, however we must take great care that our custom strategies are *describing our data correctly*.
+That is, we must be sure that we do not have bugs in our strategies.
+It is easy to imagine the frustration that we would experience when we find that our code is failing a test and searching for the bug in our code, only to find that the Hypothesis strategy that we wrote isn't actually behaving as we had intended.
+
+With this scenario in mind, we are motivated to be extra-fastidious when writing our own strategies.
+There are several steps that we can take to help ensure that we are not making our lives harder by writing buggy strategies:
+
+ 1. Validate user inputs to ensure that mutually-inconsistent parameters are not permitted, and include clear, verbose error messages when invalid inputs are provided.
+ 2. Make liberal use of internal assertion statements within a strategy's definition to double check the consistency of our own logic
+ 3. Write tests that exercise our custom strategies in controlled and contrived ways, to ensure that their expected properties hold. (That's right, we ought to write tests for the tooling that helps us write our tests!)
+
+Let's rewrite `interval_bounds` to exercise points 1 and 2.
+
+```python
+@composite
+def interval_bounds(draw, left_bnd, right_bnd, min_size=0.0, max_size=None):
+ """A Hypothesis data strategy for generating ordered bounds on the real number line."""
+
+ if right_bnd < left_bnd + min_size:
+ raise ValueError(
+ f"Unsatisfiable bounds: [left_bnd={left_bnd}, right_bnd={right_bnd}], "
+ f"min-interval size: {min_size}"
+ )
+
+ if max_size is None:
+ max_size = right_bnd - left_bnd
+ elif max_size < min_size:
+ raise ValueError(
+ f"Unsatisfiable interval bounds: min_size={min_size}, max_size={max_size}"
+ )
+
+ drawn_left = draw(st.floats(min_value=left_bnd, max_value=right_bnd - min_size))
+
+ drawn_right = draw(
+ st.floats(
+ min_value=drawn_left + min_size,
+ max_value=min(right_bnd, drawn_left + max_size),
+ )
+ )
+
+ # ensure that strategy behaves as-promised
+ assert left_bnd <= drawn_left <= right_bnd
+ assert left_bnd <= drawn_right <= right_bnd
+ assert min_size <= drawn_right - drawn_left <= max_size
+
+ return (drawn_left, drawn_right)
+```
+
+See that the `ValueError` exceptions that we have included make it clear to the user when they have passed invalid inputs to our strategy; this makes for much clearer feedback than to have `st.floats()` raise an exception because it received invalid bounds.
+
+The inclusion of assertion expressions ensure, critically, that our strategy is obeying the constraints that the user provided.
+It is better to see that our strategy is raising an internal assertion error than it is for it to "silently" generate bounds that ought to be invalid.
+
+It is important that we recognize that these measures help to ensure that our strategy is never generating bad data.
+That being said, *these do not provide any guarantees that the strategy is generating appropriately diverse data*.
+It could be that, had we unwittingly introduced bad logic into our strategy, it generates a pair of valid bounds, but that these bounds never change during the data generation process.
+This is where some level of manual inspection of our strategy's outputs, or tests that ensure that our strategy is capable of generating sufficiently-general data is important.
+</div>
+<!-- #endregion -->
+
+```python
+@given(x=interval_bounds(-10, 10, min_size=0, max_size=1))
+def test(x):
+ print(x, x[1] - x[0])
+```
+
+```python
+test()
+```
+
+<!-- #region -->
+```python
+# A basic introduction to Hypothesis
+
+from hypothesis import given
+
+# Hypothesis provides so-called "strategies" for us
+# to describe our data
+import hypothesis.strategies as st
+
+# Using hypothesis to test any integer value in [0, 10 ** 10]
+@given(size=st.integers(min_value=0, max_value=1E10))
+def test_range_length(size):
+ assert len(range(size)) == size
+```
+<!-- #endregion -->
+
+<!-- #region -->
+Here we have specified that the value of `size` in our test *should be able to take on any integer value within* $[0, 10^{10}]$.
+We did this by using the `integers` "strategy" that is provided by Hypothesis: `st.integers(min_value=0, max_value=1E10)`.
+When we execute the resulting test (which can simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
+by default, Hypothesis will generate 100 test cases - an amount that we can configure - and will execute our test function for each one of them.
+
+```python
+# Running this test once will trigger Hypothesis to
+# generate 100 values based on the description of our data,
+# and it will execute the test using each one of those values
+>>> test_range_length()
+```
+
+With great ease, we were able to replace our pytest-parameterized test, which only very narrowly tested the property at hand, with a much more robust, Hypothesis-driven test.
+This will be a recurring trend: we will generally produce much more robust tests by _describing_ our data with Hypothesis, rather than manually specifying test values.
+
+The rest of this section will be dedicated to learning about the Hypothesis library and how we can leverage it to write powerful tests.
+<!-- #endregion -->
+
+<!-- #region -->
+## The `given` Decorator
+
+Hypothesis' [given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters) is responsible for:
+
+ - drawing values from Hypothesis' so-called "strategies" for describing input data for our test function
+ - running the test function many times (up to 100 times, by default) given different input values drawn from the strategy
+ - "shrinking" the drawn inputs to identify simple fail cases: if an error is raised by the test function during one of the many executions, the `given` decorator will attempt to "shrink" (i.e. simplify) the inputs that produce that same error before reporting them to the user
+ - reporting the input values that caused the test function to raise an error
+
+
+Let's see the `given` decorator in action by writing a simple "test" for which `x` should be integers between 0 and 10, and `y` should be integers between 20 and 30.
+To do this we will make use of the `integers` Hypothesis strategy.
+Let's include a bad assertion statement – that `y` can't be larger than 25 – to see how Hypothesis reports this fail case.
+Note that we aren't really testing any code here, we are simply exercising some of the tools that Hypothesis provides us with.
+
+```python
+from hypothesis import given
+import hypothesis.strategies as st
+
+# using `given` with multiple parameters
+# `x` is an integer drawn from [0, 10]
+# `y` is an integer drawn from [20, 30]
+@given(x=st.integers(0, 10), y=st.integers(20, 30))
+def test_demonstrating_the_given_decorator(x, y):
+ assert 0 <= x <= 10
+
+ # `y` can be any value in [20, 30]
+ # this is a bad assertion: it should fail!
+ assert 20 <= y <= 25
+```
+
+See that the names of the parameters specified in `given` — `x` and `y` in this instance — must match those in the signature of the test function.
+
+To run this test function, we simply call `test_demonstrating_the_given_decorator()`.
+Note that, unlike with a typical function, we do not pass values for `x` and `y` to this function – *the* `given` *decorator will pass these values to the function for us*.
+Executing this `given`-decorated function will prompt Hypothesis to draw 100 pairs of values for `x` and `y`, according to their respective strategies, and the body of the test will be executed for each such pair.
+
+```python
+# running the test
+>>> test_demonstrating_the_given_decorator()
+Falsifying example: test_demonstrating_the_given_decorator(
+ x=0, y=26,
+)
+---------------------------------------------------------------------------
+AssertionError Traceback (most recent call last)
+<ipython-input-29-ea20353bbef5> in test_demonstrating_the_given_decorator(x, y)
+ 10 # `y` can be any value in [20, 30]
+ 11 # this is a bad assertion: it should fail!
+---> 12 assert 20 <= y <= 25
+
+AssertionError:
+
+```
+
+(Note: some of the "Traceback" error message has been removed to improve legibility)
+
+See that the error message here indicates that Hypothesis identified a "falsifying example", or a set of input values, `x=0` and `y=26`, which caused our test function to raise an error. The proceeding "Traceback" message indicates that it is indeed the second assertion statement that is responsible for raising the error.
+
+### Shrinking: Simplifying Falsifying Inputs
+
+The `given` decorator strives to report the "simplest" set of input values that produce a given error.
+It does this through the process of "shrinking".
+Each of Hypothesis' strategies has its own prescribed shrinking behavior.
+For the `integers` strategy, this means identifying the integer closest to 0 that produces the error at hand.
+For instance, `x=12` and `y=29` may have been the first drawn values to trigger the assertion error.
+These values were then incrementally reduced by the `given` decorator until `x=0` and `y=26` were identified as the smallest set of values to reproduce this fail case.
+
+We can print out the examples that Hypothesis generated:
+
+```
+x=0 y=20 - PASSED
+x=0 y=20 - PASSED
+x=0 y=20 - PASSED
+x=9 y=20 - PASSED
+x=9 y=21 - PASSED
+x=3 y=20 - PASSED
+x=3 y=20 - PASSED
+x=9 y=26 - FAILED
+x=3 y=26 - FAILED
+x=6 y=26 - FAILED
+x=10 y=27 - FAILED
+x=7 y=27 - FAILED
+x=3 y=30 - FAILED
+x=3 y=23 - PASSED
+x=10 y=30 - FAILED
+x=3 y=27 - FAILED
+x=3 y=27 - FAILED
+x=2 y=27 - FAILED
+x=0 y=27 - FAILED
+x=0 y=26 - FAILED
+x=0 y=21 - PASSED
+x=0 y=25 - PASSED
+x=0 y=22 - PASSED
+x=0 y=23 - PASSED
+x=0 y=24 - PASSED
+x=0 y=26 - FAILED
+```
+
+See that Hypothesis has to do a semi-random search to identify the boundaries of the fail case; it doesn't know if `x` is causing the error, or if `y` is the culprit, or if it is specific *combinations* of `x` and `y` that causes the failure!
+Despite this complexity, the pairs of variables are successfully shrunk to the simplest fail case.
+<!-- #endregion -->
+
+<div class="alert alert-warning">
+
+**Hypothesis will Save Falsifying Examples**:
+
+Albeit an advanced detail, it is important to note that Hypothesis does not have to search for falsifying examples from scratch every time we run a test function.
+Instead, Hypothesis will save a database of falsifying examples associated with each of your project's test functions.
+The database is saved under `.hypothesis` in whatever directory your test functions were run from.
+
+This ensures that, once Hypothesis finds a falsifying example for a test, the falsifying example will be passed to your test function each time you run it, until it no longer raises an error in your code (e.g. you update your code to fix the bug that was causing the test failure).
+</div>
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Understanding How Hypothesis Works**
+
+Define the `test_demonstrating_the_given_decorator` function as above, complete with the failing assertion, and add a print statement to the body of the function, which prints out the value for `x` and `y`.
+
+Run the test once and make note of the output that is printed. Consider copying and pasting the output to a notepad for reference. Next, rerun the test multiple times and make careful note of the printed output. What do you see? Is the output different from the first run? Does it differ between subsequent runs? Try to explain this behavior.
+
+In your file browser, navigate to the directory from which you are running this test; if you are following along in a Jupyter notebook, this is simply the directory containing said notebook. You should see a `.hypothesis` directory. As noted above, this is the database that contains the falsifying examples that Hypothesis has identified. Delete the `.hypothesis` directory and try re-running your test? What do you notice about the output now? You should also see that the `.hypothesis` directory has reappeared. Explain what is going on here.
+
+</div>
+
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Fixing the Failing Test**
+
+Update the body of `test_demonstrating_the_given_decorator` so that it no longer fails. Run the fixed test function. How many times is the test function actually be executed when you run it?
+
+</div>
+
+
+
+## Describing Data: Hypothesis Strategies
+
+Hypothesis provides us with so-called "strategies" for describing our data.
+We are already familiar with the `integers` strategy;
+Hypothesis' core strategies are all located in the `hypothesis.strategies` module.
+The official documentation for the core strategies can be found [here](https://hypothesis.readthedocs.io/en/latest/data.html).
+
+Here, we will familiarize ourselves with these core strategies and will explore some of the powerful methods that can be used to customize their behaviors.
+
+<!-- #region -->
+### Drawing examples from strategies
+
+Hypothesis provides a useful mechanism for developing an intuition for the data produced by a strategy: a strategy, once initialized, has a `.example()` method that will randomly draw a representative value from the strategy. For example:
+
+```python
+# demonstrating usage of `<strategy>.example()`
+>>> st.integers(-1, 1).example()
+-1
+
+>>> st.integers(-1, 1).example()
+1
+```
+
+**Note: the** `.example()` **mechanism is only meant to be used for pedagogical purposes. You should never use this in your test suite**
+because (among other reasons) `.example()` biases towards smaller and simpler examples than `@given`, and lacks the features to ensure any test failures are reproducible.
+
+We will be leveraging the `.example()` method throughout the rest of this section to help provide an intuition for the data that Hypothesis' various strategies generate.
+<!-- #endregion -->
+
+<!-- #region -->
+### Exploring Strategies
+
+There are a number critical Hypothesis strategies for us to become familiar with. It is worthwhile to peruse through all of Hypothesis' [core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies), but we will take time to highlight a few here.
+
+#### `st.booleans ()`
+
+[st.booleans()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.booleans) generates either `True` or `False`. This strategy will shrink towards `False`
+
+```python
+>>> st.booleans().example()
+False
+```
+<!-- #endregion -->
+
+<!-- #region -->
+
+#### `st.lists ()`
+
+[st.lists](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.lists) accepts *another* strategy, which describes the elements of the lists being generated. You can also specify:
+ - bounds on the length of the list
+ - if we want the elements to be unique
+ - a mechanism for defining "uniqueness"
+
+For example, the following strategy describes lists whose length varies from 2 to 10, and whose entries are integers on the domain $[-10, 20]$:
+
+```python
+>>> st.lists(st.integers(-10, 20), min_size=2, max_size=10).example()
+[-10, 0, 5]
+```
+
+**`st.lists(...)` is the strategy of choice anytime we want to generate sequences of varying lengths with elements that are, themselves, described by strategies**.
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.floats()`
+
+[st.floats](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.floats) is a powerful strategy that generates all variety of floats, including `math.inf` and `math.nan`. You can also specify:
+ - whether `math.inf` and `math.nan`, respectively, should be included in the data description
+ - bounds (either inclusive or exclusive) on the floats being generated; this will naturally preclude `math.nan` from being generated
+ - the "width" of the floats; e.g. if you want to generate 16-bit or 32-bit floats vs 64-bit
+ (while Python's `float` is (usually) 64-bit, `width=32` ensures that the generated values can
+ always be losslessly represented in 32 bits. This is mostly useful for NumPy arrays.)
+
+For example, the following strategy 64-bit floats that reside in the domain $[-100, 1]$:
+
+```python
+>>> st.floats(-100, 1).example()
+0.3670816313319896
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.tuples()`
+
+The [st.tuples](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.tuples) strategy accepts $N$ Hypothesis strategies, and will generate length-$N$ tuples whose elements are drawn from the respective strategies that were specified as inputs.
+
+For example, the following strategy will generate length-3 tuples whose entries are: integers, booleans, and floats:
+
+```python
+>>> st.tuples(st.integers(), st.booleans(), st.floats()).example()
+(4628907038081558014, False, -inf)
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.text()`
+
+The [st.text](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) accepts an "alphabet" – a collection of string-characters – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
+
+For example, the following strategy will strings of lowercase vowels from length 2 to length 10:
+
+```python
+>>> st.text("aeiouy", min_size=2, max_size=10).example()
+'oouoyoye'
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.just()`
+
+[st.just](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.just) is a strategy that "just" returns the value that you fed it. This is a convenient strategy that helps us to avoid having to manipulate our data before using it.
+
+Suppose that we want a strategy that describes the shape of an array (i.e. a tuple of integers) that contains 1-20 two-dimensional vectors. E.g. `(5, 2)` is the shape of the array containing five two-dimensional vectors. We can leverage `st.just`, in conjunction with `st.integers` and `st.tuples`, towards this end:
+
+```python
+>>> st.tuples(st.integers(1, 20), st.just(2)).example()
+(7, 2)
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.one_of()`
+
+The [st.one_of](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.one_of) strategy allows us to specify a collection of strategies and any given datum will be drawn from "one of" them. For example:
+
+```python
+# demonstrating st.one_of()
+st.one_of(st.integers(), st.lists(st.integers()))
+```
+
+will draw values that are *either* integers or lists of integers:
+
+```python
+>>> st.one_of(st.integers(), st.lists(st.integers())).example()
+144
+
+>>> st.one_of(st.integers(), st.lists(st.integers())).example()
+[0, -22]
+```
+
+The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` calls:
+
+```python
+# Using the pipe operator, | , in place of `st.one_of`
+# This strategy generates integers or floats
+# or lists that contain just the word "hello"
+
+>>> (st.integers() | st.floats() | st.lists(st.just("hello"))).example()
+['hello', 'hello']
+
+>>> (st.integers() | st.floats() | st.lists(st.just("hello"))).example()
+0
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### `st.sampled_from`
+
+[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and NumPy arrays). The strategy will return values that are sampled from this collection.
+
+For example, the following strategy will sample a value `0`, `"a"`, or `(2, 2)` from a list:
+
+```python
+>>> st.sampled_from([0, "a", (2, 2)]).example()
+'a'
+```
+<!-- #endregion -->
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Exploring other Core Strategies**
+
+Review the [rest of Hypothesis' core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies).
+Write down a strategy, and print out a representative example, that describes the the data according to each of the following conditions:
+
+ 1. Dictionaries of arbitrary size whose keys are positive-valued integers and whose values are `True` or `False.
+ 2. Length-4 strings whose elements are only lowercase vowels
+ 3. Permutations of the list `[1, 2, 3, 4]`
+
+</div>
+
+
+<!-- #region -->
+<div class="alert alert-info">
+
+**Reading Comprehension: Improving our tests using Hypothesis**
+
+We will be writing improved tests for the basic functions – `count_vowels` and `merge_max_mappings` – by leveraging Hypothesis.
+This reading comprehension question will require more substantial work than usual.
+That being said, the experience that we will gain from this will be well worth the work.
+Keep in mind that solutions are included at the end of this page, and that these can provide guidance if we get stuck.
+
+Part 1: Testing correctness by construction
+
+Write a Hypothesis-driven test for the `count_vowels`; include this test in `tests/test_basic_functions`.
+This is a test function where we can explicitly construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
+And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
+We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
+The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
+
+We should ask ourselves: how general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
+
+
+Part 2: Property-based testing
+
+Write a Hypothesis-driven test for `merge_max_mappings` ; include this test in `tests/test_basic_functions`.
+Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
+Instead, we should *test the expected properties* of the merged dictionary.
+For example, one such property is that the merged dictionary should only contain maximum values.
+Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
+Take some time to think of other such properties that we should test for.
+Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
+
+We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
+Although dictionary keys can be any [hashable object](https://docs.python.org/3/glossary.html#term-hashable), suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
+
+
+**We must remember to temporarily mutate our original functions to verify that these tests can actually catch bugs!**
+
+Once we have added these tests to our test suite, we should re-run the entire test suite using `pytest tests` and check that our new Hypothesis-based tests are among the tests being run.
+</div>
+
+<!-- #endregion -->
+
+<!-- #region -->
+## Extending the Functionality of Strategies
+
+Hypothesis strategies can be enriched through the use of two methods: `.map()` and `.filter()`.
+These will permit us to leverage Hypothesis' core strategies to describe much more distinctive and diverse varieties of data.
+We also will see that there is a `st.data()` strategy, which will enable us to draw from strategies interactively from within our tests.
+
+
+### The `.map` method
+
+Hypothesis strategies have the `.map` method, which permits us to [perform a mapping on the data](https://hypothesis.readthedocs.io/en/latest/data.html#mapping) being produced by a strategy.
+This is achieved by passing the `.map` method a function (or any "callable");
+upon drawing a value from a strategy, Hypothesis will feed that value to the function held by `.map()`, and the strategy
+will return the value that was returned by the function.
+In this way the strategy's output is automatically "mapped" to a transformed value via the function that we provided.
+
+For example, if we want to draw only even-valued integers, we can simply use the following mapped strategy:
+
+```python
+# demonstrating the `.map()` method
+
+def make_even(x):
+ return 2 * x
+
+# `even_integers` is now a strategy that will only return even
+# valued integers. This is achieved by ensuring that any integer
+# drawn by `st.integers()` is "mapped" to an even value
+# via the function x -> 2 * x
+even_integers = st.integers().map(make_even)
+```
+
+```python
+>>> even_integers.example()
+-15414
+```
+<!-- #endregion -->
+
+<!-- #region -->
+#### A Brief Aside: Lambda Expressions
+
+Python has a syntax, which we have yet to discuss, that permits us to conveniently define functions "on the fly".
+A "lambda expression" is a syntax for defining a simple one-line function, making that function available for use in-place.
+The key here is that, whereas standard functions first must be formally defined before they can be referenced in code, *a lambda expression can be written wherever a function is expected*.
+
+For example, we can simplify the above mapping example by defining our mapping-function _within_ the `.map()` method:
+
+```python
+# Using a lambda expression to define a function
+# "on the fly"
+even_integers = st.integers().map(lambda x: 2 * x)
+```
+
+```python
+>>> even_integers.example()
+220
+```
+
+In general, the syntax for defining a lambda expression is:
+
+```
+lambda <comma-separated variable names>: <expression using variables>
+```
+
+Note that lambda expressions are restricted compared to typical function definitions: they do not permit default values or keyword arguments in their signatures, and a lambda's "body" must fit on one line.
+
+Here are some examples of lambda expressions:
+
+```python
+# function definition
+def add(x, y):
+ return x + y
+
+>>> add(2, 3)
+5
+
+# equivalent lambda expression
+>>> (lambda x, y: x + y)(2, 3)
+5
+
+# function definition
+def get_first_and_last_items(x):
+ return x[0], x[-1]
+
+>>> get_first_and_last_items(range(11))
+(0, 10)
+
+# equivalent lambda expression
+>>> (lambda x: x[0], x[-1])(range(11))
+(0, 10)
+```
+
+We will make keen use of lambdas in order to enrich our Hypothesis strategies.
+<!-- #endregion -->
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Using the `.map` method to create a sorted list**
+
+Using the `.map()` method, construct a Hypothesis strategy that produces a sorted list of integers.
+Generate some examples from your strategy and check that they are sorted (we may have to generate quite a few examples to see a diverse set of values)
+
+</div>
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Getting creative with the `.map` method**
+
+Construct a Hypothesis strategy that produces either the string `"cat"` or the string `"dog"`.
+Then, write a test that uses this strategy;
+it should simply test that either `"cat"` or `"dog"` was indeed produced by the strategy.
+Run the test.
+
+</div>
+
+
+<!-- #region -->
+### The `.filter` method
+
+Hypothesis strategies can also [have their data filtered](https://hypothesis.readthedocs.io/en/latest/data.html#filtering) via the `.filter` method.
+`.filter` takes a function (or any "callable") that accepts as input the data generated by the strategy, and returns:
+
+ - `True` if the data should pass through the filter
+ - `False` if the data should be rejected by the filter
+
+Consider, for instance, that you want to generate all integers other than `0`.
+You can write the filtered strategy:
+
+```python
+# Demonstrating the `.filter()` method
+non_zero_integers = st.integers().filter(lambda x: x != 0)
+```
+
+The `.filter` method is not magic – it is not able to "just know" how to avoid generating all barred values.
+A strategy that filters our too much data will prompt Hypothesis to raise an error.
+For example, let's try to filter `st.integers()` so that it only produces values on $[10, 20]$.
+
+```python
+# Using `.filter()` to filter out a large proportion of generated
+# values will result in an error
+@given(st.integers().filter(lambda x: 10 <= x <= 20))
+def test_aggressive_filter(x):
+ pass
+```
+
+```python
+>>> test_aggressive_filter()
+---------------------------------------------------------------------------
+FailedHealthCheck
+
+FailedHealthCheck: It looks like your strategy is filtering out a lot of data. Health check found 50 filtered examples but only 2 good ones. This will make your tests much slower, and also will probably distort the data generation quite a lot. You should adapt your strategy to filter less. This can also be caused by a low max_leaves parameter in recursive() calls
+See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.filter_too_much to the suppress_health_check settings for this test
+```
+
+Clearly, in this instance, we should have simply used the strategy `st.integers(min_value=10, max_value=20)`.
+<!-- #endregion -->
+
+<!-- #region -->
+### Drawing From Strategies Within a Test
+
+We will often need to draw from a Hypothesis strategy in a context-dependent manner within our test.
+Suppose, for example, that we want to describe two lists of integers, but we want to be sure that the second list is longer than the first.
+[We can use the st.data() strategy to use strategies "interactively"](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests) in this sort of way.
+
+Let's see it in action.
+Suppose that we want to generate two non-empty lists of integers, `x` and `y`, but we want to ensure that the values stored in `y` values are *larger than all of the values in* `x`.
+The following test shows how we can leverage Hypothesis to describe these lists
+
+```python
+# We want all of `y`'s entries to be larger than `max(x)`
+from typing import List
+
+# Defining our test function:
+# - `x` is a non-empty list of integers.
+# - `data` is an object provided by Hypothesis that permits us
+# to draw interactively from other strategies within our test
+@given(x=st.lists(st.integers(), min_size=1), data=st.data())
+def test_two_constrained_lists(x, data):
+ # We pass `data.draw(...)` a hypothesis strategy - it will draw a value from it.
+ # Thus `y` is a non-empty list of integers, whose values are guaranteed to be
+ # larger than `max(x)`
+ y = data.draw(st.lists(st.integers(min_value=max(x) + 1), min_size=1), label="y")
+
+ largest_x = max(x)
+ assert all(largest_x < item for item in y)
+```
+```python
+# Running the test
+>>> test_two_constrained_lists()
+```
+<!-- #endregion -->
+
+The `given` operator is told to pass two values to our test:
+
+ - `x`, which is a list of integers drawn from strategies
+ - `data`, which is an instance of the [st.DataObject](https://hypothesis.readthedocs.io/en/latest/_modules/hypothesis/strategies/_internal/core.html#DataObject) class; this instance is what gets drawn from the `st.data()` strategy
+
+The only thing that you need to know about `st.DataObject` is that it's `draw` method expects a hypothesis search strategy, and that it will immediately draw a value from said strategy during the test.
+You can also, optionally, pass a string to `label` argument to the `draw` method.
+This simply permits you to provide a name for the item that was drawn, so that any stack-trace that your test produces is easy to interpret.
+
+
+<div class="alert alert-info">
+
+**Reading Comprehension: Drawing from a strategy interactively**
+
+Write a test that is fed a list (of varying length) of non-negative integers.
+Then, draw a [set](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/DataStructures_III_Sets_and_More.html#The-%E2%80%9CSet%E2%80%9D-Data-Structure) of non-negative integers whose sum is at least as large as the sum of the list.
+Assert that the expected inequality between the sums hold.
+Run the test.
+
+Hint: use the `.filter()` method.
+
+</div>
+
+
+
+## The `example` Decorator
+
+As mentioned before, Hypothesis strategies will draw values (pseudo)*randomly*.
+Thus our test will potentially encounter different values every time it is run.
+There are times where we want to be sure that, in addition the values produced by a strategy, specific values will tested.
+These might be known edge cases, critical use cases, or regression cases (i.e. values that were representative of passed bugs).
+Hypothesis provides [the example decorator](https://hypothesis.readthedocs.io/en/latest/reproducing.html#providing-explicit-examples), which is to be used in conjunction with the `given` decorator, towards this end.
+
+Let's suppose, for example, that we want to write a test whose data are pairs of perfect-squares (e.g. 4, 16, 25, ...), and that we want to be sure that the pairs `(100, 144)`, `(16, 25)`, and `(36, 36)` are tested *every* time the test is run.
+Let's use `example` to guarantee this.
+
+<!-- #region -->
+```python
+# Using the `example` decorator to ensure that specific examples
+# will always be passed as inputs to our test function
+
+from hypothesis import example
+
+# A hypothesis strategy that generates integers that are
+# perfect squares
+perfect_squares = st.integers().map(lambda x: x ** 2)
+
+
+def is_square(x):
+ """Returns True if `x` is a perfect square"""
+ return int(x ** 0.5) == x ** 0.5
+
+
+@example(a=36, b=36)
+@example(a=16, b=25)
+@example(a=100, b=144)
+@given(a=perfect_squares, b=perfect_squares)
+def test_pairs_of_squares(a, b):
+ assert is_square(a)
+ assert is_square(b)
+```
+```python
+# running the test
+>>> test_pairs_of_squares()
+```
+<!-- #endregion -->
+
+Executing this test runs 103 cases: the three specified examples and one hundred pairs of values drawn via `given`.
+
+
+## Links to Official Documentation
+
+- [Hypothesis](https://hypothesis.readthedocs.io/)
+- [The given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters)
+- [The Hypothesis example database](https://hypothesis.readthedocs.io/en/latest/database.html)
+- [Core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies)
+- [The .map method](https://hypothesis.readthedocs.io/en/latest/data.html#mapping)
+- [The .filter method](https://hypothesis.readthedocs.io/en/latest/data.html#filtering)
+- [Using data() to draw interactively in tests](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests)
+- [The example decorator](https://hypothesis.readthedocs.io/en/latest/reproducing.html#providing-explicit-examples)
+
+
+
+## Reading Comprehension Solutions
+
+<!-- #region -->
+**Understanding How Hypothesis Works: Solution**
+
+Define the `test_demonstrating_the_given_decorator` function as above, complete with the failing assertion, and add a print statement to the body of the function, which prints out the value for `x` and `y`.
+
+```python
+@given(x=st.integers(0, 10), y=st.integers(20, 30))
+def test_demonstrating_the_given_decorator(x, y):
+ print(x, y)
+ assert 0 <= x <= 10
+
+ # `y` can be any value in [20, 30]
+ # this is a bad assertion: it should fail!
+ assert 20 <= y <= 25
+```
+
+Run the test once and make note of the output that is printed. Consider copying and pasting the output to a notepad for reference. Next, rerun the test multiple times and make careful note of the printed output. What do you see? Is the output different from the first run? Does it differ between subsequent runs? Try to explain this behavior.
+
+> The printed outputs between the first and second run differ. The first set out outputs is typically longer than that of the second run. After the second run, the printed outputs are always the exact same. What is happening here is that Hypothesis has to search for the falsifying example during the first run. Once it is identified, the example is recorded in the `.hypothesis` database. All of the subsequent runs are simply re-running this saved case, which is why their inputs are not changing.
+
+In your file browser, navigate to the directory from which you are running this test; if you are following along in a Jupyter notebook, this is simply the directory containing said notebook. You should see a `.hypothesis` directory. As noted above, this is the database that contains the falsifying examples that Hypothesis has identified. Delete the `.hypothesis` directory and try re-running your test? What do you notice about the output now? You should also see that the `.hypothesis` directory has reappeared. Explain what is going on here.
+
+> Deleting `.hypothesis` removes all of the falsifying examples that Hypothesis found for tests that were run from this particular directory. Thus running the test again means that Hypothesis has to find the falsifying example again from scratch. Once it does this, it creates a new database in `.hypothesis`, which is why this directory "reappears".
+<!-- #endregion -->
+
+<!-- #region -->
+**Fixing the Failing Test: Solution**
+
+Update the body of `test_demonstrating_the_given_decorator` so that it no longer fails.
+
+> We simply need to fix the second assertion statement, specifying the bounds on `y`, so that it agrees with what is being drawn from the `integers` strategy.
+
+```python
+@given(x=st.integers(0, 10), y=st.integers(20, 30))
+def test_demonstrating_the_given_decorator(x, y):
+ assert 0 <= x <= 10
+ assert 20 <= y <= 30
+```
+
+Run the fixed test function. How many times is the test function actually be executed when you run it?
+
+> The `given` decorator, by default, will draw 100 sets of example values from the strategies that are passed to it and will thus execute the decorated test function 100 times.
+
+```python
+# no output (the function returns `None`) means that the test passed
+>>> test_demonstrating_the_given_decorator()
+```
+<!-- #endregion -->
+
+<!-- #region -->
+**Exploring other Core Strategies: Solution**
+
+Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False.
+
+```python
+>>> st.dictionaries(st.integers(min_value=1), st.booleans()).example()
+{110: True, 19091: True, 136348032: False, 78: False, 9877: False}
+```
+
+Length-4 strings whose elements are only lowercase vowels
+
+```python
+>>> st.text(alphabet="aeiou", min_size=4, max_size=4).example()
+'uiai'
+```
+
+Permutations of the list `[1, 2, 3, 4]`
+
+```python
+>>> st.permutations([1, 2, 3, 4]).example()
+[2, 3, 1, 4]
+```
+<!-- #endregion -->
+
+**Improving our tests using Hypothesis: Solution**
+
+Part 1: Testing correctness by construction
+
+Write a hypothesis-driven test for the `count_vowels`; include this test in `test/test_basic_functions`.
+This is a test function where we can explicit construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
+And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
+We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
+The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
+
+We should ask ourselves: How general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
+
+<!-- #region -->
+```python
+from string import printable
+from random import shuffle
+
+import hypothesis.strategies as st
+from hypothesis import given, note
+
+# a list of all printable non-vowel characters
+_not_vowels = "".join([l for l in printable if l.lower() not in set("aeiouy")])
+
+
+@given(
+ not_vowels=st.text(alphabet=_not_vowels),
+ vowels_but_not_ys=st.text(alphabet="aeiouAEIOU"),
+ ys=st.text(alphabet="yY"),
+)
+def test_count_vowels_hypothesis(not_vowels, vowels_but_not_ys, ys):
+ """
+ Constructs an input string with a known number of:
+ - non-vowel characters
+ - non-y vowel characters
+ - y characters
+
+ and thus, by constructions, we can test that the output
+ of `count_vowels` agrees with the known number of vowels
+ """
+ # list of characters
+ letters = list(not_vowels) + list(vowels_but_not_ys) + list(ys)
+
+ # We need to shuffle the ordering of our characters so that
+ # our input string isn't unnaturally patterned; e.g. always
+ # have its vowels at the end
+ shuffle(letters)
+ in_string = "".join(letters)
+
+ # Hypothesis provides a `note` function that will print out
+ # whatever input you give it, but only in the case that the
+ # test fails.
+ # This way we can see the exact string that we fed to `count_vowels`,
+ # if it caused our test to fail
+ note("in_string: " + in_string)
+
+ # testing that `count_vowels` produces the expected output
+ # both including and excluding y's in the count
+ assert count_vowels(in_string, include_y=False) == len(vowels_but_not_ys)
+ assert count_vowels(in_string, include_y=True) == len(vowels_but_not_ys) + len(ys)
+```
+<!-- #endregion -->
+
+Part 2: Property-based testing
+
+Write a hypothesis-driven test for `merge_max_mappings` ; include this test in `test/test_basic_functions`.
+Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
+Instead, we should *test the expected properties* of the merged dictionary.
+For example, one such property is that the merged dictionary should only contain maximum values.
+Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
+Take some time to think of other such properties that we should test for.
+Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
+
+We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
+Although dictionary keys can be any [hashable object](https://docs.python.org/3/glossary.html#term-hashable), suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
+
+<!-- #region -->
+```python
+@given(
+ dict1=st.dictionaries(
+ keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
+ ),
+ dict2=st.dictionaries(
+ keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
+ ),
+)
+def test_merge_max_mappings_hypothesis(dict1, dict2):
+ merged_dict = merge_max_mappings(dict1, dict2)
+
+ # property: `merged_dict` contains all of the keys among
+ # `dict1` and `dict2`
+ assert set(merged_dict) == set(dict1).union(dict2), \
+ "novel keys were introduced or lost"
+
+ # property: `merged_dict` only contains values that appear
+ # among `dict1` and `dict2`
+ assert set(merged_dict.values()) <= set(dict1.values()).union(
+ dict2.values()
+ ), "novel values were introduced"
+
+ # property: `merged_dict` only contains key-value pairs with
+ # the largest value represented among the pairs in `dict1`
+ # and `dict2`
+ assert all(dict1[k] <= merged_dict[k] for k in dict1) and \
+ all(dict2[k] <= merged_dict[k] for k in dict2), \
+ "`merged_dict` contains a non-max value"
+
+ # property: `merged_dict` only contains key-value pairs that
+ # appear among `dict1` and `dict2`
+ for k, v in merged_dict.items():
+ assert (k, v) in dict1.items() or \
+ (k, v) in dict2.items(), \
+ "`merged_dict` did not preserve the key-value pairings"
+```
+<!-- #endregion -->
+
+**Using the `.map` method to create a sorted list: Solution**
+
+Using the `.map()` method, construct a Hypothesis strategy that produces a sorted list of integers.
+Generate some examples from your strategy and check that they are sorted (we may have to generate quite a few examples to see a diverse set of values)
+
+<!-- #region -->
+```python
+# Note that the built-in `sorted` function can be supplied
+# directly to the `.map()` method - there is no need to define
+# a function or use a lambda expression here
+sorted_list_of_ints = st.lists(st.integers()).map(sorted)
+```
+```python
+>>> sorted_list_of_ints.example()
+[-27120, 97, 12805]
+```
+<!-- #endregion -->
+
+
+**Getting creative with the `.map` method: Solution**
+
+Construct a Hypothesis strategy that produces either the string `"cat"` or the string `"dog"`.
+Then, write a test that uses this strategy;
+it should simply test that either `"cat"` or `"dog"` was indeed produced by the strategy.
+Run the test
+
+<!-- #region -->
+```python
+# We "hijack" the `st.booleans()` strategy, which only generates
+# `True` or `False`, and use the `.map` method to transform these
+# two outputs to `"cat"` or `"dog"`.
+#
+# This is only one of many ways that you could have created this
+# strategy
+cat_or_dog = st.booleans().map(lambda x: "cat" if x else "dog")
+
+@given(cat_or_dog)
+def test_cat_dog(x):
+ assert x in {"cat", "dog"}
+```
+```python
+# running the test
+>>> test_cat_dog()
+```
+<!-- #endregion -->
+
+<!-- #region -->
+**Drawing from a strategy interactively: Solution**
+
+Write a test that is fed a list (of varying length) of non-negative integers.
+Then, draw a [set](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/DataStructures_III_Sets_and_More.html#The-%E2%80%9CSet%E2%80%9D-Data-Structure) of non-negative integers whose sum is at least as large as the sum of the list.
+Assert that the expected inequality between the sums hold.
+Run the test.
+
+
+Hint: use the `.filter()` method.
+<!-- #endregion -->
+
+<!-- #region -->
+```python
+@given(the_list=st.lists(st.integers(min_value=0)), data=st.data())
+def test_interactive_draw_skills(the_list, data):
+ the_set = data.draw(
+ st.sets(elements=st.integers(min_value=0)).filter(
+ lambda x: sum(x) >= sum(the_list)
+ )
+ )
+ assert sum(the_list) <= sum(the_set)
+```
+```python
+# running the test
+>>> test_interactive_draw_skills()
+```
+<!-- #endregion -->
From 674361a2715dacf40ea529a26a9ed7b7bb85804e Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ryan.soklaski@gmail.com>
Date: Thu, 11 Feb 2021 19:35:21 -0500
Subject: [PATCH 137/152] Update
Python/Module6_Testing/Hypothesis_Practice_Exercises.md
Co-authored-by: Zac Hatfield-Dodds <zac.hatfield.dodds@gmail.com>
---
Python/Module6_Testing/Hypothesis_Practice_Exercises.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
index 67c6ca95..cbf9885d 100644
--- a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
+++ b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
@@ -15,7 +15,7 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
:description: Topic: Practice exercises using Hypothesis, Difficulty: Easy, Category: Section
- :keywords: hypotehsis, practice, exercise, test
+ :keywords: hypothesis, practice, exercise, test
<!-- #endraw -->
# Practice Exercises Using Hypothesis
From 20f2bf8dc30007f0b6efd7c845546f1681489528 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <rsoklaski@gmail.com>
Date: Sun, 21 Aug 2022 11:32:32 -0400
Subject: [PATCH 138/152] fix header level
---
Python/Module6_Testing/Intro_to_Testing.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index 35231320..e7b2b55a 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -295,7 +295,7 @@ Write two assertion statements with the respective behaviors:
<!-- #endregion -->
<!-- #region -->
-#### What is the Purpose of an Assertion?
+### What is the Purpose of an Assertion?
In our code, an assertion should be used as _a statement that is true unless there is a bug in our code_.
It is plain to see that the assertions in `test_count_vowels_basic` fit this description.
However, it can also be useful to include assertions within our source code itself.
From 9157db1ee269742ba2e2606d5a77b1b1192f0768 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <rsoklaski@gmail.com>
Date: Sun, 21 Aug 2022 11:33:08 -0400
Subject: [PATCH 139/152] fix kernel info
---
Python/Module6_Testing/Writing_Your_Own_Strategy.md | 13 ++-----------
1 file changed, 2 insertions(+), 11 deletions(-)
diff --git a/Python/Module6_Testing/Writing_Your_Own_Strategy.md b/Python/Module6_Testing/Writing_Your_Own_Strategy.md
index 85f318ca..c04c9064 100644
--- a/Python/Module6_Testing/Writing_Your_Own_Strategy.md
+++ b/Python/Module6_Testing/Writing_Your_Own_Strategy.md
@@ -7,9 +7,9 @@ jupyter:
format_version: '1.2'
jupytext_version: 1.3.0
kernelspec:
- display_name: Python [conda env:scicomp]
+ display_name: Python 3
language: python
- name: conda-env-scicomp-py
+ name: python3
---
<!-- #raw raw_mimetype="text/restructuredtext" -->
@@ -241,15 +241,6 @@ This is where some level of manual inspection of our strategy's outputs, or test
</div>
<!-- #endregion -->
-```python
-@given(x=interval_bounds(-10, 10, min_size=0, max_size=1))
-def test(x):
- print(x, x[1] - x[0])
-```
-
-```python
-test()
-```
<!-- #region -->
```python
From cd380876e32ad493dd8ca26e0812fd36330eb4ad Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <rsoklaski@gmail.com>
Date: Sun, 21 Aug 2022 11:58:49 -0400
Subject: [PATCH 140/152] Reformat decorator aside and fix clunky wording
---
Python/Module6_Testing/Pytest.md | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 1ba87276..2b714775 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -18,9 +18,11 @@ jupyter:
:keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
<!-- #endraw -->
-# The pytest Framework
+# Designing a Test Suite for a Python Project
-Thus far, our process for running tests has been an entirely manual one. It is time for us to arrange our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
+Thus far, our process for running tests has been an entirely manual one – we have been responsible for running each test-function and noting whether or not an assertion error was raised.
+It is time for us to arrange our test functions into a proper "test suite" and to learn to leverage [the pytest framework](https://docs.pytest.org/en/latest/) to run them.
+This will make it trivial for us to run all (or a subset) of our tests with a single command and to view the pass/fail status of each test in a nice list.
We will begin by reorganizing our source code to create an installable [Python package](https://www.pythonlikeyoumeanit.com/Module5_OddsAndEnds/Modules_and_Packages.html#Packages).
We will then learn how to structure and run a test suite for this Python package, using pytest.
@@ -69,8 +71,8 @@ While [unittest.mock](https://docs.python.org/3/library/unittest.mock.html) prov
There is a project, "Nose2", which is carrying the torch of `nose`. However, this is a fledgling project in comparison with `pytest`.
As of writing this, `pytest` was downloaded 12 million times last month versus `nose2`'s 150 thousand downloads.
-The takeaway here is that, when it comes to picking a testing framework for Python, `pytest` is the clear choice.
-Any discussion that you come across to the contrary is likely outdated.
+The takeaway here is that `pytest` is the clear choice when it comes to picking a testing framework for Python.
+Any discussion that you come across to the contrary is likely outdated or is lending too much legitimacy to out-dated frameworks.
</div>
<!-- #region -->
@@ -418,10 +420,13 @@ Furthermore, the four assertions are now being run independently from one anothe
<!-- #endregion -->
<!-- #region -->
-#### Decorators
+
+<div class="alert alert-warning">
+
+**Decorators**
The syntax used to parameterize this test may look alien to us, as we have yet to encounter this construct thus far.
-`pytest.mark.parameterize(...)` is a _decorator_ - an object that is used to "wrap" a function in order to transform its behavior.
+`pytest.mark.parameterize(...)` is a _decorator_ – an object that is used to "wrap" a function in order to transform its behavior.
The `pytest.mark.parameterize(...)` decorator wraps our test function so that pytest can call it multiple times, once for each parameter value.
The `@` character, in this context, denotes the application of a decorator:
@@ -434,6 +439,9 @@ def the_function_being_decorated(<arguments_for_function>):
```
For an in-depth discussion of decorators, please refer to [Real Python's Primer on decorators](https://realpython.com/primer-on-python-decorators/#simple-decorators).
+</div>
+
+
<!-- #endregion -->
<!-- #region -->
From 590e2cbb922a547901235ea14f64594c2b3a94a9 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <rsoklaski@gmail.com>
Date: Sun, 21 Aug 2022 11:59:23 -0400
Subject: [PATCH 141/152] Introduction to Testing -> The Basics of Writing
Tests for Python Code
---
Python/Module6_Testing/Intro_to_Testing.md | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/Python/Module6_Testing/Intro_to_Testing.md b/Python/Module6_Testing/Intro_to_Testing.md
index e7b2b55a..235b46f2 100644
--- a/Python/Module6_Testing/Intro_to_Testing.md
+++ b/Python/Module6_Testing/Intro_to_Testing.md
@@ -14,13 +14,12 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
- :description: Topic: Writing tests for your code, Difficulty: Easy, Category: Section
- :keywords: test, automated, unit, assert
+ :description: A basic introduction to writing tests for Python code
<!-- #endraw -->
-# Introduction to Testing
+# The Basics of Writing Tests for Python Code
-This section will show us just how simple it is to write rudimentary tests. We need only recall some of Python's basic scoping rules and introduce ourselves to the `assert` statement to write a genuine test function. That being said, we will quickly encounter some important questions to ponder. How do we know that our tests work? And, how do we know that our tests are effective? These questions will drive us deeper into the world of testing.
+This section will show us just how simple it is to write rudimentary tests for our Python code. We need only recall some of Python's [basic scoping rules](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Scope.html) and introduce ourselves to the `assert` statement to write a genuine test function. That being said, we will quickly encounter some important questions to ponder. How do we know that our tests work? And, how do we know that our tests are effective? These questions will drive us deeper into the world of testing.
Before we hit the ground running, let's take a moment to consider some motivations for testing out code.
From 624ca60956b2061c09f833e9ad5efd4921a9d39a Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <rsoklaski@gmail.com>
Date: Sun, 21 Aug 2022 12:15:02 -0400
Subject: [PATCH 142/152] Fix meta descriptions
---
Python/Module6_Testing/Hypothesis.md | 3 +--
Python/Module6_Testing/Hypothesis_Practice_Exercises.md | 3 +--
Python/Module6_Testing/Pytest.md | 3 +--
Python/Module6_Testing/Writing_Your_Own_Strategy.md | 3 +--
4 files changed, 4 insertions(+), 8 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 28ab7517..0ae2491a 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -14,8 +14,7 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
- :description: Topic: Writing tests for your code, Difficulty: Easy, Category: Section
- :keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
+ :description: A basic introduction for using Hypothesis testing library
<!-- #endraw -->
<!-- #region -->
diff --git a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
index cbf9885d..323c44c6 100644
--- a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
+++ b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
@@ -14,8 +14,7 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
- :description: Topic: Practice exercises using Hypothesis, Difficulty: Easy, Category: Section
- :keywords: hypothesis, practice, exercise, test
+ :description: Practice exercises using the Hypothesis testing library
<!-- #endraw -->
# Practice Exercises Using Hypothesis
diff --git a/Python/Module6_Testing/Pytest.md b/Python/Module6_Testing/Pytest.md
index 2b714775..613cafc0 100644
--- a/Python/Module6_Testing/Pytest.md
+++ b/Python/Module6_Testing/Pytest.md
@@ -14,8 +14,7 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
- :description: Topic: Introducing the pytest framework, Difficulty: Easy, Category: Section
- :keywords: test, automated, pytest, parametrize, fixture, suite, decorator, clean directory
+ :description: The basics of using pytest to create a test suite for a Python project
<!-- #endraw -->
# Designing a Test Suite for a Python Project
diff --git a/Python/Module6_Testing/Writing_Your_Own_Strategy.md b/Python/Module6_Testing/Writing_Your_Own_Strategy.md
index c04c9064..01225cd8 100644
--- a/Python/Module6_Testing/Writing_Your_Own_Strategy.md
+++ b/Python/Module6_Testing/Writing_Your_Own_Strategy.md
@@ -14,8 +14,7 @@ jupyter:
<!-- #raw raw_mimetype="text/restructuredtext" -->
.. meta::
- :description: Topic: Writing Your Own Hypothesis Strategy, Difficulty: Medium, Category: Section
- :keywords: test, python, hypotheis, composite, decorator, strategy, property-based
+ :description: Writing your own strategies for describing data in Hypothesis tests
<!-- #endraw -->
<!-- #region -->
From 4bded9eb65eeb50a8c2a057ba55d6863bd931b86 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <rsoklaski@gmail.com>
Date: Sun, 21 Aug 2022 12:34:24 -0400
Subject: [PATCH 143/152] Add hypothesis practice exercises
---
Python/Module6_Testing/Hypothesis.md | 88 +-
.../Hypothesis_Practice_Exercises.md | 2 +-
.../Writing_Your_Own_Strategy.md | 1182 -----------------
Python/module_6.rst | 1 +
4 files changed, 13 insertions(+), 1260 deletions(-)
delete mode 100644 Python/Module6_Testing/Writing_Your_Own_Strategy.md
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 0ae2491a..0eebbd66 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -27,16 +27,10 @@ Indeed, these are questions that you may have been asking yourself when writing
It will then *generate* test cases that satisfy this description and will run our test on these cases.
Ultimately, this an extremely powerful tool for enabling us to write high-quality automated tests for our code.
-Hypothesis can be installed via conda:
+Hypothesis can be installed pip:
-```shell
-conda install hypothesis
-```
-
-or pip:
-
-```shell
-pip install hypothesis
+```console
+$ pip install hypothesis
```
@@ -472,7 +466,7 @@ This reading comprehension question will require more substantial work than usua
That being said, the experience that we will gain from this will be well worth the work.
Keep in mind that solutions are included at the end of this page, and that these can provide guidance if we get stuck.
-Part 1: Testing correctness by construction
+Testing correctness by construction:
Write a Hypothesis-driven test for the `count_vowels`; include this test in `tests/test_basic_functions`.
This is a test function where we can explicitly construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
@@ -483,20 +477,6 @@ The standard library's built-in `string` module provides a string of all printab
We should ask ourselves: how general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
-Part 2: Property-based testing
-
-Write a Hypothesis-driven test for `merge_max_mappings` ; include this test in `tests/test_basic_functions`.
-Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
-Instead, we should *test the expected properties* of the merged dictionary.
-For example, one such property is that the merged dictionary should only contain maximum values.
-Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
-Take some time to think of other such properties that we should test for.
-Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
-
-We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
-Although dictionary keys can be any hashable object, suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
-
-
**We must remember to temporarily mutate our original functions to verify that these tests can actually catch bugs!**
Once we have added these tests to our test suite, we should re-run the entire test suite using `pytest tests` and check that our new Hypothesis-based tests are among the tests being run.
@@ -765,6 +745,12 @@ def test_pairs_of_squares(a, b):
Executing this test runs 103 cases: the three specified examples and one hundred pairs of values drawn via `given`.
+## Next Steps
+
+Thus far we have learned about the basic anatomy of a test, how to use pytest to create an automated test-suite for our code base, and how to leverage Hypothesis to generate diverse and randomized inputs to our test functions.
+In the final section of this module, we will discuss three testing methods: example-based testing, fuzzing, and property-based testing (Hypothesis will prove to be indispensable for facilitating these last two methods).
+These strategies will equip us with ability to "test the untestable": we will be able to write effective tests for code where we are unable to discern what the exact appropriate behavior is for the code under arbitrary inputs.
+
## Links to Official Documentation
@@ -857,7 +843,7 @@ Permutations of the list `[1, 2, 3, 4]`
**Improving our tests using Hypothesis: Solution**
-Part 1: Testing correctness by construction
+Testing correctness by construction
Write a hypothesis-driven test for the `count_vowels`; include this test in `test/test_basic_functions`.
This is a test function where we can explicit construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
@@ -917,58 +903,6 @@ def test_count_vowels_hypothesis(not_vowels, vowels_but_not_ys, ys):
```
<!-- #endregion -->
-Part 2: Property-based testing
-
-Write a hypothesis-driven test for `merge_max_mappings` ; include this test in `test/test_basic_functions`.
-Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
-Instead, we should *test the expected properties* of the merged dictionary.
-For example, one such property is that the merged dictionary should only contain maximum values.
-Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
-Take some time to think of other such properties that we should test for.
-Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
-
-We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
-Although dictionary keys can be any hashable object, suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
-
-<!-- #region -->
-```python
-@given(
- dict1=st.dictionaries(
- keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
- ),
- dict2=st.dictionaries(
- keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
- ),
-)
-def test_merge_max_mappings_hypothesis(dict1, dict2):
- merged_dict = merge_max_mappings(dict1, dict2)
-
- # property: `merged_dict` contains all of the keys among
- # `dict1` and `dict2`
- assert set(merged_dict) == set(dict1).union(dict2), \
- "novel keys were introduced or lost"
-
- # property: `merged_dict` only contains values that appear
- # among `dict1` and `dict2`
- assert set(merged_dict.values()) <= set(dict1.values()).union(
- dict2.values()
- ), "novel values were introduced"
-
- # property: `merged_dict` only contains key-value pairs with
- # the largest value represented among the pairs in `dict1`
- # and `dict2`
- assert all(dict1[k] <= merged_dict[k] for k in dict1) and \
- all(dict2[k] <= merged_dict[k] for k in dict2), \
- "`merged_dict` contains a non-max value"
-
- # property: `merged_dict` only contains key-value pairs that
- # appear among `dict1` and `dict2`
- for k, v in merged_dict.items():
- assert (k, v) in dict1.items() or \
- (k, v) in dict2.items(), \
- "`merged_dict` did not preserve the key-value pairings"
-```
-<!-- #endregion -->
**Using the `.map` method to create a sorted list: Solution**
diff --git a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
index 323c44c6..035b079c 100644
--- a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
+++ b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
@@ -17,7 +17,7 @@ jupyter:
:description: Practice exercises using the Hypothesis testing library
<!-- #endraw -->
-# Practice Exercises Using Hypothesis
+# Additional Practice Exercises Using Hypothesis
Hypothesis will not only improve the quality of our tests, but it should also save us time and cognitive load as it simplifies the process for describing the data that we want to pass to our code.
That being said, it can take some practice to learn one's way around [Hypothesis' core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies);
diff --git a/Python/Module6_Testing/Writing_Your_Own_Strategy.md b/Python/Module6_Testing/Writing_Your_Own_Strategy.md
deleted file mode 100644
index 01225cd8..00000000
--- a/Python/Module6_Testing/Writing_Your_Own_Strategy.md
+++ /dev/null
@@ -1,1182 +0,0 @@
----
-jupyter:
- jupytext:
- text_representation:
- extension: .md
- format_name: markdown
- format_version: '1.2'
- jupytext_version: 1.3.0
- kernelspec:
- display_name: Python 3
- language: python
- name: python3
----
-
-<!-- #raw raw_mimetype="text/restructuredtext" -->
-.. meta::
- :description: Writing your own strategies for describing data in Hypothesis tests
-<!-- #endraw -->
-
-<!-- #region -->
-# Writing Your Own Hypothesis Strategy
-
-Hypothesis' key role in our test-writing practice is that it permits us to describe the data, and the relationships between the data, that we want to pass to our code, and to generate data according to these descriptions.
-It is often the case that a Python project will use specialized data structures and patterns of data, and thus testing this project's code requires that we can describe this specialized data.
-Towards this end, Hypothesis provides us with the [composite](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.composite) decorator, which permits us to form our own strategies for describing data by composing Hypothesis' built-in strategies.
-
-The general syntax for defining a custom Hypothesis strategy using the `composite` decorator is as follows:
-
-```python
-from hypothesis import composite
-
-@composite
-def custom_strategy(draw, <user-facing parameters>):
- # Use `draw(...)` to draw values from other hypothesis
- # strategies. We can leverage the values passed in via
- # <user-facing parameters>
- # E.g.
- # `x = draw(st.integers(1, 4))`
- # will result in `x` being assigned an integer value
- # between 1 and 4.
-
- # A composite strategy should be written to
- # return standard Python objects (e.g. ints, lists, etc.);
- # it is *not* meant to return other Hypothesis strategies
- return <value(s) to be returned by strategy>
-
-```
-<!-- #endregion -->
-
-Let's see this in action by writing a Hypothesis strategy that will produce the left and right bounds for an interval on the number line, and that permits the user to constrain various aspects of the intervals that can be generated by the strategy.
-
-<!-- #region -->
-```python
-from hypothesis.strategies import composite
-import hypothesis.strategies as st
-
-
-@composite
-def interval_bounds(draw, left_bnd, right_bnd, min_size=0.0, max_size=None):
- """A Hypothesis data strategy for generating ordered bounds on the real number line.
-
- Note: The `draw` parameter it reserved by Hypothesis and is not exposed by
- the function signature.
-
- Parameters
- ----------
- left_bnd : float
- The smallest value that the left-bound can take on.
-
- right_bnd : float
- The largest value that the right-bound can take on.
-
- min_size : float, optional (default=0.0)
- The guaranteed minimum separation of the bounds.
-
- max_size : Optional[float]
- The guaranteed maximum separation of the bounds. Defaults to `right_bnd - left_bnd`
-
- Returns
- -------
- st.SearchStrategy[Tuple[float, float]]
- """
- # Let `a` represent left-bound and `b` represent right-bound
- # let `d` represent the smalled permitted interval
-
- # `drawn_left` is a float
- # Is a number in [a, b - d]
- drawn_left = draw(st.floats(min_value=left_bnd, max_value=right_bnd - min_size))
-
- # `drawn_right` is a float
- # Is a number in [a + d, b]
- drawn_right = draw(
- st.floats(
- min_value=drawn_left + min_size,
- max_value=min(right_bnd, drawn_left + max_size),
- )
- )
-
- # Note that a composite strategy definition should return the drawn values,
- # and *not* Hypothesis strategies.
- #
- # E.g. here we return a tuple of floating point numbers, and
- # *not* `st.tuples(st.floats(), st.floats())`
- return (drawn_left, drawn_right)
-```
-
-<!-- #endregion -->
-
-The first argument, `draw`, is required to be present in our function definition by the `@composite` decorator.
-It is a function that is used by Hypothesis to draw values from strategies in order to generate data from our composite strategy. Each `draw(...)` call simply produces a value from that strategy.
-Thus `drawn_left` and `drawn_right` are floating point numbers.
-We then return a tuple of these floats, as expected from this strategy.
-That being said, *the* `draw` *parameter is not exposed to users*.
-See that it is not present in the function signature for our strategy when we query it via `help()`.
-
-
-
-<!-- #region -->
-```python
-# The `draw` is not exposed to users. It is only meant
-# to be leveraged when defining the custom strategy
->>> help(interval_bounds)
-Help on function interval_bounds in module __main__:
-
-interval_bounds(left_bnd, right_bnd, min_size=0.0, max_size=None)
-[...]
-```
-<!-- #endregion -->
-
-Note that, even though the resulting function `interval_bounds(...)` is a Hypothesis search strategy, the return statement in the body of the definition *specifies what values are to be returned by the strategy*.
-That is, it does *not* return strategy-instances.
-This is at odds with the "Returns" section of the docstring, which informs the user that calling `interval_bounds(...)` in face returns a search strategy.
-
-Let's consider some usages of this strategy:
-
-<!-- #region -->
-```python
-# The following pieces of code generate a Hypothesis search
-# strategy that...
-
-# generates any pair of bounds on [-10, 10]
->>> interval_bounds(left_bnd=-10, right_bnd=10)
-
-# generates any pair of bounds on [0, 1] whose difference is
-# at least 0.5
->>> interval_bounds(left_bnd=0, right_bnd=1, min_size=.5)
-
-# generates any pair of bounds on [-100, 100] whose difference is
-# at least 1 and is no larger than 5
->>> interval_bounds(left_bnd=-100, right_bnd=100, min_size=1, max_size=5)
-```
-<!-- #endregion -->
-
-Now that we have created our own strategy, we can draw values from it in a test as we would any other Hypothesis strategy:
-
-<!-- #region -->
-```python
-# Using our custom strategy in a test
-
-from hypothesis import given
-
-@given(bnd=interval_bounds(left_bnd=0, right_bnd=1000))
-def test_a_function(bnd):
- left_bnd, right_bnd = bnd
- # ...
- # conduct a test using the bounds drawn from our
- # custom strategy
-```
-<!-- #endregion -->
-
-Note the power of the `composite` operator; we can return *any* sort of value from our custom strategy, generated via arbitrarily-complicated logic.
-To this end, we can leverage this decorator to generate instances of custom classes, patterned NumPy arrays, specialized rest API calls, etc.
-That being said, we must keep in mind that Hypothesis will, by default, make this function call one hundred times for each test evaluation that uses our custom strategy.
-Thus we should be conscientious about the computational cost associated with our strategy's data generation process.
-This is a place where writing efficient code really counts.
-
-<!-- #region -->
-<div class="alert alert-warning">
-
-**With Great Power Comes Great Responsibility**:
-
-Hypothesis' `composite` enables us to create rich strategies for describing the data that we want to use to test our code.
-This is empowering indeed, however we must take great care that our custom strategies are *describing our data correctly*.
-That is, we must be sure that we do not have bugs in our strategies.
-It is easy to imagine the frustration that we would experience when we find that our code is failing a test and searching for the bug in our code, only to find that the Hypothesis strategy that we wrote isn't actually behaving as we had intended.
-
-With this scenario in mind, we are motivated to be extra-fastidious when writing our own strategies.
-There are several steps that we can take to help ensure that we are not making our lives harder by writing buggy strategies:
-
- 1. Validate user inputs to ensure that mutually-inconsistent parameters are not permitted, and include clear, verbose error messages when invalid inputs are provided.
- 2. Make liberal use of internal assertion statements within a strategy's definition to double check the consistency of our own logic
- 3. Write tests that exercise our custom strategies in controlled and contrived ways, to ensure that their expected properties hold. (That's right, we ought to write tests for the tooling that helps us write our tests!)
-
-Let's rewrite `interval_bounds` to exercise points 1 and 2.
-
-```python
-@composite
-def interval_bounds(draw, left_bnd, right_bnd, min_size=0.0, max_size=None):
- """A Hypothesis data strategy for generating ordered bounds on the real number line."""
-
- if right_bnd < left_bnd + min_size:
- raise ValueError(
- f"Unsatisfiable bounds: [left_bnd={left_bnd}, right_bnd={right_bnd}], "
- f"min-interval size: {min_size}"
- )
-
- if max_size is None:
- max_size = right_bnd - left_bnd
- elif max_size < min_size:
- raise ValueError(
- f"Unsatisfiable interval bounds: min_size={min_size}, max_size={max_size}"
- )
-
- drawn_left = draw(st.floats(min_value=left_bnd, max_value=right_bnd - min_size))
-
- drawn_right = draw(
- st.floats(
- min_value=drawn_left + min_size,
- max_value=min(right_bnd, drawn_left + max_size),
- )
- )
-
- # ensure that strategy behaves as-promised
- assert left_bnd <= drawn_left <= right_bnd
- assert left_bnd <= drawn_right <= right_bnd
- assert min_size <= drawn_right - drawn_left <= max_size
-
- return (drawn_left, drawn_right)
-```
-
-See that the `ValueError` exceptions that we have included make it clear to the user when they have passed invalid inputs to our strategy; this makes for much clearer feedback than to have `st.floats()` raise an exception because it received invalid bounds.
-
-The inclusion of assertion expressions ensure, critically, that our strategy is obeying the constraints that the user provided.
-It is better to see that our strategy is raising an internal assertion error than it is for it to "silently" generate bounds that ought to be invalid.
-
-It is important that we recognize that these measures help to ensure that our strategy is never generating bad data.
-That being said, *these do not provide any guarantees that the strategy is generating appropriately diverse data*.
-It could be that, had we unwittingly introduced bad logic into our strategy, it generates a pair of valid bounds, but that these bounds never change during the data generation process.
-This is where some level of manual inspection of our strategy's outputs, or tests that ensure that our strategy is capable of generating sufficiently-general data is important.
-</div>
-<!-- #endregion -->
-
-
-<!-- #region -->
-```python
-# A basic introduction to Hypothesis
-
-from hypothesis import given
-
-# Hypothesis provides so-called "strategies" for us
-# to describe our data
-import hypothesis.strategies as st
-
-# Using hypothesis to test any integer value in [0, 10 ** 10]
-@given(size=st.integers(min_value=0, max_value=1E10))
-def test_range_length(size):
- assert len(range(size)) == size
-```
-<!-- #endregion -->
-
-<!-- #region -->
-Here we have specified that the value of `size` in our test *should be able to take on any integer value within* $[0, 10^{10}]$.
-We did this by using the `integers` "strategy" that is provided by Hypothesis: `st.integers(min_value=0, max_value=1E10)`.
-When we execute the resulting test (which can simply be run within a Jupyter cell or via pytest), this will trigger Hypothesis to generate test cases based on this specification;
-by default, Hypothesis will generate 100 test cases - an amount that we can configure - and will execute our test function for each one of them.
-
-```python
-# Running this test once will trigger Hypothesis to
-# generate 100 values based on the description of our data,
-# and it will execute the test using each one of those values
->>> test_range_length()
-```
-
-With great ease, we were able to replace our pytest-parameterized test, which only very narrowly tested the property at hand, with a much more robust, Hypothesis-driven test.
-This will be a recurring trend: we will generally produce much more robust tests by _describing_ our data with Hypothesis, rather than manually specifying test values.
-
-The rest of this section will be dedicated to learning about the Hypothesis library and how we can leverage it to write powerful tests.
-<!-- #endregion -->
-
-<!-- #region -->
-## The `given` Decorator
-
-Hypothesis' [given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters) is responsible for:
-
- - drawing values from Hypothesis' so-called "strategies" for describing input data for our test function
- - running the test function many times (up to 100 times, by default) given different input values drawn from the strategy
- - "shrinking" the drawn inputs to identify simple fail cases: if an error is raised by the test function during one of the many executions, the `given` decorator will attempt to "shrink" (i.e. simplify) the inputs that produce that same error before reporting them to the user
- - reporting the input values that caused the test function to raise an error
-
-
-Let's see the `given` decorator in action by writing a simple "test" for which `x` should be integers between 0 and 10, and `y` should be integers between 20 and 30.
-To do this we will make use of the `integers` Hypothesis strategy.
-Let's include a bad assertion statement – that `y` can't be larger than 25 – to see how Hypothesis reports this fail case.
-Note that we aren't really testing any code here, we are simply exercising some of the tools that Hypothesis provides us with.
-
-```python
-from hypothesis import given
-import hypothesis.strategies as st
-
-# using `given` with multiple parameters
-# `x` is an integer drawn from [0, 10]
-# `y` is an integer drawn from [20, 30]
-@given(x=st.integers(0, 10), y=st.integers(20, 30))
-def test_demonstrating_the_given_decorator(x, y):
- assert 0 <= x <= 10
-
- # `y` can be any value in [20, 30]
- # this is a bad assertion: it should fail!
- assert 20 <= y <= 25
-```
-
-See that the names of the parameters specified in `given` — `x` and `y` in this instance — must match those in the signature of the test function.
-
-To run this test function, we simply call `test_demonstrating_the_given_decorator()`.
-Note that, unlike with a typical function, we do not pass values for `x` and `y` to this function – *the* `given` *decorator will pass these values to the function for us*.
-Executing this `given`-decorated function will prompt Hypothesis to draw 100 pairs of values for `x` and `y`, according to their respective strategies, and the body of the test will be executed for each such pair.
-
-```python
-# running the test
->>> test_demonstrating_the_given_decorator()
-Falsifying example: test_demonstrating_the_given_decorator(
- x=0, y=26,
-)
----------------------------------------------------------------------------
-AssertionError Traceback (most recent call last)
-<ipython-input-29-ea20353bbef5> in test_demonstrating_the_given_decorator(x, y)
- 10 # `y` can be any value in [20, 30]
- 11 # this is a bad assertion: it should fail!
----> 12 assert 20 <= y <= 25
-
-AssertionError:
-
-```
-
-(Note: some of the "Traceback" error message has been removed to improve legibility)
-
-See that the error message here indicates that Hypothesis identified a "falsifying example", or a set of input values, `x=0` and `y=26`, which caused our test function to raise an error. The proceeding "Traceback" message indicates that it is indeed the second assertion statement that is responsible for raising the error.
-
-### Shrinking: Simplifying Falsifying Inputs
-
-The `given` decorator strives to report the "simplest" set of input values that produce a given error.
-It does this through the process of "shrinking".
-Each of Hypothesis' strategies has its own prescribed shrinking behavior.
-For the `integers` strategy, this means identifying the integer closest to 0 that produces the error at hand.
-For instance, `x=12` and `y=29` may have been the first drawn values to trigger the assertion error.
-These values were then incrementally reduced by the `given` decorator until `x=0` and `y=26` were identified as the smallest set of values to reproduce this fail case.
-
-We can print out the examples that Hypothesis generated:
-
-```
-x=0 y=20 - PASSED
-x=0 y=20 - PASSED
-x=0 y=20 - PASSED
-x=9 y=20 - PASSED
-x=9 y=21 - PASSED
-x=3 y=20 - PASSED
-x=3 y=20 - PASSED
-x=9 y=26 - FAILED
-x=3 y=26 - FAILED
-x=6 y=26 - FAILED
-x=10 y=27 - FAILED
-x=7 y=27 - FAILED
-x=3 y=30 - FAILED
-x=3 y=23 - PASSED
-x=10 y=30 - FAILED
-x=3 y=27 - FAILED
-x=3 y=27 - FAILED
-x=2 y=27 - FAILED
-x=0 y=27 - FAILED
-x=0 y=26 - FAILED
-x=0 y=21 - PASSED
-x=0 y=25 - PASSED
-x=0 y=22 - PASSED
-x=0 y=23 - PASSED
-x=0 y=24 - PASSED
-x=0 y=26 - FAILED
-```
-
-See that Hypothesis has to do a semi-random search to identify the boundaries of the fail case; it doesn't know if `x` is causing the error, or if `y` is the culprit, or if it is specific *combinations* of `x` and `y` that causes the failure!
-Despite this complexity, the pairs of variables are successfully shrunk to the simplest fail case.
-<!-- #endregion -->
-
-<div class="alert alert-warning">
-
-**Hypothesis will Save Falsifying Examples**:
-
-Albeit an advanced detail, it is important to note that Hypothesis does not have to search for falsifying examples from scratch every time we run a test function.
-Instead, Hypothesis will save a database of falsifying examples associated with each of your project's test functions.
-The database is saved under `.hypothesis` in whatever directory your test functions were run from.
-
-This ensures that, once Hypothesis finds a falsifying example for a test, the falsifying example will be passed to your test function each time you run it, until it no longer raises an error in your code (e.g. you update your code to fix the bug that was causing the test failure).
-</div>
-
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Understanding How Hypothesis Works**
-
-Define the `test_demonstrating_the_given_decorator` function as above, complete with the failing assertion, and add a print statement to the body of the function, which prints out the value for `x` and `y`.
-
-Run the test once and make note of the output that is printed. Consider copying and pasting the output to a notepad for reference. Next, rerun the test multiple times and make careful note of the printed output. What do you see? Is the output different from the first run? Does it differ between subsequent runs? Try to explain this behavior.
-
-In your file browser, navigate to the directory from which you are running this test; if you are following along in a Jupyter notebook, this is simply the directory containing said notebook. You should see a `.hypothesis` directory. As noted above, this is the database that contains the falsifying examples that Hypothesis has identified. Delete the `.hypothesis` directory and try re-running your test? What do you notice about the output now? You should also see that the `.hypothesis` directory has reappeared. Explain what is going on here.
-
-</div>
-
-
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Fixing the Failing Test**
-
-Update the body of `test_demonstrating_the_given_decorator` so that it no longer fails. Run the fixed test function. How many times is the test function actually be executed when you run it?
-
-</div>
-
-
-
-## Describing Data: Hypothesis Strategies
-
-Hypothesis provides us with so-called "strategies" for describing our data.
-We are already familiar with the `integers` strategy;
-Hypothesis' core strategies are all located in the `hypothesis.strategies` module.
-The official documentation for the core strategies can be found [here](https://hypothesis.readthedocs.io/en/latest/data.html).
-
-Here, we will familiarize ourselves with these core strategies and will explore some of the powerful methods that can be used to customize their behaviors.
-
-<!-- #region -->
-### Drawing examples from strategies
-
-Hypothesis provides a useful mechanism for developing an intuition for the data produced by a strategy: a strategy, once initialized, has a `.example()` method that will randomly draw a representative value from the strategy. For example:
-
-```python
-# demonstrating usage of `<strategy>.example()`
->>> st.integers(-1, 1).example()
--1
-
->>> st.integers(-1, 1).example()
-1
-```
-
-**Note: the** `.example()` **mechanism is only meant to be used for pedagogical purposes. You should never use this in your test suite**
-because (among other reasons) `.example()` biases towards smaller and simpler examples than `@given`, and lacks the features to ensure any test failures are reproducible.
-
-We will be leveraging the `.example()` method throughout the rest of this section to help provide an intuition for the data that Hypothesis' various strategies generate.
-<!-- #endregion -->
-
-<!-- #region -->
-### Exploring Strategies
-
-There are a number critical Hypothesis strategies for us to become familiar with. It is worthwhile to peruse through all of Hypothesis' [core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies), but we will take time to highlight a few here.
-
-#### `st.booleans ()`
-
-[st.booleans()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.booleans) generates either `True` or `False`. This strategy will shrink towards `False`
-
-```python
->>> st.booleans().example()
-False
-```
-<!-- #endregion -->
-
-<!-- #region -->
-
-#### `st.lists ()`
-
-[st.lists](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.lists) accepts *another* strategy, which describes the elements of the lists being generated. You can also specify:
- - bounds on the length of the list
- - if we want the elements to be unique
- - a mechanism for defining "uniqueness"
-
-For example, the following strategy describes lists whose length varies from 2 to 10, and whose entries are integers on the domain $[-10, 20]$:
-
-```python
->>> st.lists(st.integers(-10, 20), min_size=2, max_size=10).example()
-[-10, 0, 5]
-```
-
-**`st.lists(...)` is the strategy of choice anytime we want to generate sequences of varying lengths with elements that are, themselves, described by strategies**.
-<!-- #endregion -->
-
-<!-- #region -->
-#### `st.floats()`
-
-[st.floats](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.floats) is a powerful strategy that generates all variety of floats, including `math.inf` and `math.nan`. You can also specify:
- - whether `math.inf` and `math.nan`, respectively, should be included in the data description
- - bounds (either inclusive or exclusive) on the floats being generated; this will naturally preclude `math.nan` from being generated
- - the "width" of the floats; e.g. if you want to generate 16-bit or 32-bit floats vs 64-bit
- (while Python's `float` is (usually) 64-bit, `width=32` ensures that the generated values can
- always be losslessly represented in 32 bits. This is mostly useful for NumPy arrays.)
-
-For example, the following strategy 64-bit floats that reside in the domain $[-100, 1]$:
-
-```python
->>> st.floats(-100, 1).example()
-0.3670816313319896
-```
-<!-- #endregion -->
-
-<!-- #region -->
-#### `st.tuples()`
-
-The [st.tuples](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.tuples) strategy accepts $N$ Hypothesis strategies, and will generate length-$N$ tuples whose elements are drawn from the respective strategies that were specified as inputs.
-
-For example, the following strategy will generate length-3 tuples whose entries are: integers, booleans, and floats:
-
-```python
->>> st.tuples(st.integers(), st.booleans(), st.floats()).example()
-(4628907038081558014, False, -inf)
-```
-<!-- #endregion -->
-
-<!-- #region -->
-#### `st.text()`
-
-The [st.text](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) accepts an "alphabet" – a collection of string-characters – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
-
-For example, the following strategy will strings of lowercase vowels from length 2 to length 10:
-
-```python
->>> st.text("aeiouy", min_size=2, max_size=10).example()
-'oouoyoye'
-```
-<!-- #endregion -->
-
-<!-- #region -->
-#### `st.just()`
-
-[st.just](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.just) is a strategy that "just" returns the value that you fed it. This is a convenient strategy that helps us to avoid having to manipulate our data before using it.
-
-Suppose that we want a strategy that describes the shape of an array (i.e. a tuple of integers) that contains 1-20 two-dimensional vectors. E.g. `(5, 2)` is the shape of the array containing five two-dimensional vectors. We can leverage `st.just`, in conjunction with `st.integers` and `st.tuples`, towards this end:
-
-```python
->>> st.tuples(st.integers(1, 20), st.just(2)).example()
-(7, 2)
-```
-<!-- #endregion -->
-
-<!-- #region -->
-#### `st.one_of()`
-
-The [st.one_of](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.one_of) strategy allows us to specify a collection of strategies and any given datum will be drawn from "one of" them. For example:
-
-```python
-# demonstrating st.one_of()
-st.one_of(st.integers(), st.lists(st.integers()))
-```
-
-will draw values that are *either* integers or lists of integers:
-
-```python
->>> st.one_of(st.integers(), st.lists(st.integers())).example()
-144
-
->>> st.one_of(st.integers(), st.lists(st.integers())).example()
-[0, -22]
-```
-
-The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` calls:
-
-```python
-# Using the pipe operator, | , in place of `st.one_of`
-# This strategy generates integers or floats
-# or lists that contain just the word "hello"
-
->>> (st.integers() | st.floats() | st.lists(st.just("hello"))).example()
-['hello', 'hello']
-
->>> (st.integers() | st.floats() | st.lists(st.just("hello"))).example()
-0
-```
-<!-- #endregion -->
-
-<!-- #region -->
-#### `st.sampled_from`
-
-[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and NumPy arrays). The strategy will return values that are sampled from this collection.
-
-For example, the following strategy will sample a value `0`, `"a"`, or `(2, 2)` from a list:
-
-```python
->>> st.sampled_from([0, "a", (2, 2)]).example()
-'a'
-```
-<!-- #endregion -->
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Exploring other Core Strategies**
-
-Review the [rest of Hypothesis' core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies).
-Write down a strategy, and print out a representative example, that describes the the data according to each of the following conditions:
-
- 1. Dictionaries of arbitrary size whose keys are positive-valued integers and whose values are `True` or `False.
- 2. Length-4 strings whose elements are only lowercase vowels
- 3. Permutations of the list `[1, 2, 3, 4]`
-
-</div>
-
-
-<!-- #region -->
-<div class="alert alert-info">
-
-**Reading Comprehension: Improving our tests using Hypothesis**
-
-We will be writing improved tests for the basic functions – `count_vowels` and `merge_max_mappings` – by leveraging Hypothesis.
-This reading comprehension question will require more substantial work than usual.
-That being said, the experience that we will gain from this will be well worth the work.
-Keep in mind that solutions are included at the end of this page, and that these can provide guidance if we get stuck.
-
-Part 1: Testing correctness by construction
-
-Write a Hypothesis-driven test for the `count_vowels`; include this test in `tests/test_basic_functions`.
-This is a test function where we can explicitly construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
-And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
-We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
-The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
-
-We should ask ourselves: how general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
-
-
-Part 2: Property-based testing
-
-Write a Hypothesis-driven test for `merge_max_mappings` ; include this test in `tests/test_basic_functions`.
-Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
-Instead, we should *test the expected properties* of the merged dictionary.
-For example, one such property is that the merged dictionary should only contain maximum values.
-Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
-Take some time to think of other such properties that we should test for.
-Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
-
-We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
-Although dictionary keys can be any [hashable object](https://docs.python.org/3/glossary.html#term-hashable), suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
-
-
-**We must remember to temporarily mutate our original functions to verify that these tests can actually catch bugs!**
-
-Once we have added these tests to our test suite, we should re-run the entire test suite using `pytest tests` and check that our new Hypothesis-based tests are among the tests being run.
-</div>
-
-<!-- #endregion -->
-
-<!-- #region -->
-## Extending the Functionality of Strategies
-
-Hypothesis strategies can be enriched through the use of two methods: `.map()` and `.filter()`.
-These will permit us to leverage Hypothesis' core strategies to describe much more distinctive and diverse varieties of data.
-We also will see that there is a `st.data()` strategy, which will enable us to draw from strategies interactively from within our tests.
-
-
-### The `.map` method
-
-Hypothesis strategies have the `.map` method, which permits us to [perform a mapping on the data](https://hypothesis.readthedocs.io/en/latest/data.html#mapping) being produced by a strategy.
-This is achieved by passing the `.map` method a function (or any "callable");
-upon drawing a value from a strategy, Hypothesis will feed that value to the function held by `.map()`, and the strategy
-will return the value that was returned by the function.
-In this way the strategy's output is automatically "mapped" to a transformed value via the function that we provided.
-
-For example, if we want to draw only even-valued integers, we can simply use the following mapped strategy:
-
-```python
-# demonstrating the `.map()` method
-
-def make_even(x):
- return 2 * x
-
-# `even_integers` is now a strategy that will only return even
-# valued integers. This is achieved by ensuring that any integer
-# drawn by `st.integers()` is "mapped" to an even value
-# via the function x -> 2 * x
-even_integers = st.integers().map(make_even)
-```
-
-```python
->>> even_integers.example()
--15414
-```
-<!-- #endregion -->
-
-<!-- #region -->
-#### A Brief Aside: Lambda Expressions
-
-Python has a syntax, which we have yet to discuss, that permits us to conveniently define functions "on the fly".
-A "lambda expression" is a syntax for defining a simple one-line function, making that function available for use in-place.
-The key here is that, whereas standard functions first must be formally defined before they can be referenced in code, *a lambda expression can be written wherever a function is expected*.
-
-For example, we can simplify the above mapping example by defining our mapping-function _within_ the `.map()` method:
-
-```python
-# Using a lambda expression to define a function
-# "on the fly"
-even_integers = st.integers().map(lambda x: 2 * x)
-```
-
-```python
->>> even_integers.example()
-220
-```
-
-In general, the syntax for defining a lambda expression is:
-
-```
-lambda <comma-separated variable names>: <expression using variables>
-```
-
-Note that lambda expressions are restricted compared to typical function definitions: they do not permit default values or keyword arguments in their signatures, and a lambda's "body" must fit on one line.
-
-Here are some examples of lambda expressions:
-
-```python
-# function definition
-def add(x, y):
- return x + y
-
->>> add(2, 3)
-5
-
-# equivalent lambda expression
->>> (lambda x, y: x + y)(2, 3)
-5
-
-# function definition
-def get_first_and_last_items(x):
- return x[0], x[-1]
-
->>> get_first_and_last_items(range(11))
-(0, 10)
-
-# equivalent lambda expression
->>> (lambda x: x[0], x[-1])(range(11))
-(0, 10)
-```
-
-We will make keen use of lambdas in order to enrich our Hypothesis strategies.
-<!-- #endregion -->
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Using the `.map` method to create a sorted list**
-
-Using the `.map()` method, construct a Hypothesis strategy that produces a sorted list of integers.
-Generate some examples from your strategy and check that they are sorted (we may have to generate quite a few examples to see a diverse set of values)
-
-</div>
-
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Getting creative with the `.map` method**
-
-Construct a Hypothesis strategy that produces either the string `"cat"` or the string `"dog"`.
-Then, write a test that uses this strategy;
-it should simply test that either `"cat"` or `"dog"` was indeed produced by the strategy.
-Run the test.
-
-</div>
-
-
-<!-- #region -->
-### The `.filter` method
-
-Hypothesis strategies can also [have their data filtered](https://hypothesis.readthedocs.io/en/latest/data.html#filtering) via the `.filter` method.
-`.filter` takes a function (or any "callable") that accepts as input the data generated by the strategy, and returns:
-
- - `True` if the data should pass through the filter
- - `False` if the data should be rejected by the filter
-
-Consider, for instance, that you want to generate all integers other than `0`.
-You can write the filtered strategy:
-
-```python
-# Demonstrating the `.filter()` method
-non_zero_integers = st.integers().filter(lambda x: x != 0)
-```
-
-The `.filter` method is not magic – it is not able to "just know" how to avoid generating all barred values.
-A strategy that filters our too much data will prompt Hypothesis to raise an error.
-For example, let's try to filter `st.integers()` so that it only produces values on $[10, 20]$.
-
-```python
-# Using `.filter()` to filter out a large proportion of generated
-# values will result in an error
-@given(st.integers().filter(lambda x: 10 <= x <= 20))
-def test_aggressive_filter(x):
- pass
-```
-
-```python
->>> test_aggressive_filter()
----------------------------------------------------------------------------
-FailedHealthCheck
-
-FailedHealthCheck: It looks like your strategy is filtering out a lot of data. Health check found 50 filtered examples but only 2 good ones. This will make your tests much slower, and also will probably distort the data generation quite a lot. You should adapt your strategy to filter less. This can also be caused by a low max_leaves parameter in recursive() calls
-See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.filter_too_much to the suppress_health_check settings for this test
-```
-
-Clearly, in this instance, we should have simply used the strategy `st.integers(min_value=10, max_value=20)`.
-<!-- #endregion -->
-
-<!-- #region -->
-### Drawing From Strategies Within a Test
-
-We will often need to draw from a Hypothesis strategy in a context-dependent manner within our test.
-Suppose, for example, that we want to describe two lists of integers, but we want to be sure that the second list is longer than the first.
-[We can use the st.data() strategy to use strategies "interactively"](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests) in this sort of way.
-
-Let's see it in action.
-Suppose that we want to generate two non-empty lists of integers, `x` and `y`, but we want to ensure that the values stored in `y` values are *larger than all of the values in* `x`.
-The following test shows how we can leverage Hypothesis to describe these lists
-
-```python
-# We want all of `y`'s entries to be larger than `max(x)`
-from typing import List
-
-# Defining our test function:
-# - `x` is a non-empty list of integers.
-# - `data` is an object provided by Hypothesis that permits us
-# to draw interactively from other strategies within our test
-@given(x=st.lists(st.integers(), min_size=1), data=st.data())
-def test_two_constrained_lists(x, data):
- # We pass `data.draw(...)` a hypothesis strategy - it will draw a value from it.
- # Thus `y` is a non-empty list of integers, whose values are guaranteed to be
- # larger than `max(x)`
- y = data.draw(st.lists(st.integers(min_value=max(x) + 1), min_size=1), label="y")
-
- largest_x = max(x)
- assert all(largest_x < item for item in y)
-```
-```python
-# Running the test
->>> test_two_constrained_lists()
-```
-<!-- #endregion -->
-
-The `given` operator is told to pass two values to our test:
-
- - `x`, which is a list of integers drawn from strategies
- - `data`, which is an instance of the [st.DataObject](https://hypothesis.readthedocs.io/en/latest/_modules/hypothesis/strategies/_internal/core.html#DataObject) class; this instance is what gets drawn from the `st.data()` strategy
-
-The only thing that you need to know about `st.DataObject` is that it's `draw` method expects a hypothesis search strategy, and that it will immediately draw a value from said strategy during the test.
-You can also, optionally, pass a string to `label` argument to the `draw` method.
-This simply permits you to provide a name for the item that was drawn, so that any stack-trace that your test produces is easy to interpret.
-
-
-<div class="alert alert-info">
-
-**Reading Comprehension: Drawing from a strategy interactively**
-
-Write a test that is fed a list (of varying length) of non-negative integers.
-Then, draw a [set](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/DataStructures_III_Sets_and_More.html#The-%E2%80%9CSet%E2%80%9D-Data-Structure) of non-negative integers whose sum is at least as large as the sum of the list.
-Assert that the expected inequality between the sums hold.
-Run the test.
-
-Hint: use the `.filter()` method.
-
-</div>
-
-
-
-## The `example` Decorator
-
-As mentioned before, Hypothesis strategies will draw values (pseudo)*randomly*.
-Thus our test will potentially encounter different values every time it is run.
-There are times where we want to be sure that, in addition the values produced by a strategy, specific values will tested.
-These might be known edge cases, critical use cases, or regression cases (i.e. values that were representative of passed bugs).
-Hypothesis provides [the example decorator](https://hypothesis.readthedocs.io/en/latest/reproducing.html#providing-explicit-examples), which is to be used in conjunction with the `given` decorator, towards this end.
-
-Let's suppose, for example, that we want to write a test whose data are pairs of perfect-squares (e.g. 4, 16, 25, ...), and that we want to be sure that the pairs `(100, 144)`, `(16, 25)`, and `(36, 36)` are tested *every* time the test is run.
-Let's use `example` to guarantee this.
-
-<!-- #region -->
-```python
-# Using the `example` decorator to ensure that specific examples
-# will always be passed as inputs to our test function
-
-from hypothesis import example
-
-# A hypothesis strategy that generates integers that are
-# perfect squares
-perfect_squares = st.integers().map(lambda x: x ** 2)
-
-
-def is_square(x):
- """Returns True if `x` is a perfect square"""
- return int(x ** 0.5) == x ** 0.5
-
-
-@example(a=36, b=36)
-@example(a=16, b=25)
-@example(a=100, b=144)
-@given(a=perfect_squares, b=perfect_squares)
-def test_pairs_of_squares(a, b):
- assert is_square(a)
- assert is_square(b)
-```
-```python
-# running the test
->>> test_pairs_of_squares()
-```
-<!-- #endregion -->
-
-Executing this test runs 103 cases: the three specified examples and one hundred pairs of values drawn via `given`.
-
-
-## Links to Official Documentation
-
-- [Hypothesis](https://hypothesis.readthedocs.io/)
-- [The given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters)
-- [The Hypothesis example database](https://hypothesis.readthedocs.io/en/latest/database.html)
-- [Core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies)
-- [The .map method](https://hypothesis.readthedocs.io/en/latest/data.html#mapping)
-- [The .filter method](https://hypothesis.readthedocs.io/en/latest/data.html#filtering)
-- [Using data() to draw interactively in tests](https://hypothesis.readthedocs.io/en/latest/data.html#drawing-interactively-in-tests)
-- [The example decorator](https://hypothesis.readthedocs.io/en/latest/reproducing.html#providing-explicit-examples)
-
-
-
-## Reading Comprehension Solutions
-
-<!-- #region -->
-**Understanding How Hypothesis Works: Solution**
-
-Define the `test_demonstrating_the_given_decorator` function as above, complete with the failing assertion, and add a print statement to the body of the function, which prints out the value for `x` and `y`.
-
-```python
-@given(x=st.integers(0, 10), y=st.integers(20, 30))
-def test_demonstrating_the_given_decorator(x, y):
- print(x, y)
- assert 0 <= x <= 10
-
- # `y` can be any value in [20, 30]
- # this is a bad assertion: it should fail!
- assert 20 <= y <= 25
-```
-
-Run the test once and make note of the output that is printed. Consider copying and pasting the output to a notepad for reference. Next, rerun the test multiple times and make careful note of the printed output. What do you see? Is the output different from the first run? Does it differ between subsequent runs? Try to explain this behavior.
-
-> The printed outputs between the first and second run differ. The first set out outputs is typically longer than that of the second run. After the second run, the printed outputs are always the exact same. What is happening here is that Hypothesis has to search for the falsifying example during the first run. Once it is identified, the example is recorded in the `.hypothesis` database. All of the subsequent runs are simply re-running this saved case, which is why their inputs are not changing.
-
-In your file browser, navigate to the directory from which you are running this test; if you are following along in a Jupyter notebook, this is simply the directory containing said notebook. You should see a `.hypothesis` directory. As noted above, this is the database that contains the falsifying examples that Hypothesis has identified. Delete the `.hypothesis` directory and try re-running your test? What do you notice about the output now? You should also see that the `.hypothesis` directory has reappeared. Explain what is going on here.
-
-> Deleting `.hypothesis` removes all of the falsifying examples that Hypothesis found for tests that were run from this particular directory. Thus running the test again means that Hypothesis has to find the falsifying example again from scratch. Once it does this, it creates a new database in `.hypothesis`, which is why this directory "reappears".
-<!-- #endregion -->
-
-<!-- #region -->
-**Fixing the Failing Test: Solution**
-
-Update the body of `test_demonstrating_the_given_decorator` so that it no longer fails.
-
-> We simply need to fix the second assertion statement, specifying the bounds on `y`, so that it agrees with what is being drawn from the `integers` strategy.
-
-```python
-@given(x=st.integers(0, 10), y=st.integers(20, 30))
-def test_demonstrating_the_given_decorator(x, y):
- assert 0 <= x <= 10
- assert 20 <= y <= 30
-```
-
-Run the fixed test function. How many times is the test function actually be executed when you run it?
-
-> The `given` decorator, by default, will draw 100 sets of example values from the strategies that are passed to it and will thus execute the decorated test function 100 times.
-
-```python
-# no output (the function returns `None`) means that the test passed
->>> test_demonstrating_the_given_decorator()
-```
-<!-- #endregion -->
-
-<!-- #region -->
-**Exploring other Core Strategies: Solution**
-
-Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False.
-
-```python
->>> st.dictionaries(st.integers(min_value=1), st.booleans()).example()
-{110: True, 19091: True, 136348032: False, 78: False, 9877: False}
-```
-
-Length-4 strings whose elements are only lowercase vowels
-
-```python
->>> st.text(alphabet="aeiou", min_size=4, max_size=4).example()
-'uiai'
-```
-
-Permutations of the list `[1, 2, 3, 4]`
-
-```python
->>> st.permutations([1, 2, 3, 4]).example()
-[2, 3, 1, 4]
-```
-<!-- #endregion -->
-
-**Improving our tests using Hypothesis: Solution**
-
-Part 1: Testing correctness by construction
-
-Write a hypothesis-driven test for the `count_vowels`; include this test in `test/test_basic_functions`.
-This is a test function where we can explicit construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
-And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
-We will want to read about the [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy to construct the different parts of the string.
-The standard library's built-in `string` module provides a string of all printable characters (`string.printable`).
-
-We should ask ourselves: How general are input strings that we are constructing? Are there regular patterns in the strings that might prevent our test from identifying edge case bugs in `count_vowels`?
-
-<!-- #region -->
-```python
-from string import printable
-from random import shuffle
-
-import hypothesis.strategies as st
-from hypothesis import given, note
-
-# a list of all printable non-vowel characters
-_not_vowels = "".join([l for l in printable if l.lower() not in set("aeiouy")])
-
-
-@given(
- not_vowels=st.text(alphabet=_not_vowels),
- vowels_but_not_ys=st.text(alphabet="aeiouAEIOU"),
- ys=st.text(alphabet="yY"),
-)
-def test_count_vowels_hypothesis(not_vowels, vowels_but_not_ys, ys):
- """
- Constructs an input string with a known number of:
- - non-vowel characters
- - non-y vowel characters
- - y characters
-
- and thus, by constructions, we can test that the output
- of `count_vowels` agrees with the known number of vowels
- """
- # list of characters
- letters = list(not_vowels) + list(vowels_but_not_ys) + list(ys)
-
- # We need to shuffle the ordering of our characters so that
- # our input string isn't unnaturally patterned; e.g. always
- # have its vowels at the end
- shuffle(letters)
- in_string = "".join(letters)
-
- # Hypothesis provides a `note` function that will print out
- # whatever input you give it, but only in the case that the
- # test fails.
- # This way we can see the exact string that we fed to `count_vowels`,
- # if it caused our test to fail
- note("in_string: " + in_string)
-
- # testing that `count_vowels` produces the expected output
- # both including and excluding y's in the count
- assert count_vowels(in_string, include_y=False) == len(vowels_but_not_ys)
- assert count_vowels(in_string, include_y=True) == len(vowels_but_not_ys) + len(ys)
-```
-<!-- #endregion -->
-
-Part 2: Property-based testing
-
-Write a hypothesis-driven test for `merge_max_mappings` ; include this test in `test/test_basic_functions`.
-Here, we can't simply contrive the inputs to `merge_max_mappings` in a general way and know what its output should be – we would have to re-implement the function to do that.
-Instead, we should *test the expected properties* of the merged dictionary.
-For example, one such property is that the merged dictionary should only contain maximum values.
-Another property would be that all of the keys among the input dictionaries should be present in the merged dictionary.
-Take some time to think of other such properties that we should test for.
-Ultimately we want to arrive at a comprehensive set of properties to test for such that we can be confident that our merged dictionary is correct.
-
-We will want to use [st.dictionaries()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.dictionaries) to describe the inputs to `merge_max_mappings`.
-Although dictionary keys can be any [hashable object](https://docs.python.org/3/glossary.html#term-hashable), suffice it to use both integers and text for the keys, and integers for the dictionary values in this test, for simplicity's sake.
-
-<!-- #region -->
-```python
-@given(
- dict1=st.dictionaries(
- keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
- ),
- dict2=st.dictionaries(
- keys=st.integers(-10, 10) | st.text(), values=st.integers(-10, 10)
- ),
-)
-def test_merge_max_mappings_hypothesis(dict1, dict2):
- merged_dict = merge_max_mappings(dict1, dict2)
-
- # property: `merged_dict` contains all of the keys among
- # `dict1` and `dict2`
- assert set(merged_dict) == set(dict1).union(dict2), \
- "novel keys were introduced or lost"
-
- # property: `merged_dict` only contains values that appear
- # among `dict1` and `dict2`
- assert set(merged_dict.values()) <= set(dict1.values()).union(
- dict2.values()
- ), "novel values were introduced"
-
- # property: `merged_dict` only contains key-value pairs with
- # the largest value represented among the pairs in `dict1`
- # and `dict2`
- assert all(dict1[k] <= merged_dict[k] for k in dict1) and \
- all(dict2[k] <= merged_dict[k] for k in dict2), \
- "`merged_dict` contains a non-max value"
-
- # property: `merged_dict` only contains key-value pairs that
- # appear among `dict1` and `dict2`
- for k, v in merged_dict.items():
- assert (k, v) in dict1.items() or \
- (k, v) in dict2.items(), \
- "`merged_dict` did not preserve the key-value pairings"
-```
-<!-- #endregion -->
-
-**Using the `.map` method to create a sorted list: Solution**
-
-Using the `.map()` method, construct a Hypothesis strategy that produces a sorted list of integers.
-Generate some examples from your strategy and check that they are sorted (we may have to generate quite a few examples to see a diverse set of values)
-
-<!-- #region -->
-```python
-# Note that the built-in `sorted` function can be supplied
-# directly to the `.map()` method - there is no need to define
-# a function or use a lambda expression here
-sorted_list_of_ints = st.lists(st.integers()).map(sorted)
-```
-```python
->>> sorted_list_of_ints.example()
-[-27120, 97, 12805]
-```
-<!-- #endregion -->
-
-
-**Getting creative with the `.map` method: Solution**
-
-Construct a Hypothesis strategy that produces either the string `"cat"` or the string `"dog"`.
-Then, write a test that uses this strategy;
-it should simply test that either `"cat"` or `"dog"` was indeed produced by the strategy.
-Run the test
-
-<!-- #region -->
-```python
-# We "hijack" the `st.booleans()` strategy, which only generates
-# `True` or `False`, and use the `.map` method to transform these
-# two outputs to `"cat"` or `"dog"`.
-#
-# This is only one of many ways that you could have created this
-# strategy
-cat_or_dog = st.booleans().map(lambda x: "cat" if x else "dog")
-
-@given(cat_or_dog)
-def test_cat_dog(x):
- assert x in {"cat", "dog"}
-```
-```python
-# running the test
->>> test_cat_dog()
-```
-<!-- #endregion -->
-
-<!-- #region -->
-**Drawing from a strategy interactively: Solution**
-
-Write a test that is fed a list (of varying length) of non-negative integers.
-Then, draw a [set](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/DataStructures_III_Sets_and_More.html#The-%E2%80%9CSet%E2%80%9D-Data-Structure) of non-negative integers whose sum is at least as large as the sum of the list.
-Assert that the expected inequality between the sums hold.
-Run the test.
-
-
-Hint: use the `.filter()` method.
-<!-- #endregion -->
-
-<!-- #region -->
-```python
-@given(the_list=st.lists(st.integers(min_value=0)), data=st.data())
-def test_interactive_draw_skills(the_list, data):
- the_set = data.draw(
- st.sets(elements=st.integers(min_value=0)).filter(
- lambda x: sum(x) >= sum(the_list)
- )
- )
- assert sum(the_list) <= sum(the_set)
-```
-```python
-# running the test
->>> test_interactive_draw_skills()
-```
-<!-- #endregion -->
diff --git a/Python/module_6.rst b/Python/module_6.rst
index 2b113105..acd45fe6 100644
--- a/Python/module_6.rst
+++ b/Python/module_6.rst
@@ -17,5 +17,6 @@ This will take us down a bit of a rabbit hole, where we will find the powerful p
Module6_Testing/Intro_to_Testing.md
Module6_Testing/Pytest.md
Module6_Testing/Hypothesis.md
+ Module6_Testing/Hypothesis_Practice_Exercises.md
From 25e25f4eb45ef23dae2bddd76bdffe8c4f766df6 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 19:50:19 -0400
Subject: [PATCH 144/152] remove distutils ref
---
Python/Module5_OddsAndEnds/Modules_and_Packages.md | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/Python/Module5_OddsAndEnds/Modules_and_Packages.md b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
index 8f0034a7..41521504 100644
--- a/Python/Module5_OddsAndEnds/Modules_and_Packages.md
+++ b/Python/Module5_OddsAndEnds/Modules_and_Packages.md
@@ -408,7 +408,7 @@ It must be mentioned that we are sweeping some details under the rug here. Insta
Suppose that we are happy with the work we have done on our `face_detector` project. We will want to install this package - placing it in our site-packages directory so that we can import it irrespective of our Python interpreter's working directory. Here we will construct a basic setup script that will allow us to accomplish this. For completeness, we will also indicate how one would include a test suite alongside the source code in this directory structure.
-We note outright that the purpose of this section is strictly to provide you with the minimum set of instructions needed to install a package. We will not be diving into what is going on under the hood at all. Please refer to [An Introduction to Distutils](https://docs.python.org/3/distutils/introduction.html#an-introduction-to-distutils) and [Packaging Your Project](https://packaging.python.org/tutorials/packaging-projects/#packaging-your-project) for a deeper treatment of this topic.
+We note outright that the purpose of this section is strictly to provide you with the minimum set of instructions needed to install a package. We will not be diving into what is going on under the hood at all. Please refer [the Python packaging user guide](https://packaging.python.org/en/latest/) for a deeper treatment of this topic.
Carrying on, we will want to create a setup-script, `setup.py`, *in the same directory as our package*. That is, our directory structure should look like:
@@ -503,8 +503,7 @@ You are free to install some packages using `conda` and others with `pip`. Just
## Links to Official Documentation
- [Python Tutorial: Modules](https://docs.python.org/3/tutorial/modules.html)
-- [An Introduction to Distutils](https://docs.python.org/3/distutils/introduction.html#an-introduction-to-distutils)
-- [Packaging Your Project](https://packaging.python.org/tutorials/packaging-projects/#packaging-your-project)
+- [The Python Packaging User Guids](https://packaging.python.org/en/latest/)
- [PyPi](https://pypi.org/)
From 407dc7d384bfbfc985aef410dfdbb7c78d403195 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 19:53:23 -0400
Subject: [PATCH 145/152] Apply suggestions from code review
Co-authored-by: Zac Hatfield-Dodds <zac.hatfield.dodds@gmail.com>
---
Python/Module6_Testing/Hypothesis_Practice_Exercises.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
index 035b079c..8df7b5d8 100644
--- a/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
+++ b/Python/Module6_Testing/Hypothesis_Practice_Exercises.md
@@ -26,7 +26,7 @@ thus this section is dedicated to providing some useful exercises towards this e
<div class="alert alert-info">
-**Exercise: Describing data with `st.lists`**
+**Exercise: Describing data with `st.lists()`**
Write a strategy that generates lists of even-valued integers, ranging from length-0 to length-10.
@@ -40,7 +40,7 @@ Write a test that checks these properties and run the test.
**Exercise: Using Hypothesis to learn about floats.. Part 1**
-Use the `st.floats` strategy to identify which float(s) violate the identity: `x == x`.
+Use the `st.floats()` strategy to identify which float(s) violate the identity: `x == x`.
That is, write a hypothesis-driven test for which `assert x == x` *fails*, run the test, and identify the input that causes the failure.
Then, revise your usage of `st.floats` such that it only describes values that satisfy the identity.
From 0f74df1d94b56f76b27f344ce2ff1117ad694d3c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 19:55:56 -0400
Subject: [PATCH 146/152] Ease up on discussion of len esge case
---
Python/Module6_Testing/Hypothesis.md | 1 -
1 file changed, 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 0eebbd66..f92aebcf 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -129,7 +129,6 @@ I did not want this error to distract from what is otherwise merely a simple exa
Hypothesis has a knack for catching these sorts of unexpected edge cases.
Now we know that `len(range(size)) == size` _does not_ hold for "arbitrary" non-negative integers!
-(This overflow behavior is now documented in the [CPython source code](https://github.com/python/cpython) because it was discovered while writing this material 😄).
</div>
From 56366e97c8af82b3bb1984b4a31c7e46f4235d0c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 19:57:51 -0400
Subject: [PATCH 147/152] Update Python/Module6_Testing/Hypothesis.md
Co-authored-by: Zac Hatfield-Dodds <zac.hatfield.dodds@gmail.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 0eebbd66..464743f1 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -308,7 +308,7 @@ We will be leveraging the `.example()` method throughout the rest of this sectio
There are a number critical Hypothesis strategies for us to become familiar with. It is worthwhile to peruse through all of Hypothesis' [core strategies](https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies), but we will take time to highlight a few here.
-#### `st.booleans ()`
+#### `st.booleans()`
[st.booleans()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.booleans) generates either `True` or `False`. This strategy will shrink towards `False`
From 36262f784f75a24931b4d5a5f9eecc36ea63dd0c Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 19:57:57 -0400
Subject: [PATCH 148/152] Update Python/Module6_Testing/Hypothesis.md
Co-authored-by: Zac Hatfield-Dodds <zac.hatfield.dodds@gmail.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 464743f1..29651f85 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -320,7 +320,7 @@ False
<!-- #region -->
-#### `st.lists ()`
+#### `st.lists()`
[st.lists](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.lists) accepts *another* strategy, which describes the elements of the lists being generated. You can also specify:
- bounds on the length of the list
From 0b1f2dc821c490387df9e2429d27fef6a1ca4610 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 19:58:10 -0400
Subject: [PATCH 149/152] Update Python/Module6_Testing/Hypothesis.md
Co-authored-by: Zac Hatfield-Dodds <zac.hatfield.dodds@gmail.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 29651f85..2ff206b7 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -373,7 +373,7 @@ For example, the following strategy will generate length-3 tuples whose entries
The [st.text](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) accepts an "alphabet" – a collection of string-characters – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
-For example, the following strategy will strings of lowercase vowels from length 2 to length 10:
+For example, the following strategy will generate strings of lowercase vowels from length 2 to length 10:
```python
>>> st.text("aeiouy", min_size=2, max_size=10).example()
From 2baed1d46a0e4d3e44eb07a8589110f8d19146c7 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 20:00:04 -0400
Subject: [PATCH 150/152] be more precise in description of st.text()
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index db140fc5..4582a87e 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -370,7 +370,7 @@ For example, the following strategy will generate length-3 tuples whose entries
<!-- #region -->
#### `st.text()`
-The [st.text](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) accepts an "alphabet" – a collection of string-characters – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
+The [st.text](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) accepts an "alphabet" – a collection of length-one strings or a strategy for generating such values (such as `st.characters()`). – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
For example, the following strategy will generate strings of lowercase vowels from length 2 to length 10:
From c97ee7a60d3a73a01096816a65608962fff0a5b5 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 21:14:10 -0400
Subject: [PATCH 151/152] Update Python/Module6_Testing/Hypothesis.md
Co-authored-by: Zac Hatfield-Dodds <zac.hatfield.dodds@gmail.com>
---
Python/Module6_Testing/Hypothesis.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 2ff206b7..c3249b9d 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -430,7 +430,7 @@ The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` ca
<!-- #endregion -->
<!-- #region -->
-#### `st.sampled_from`
+#### `st.sampled_from()`
[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and NumPy arrays). The strategy will return values that are sampled from this collection.
From b643bb31819a1193db85ae874d8f226914737b46 Mon Sep 17 00:00:00 2001
From: Ryan Soklaski <ry26099@mit.edu>
Date: Mon, 22 Aug 2022 21:33:06 -0400
Subject: [PATCH 152/152] Fix description of lambda, and make use of term
'strategy' consistent
---
Python/Module6_Testing/Hypothesis.md | 36 ++++++++++++++--------------
1 file changed, 18 insertions(+), 18 deletions(-)
diff --git a/Python/Module6_Testing/Hypothesis.md b/Python/Module6_Testing/Hypothesis.md
index 4582a87e..e81586d7 100644
--- a/Python/Module6_Testing/Hypothesis.md
+++ b/Python/Module6_Testing/Hypothesis.md
@@ -276,7 +276,7 @@ Update the body of `test_demonstrating_the_given_decorator` so that it no longer
## Describing Data: Hypothesis Strategies
Hypothesis provides us with so-called "strategies" for describing our data.
-We are already familiar with the `integers` strategy;
+We are already familiar with the `integers()` strategy;
Hypothesis' core strategies are all located in the `hypothesis.strategies` module.
The official documentation for the core strategies can be found [here](https://hypothesis.readthedocs.io/en/latest/data.html).
@@ -289,15 +289,17 @@ Hypothesis provides a useful mechanism for developing an intuition for the data
```python
# demonstrating usage of `<strategy>.example()`
->>> st.integers(-1, 1).example()
+>>> for _ in range(5):
+... print(st.integers(-1, 1).example())
+0
+1
-1
-
->>> st.integers(-1, 1).example()
1
+0
```
**Note: the** `.example()` **mechanism is only meant to be used for pedagogical purposes. You should never use this in your test suite**
-because (among other reasons) `.example()` biases towards smaller and simpler examples than `@given`, and lacks the features to ensure any test failures are reproducible.
+because (among other reasons) `.example()` lacks the features to ensure any test failures are reproducible.
We will be leveraging the `.example()` method throughout the rest of this section to help provide an intuition for the data that Hypothesis' various strategies generate.
<!-- #endregion -->
@@ -321,7 +323,7 @@ False
#### `st.lists()`
-[st.lists](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.lists) accepts *another* strategy, which describes the elements of the lists being generated. You can also specify:
+[st.lists()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.lists) accepts *another* strategy, which describes the elements of the lists being generated. You can also specify:
- bounds on the length of the list
- if we want the elements to be unique
- a mechanism for defining "uniqueness"
@@ -333,13 +335,13 @@ For example, the following strategy describes lists whose length varies from 2 t
[-10, 0, 5]
```
-**`st.lists(...)` is the strategy of choice anytime we want to generate sequences of varying lengths with elements that are, themselves, described by strategies**.
+**st.lists(...) is our go-to anytime we want to create a strategy that generates sequences of varying lengths with elements that are, themselves, described by strategies**.
<!-- #endregion -->
<!-- #region -->
#### `st.floats()`
-[st.floats](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.floats) is a powerful strategy that generates all variety of floats, including `math.inf` and `math.nan`. You can also specify:
+[st.floats()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.floats) is a powerful strategy that generates all variety of floats, including `math.inf` and `math.nan`. You can also specify:
- whether `math.inf` and `math.nan`, respectively, should be included in the data description
- bounds (either inclusive or exclusive) on the floats being generated; this will naturally preclude `math.nan` from being generated
- the "width" of the floats; e.g. if you want to generate 16-bit or 32-bit floats vs 64-bit
@@ -370,7 +372,7 @@ For example, the following strategy will generate length-3 tuples whose entries
<!-- #region -->
#### `st.text()`
-The [st.text](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) accepts an "alphabet" – a collection of length-one strings or a strategy for generating such values (such as `st.characters()`). – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
+The [st.text()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) strategy accepts an "alphabet" – a collection of length-one strings or a strategy for generating such values (such as `st.characters()`) – from which it will construct strings of varying lengths, whose bounds can be specified by the user.
For example, the following strategy will generate strings of lowercase vowels from length 2 to length 10:
@@ -383,7 +385,7 @@ For example, the following strategy will generate strings of lowercase vowels fr
<!-- #region -->
#### `st.just()`
-[st.just](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.just) is a strategy that "just" returns the value that you fed it. This is a convenient strategy that helps us to avoid having to manipulate our data before using it.
+[st.just()](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.just) is a strategy that "just" returns the value that you fed it. This is a convenient strategy that helps us to avoid having to manipulate our data before using it.
Suppose that we want a strategy that describes the shape of an array (i.e. a tuple of integers) that contains 1-20 two-dimensional vectors. E.g. `(5, 2)` is the shape of the array containing five two-dimensional vectors. We can leverage `st.just`, in conjunction with `st.integers` and `st.tuples`, towards this end:
@@ -429,9 +431,9 @@ The "pipe" operator, `|` can be used between strategies, to chain `st.one_of` ca
<!-- #endregion -->
<!-- #region -->
-#### `st.sampled_from`
+#### `st.sampled_from()`
-[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and NumPy arrays). The strategy will return values that are sampled from this collection.
+[st.sampled_from](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.sampled_from) accepts a collection of objects (anything that has a length and supports integer-based indexing is a collection; e.g. lists, tuples, strings, and NumPy arrays) and returns a strategy that are randomly samples values from this collection.
For example, the following strategy will sample a value `0`, `"a"`, or `(2, 2)` from a list:
@@ -458,15 +460,13 @@ Write down a strategy, and print out a representative example, that describes th
<!-- #region -->
<div class="alert alert-info">
-**Reading Comprehension: Improving our tests using Hypothesis**
+**Reading Comprehension: Testing correctness by construction**
-We will be writing improved tests for the basic functions – `count_vowels` and `merge_max_mappings` – by leveraging Hypothesis.
+We will be writing improved tests for `count_vowels` by leveraging Hypothesis.
This reading comprehension question will require more substantial work than usual.
That being said, the experience that we will gain from this will be well worth the work.
Keep in mind that solutions are included at the end of this page, and that these can provide guidance if we get stuck.
-Testing correctness by construction:
-
Write a Hypothesis-driven test for the `count_vowels`; include this test in `tests/test_basic_functions`.
This is a test function where we can explicitly construct a string in parts: its non-vowel characters, non-y vowels, and y-vowels.
And thus, by constructing a string with a known number of vowel and non-vowel characters, we can know what the output of `count_vowels` *should* be for that input, and we can thus test for correctness in this way.
@@ -546,7 +546,7 @@ In general, the syntax for defining a lambda expression is:
lambda <comma-separated variable names>: <expression using variables>
```
-Note that lambda expressions are restricted compared to typical function definitions: they do not permit default values or keyword arguments in their signatures, and a lambda's "body" must fit on one line.
+Note that lambda expressions are restricted compared to typical function definitions: their body must consist only of a single Python expression, and the lambda function returns whatever that expression returns.
Here are some examples of lambda expressions:
@@ -748,7 +748,7 @@ Executing this test runs 103 cases: the three specified examples and one hundred
Thus far we have learned about the basic anatomy of a test, how to use pytest to create an automated test-suite for our code base, and how to leverage Hypothesis to generate diverse and randomized inputs to our test functions.
In the final section of this module, we will discuss three testing methods: example-based testing, fuzzing, and property-based testing (Hypothesis will prove to be indispensable for facilitating these last two methods).
-These strategies will equip us with ability to "test the untestable": we will be able to write effective tests for code where we are unable to discern what the exact appropriate behavior is for the code under arbitrary inputs.
+These strategies will equip us with ability to "test the untestable": we will be able to write effective tests for code even when we can't predict what the exact behavior of a function should be for an arbitrary input.
## Links to Official Documentation