Ensure body is consumed only once (alternative to #847) #851
Conversation
|
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #851 +/- ##
==========================================
- Coverage 92.32% 91.96% -0.37%
==========================================
Files 27 27
Lines 1811 1829 +18
Branches 338 293 -45
==========================================
+ Hits 1672 1682 +10
- Misses 92 96 +4
- Partials 47 51 +4 ☔ View full report in Codecov by Sentry. |
tests/unit/test_stubs.py
Outdated
| data2, | ||
| data3, | ||
| ): | ||
| testpath = str(tmpdir.join(testfile)) |
There was a problem hiding this comment.
Any particular reason why we want control over the precise filename here? If not, with NamedTemporaryFile(...) would work:
suffix=".yml"allows to control the file extension if neededdir=tmpdirallows using the directory we have been given.
It would resolve one no-longer-necessary parameter. I think it could help simplicity.
There was a problem hiding this comment.
I had to use TemporaryDirectory instead, because vcrpy fails when cassette is an empty file.
There was a problem hiding this comment.
@sathieu interesting! I got curious, played with it and can confirm that the file cannot exits already when we start. Here's what I did to make it work still:
# git --no-pager diff
diff --git a/tests/unit/test_stubs.py b/tests/unit/test_stubs.py
index c177061..8373923 100644
--- a/tests/unit/test_stubs.py
+++ b/tests/unit/test_stubs.py
@@ -1,7 +1,7 @@
import contextlib
import http.client as httplib
from io import BytesIO
-from tempfile import TemporaryDirectory
+from tempfile import NamedTemporaryFile
from unittest import mock
from pytest import mark
@@ -52,8 +52,12 @@ class TestVCRConnection:
data2,
data3,
):
- with TemporaryDirectory(dir=tmpdir) as cassete_dir:
- testpath = str(cassete_dir.join("cass.yml"))
+ with NamedTemporaryFile(dir=tmpdir, suffix='.yml') as f:
+ testpath = f.name
+ # NOTE: ``use_cassette`` is not okay with the file existing
+ # already. So we are using ``.close()`` to not only
+ # close but also delete the empty file, before we start.
+ f.close()
host, port = httpbin.host, httpbin.port
match_on = ["method", "uri", "body"]
with use_cassette(testpath, match_on=match_on):I'm not saying that's better, maybe it's not 🤷 How about adding a sentence of comment why TemporaryDirectory is prefered over NamedTemporaryFile in our case? Something like that would maybe be best?
There was a problem hiding this comment.
Yes, changed to your suggestion. thanks!
vcr/util.py
Outdated
| @@ -92,6 +92,21 @@ def composed(incoming): | |||
| def read_body(request): | |||
| if hasattr(request.body, "read"): | |||
| return request.body.read() | |||
| if hasattr(request.body, "__iter__") and not isinstance( | |||
| request.body, | |||
| (bytearray, bytes, dict, list, str), | |||
There was a problem hiding this comment.
This very 5-types list exists in two places in the code base now, so there is risk of the two going out of sync. Could this be (a) extracted to a non-public constant-like symbol somewhere or (b) commented about keeping in sync on both sides? (a) would likely be preferable. What do you think?
There was a problem hiding this comment.
added a new is_iterator function. WDYT?
There was a problem hiding this comment.
@sathieu the function idea and the function body I like, but the name is a bit misleading and I'm worried that this de-facto internal helper function is becoming part of the public API this way. How about _is_nonsequence_iterator? The naming idea is based on collections.abc.Sequence and it makes me realize as a side effect that we could make replace isinstance(request.body, (bytearray, bytes, dict, list, str)) by isinstance(request.body, Sequence). What do you think?
There was a problem hiding this comment.
Renamed to _is_nonsequence_iterator.
Unfortunately, isinstance(request.body, Sequence) is breaking a few tests...
hartwork
changed the title
sathieu
force-pushed
the
consume-body-once2
branch
from
ffb8c07
to
94ab4aa
Compare
tests/unit/test_stubs.py
Outdated
| data2, | ||
| data3, | ||
| ): | ||
| testpath = str(tmpdir.join(testfile)) |
There was a problem hiding this comment.
I had to use TemporaryDirectory instead, because vcrpy fails when cassette is an empty file.
vcr/util.py
Outdated
| @@ -92,6 +92,21 @@ def composed(incoming): | |||
| def read_body(request): | |||
| if hasattr(request.body, "read"): | |||
| return request.body.read() | |||
| if hasattr(request.body, "__iter__") and not isinstance( | |||
| request.body, | |||
| (bytearray, bytes, dict, list, str), | |||
There was a problem hiding this comment.
added a new is_iterator function. WDYT?
Fixes: kevin1024#846 Signed-off-by: Mathieu Parent <math.parent@gmail.com>
sathieu
force-pushed
the
consume-body-once2
branch
from
94ab4aa
to
241b0bb
Compare
|
@hartwork Sorry for the delay, the mail was lost in my inbox 😉 ... Back to you, see my above comments ☝️. |
There was a problem hiding this comment.
@sathieu thanks for the adjustements! This looks ready-to-merge to me. Any concerns from @kevin1024 or @jairhenrique with regard to merging?
|
The only contributions I have are (1) Thank you for your work, this looks great! and (2) this PR title sounds like some kind of strange horror movie. |
Fixes: #846