fix: small InputSource related issues #2255
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aucampia
force-pushed
the
iwana-20230307T1924-fix_iri2uri
branch
from
372a8fc
to
0fd7567
Compare
aucampia
force-pushed
the
iwana-20230307T1924-fix_iri2uri
branch
from
0fd7567
to
dbd7cb0
Compare
|
I did this while looking at #1844 - but this does not quite address anything there. I plan to make a separate PR with some security documentation and warnings, and maybe an example of how to use python auditing and |
aucampia
force-pushed
the
iwana-20230307T1924-fix_iri2uri
branch
2 times, most recently
from
5584d45
to
415669e
Compare
|
Reduced the size of changes a bit, will merge with no review as this is pretty well tested code and the non-test changes are very minor. |
aucampia
force-pushed
the
iwana-20230307T1924-fix_iri2uri
branch
2 times, most recently
from
a9a2173
to
9060b7c
Compare
I have added a bunch of tests for `InputSource` handling, checking every kind of input source with every parser. During this, I detected the following issues that I fixed: - `rdflib.util._iri2uri()` was URL quoting the `netloc` parameter, but this is wrong and the `idna` encoding already takes care of special characters. I removed the URL quoting of `netloc`. - HexTuple parsing was handling the input source in a way that would only work for some input sources, and not raising errors for other input sources. I changed the input source handling to be more generic. - `rdflib.parser.create_input_source()` incorrectly used `file.buffer` instead of `source.buffer` when dealing with IO stream sources. Other changes with no runtime impact include: - Changed the HTTP mocking stuff in test slightly to accommodate serving arbitrary files, as I used this in the `InputSource` tests. - Don't use google in tests as we keep getting `urllib.error.HTTPError: HTTP Error 429: Too Many Requests` from it.
aucampia
force-pushed
the
iwana-20230307T1924-fix_iri2uri
branch
from
9060b7c
to
ec2e5c6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary of changes
I have added a bunch of tests for
InputSourcehandling, checking every kind of input source with most of the parsers. During this, I detected the following issues that I fixed:rdflib.util._iri2uri()should not URL quote thenetlocparameter, theidnaencoding already takes care of special characters. I removed the URL quoting ofnetlocrdflib.parser.create_input_source()incorrectly usesfile.bufferinstead ofsource.bufferwhen dealing with IO stream sources.Other changes with no runtime impact include:
InputSourcetests.urllib.error.HTTPError: HTTP Error 429: Too Many Requestsfrom it.
Checklist
the same change.
so maintainers can fix minor issues and keep your PR up to date.