Skip to content

Loading…

if query result source has no encoding set, fall back to utf-8 encoding. #366

Closed
wants to merge 2 commits into from

1 participant

@gweis
RDFLib member

It's a bit tricky to do this properly for python 2 and 3, but I think this patch goes into the right direction.

For later on I think rdflib should be clear about what source is, and what attributes, methods and return values are expected.

If that would be defined, then a user of the library would know when to wrap a source within a codec or not.

resolves #344

@gweis
RDFLib member

Not quite sure about it, but the failing travis tests seem to be unrelated.

I hope this patch can go into 4.1.1 release, if this works for everyone

@gweis gweis closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
View
5 rdflib/plugins/sparql/results/csvresults.py
@@ -23,7 +23,10 @@ def parse(self, source):
r = Result('SELECT')
- if hasattr(source, 'mode') and 'b' in source.mode:
+ if not hasattr(source, 'mode') or 'b' in source.mode and not getattr(source, 'encoding', None):
+ # if there is no mode, or source is in binary mode
+ # assume we need to decode from utf-8.
+ # if there is a mode set and it's not binary, check if source has encoding set.
source = codecs.getreader('utf-8')(source)
reader = csv.reader(source, delimiter=self.delim)
View
5 rdflib/plugins/sparql/results/tsvresults.py
@@ -40,7 +40,10 @@
class TSVResultParser(ResultParser):
def parse(self, source):
- if hasattr(source, 'mode') and 'b' in source.mode:
+ if not hasattr(source, 'mode') or 'b' in source.mode and not getattr(source, 'encoding', None):
+ # if there is no mode, or source is in binary mode
+ # assume we need to decode from utf-8.
+ # if there is a mode set and it's not binary, check if source has encoding set.
source = codecs.getreader('utf-8')(source)
try:
View
2 test/test_dawg.py
@@ -392,7 +392,7 @@ def skip(reason='(none)'):
s = BytesIO()
res2.serialize(s, format='csv')
print s.getvalue()
- s = StringIO(s.getvalue().decode('utf-8')) # hmm ?
+ s = StringIO(s.getvalue()) # hmm ?
res2 = Result.parse(s, format='csv')
else:
Something went wrong with that request. Please try again.