ISQL CSV encoding is weird...

[joernhees]

Is it just me or is the isql CSV encoding weird?

SQL> SET CSV=ON;
SQL> sparql select ?s ("hallo you; % jörn" as ?foo) ?p  where {?s ?p ?o} limit 1;
s;foo;p
http://www.openlinksw.com/virtrdf-data-formats#default-iid;hallo you%3B %25 j%FFF6rn;http://www.w3.org/1999/02/22-rdf-syntax-ns#type

1 Rows. -- 2 msec.

I guess that this is some kind of ASCII-%-URI-like-encoding, but it's not very parseable, especially the " % " being equal to "%3B %25 " and the "ö" being "%FFF6".

[roelj]

Digging up an old issue..

I am wondering how the encoding actually works. I have a value in which a single ö is turned into %FFC3%FFB6.
The command I used is:

isql 1111 csv=on exec='SPARQL SELECT ?s ?o FROM <some-graph> WHERE { ?s rdfs:label ?o }'

Not using the csv=on produces the ö just fine:

isql 1111 exec='SPARQL SELECT ?s ?o FROM <some-graph> WHERE { ?s rdfs:label ?o }'

[HughWilliams]

The "isql" SET CSV_RFC4180 ON; command is the correct way to enable CSV results output:

SQL> SET CSV_RFC4180 ON;
SQL> SPARQL SELECT * WHERE {?s ?p ?o} LIMIT 2;
"s","p","o"
"http://www.w3.org/1999/02/22-rdf-syntax-ns#type","http://www.w3.org/1999/02/22-rdf-syntax-ns#type","http://www.w3.org/1999/02/22-rdf-syntax-ns#Property"
"http://www.w3.org/1999/02/22-rdf-syntax-ns#type","http://www.w3.org/1999/02/22-rdf-syntax-ns#type","http://www.w3.org/1999/02/22-rdf-syntax-ns#Property"

2 Rows. -- 81 msec.
SQL>

[roelj]

That doesn't seem to affect the encoding:

/opt/sparqling-genomics/bin/isql 1111 verbose=off csv_rfc4180=on csv_rfc4180_field_separator=, exec='SPARQL SELECT ?s ?o FROM <some-graph> WHERE { ?s rdfs:label ?o }'

Produces the same %FFC3%FFB6.

[HughWilliams]

We shall look into this encoding issue in "isql" ...

[roelj]

Thanks a lot!

Perhaps an easy way to avoid breaking anything is to introduce a no_encode option in binsrc/tests/isql.c and use field_print_normal in field_print_csv_rfc4180 when no_encode is set to on.

If you'd like I could prepare a patch that implements this.

[HughWilliams]

@roelj: patch contributions are always welcomed ...

[roelj]

Just letting you know that my initial idea does not seem to have any effect: roelj@4908939. So I'm working on a second version of the patch.