RDFParser throws an error "Not a valid token for an RDF term: [LBRACE]" while parsing valid Turtle file

[mpro7]

Version

4.9.0

What happened?

After upgrading Jena from v4.8.0 to v4.9.0 one of tests which process streamed RDF from valid TTL file and write the formatted result to an output stream failed with an error:
sbt.ForkMain$ForkError: org.apache.jena.riot.RiotException: [line: 11, col: 26] Not a valid token for an RDF term: [LBRACE].

I've noticed that v4.9.0 introduced new Turtle format, so maybe btw of that change a bug sneaked into the RDF parser code too?

Relevant output and stacktrace

[line: 11, col: 26] Not a valid token for an RDF term: [LBRACE]
org.apache.jena.riot.RiotException: [line: 11, col: 26] Not a valid token for an RDF term: [LBRACE]
	at org.apache.jena.riot.system.ErrorHandlerFactory$ErrorHandlerStrict.fatal(ErrorHandlerFactory.java:235)
	at org.apache.jena.riot.system.ParserProfileStd.create(ParserProfileStd.java:310)
	at org.apache.jena.riot.lang.LangTurtleBase.tokenAsNode(LangTurtleBase.java:577)
	at org.apache.jena.riot.lang.LangTurtleBase.node(LangTurtleBase.java:410)
	at org.apache.jena.riot.lang.LangTurtleBase.predicate(LangTurtleBase.java:376)
	at org.apache.jena.riot.lang.LangTurtleBase.predicateObjectItem(LangTurtleBase.java:350)
	at org.apache.jena.riot.lang.LangTurtleBase.predicateObjectList(LangTurtleBase.java:333)
	at org.apache.jena.riot.lang.LangTurtleBase.triples(LangTurtleBase.java:314)
	at org.apache.jena.riot.lang.LangTurtleBase.triplesSameSubject(LangTurtleBase.java:178)
	at org.apache.jena.riot.lang.LangTurtle.oneTopLevelElement(LangTurtle.java:46)
	at org.apache.jena.riot.lang.LangTurtleBase.runParser(LangTurtleBase.java:79)
	at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:43)
	at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTLang.read(RDFParserRegistry.java:204)
	at org.apache.jena.riot.RDFParser.read(RDFParser.java:416)
	at org.apache.jena.riot.RDFParser.parseNotUri(RDFParser.java:406)
	at org.apache.jena.riot.RDFParser.parse(RDFParser.java:356)
	at org.apache.jena.riot.RDFParserBuilder.parse(RDFParserBuilder.java:570)

Are you interested in making a pull request?

None

[rvesse]

I'm unable to reproduce this, the given input file validates successfully riot --validate with Jena 4.8.0, 4.9.0, 4.10.0

Also the error stack trace you show refers to a Line/Column that is a blank line in your linked file so that isn't the actual data you are encountering the error on. Your bug report implies that there is some data transformation happening in your environment so you need to provide all the steps necessary to reproduce this issue in order for us to help you.

[mpro7]

@rvesse thanks for the replay. Sorry, the stacktrace was somehow truncated. Here is the link how it exactly looks like (the bottom error). In the test the file is read and process with parseWithStreamProcessor method which uses jena.riot.RDFParser to do so. Result then should be written into ByteArrayOutputStream but it stucks on RDFParser.parse() method.

[rvesse]

Here is the link

Again no idea what that link is for, it's to a diff in one of your projects PRs and there's no error message present there (nor any Jena related code) 😕

[mpro7]

@rvesse sorry, this is because I've updated the branch and instead of commit link I've pasted some failing tests link, which seems to change after the branch update. Here is the correct link. To see the full stack trace, scroll down to last error and click raw output.

[afs]

I've noticed that v4.9.0 introduced new Turtle format,

It is a new output format and it isn't used by default.

I can't see what's happening other than it seems the parsing is from an input stream given the parser - setup at line 139.

There isn't an LBRACE (a { ) in the file.

This would be the error if the input stream is actually TriG syntax, the parser syntax is set to Turtle, and a named graph is in the input.

Please could we have a complete, reproducible, minimal example.

[afs]

@mpro7 Any progress on a reproducible, minimal example?

[afs]

Ping @mpro7

[afs]

We haven't heard back on this and can't reproduce it.

@mpro7 Please do reopen this issue if there is new information.

[mpro7]

@afs sorry for coming back late, we solved the issue which was on our side. Our code did some additional transformations which started to fail after upgrading to Jena v4.9.0. Once we cleaned up the code and started calling Jena functions more directly, the problem was gone. However it shows that some functionalities on Jena side changed.

If you still would like to have more detailed information, please let me know.

[afs]

@mpro7 - Yes, I'd like to know more.

Very little has changed between 4.8.0 (several versions before that) to 4.9.0 in this area.

[BalduinLandolt]

Hi, since I worked with @mpro7 on this, and I had some spare time to reproduce it, let me provide additional information:

The following (admittedly nonsensical!) Scala code:

import org.apache.jena

import java.io.{ByteArrayInputStream, ByteArrayOutputStream, InputStream, OutputStream}
import java.nio.charset.StandardCharsets
import scala.util.Try

object Run extends App {
  val ttl =
    """|
       |@prefix xsd:        <http://www.w3.org/2001/XMLSchema#> .
       |@prefix rdf:        <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
       |@prefix rdfs:       <http://www.w3.org/2000/01/rdf-schema#> .
       |@prefix owl:        <http://www.w3.org/2002/07/owl#> .
       |@prefix knora-base: <http://www.knora.org/ontology/knora-base#> .
       |
       |<http://www.knora.org/ontology/0001/anything>
       |    rdf:type                        owl:Ontology ;
       |    rdfs:label                      "The anything ontology" ;
       |    knora-base:attachedToProject    <http://rdfh.ch/projects/0001> ;
       |    knora-base:lastModificationDate "2017-12-19T15:23:42.166Z"^^xsd:dateTime .
       |
       |""".stripMargin

  val is = new ByteArrayInputStream(ttl.getBytes(StandardCharsets.UTF_8))
  val os = new ByteArrayOutputStream()

  StreamParser.parse(is, os)
  val res = os.toString

  println(res)
}

object StreamParser {

  def parse(inputStream: InputStream, outputStream: OutputStream): Unit = {

    val streamRDF = new jena.riot.system.StreamRDF {
      val inner = jena.riot.system.StreamRDFWriter.getWriterStream(outputStream, jena.riot.RDFLanguages.TURTLE)

      override def start(): Unit                             = inner.start()
      override def finish(): Unit                            = inner.finish()
      override def base(base: String): Unit                  = {}
      override def prefix(prefix: String, iri: String): Unit = inner.prefix(prefix, iri)
      override def quad(quad: jena.sparql.core.Quad): Unit   = inner.quad(quad)
      // ----- !!! here !!! -----
      override def triple(triple: jena.graph.Triple): Unit =
        inner.quad(jena.sparql.core.Quad.create(jena.sparql.core.Quad.defaultGraphIRI, triple))
    }
    val parser = jena.riot.RDFParser.create()
    parser.source(inputStream)

    val parseTry: Try[Unit] = Try {
      parser
        .lang(jena.riot.RDFLanguages.TURTLE)
        .errorHandler(jena.riot.system.ErrorHandlerFactory.errorHandlerStrictNoLogging)
        .parse(streamRDF)
    }
    inputStream.close()
    outputStream.close()
    parseTry.get
  }

}

when executed with Jena 4.8.0, produces

@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix knora-base: <http://www.knora.org/ontology/knora-base#> .

<http://www.knora.org/ontology/0001/anything>
        rdf:type                      owl:Ontology ;
        rdfs:label                    "The anything ontology" ;
        knora-base:attachedToProject  <http://rdfh.ch/projects/0001> ;
        knora-base:lastModificationDate  "2017-12-19T15:23:42.166Z"^^xsd:dateTime .

but with Jena 4.9.0 it outputs

@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix knora-base: <http://www.knora.org/ontology/knora-base#> .

<urn:x-arq:DefaultGraph> {
    <http://www.knora.org/ontology/0001/anything>
            rdf:type                      owl:Ontology ;
            rdfs:label                    "The anything ontology" ;
            knora-base:attachedToProject  <http://rdfh.ch/projects/0001> ;
            knora-base:lastModificationDate  "2017-12-19T15:23:42.166Z"^^xsd:dateTime .
}

the interesting part is probably where the implementation of StreamRDF implements triple() as quad() with default graph.
From the looks of it, in Jena 4.8 this produced TTL, while in Jena 4.9 it seems to produce TRIG output.

I hope this information helps, and I apologise for providing Scala rather than Java code (but it was painful enough to get to this reproduction of the issue).

If you have any questions, please feel free to ask!

[afs]

@BalduinLandolt -- thank you for the information.

      override def triple(triple: jena.graph.Triple): Unit =
        inner.quad(jena.sparql.core.Quad.create(jena.sparql.core.Quad.defaultGraphIRI, triple))

Could I ask what the code is trying to achieve by converting a triple to a quad if the intended output is Turtle (triples only)?