Initial High Level spec including Dataset interface

[bergos]

This is mainly an entry point for other PR. The Datasetinterface is a copy of the wiki page without the context, questions and links to RDF-Ext.

[dmitrizagidulin]

What's the factory parameter again?

[elf-pavlik]

I think it forwards data factory which dataset should use https://github.com/rdfjs/representation-task-force/issues/79 but LowLevel API defines factory as Object with multiple methods not Function

[elf-pavlik]

I think we should clarify how Store and Dataset relate to each other. I think sometimes the same object might need to have both Store and Dataset interface...

Related - rdf-ext-archive/discussions#21

1. match

• Source: Stream .match([Term|RegExp subject], [Term|RegExp predicate], [Term|RegExp object], [Term|RegExp graph]) Returns a stream that processes all quads matching the pattern.
• Dataset: Dataset match (Term subject, Term predicate, Term object, Term graph) synchronous method, returns a Dataset containing all the quads that match the given pattern. See also the RDFJS spec's Stream.match() method.

2. import

• Sink: undefined .import(Stream stream) Writes all quads from the stream to the sink.
• Dataset: Promise<Dataset> import (Stream stream)

3. remove

• Store: EventEmitter .remove(Stream stream) Removes all streamed quads. The end and error events are used like described in the Stream interface.
• Dataset: Dataset remove(Quad quad) removes the given quad from this dataset, returns itself (this dataset)

4. removeMatches

• Store: EventEmitter .removeMatches([Term|RegExp subject], [Term|RegExp predicate], [Term|RegExp object], [Term|RegExp graph]) All quads matching the pattern will be removed. The end and error events are used like described in the Stream interface.
• Dataset: Dataset removeMatches (Term subject, Term predicate, Term object, Term graph) removes all quads that match the given pattern from this dataset, returns itself (this dataset)

Maybe Dataset could expose LowLevel Stream interfaces with properties

• Source .source
• Sink .sink
• Store .store

[elf-pavlik]

This gitter comment brought to my attention one of the distinctions between Dataset and Store https://gitter.im/linkeddata/rdflib.js?at=580e26dc684ccefd38a081fa

Stores could do indexing, but Dataset would just provide more lightweight inteface for a set of quads, in many cases more convenient to use than a raw stream of quads.

[RubenVerborgh]

Isn't a Store an implementation of a Dataset then?

[elf-pavlik]

As I understood @dmitrizagidulin rdflib.js would not implement distinct Store and Dataset.

[jacoscaz]

I think some sort of easy-to-use abstraction for dealing with sets of quads being passed around between functions is necessary. I find that one of the hardest thing to do in RDF land is creating the equivalent of simple objects like {power: true, dimming: 7} in a straight-forward way.

We experimented with (de)serialising RDF data to instances of a custom wrapper around N3's Store that adds chain-based features like

set.subject('<http://example.com/s>')
  .predicate('<http://example.com/power>', true)
  .predicate('<http://example.com/dimming>', 7)

but felt that it still becomes too complex. We're now looking at a more descriptive approach via something like simplerdf + RDF/JS interface.

In any case, I think the API between Dataset and Store should be kept as different as possible to make life easier for implementors.

[RubenVerborgh]

@jacoscaz Indeed, simplerdf was actually the final trigger to get the RDF/JS work started. If all low-level libraries implement this spec, then simplerdf can be compatible with all of them.

[dmitrizagidulin]

In any case, I think the API between Dataset and Store should be kept as different as possible to make life easier for implementors.

Can you explain a bit more?

[elf-pavlik]

Besides conversation in rdf-ext-archive/discussions#21

@bergos: The difference between a Store and Graph is more the sync vs. async interface.

We also had some discussion with @bergos during TF calls about possible requirement for Store interface anticipate using it for stores that provide interface to remote servers (LDP, SPARQL, Triple Patter Fragmetns ...) Which will have different requirements to Dataset which to my understanding supposed to just represent set of quads available in memory.

[blake-regalia]

@jacoscaz

I think some sort of easy-to-use abstraction for dealing with sets of quads being passed around between functions is necessary.

This is where libraries can take their own approach to simplifying the process, such as with creating/inserting triples.

I find that one of the hardest thing to do in RDF land is creating the equivalent of simple objects

Like this? The output is shown here. I'll be releasing this w/ next version of graphy.

[niklasl]

@jacoscaz @blake-regalia Without specific application needs I cannot really judge what's required, but why not just JSON-LD? Like:

{
  "@context": {"@vocab": "http://example.com/"},
  "@id": "http://example.com/s",
  "power": true,
  "dimming": 7
}

[bergos]

NamedNode, BlankNode, Literal and Quad are all in memory representations of the RDF model. Dataset completes the RDF model. It's aligned to the JS Array API. It's not designed to connect to external stores like the Store interface. It's designed to pass multiple quads in JS code and sync access. But I agree that the label high level is confusing.

I would like to shift the discussion, if and how we add it to the spec, to a later time and focus now on the stream interfaces. Of course everybody who want's to use or implement the API is invited to discuss technical details.

This is not about about an application specific high level API like SimpleRDF or other mentioned libraries. We can make a wiki page for it and an issue to discuss it, but let's not mix topics here.

[jacoscaz]

@niklasl @blake-regalia yes, we're also experimenting with pre-compacting JSON-LD documents down to plain old JSON objects using shared contexts before passing data from RDF-aware components to non-RDF-aware components.

[l00mi]

How do we want to proceed here?
👍 Merge and open multiple Issues/PR to adopt the initial proposition.
👎 Close the request and start off with a more basic version.

[elf-pavlik]

If we decide to merge it let's create right away an Issue (or PR if someone feels up to it) which clarifies similarities and differences between Store and Dataset

[l00mi]

Can please everybody cast your vote to go with this proposition as initial draft for the high-level spec?

This will lay the ground for the future topics discussed in #123, https://github.com/rdfjs/representation-task-force/issues/108, #100, #98.

[bergos]

In favor of the new Dataset Spec, I would like to close this PR.