Feature/import api

[bplatz]

This adds a new 'import' API to fluree/server which accepts TURTLE (.ttl) and JSON-LD.

The main feature of this is that raw RDF data can be imported without converting it to JSON-LD... and even raw JSON-LD can be imported without having to reformat it by wrapping your JSON-LD file in {"insert": }.

The RDF data is just sent in the HTTP body, regardless of format.

API endpoint:

<server>/fluree/import

To use the new API with TURTLE, the HTTP Headers must include:

{"Content-Type":  "text/turtle",
 "Fluree-Ledger": "<fluree-ledger-name-here>"}

To use the API with JSON-LD, the HTTP Headers must include:

{"Content-Type":  "application/json",
 "Fluree-Ledger": "<fluree-ledger-name-here>"}

For now, the ledger must already exist, and you are just 'importing' into an existing ledger. Having the ability to create a new ledger from an import is likely desirable, and it can be added fairly simply.

[dpetran]

Is it possible to sign a turtle import? I think you'd use a JWT, but cred/verify will not handle turtle correctly.

[bplatz]

I think so, but it wasn't the priority and I think there is a good chance this middleware dissapears, but not 100% on that.

The db api now has *-credential APIs which ideally we'd mostly leverage instead of fluree/server doing its own thing.

But for now it doesn't have signatures for TTL imports, although I should make a ticket for it as I think it should at some point.

[dpetran]

Just checking to make sure I understand, these schemas correspond to:

• turtle string
• json-ld map
• vector of json-ld maps

Is that right?

[bplatz]

Yes, correct.

In this case we are doing a validation that doesn't offer much value, but it is how things are set up. I don't know how to bypass it, but would rather we did.

For example, here we have to look at every element of a json-ld vector and check map?. But in a moment we'll be running json-ld/expand which does the same check (and more). Not a big deal for a small tx, but meaningful for a very large one.

Or, with ttl, it has to perform a (count ttl) on potentially a large file to make sure it is > 1, but then we turn around and run it through a ttl import which would throw anyhow if malformed.

In some cases these validations seem to offer a lot of value, but in this case it just slows things down from what I can tell.

[dpetran]

I wonder if we should name this add instead of import, sort of mirroring the SPARQL Update API. It leaves room for us to follow those conventions for other graph-level operations. We already have create, we could add copy, move, and drop as well, operations that I think could be especially useful for doing git-like branching workflows, or moving graphs between different types of storage.