create edges to store xid while importing data

[peternavr]

I'm importing data files from dbpedia and I guess I'm expecting dgraph to create edges for these things and they're not.

file: category_labels_en.ttl.gz

<http://dbpedia.org/resource/Category:German_money_launderers> <http://www.w3.org/2000/01/rdf-schema#label> "German money launderers"@en .

it's not a reverse edge,  you can probably say create edges to store xid while importing data

[10:31] 
<http://dbpedia.org/resource/Category:German_money_launderers> is an external id for us

new messages
[10:32] 
even if you create a reverse edge, we throw the external id and convert it to uid and store it

[manishrjain]

Hey @clanstyles,

We're planning to remove XID support altogether. They are not native to Dgraph and add significant complexity and slowness to the code base.

Instead, users would need to create Dgraph nodes, and then they can assign these XIDs in an edge, with string type, just like other data. That'd be closer to how Dgraph works, and avoid this half-support for XIDs.

[peternavr]

This seems like a pretty large change. When do you plan on doing this?

How do you see this working when importing data? A curious approach might be to list all the different datasets and during import let dgraph determine how many edges/properties exists to join them all under one node/XID?

[manishrjain]

We had only partial support for XID. We tried to implement full support but then realized that piece doesn't directly fit into Dgraph and slows it down considerably. We use RDF format, but Dgraph isn't really an RDF store.

We're working on the change right now. What would need to happen when loading data is to create blank nodes, and then assign them the xid via an edge. We'll bake support for dealing with XIDs into our client, i.e. outside of Dgraph, not within.

[ashwin95r]

@MichaelJCompton Just for completeness, can you add the doc explaining the new way to load RDF NQuads into Dgraph.

[MichaelJCompton]

Hi @clanstyles, the master branch now has support for loading RDF URIs / XID.

dgraphloader has a -x option to auto create _xid_ edges.

There's docs at
https://docs.dgraph.io/master/query-language/#external-ids
and
https://docs.dgraph.io/master/deploy/#bulk-data-loading