Handling the Graphs Lifecycle during Commits resp. Time

[Kyrosh]

Parts of this issue are fixed in different Pull Request:

• Add support of new files/graphs via Insert Data/Insert Where/Insert Delete Where #161 split of configuration into store/app configuration and graph configuration (per commit)
• Add support of new files/graphs via Insert Data/Insert Where/Insert Delete Where #161 add new files (nq + graph) or add/update files (nq and config.ttl) if untracked named graphs appear in INSERT/INSERT DATA/INSERT (DELETE) WHERE
• Add support of new files/graphs via Insert Data/Insert Where/Insert Delete Where #161 Remove unassigned.nq
• Add support for SPARQL Update graph management CREATE/DROP/COPY/MOVE/ADD

We still have a few things to discuss concerning finding named graphs:

• We currently have 2 modes to find graphes namely 'by config' or 'by .graph files'.
  Both only run during initialization and create a file to graph mapping. Additionally .graph files are only taken from the working directory, which isn't working for bare repositories. When an instance is requested the store checks all files of a commit if they exists in this mapping.
  This means we are missing on files that were moved or renamed.
• The unassigned.nq file holds any changes which couldn't be mapped but won't update the config to make new graphs available.

Any Ideas how we will approach those problems?

The .graph files and project config files are also committed to the repository. So in these cases we could build the file mapping for every instance (unassigned.nq is always included and parsed). But this won't work for a local config which isn't part of the repository.

[white-gecko]

Is this fixed by #86 ? @splattater

[splattater]

No it isn't.
It would be possible to create new graphs via Flask-API, or via SPARQL queries (! CREATE GRAPH vs. INSERT DATA { graph urn:newGraph).
Allowing graph creation leads to an update of config object, the creation of a new rdf file (+ creation of a new graphfile) and an update of the store/cache by adding a new context.
If quit is running in mode "graphfiles" it would be possible to commit this new state.
If using the config.ttl that is part of the repository we could update and commit this file too.
When running mode "localconfig" we could add and commit shadow a config file, what do you think?

[white-gecko]

This issue is also related to #30.

[white-gecko]

When this issue is fixed the test in 2372ff5 should be activated.

[Kyrosh]

@splattater I think just allowing Create Graph also creates another problem. We need a name for the nq and nq.graph file (or the entry in the config file), which could either be a generic name (not very user friendly) or extracted from the uri, where graphs could differ e.g. in the sub-domain, top-level-domain or just somewhere in the path (multiple named graphs per file?). Therefore my idea was to use the unassigned.nq and let the user sort the data afterwards (maybe add manager ui for this?). For queries and the frontend it doesn't really matter if it is one or multiple files, since all files will contribute to the instance graph.

This means a single .graph file or config entry isn't sufficient for the whole repository, only for a single commit or a series of commits. In this regard, as you mentioned, commiting and reading .graph files or reading some type of (shadow) config in each commit, depending on config mode, is needed and would also allow to move graphs inside the git repository.

Maybe we should split the config in two parts? One for real global settings which matter on startup e.g. where the repository could be found and one inside each commit which holds the graph information.
Graph information in the localconfig file wouldn't matter anymore after the first commit is done, except to maybe register graphs which are not part of the repository yet.