README.md

Deprecated

This service has been archived. The current version of the delta service, the delta-notifier can be found here.

Delta Service

The delta service can essentially be put in front of any SPARQL endpoint and will proxy calls to it. Then interested parties can subscribe to different forms of notifications about changes in the SPARQL endpoint.

Using the Delta Service

Add the config files

The standard setup for the configuration files is in the ./config/Delta Service directory. You can copy the config.properties and the contributors.json files from this repository to that location. Add the desired subscribers to the subscribers file and that is it!

Changing the docker-compose

First at the delta entry as follows (adjust if necessary), this assumes that you want to publish the Delta Service as a SPARQL endpoint on port 8890 so your micro-services can hook into changes that are introduced from outside of the system:

  delta:
    image: semtech/mu-delta-service:beta-0.7
	ports:
	 - "8890:8890"
    volumes:
      - ./config/delta-service:/config
    environment:
      CONFIGFILE: "/config/config.properties"
      SUBSCRIBERSFILE: "/config/subscribers.json"
    links:
      - db:db

And then for all micro-services for which you want to know the delta's just replace the db:database link with delta:database.

To be done

• improve the internal triple model (it's a mess now)
• sort out the update notification format
• make sure the credentials are used (config.properties)
• updates should be sendable in the bodies if too big or in the settings

Configuring

There are 2 configuratble parts for each a file has to be provided.

The first one is the location of the config.properties file for this microservice. This properties file states where the query and update endpoint are. There is also a flag to force the system to send updates in the body even when the query URL would not become too big.

There is also a subscribers.json file in which the 'static' subscribers can be set to be automatically loaded. The location of this JSON file needs to be set with an environment variable.

Environment variables for settings

• config.properties CONFIGFILE
• subscribers.json SUBSCRIBERSFILE

In subscribers.json you enumerate for both potentials and effectives subscribers by specifying the list of URL's to update to. So the delta service will simply post to those URLs when required. For example:

subscribers.json

{
  "potentials":[
    "http://push-service/update",
    "http://initdaemon/process_delta"
  ],
  "effectives":[
    "http://initdaemon/process_delta"
  ]
}

Notifications

Subscribers can be set with the subscribers.json file (see above) or through the subscription endpoint as follows. To subscribe for notifications you can send a http request to the "/registerForPotentialDifferences" or the "/registerForEffectiveDifferences" route. The target of the callback should be identified in the body as follows:

{
"callback":"http://localhost:3000/print"
}

The callbacks will be notified with a HTTP request that has a body like this:

[
  {
    "graph": "http://graph1",
    "delta": {
      "inserts": [
        {
          "o": "object",
          "p": "predicate",
          "s": "subject"
        }
      ],
      "deletes": []
    }
  }
]

Different types of notifications

Effective differences

Those are triples that are really added to the datastore, these will always be a subset of the 'potential differences' for the same query. To register for it you can send a request to the "/registerForEffectiveDifferences" route.

Potential differences

These are the inserts and delete generated by a certain query. However if a triple would be inserted that is already in the SPARQL store or a triple would be deleted that is not in the SPARQL store then these changes do not effectively change the SPARQL endpoint. For instance the first time the following query would be executed:

with <http://graph1>
insert
{
<http://uri1> <http://predicate1> <http://object1>
}

It is not unlikely that the effective and potential delta for this operation would be:

{
  "graph": "http://graph1",
  "delta": {
    "inserts": [
      {
        "s": "http://uri1",
        "p": "http://predicate1",
        "o": "http://object1"
      }
    ],
    "deletes": [

    ]
  }
}

The second time this query is executed however (unless someone deletes the above triple of course) the potential differences are still:

{
  "graph": "http://graph1",
  "delta": {
    "inserts": [
      {
        "s": "http://uri1",
        "p": "http://predicate1",
        "o": "http://object1"
      }
    ],
    "deletes": [

    ]
  }
}

but the effective differences would be:

{
  "graph": "http://graph1",
  "delta": {
    "inserts": [],
    "deletes": []
  }
}

To register you can send a HTTP request to the "/registerForPotentialDifferences" route.