The Smart Data Management Platform to feed https://inps.g0v.it/ application
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
apis
doc
sdaas
.gitignore
LICENSE
README.md
_config.yml
docker-compose.yml

README.md

copernicani

g0v data-inps

A simple Smart Data Management Platform to feed the http://inps.g0v.it/ web application.

Reference implementation:

Development

The project contains the two "core" logical components:

  • sdaas (smart data as a service): the data management platform core providing a RDF store, a SPARQL endpoint, a data ingestion engine, a set of gateways to transform raw data in linked data and a build script that populates the RDF store. See files and docs in sdaas directory
  • a set of apis that query the SPARQL endpoint and produce json data with a schema suitable to be used with the BubbleGraph Component. See files and docs in apis directory

Beside these, two optional components may be needed to complete a production system:

  • LODMAP server: a linked data browser to navigate the RDF store;
  • a router that provides a single acces point to all other services with caching and ssl features.

This picture shows the components interactions:

architecture

To deploy the platform deploy a stack of some services is required:

stack

The platform is shipped with a Docker setup that makes it easy to get a containerized development environment up and running. If you do not already have Docker on your computer, it's the right time to install it.

To start all services using docker Compose type:

docker-compose up -d

This starts locally the following services:

Name Description Port
sdaas a server that manages the datastore and the ingestion engine 29311
api a server that manages the web-budget api 29312

Try http://localhost:29311/sdaas to access blazegraph workbench Try http://localhost:29312/ to test api endpoint

The first time you start the containers, Docker downloads and builds images for you. It will take some time, but don't worry this is done only once. Starting servers will then be lightning fast

To shudown the platform type:

docker-compose down

Production deployment hints

For production deployment a SSL reverse proxy server, with caching capability is strongly suggested. Here is a snippet of apache virtual host configuration for http://data.example.org/ that provides two public points:

  • **/ldp/ *** that acts as a frontend to the api container
  • /sparql that acts as a frontend to the sparql service
<VirtualHost *:80>
    ServerName data.example.org
    ServerAdmin webmaster@example.org
    ProxyRequests Off
    ProxyVia Off
    ProxyPreserveHost On
    <Proxy *>
      AddDefaultCharset off
      Order deny,allow
      Allow from all
    </Proxy>
    ProxyPass /sparql http://docker.example.org:29321/sdaas/sparql
    ProxyPass /ldp/ http://docker.example.org:29322/

You could also run a service to manage uri dereferencing and linked data content negotiation. For instance the LODVIEW application

Support

For answers you may not find in here or in the Wiki, avoid posting issues. Feel free to ask for support on the Slack general room. Make sure to mention @enrico so he is notified

Credits

Thanks to all project contributors, to the Copernicani community and to the g0v asia community for ideas and support.

The URI dereferencing platform is derived from the LODView project

License

The MIT License (MIT). Please see License File for more information.