quangis-web

The web interface that brings all the different components of the QuAnGIS project together into a single pipeline.

1. Overview
   • Workflow generator
   • Algebra abstractor
   • Query formulator
   • Query translator
   • Query executor
   • Data reifier
2. Usage
   • Setting up the environment
   • Setting up JavaScript libraries
   • Loading data

Overview

Workflow generator

The workflow repository contains possible GIS workflows that might solve a user's task. At the moment, it contains only example workflows. However, eventually, the workflow-synthesis module will be used to pre-generate workflows. This module interfaces with the Automated Pipeline Explorer via apey to discover sensible workflows. It does this by matching core concept data types of the GIS tools' inputs and outputs.

(The workflow specifier would find meaningful combinations of input/output CCD types for the transformation algebra query at hand, suggesting that workflows are also generated on-the-fly.)

Algebra abstractor

Each workflow in the workflow repository is enriched with a transformation graph. This is a directed acyclic graph with core concepts of geographical information encoded as CCT types at the nodes. The edges represent transformations between these concepts.

These graphs are constructed automatically. We have manually annotated the tools with CCT expressions that capture the underlying conceptual transformation. A workflow connects individual tool applications, each of them now associated with a CCT expression that is parsed into a subgraph by the transformation-algebra library. The library then stitches the subgraphs together, using type inference to find the most specific type at every node.

Query formulator

A web interface presents users with a constrained natural language, using blockly. The interface passes the user's question on to the rest of the pipeline. At the end of the pipeline, it presents a set of workflows to the user as possible solutions to their task. The implementation is in this repository, quangis-web.

Query translator

The user's natural language question is parsed into a tree and then further into a transformation query, which is essentially a transformation graph that represents only parts of the workflow. Leaf nodes may have keywords, to be matched with possible data sources. This is implemented in the geo-question-parser module.

Query executor

The transformation query is matched against the transformation graphs in the workflow repository. To this end, the transformation query is converted to a SPARQL query via the transformation-algebra library and sent to a triple store such as MarkLogic or the open-source Apache Jena Fuseki. This yields a set of workflows that may solve the user's task.

Data reifier

To be implemented. Finds suitable input data sources to concretize the queried workflow. The data repository contains sources that are manually or automatically annotated with text descriptions and CCD/CCT types.

Usage

Setting up the environment

There are myriad ways to manage Python packages and environments. We have chosen Miniconda; also see instructions for Linux. After installing it, create and activate the quangis environment:

conda env create --file environment.yml
conda activate quangis

After the environment is set up, make sure that you have downloaded relevant data files:

python -m spacy download en_core_web_sm
python -m nltk.downloader averaged_perceptron_tagger omw-1.4

Setting up JavaScript libraries

We use the blockly, cytoscape and klay JavaScript libraries. For now, these are just bundled as static JS files; we will need a maintainable front-end workflow in the future. (Also, klay is deprecated.)

npm install
cp  node_modules/blockly/blockly.min.js \
    node_modules/cytoscape/dist/cytoscape.min.js \
    node_modules/cytoscape-klay/cytoscape-klay.js \
    node_modules/klay/klay.js \
    blocklyUI/static/blocklyUI/js/

Loading data

The cct repository contains example workflows. These can be augmented with their transformation graphs using the transformation-algebra command-line tool. For example, to produce a TriG file containing all graphs, you could use:

python -m transformation_algebra graph -L cct -T tools/tools.ttl \
    workflows/*.ttl -o transformations.trig

To additionally upload these graphs to a graph store, additionally pass arguments such as these:

-b marklogic -u "<user>:<pass>" -s "https://<host>:<port>"

Running the server

During debugging, we can use the built-in Django test server. However, do set environment variables with the appropriate secrets:

DJANGO_KEY=… TDB_USER=… TDB_PASS=… python manage.py runserver 3000

Or, on Windows:

set DJANGO_KEY=… & TDB_USER=… & TDB_PASS=… & python manage.py runserver 3000