RDF Pipeline Team Processes & Documentation
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
images
Getting-Started.md
GitCheatSheet.md
Installation.md
README.md
Troubleshooting.md

README.md

Documentation for RDF Pipeline

The team-docs repository contains markdown files that describe installation, use, and development of modules used in the rdf-pipeline project.

Installation

[Tested 23-Oct-2017 on Ubuntu 16.04 by @dbooth-boston]

This is the quickstart guide. The long-form instructions are here. See also Troubleshooting.

  1. Install Prerequisites

    • (optional) Linux VM - Centos/Ubuntu
    • (optional) nvm - node version manager
    • java jdk 1.7
    • node.js LTS "argon" (version 4.8 as of this writing)
  2. Clone needed repositories, checking out 'develop' branch

git clone https://github.com/rdf-pipeline/noflo-nodejs.git -b develop
git clone https://github.com/rdf-pipeline/noflo-ui.git -b develop
git clone https://github.com/rdf-pipeline/noflo-rdf-components.git -b develop
  1. Install noflo-nodejs:
cd noflo-nodejs
npm install
npm link
cd ..
  1. Install noflo-ui:
cd noflo-ui
npm install
npm install --save-dev coffee-script
npm update
grunt build
npm link
cd ..
  1. Install noflo-rdf-components:
cd noflo-rdf-components
npm install
npm link noflo-nodejs
cd ..
  1. run an empty NoFlo graph for testing
# enter the directory for the repository containing all our custom components
cd noflo-rdf-components
# create an empty JSON graph
echo "{}" > output.json
# Start the NoFlo Engine
noflo-nodejs --register false --ide http://localhost:8080/ --secret secret --graph output.json --save-graph output.json &
#Start the NoFlo UI
noflo-ui --secret secret --host localhost --port 8080 --websocket ws://localhost:3569 &

Testing Your Installation

[Tested 23-Oct-2017 on Ubuntu 16.04 by @dbooth-boston]

The noflo-nodejs script runs a local NoFlo runtime, and--in our fork--serves up a web page to inspect the status of active nodes at localhost:3569/node/.

The noflo-ui script runs the frontend, which connects to the runtime. Browsing to localhost:3569 should automatically forward to the UI, which is served from localhost:8080. This is the interface where you will create your pipeline graph. Currently it should be empty, and look something like this:

noflo ui without a graph

By clicking the name of the graph ( where it says default/main in the top left ), you should see a list of components. You can filter this list by typing a few characters of the component you wish to find.

Some basic components included with noflo are prefixed with CORE, such as CORE/CALLBACK.

Many of our components start with the prefix RDF-COMPONENTS, such as RDF-COMPONENTS/ADD-METADATA.

If you see a long list of components that includes both of these examples, then your installation is likely successful. Let's try to create and run a graph to check!

Getting Started

[Tested 23-Oct-2017 on Ubuntu 16.04 by @dbooth-boston]

Open localhost:3569 in your browser, click the default/main label and select the CORE/Output component to create a node in the graph. Click that node, and enter a string literal, such as "Hello, JSON!" in the input field labeled in. Pressing Escape, Tab, or Enter should hide the inspector, leaving you with a canvas that looks like this:

noflo ui without a graph

When you click the blue triangle in the top right, it will run the flow, and print that text out to the same terminal where you ran noflo-nodejs.

Full Guide

Development

Developers will need to install the following packages in addition to those needed to run the RDF Pipeline Framework:

sudo apt install mocha
npm install underscore
npm install chai
npm install sinon
npm install rdfstore
npm install jsonld

Our git protocol for developers:

  • Developer creates a branch or fork
  • Developer issues pull request (PR) with changes
  • Other developers review, and post comments or "LGTM" - "Looks good to me"
  • When LGTM consensus is reached, original Developer merges PR

Best Practices

  • If you come across a "BactrianCamelCase" variable name, it may refer to an imported module, a local class, or an enum
  • When in doubt, we follow the Google JavaScript Style Guide

Glossary

  • RDF: Resource Description Framework - a DL-logic-based assertion language for expressing facts/knowledge
  • JSON-LD: JSON with Linked Data - a JSON-based serialization of RDF data
  • VNI: Virtual Node Instance. A single NoFlo node can represent multiple VNIs having the same functionality but processing different data.