Skip to content
This repository has been archived by the owner on Jun 23, 2018. It is now read-only.

Commit

Permalink
Update for immutable modules (#20)
Browse files Browse the repository at this point in the history
  • Loading branch information
mands authored and lanthias committed Mar 16, 2017
1 parent 417b715 commit f263707
Show file tree
Hide file tree
Showing 5 changed files with 49 additions and 31 deletions.
27 changes: 19 additions & 8 deletions quick_start/module.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ You should see the following output confirming that this operation was successfu

.. code:: bash
~> mkdir demo
~> cd demo
~/demo> nstack init python
python module 'demo' successfully initialised at ~/demo
~> mkdir Demo
~> cd Demo
~/Demo> nstack init python
python module 'Demo:0.0.1' successfully initialised at ~/Demo
A successful ``init`` will have created some files in the directory.

Expand Down Expand Up @@ -59,13 +59,13 @@ In ``service.py``, there is a ``Service`` class. This is where we would write th
.. code:: yaml
# Service name (a combination of lower case letters, numbers, and dashes)
name: demo
name: Demo:0.0.1
# The language stack to use
stack: python
# Parent Image
parent: com.nstack.python:24.0
parent: NStack.Python:0.24.0
api: |
interface Default {
Expand Down Expand Up @@ -99,8 +99,19 @@ We can check that our ``numChars`` method is live by running the suggested ``nst

.. code:: bash
~/demo> nstack list methods
demo.numChars : Text -> Integer
~/Demo> nstack list methods
Demo.numChars : Text -> Integer
Now that our ``numChars`` method is live on NStack, we can productionise it by connecting it to input and output data. We do this by attaching it to a event *source* and an event *sink* using NStack's Workflow Language.

Advanced: Framework Modules
---------------------------

You may want to create a common parent module that has lots of complex dependencies already installed, either to save time or for standardisation. NStack supports this with _Framework Modules_. Simply create a new module similar to above, `nstack init framework [parent]`, and modify the resulting `nstack.yaml` as needed.

You can then build this module using `nstack build`, and refer to it within your future services within the `parent` field of their `nstack.yaml` config file.





23 changes: 13 additions & 10 deletions quick_start/workflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Building your Workflow
=========================

In the previous tutorial, we built and published a Python module using NStack.
In the previous tutorial, we built and published a Python module `Demo` using NStack.
This module had a single method on it, ``numChars``, which counted the number of characters in some text. Although it has been published, we cannot talk to it until we connect it to a `source` and a `sink`.
In this tutorial, we're going to do this, and we'll be using an HTTP endpoint as a `source`, and NStack's built-in log as a `sink`. When we're finished, we will be able to send some text to an HTTP endpoint, and see the length of the text in our log.

Expand All @@ -14,7 +14,7 @@ Let's first refresh ourselves on what the input and output types of our method w
.. code:: bash
> nstack list methods
demo.numChars : Text -> Integer
Demo.numChars : Text -> Integer
This means that our method can be connected to any source which generates ``Text``, and can write to any sink which can take an ``Integer`` as input.

Expand All @@ -28,8 +28,8 @@ which is a simple sink for seeing the output from your method.
In full, our workflow is going to look like this:

.. code:: bash
sources.http : Text { http_path = "/demo" } | demo.numChars | sinks.log : Integer
import Demo:0.0.1 as Demo;
Sources.http : Text { http_path = "/demo" } | Demo.numChars | Sinks.log : Integer
NStack uses the ``|`` operator to connect statements together, just like in a shell such as ``bash``. We use it to connect together the parts to form our workflow.
Expand All @@ -39,19 +39,22 @@ Let's break these parts to see what we're doing:
=============================================== ===========
Part Description
=============================================== ===========
``sources.http : Text { http_path = "/demo" }`` Use ``http`` as a source, which creates an endpoint on ``/demo``. The ``Text`` statement means it can only accept and pass on Text.
``Sources.http : Text { http_path = "/demo" }`` Use ``http`` as a source, which creates an endpoint on ``/demo``. The ``Text`` statement means it can only accept and pass on Text.
``demo.numChars`` The name of the method which we built.
``Demo.numChars`` The name of the method which we built.
``sinks.log : Integer`` Use NStack's log as a sink. The ``Integer`` statement means it can only accept Integers.
``Sinks.log : Integer`` Use NStack's log as a sink. The ``Integer`` statement means it can only accept Integers.
=============================================== ===========
To start this workflow with NStack, we use NStack's ``start`` command:
To start this workflow with NStack, we use NStack's ``start`` command, this opens a command prompt where you can type a snippet and press Ctrl+D to submit it (you can also redirect a file):
.. code:: bash
> nstack start 'sources.http : Text { http_path = "/demo" } | demo.numChars | sinks.log : Integer'
Started sources.http : Text { http_path = "/demo" } | demo.numChars | sinks.log : Integer as process 1
> nstack start
import Demo:0.0.1 as Demo;
'Sources.http : Text { http_path = "/demo" } | Demo.numChars | Sinks.log : Integer'
[Ctrl + D]
Started Sources.http : Text { http_path = "/demo" } | Demo.numChars | Sinks.log : Integer as process 1
We now have a live HTTP endpoint on ``localhost:8080/demo``, running as process ``1`` on NStack. The HTTP endpoint is configured to accept JSON-encoded values. We defined it to use an input schema of ``Text``, so we will be able to send it any JSON ``string``. In our JSON, we put ``params`` as the key, and our input as the value:
Expand Down
6 changes: 3 additions & 3 deletions reference/module_structure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ Sample ``nstack.yaml`` file:

.. code-block:: yaml
# Module name (a combination of lower case letters, numbers, and dashes)
name: my-module
# Module name
name: MyModule
# The language stack to use
stack: python
# Parent Image
parent: com.nstack.python:24.0
parent: NStack.Python:0.24.0
api: |
interface Default {
Expand Down
8 changes: 6 additions & 2 deletions reference/nstack_toolkit.rst
Original file line number Diff line number Diff line change
Expand Up @@ -95,12 +95,16 @@ Option Description
Used to start a workflow as a process. Workflows can either be provided as an argument such as:

.. code:: bash
$ nstack start "mySource | myModule.myMethod | sink(mySink)"
$ nstack start
> import MySource:0.0.1 as MySource;
> import MySink:0.0.1 as MySink;
> import MyModule:0.0.1 as MyModule;
> MySource.src | MyModule.myMethod | MySink.snk
Or, if you have built a workflow as a module, you can start it with:

.. code:: bash
$ nstack start myWorkflow
$ nstack start "Import MyWorkflow:0.0.1 as W; W.myWorkflow"
``ps``
Expand Down
16 changes: 8 additions & 8 deletions reference/supported_integrations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
Supported Integrations
======================

NStack is built to integrate with existing infrastructure, event, and data-sources. Typically, this is by using them as *sources* and *sinks* in the NStack Workflow Language.
NStack is built to integrate with existing infrastructure, event, and data-Sources. Typically, this is by using them as *sources* and *sinks* in the NStack Workflow Language.

.. seealso:: Learn more about *sources* and *sinks* in :ref:`Concepts<concepts>`

**Sources**
- Postgres ::

sources.postgres : Text {
Sources.postgres : Text {
pg_host = "localhost", pg_port = "5432",
pg_user = "user", pg_password = "123456",
pg_database = "db", pg_query = "SELECT * FROM tbl;" }
Expand All @@ -20,11 +20,11 @@ NStack is built to integrate with existing infrastructure, event, and data-sourc

- HTTP ::

sources.http : Text { http_path = "/foo" }
Sources.http : Text { http_path = "/foo" }

- RabbitMQ (AMQP) ::

sources.amqp : Text {
Sources.amqp : Text {
amqp_host = "localhost", amqp_port = "5672",
amqp_vhost = "/", amqp_exchange = "ex",
amqp_key = "key"
Expand All @@ -36,7 +36,7 @@ NStack is built to integrate with existing infrastructure, event, and data-sourc
**Sinks**
- Postgres ::

sinks.postgres : Text {
Sinks.postgres : Text {
pg_host = "localhost", pg_port = "5432",
pg_user = "user", pg_password = "123456",
pg_database = "db", pg_table = "tbl" }
Expand All @@ -47,13 +47,13 @@ NStack is built to integrate with existing infrastructure, event, and data-sourc

- NStack Log ::

sinks.log : Text
Sinks.log : Text

The Log sink takes no parameters.

- RabbitMQ (AMQP) ::

sinks.amqp : Text {
Sinks.amqp : Text {
amqp_host = "localhost", amqp_port = "5672",
amqp_vhost = "/", amqp_exchange = "ex",
amqp_key = "key"
Expand All @@ -65,7 +65,7 @@ NStack is built to integrate with existing infrastructure, event, and data-sourc

- Firebase ::

sinks.firebase {
Sinks.firebase {
firebase_host = "localhost",
firebase_port = "111",
firebase_path = "..."
Expand Down

0 comments on commit f263707

Please sign in to comment.