This repository has been archived by the owner on Jun 23, 2018. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
4e469dd
commit 75081e9
Showing
5 changed files
with
181 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
.. _idl_language: | ||
|
||
IDL Language | ||
============ | ||
|
||
An IDL block describes the interface of an external module | ||
(which could be written in Python, for instance). | ||
It is embedded in :ref:`nstack.yaml<creating_structure_yaml>` | ||
|
||
The IDL block consists of two sections: | ||
the first section declares types, | ||
the second section declares functions. | ||
|
||
Types are declared using the ``type`` keyword: :: | ||
|
||
type PlantInfo = { petalLength : Double | ||
, petalWidth : Double | ||
, sepalLength : Double | ||
, sepalWidth : Double | ||
} | ||
type PlantSpecies = Text | ||
|
||
The left-hand side of a type declaration is the new type name; | ||
the right-hand side must be an :ref:`existing type<supported_types>`. | ||
Type declarations don't need to be delimited in any way. | ||
|
||
Function declarations have form ``name : Type`` and | ||
must be delimited by commas, for instance :: | ||
|
||
gotham : MovieRecordImage -> MovieRecordImage, | ||
kelvin : MovieRecordImage -> MovieRecordImage, | ||
lomo : MovieRecordImage -> MovieRecordImage | ||
|
||
Thus, the full IDL block might look like this: :: | ||
|
||
type PlantInfo = { petalLength : Double | ||
, petalWidth : Double | ||
, sepalLength : Double | ||
, sepalWidth : Double | ||
} | ||
type PlantSpecies = Text | ||
|
||
gotham : MovieRecordImage -> MovieRecordImage, | ||
kelvin : MovieRecordImage -> MovieRecordImage, | ||
lomo : MovieRecordImage -> MovieRecordImage |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,6 +8,8 @@ Reference | |
|
||
nstack_toolkit | ||
module_structure | ||
supported_types | ||
workflow_language | ||
idl_language | ||
supported_integrations | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
.. _supported_types: | ||
|
||
Supported Types | ||
=============== | ||
|
||
Primitive types | ||
--------------- | ||
|
||
NStack supports the following primitive types: | ||
|
||
* ``Integer`` | ||
* ``Double`` | ||
* ``Boolean`` | ||
* ``Text`` | ||
|
||
.. ByteArray | ||
More complex types can be built out of primitive ones: | ||
|
||
* Optional types: ``type1 optional`` | ||
* Tuples: ``(type1, type2, ...)``. A tuple must have at least two fields. | ||
* Structs: ``{ name1: type1, name2: type2, ... }`` | ||
* Arrays: ``[type1]`` | ||
* `Sums <https://en.wikipedia.org/wiki/Algebraic_data_type>`_: ``Name1 type1a ... | Name2 type2a ... | ...`` | ||
|
||
A user can define their own type in the :ref:`idl_language`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
.. _workflow_language: | ||
|
||
Workflow Language | ||
================= | ||
|
||
A module consists of a module header, import statements, and definitions, | ||
for instance: :: | ||
|
||
module ModuleB:0.1.0 { | ||
import ModuleA:0.1.0 as A; | ||
def x = A.y | A.z; | ||
} | ||
|
||
An import statement includes the module to be imported (``MyModule:0.1.0``) | ||
and its alias (``A``). | ||
The alias is used to qualify functions imported from the module, | ||
e.g. ``A.y``. | ||
|
||
Definitions bind function names (``x``) to expressions (``A.y | A.z``). | ||
Any function ``x`` defined in a module ``ModuleB`` can be used in | ||
any other module: :: | ||
|
||
import ModuleB:0.1.0 as B; | ||
def z = filter B.x; | ||
|
||
If a name is not prefixed by a module alias, it refers to a function defined in | ||
the current module. | ||
|
||
Expressions combine already defined functions through the following operations: | ||
|
||
* Pipe: ``A.y | A.z`` | ||
|
||
Every value produced by ``A.y`` is passed to ``A.z``. | ||
|
||
The output type of ``A.y`` must match the input type of ``A.z``. | ||
|
||
* Concat: ``concat A.y`` or ``A.y*`` | ||
|
||
``A.y`` must be a function that produces lists of values, | ||
in which case ``concat A.y`` is a function that "unpacks" the lists | ||
and yields the same values one by one. | ||
|
||
* Filter: ``filter A.y`` or ``A.y?`` | ||
|
||
``A.y`` must be a function that produces "optional" (potentially missing) values, | ||
in which case ``filter A.y`` is a function that filters out missing values. | ||
|
||
* Type application. Some functions (notably, most sources and sinks) can be specialized | ||
to multiple input or output types. | ||
This is done with type application: ``Sources.http<Text>`` specializes | ||
``Sources.http`` to the type ``Text``. | ||
|
||
* Parameter application: ``A.y { one = "...", two = "..." }``. | ||
|
||
Parameters are analogous to UNIX environment variables in the following ways: | ||
|
||
1. Parameters are inherited. E.g. in :: | ||
|
||
y = x; | ||
z = y { foo = "bar" }; | ||
|
||
both functions ``x`` and ``y`` will have access to ``foo`` when ``z`` is | ||
called. | ||
|
||
2. Parameters can be overridden. E.g. in :: | ||
|
||
y = x { foo = "baz" }; | ||
z = y { foo = "bar" }; | ||
|
||
``y`` overrides the value of ``foo`` that is passed to ``x``. | ||
Therefore, ``x`` will see the value of ``foo`` as ``baz``, not ``bar``. | ||
|
||
Parameters are used to configure sources and sinks — | ||
for instance, to specify how to connect to a PostgreSQL database. | ||
|
||
Parameters can also be used to configure user-defined modules. | ||
Inside a Python nstack method, the value of parameter ``foo`` can be accessed as | ||
``self.args["foo"]``. | ||
|
||
The workflow language supports line and block comments. | ||
Line comments start with ``//`` and extend until the end of line. | ||
Block comments are enclosed in ``/*`` and ``*/`` and cannot be nested. | ||
|
||
EBNF grammar | ||
------------ | ||
|
||
The syntax is defined in EBNF (ISO/IEC 14977) in terms of tokens. | ||
|
||
.. highlight:: ebnf | ||
|
||
:: | ||
|
||
module = 'module', module name, '{', {import}, {definition}, '}'; | ||
import = 'import', module name, 'as', module alias, ';'; | ||
definition = 'def', name, '=', expression, ';'; | ||
expression = expression1, {'|', expression1}; | ||
expression1 = application, '*' | ||
| application, '?' | ||
| 'concat', application | ||
| 'filter', application | ||
; | ||
application = term [arguments]; | ||
arguments = '{', argument binding, {',', argument binding}, '}'; | ||
argument binding = name, '=', literal; | ||
term = '(', expression, ')' | ||
| qualified name ['<', type, '>'] | ||
; |