Skip to content
This repository has been archived by the owner on Jun 23, 2018. It is now read-only.

Commit

Permalink
Update screencast and add software lifecycle section (#40)
Browse files Browse the repository at this point in the history
* Add a basic screencast

* Add a basic screencast to docs

* Update DSL snippet

* Update example

* Move screencast

* Add software lifecycle

* Update example image

* Reviewer feedback
  • Loading branch information
mands authored and lanthias committed Apr 12, 2017
1 parent 3340939 commit 2eb7e3e
Show file tree
Hide file tree
Showing 5 changed files with 158 additions and 135 deletions.
9 changes: 5 additions & 4 deletions concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,19 @@ Example

.. image:: resources/readme-flowchart-example.svg

We can express this within the NStack scripting language locally as follows (it can help to think of it akin to Bash-style piping for microservices),

.. code:: fsharp
module Demo:0.1.0 {
import NStack.Transformers:0.1.4 as T;
import Acme.Classifiers:0.3.0 as C;
import NStack.Transformers:0.1.4 as T
import Acme.Classifiers:0.3.0 as C
// our analytics workflow
def workflow = Sources.Postgresql<(Int, Int, Text, CustomerRecord)>
def workflow = Sources.Postgresql<(Text, Int)>
| T.transform { strength = 5 }
| C.classify { model = "RandomForest" }
| Sinks.S3blob<Text>;
| Sinks.S3blob<Text>
}
Expand Down
12 changes: 12 additions & 0 deletions quick_start/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,24 @@ Quick Tutorial

In this section, we're going to see how to build up a simple NStack module, deploy it to the cloud, and use it in a workflow by connecting it to a `source` and a `sink`.


Intro Screencast
^^^^^^^^^^^^^^^^

The following screencast accompanies this tutorial, demonstrating building a module and connecting it within a workflow,

.. raw:: html

<script type="text/javascript" src="https://asciinema.org/a/112733.js" id="asciicast-112733" async></script>


By the end of the tutorial, you will learn how to publish your code to NStack and connect it to event and data sources. Enjoy!

.. note:: To learn more about modules, sources, and sinks, read :ref:`Concepts<concepts>`

Make sure you have :doc:`installed NStack </installation>` and let's get going. These instructions are for the Linux and macOS versions of the NStack CLI, so adjust accordingly if you are on Windows.


.. toctree::

module
Expand Down
262 changes: 131 additions & 131 deletions resources/readme-flowchart-example.svg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified resources/readme-flowchart.odg
Binary file not shown.
10 changes: 10 additions & 0 deletions usecases.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,13 @@ Data Integration
Transform disparate and disconnected data-sources -- such as 3rd-party APIs, legacy infrastructure, or databases into streams of typed, structured records -- which can be composed together. For instance, you could set up a workflow in the cloud which pipes the Twitter Ads API into your data-lake (and even do some modelling in Python in-transit) in under 5 minutes.


Software Lifecycle
******************

NStack provides best practices from software engineering and end-to-end software life-cycle management to the data science process, including,

* **sharing and reuse** - build and deploy individual model modules to your cloud that can reused, imported, and utilised by other members, either within your team or by third-parties
* **reproducibility** - build models with guaranteed versions of system and language dependencies, that can then be shared with confidence that your code is bit-for-bit identical everywhere
* **versioning** - modules a have a immutable, globally unique, version - so when importing to use in a workflow, you can be sure the API, code, and artifacts are just as you intended, allowing you to upgrade to newer versions when ready
* **runtime isolation** - all modules in your workflow run using our container technology, meaning they are fully isolated from each other, can scale up as needed, and won't interfere with each others data

0 comments on commit 2eb7e3e

Please sign in to comment.