Seamless: a cell-based interactive workflow framework

Seamless is a framework to set up reproducible workflows and computations that respond to changes in cells. Cells define the input data as well as the source code of the computations. All cells and computations can be created, edited and connected interactively.

The main application domains are data science, scientific computing, software prototyping, and interactive web services.

Workflows, computations and results are all internally represented as trees of checksums. This makes them strongly interoperable and reproducible.

Features

Seamless workflows define both data and code in a single file. Any user can re-execute a Seamless workflow (reproducing its results) and then modify it easily: by editing workflow parameters in a web interface, or by changing the source code cells, or by extending the workflow with additional steps. Modification of the workflow takes place while it is running.

Seamless-generated web interfaces are interactive and collaborative for the user. Also, the process of workflow creation is interactive for the programmer. At all times, the status of all of the workflow is being reported. In fact, there is no sharp distinction between user and programmer. Each step of the workflow (which can be in Python, bash, C/C++, or several other languages) can be mounted to a file, so that you can use Git, diff, and text editors. If you use an IDE, live debugging with code breakpoints is supported. As a programmer, you can work collaboratively, creating a live shared session where a team of developers can work on different parts of the workflow in parallel.

Source code: https://github.com/sjdv1982/seamless

Documentation: http://sjdv1982.github.io/seamless

Try out Seamless

You can try out Seamless in your browser, without any installation, thanks to the Binder project. Click on the badge below:

Quick installation

First, install Docker and miniconda.

docker pull rpbs/seamless
conda create -n seamless -c rpbs -c conda-forge seamless-cli -y
conda activate seamless

For more details, see Installation

Basic example

First, start IPython (seamless-ipython) or Jupyter (seamless-jupyter => create a new Python Notebook).

1. Import Seamless in IPython or Jupyter

from seamless.highlevel import Context
ctx = Context()

2. Set up a simple Seamless context

def add(a, b):
    return a + b

ctx.a = 10              # ctx.a => Seamless cell
ctx.b = 20              # ctx.b => Seamless cell
ctx.add = add           # ctx.add => Seamless transformer
ctx.add.a = ctx.a
ctx.add.b = ctx.b
ctx.c = ctx.add.result  # ctx.c => Seamless cell
await ctx.computation() # in a .py file, use "ctx.compute()" instead
ctx.c.value

Out[1]: <Silk: 30 >

ctx.a += 5
await ctx.computation()
ctx.c.value

Out[2]: <Silk: 35 >

3. Define schemas and validation rules

ctx.add.example.a = 0.0  # declares that add.a must be a number
ctx.add.example.b = 0.0

def validate(self):
    assert self.a < self.b

ctx.add.add_validator(validate, name="validate")

await ctx.computation()
print(ctx.add.exception)
# Validation passes => exception is None

4. Create an API for a Seamless cell

def report(self):
    value = self.unsilk
    if value is None:
        print("Sorry, there is no result")
    else:
        print("The result is: {}".format(value))

ctx.c.example.report = report
await ctx.computation()
ctx.c.value.report()

Out[3]: The result is 35

5. Mount cells to the file system

ctx.a.celltype = "plain"
ctx.a.mount("a.json")
ctx.b.celltype = "plain"
ctx.b.mount("b.json")
ctx.c.celltype = "plain"
ctx.c.mount("c.json", mode="w")
ctx.add.code.mount("code.py")
await ctx.translation()

6. Share a cell over HTTP

ctx.c.mimetype = "text"
ctx.c.share()
await ctx.translation()

>>> curl http://localhost:5813/ctx/c
35

7. Control cells from Jupyter

from ipywidgets import IntSlider, IntText

a = IntSlider(min=-10,max=30)
b = IntSlider(min=-10,max=30)
c = ctx.c.output()
ctx.a.traitlet().link(a)
ctx.b.traitlet().link(b)
display(a)
display(b)
display(c)

Out[4]

Jupyter widgets (shown only on github.io, not on github.com)

8. Save the entire state of the context

# Graph and checksums, as JSON
ctx.save_graph("basic-example.seamless")
# Checksum-to-buffer cache, as ZIP file
ctx.save_zip("basic-example.zip")

9. In a new notebook / IPython console:

from seamless.highlevel import load_graph
ctx = load_graph(
    "basic-example.seamless",
    zip="basic-example.zip"
)
await ctx.computation()
ctx.c.value

Out[1]: <Silk: 35 >

ctx.add.code.value

Out[2]: 'def add(a, b):\n return a + b'

>>> curl http://localhost:5813/ctx/c
35

Source code

https://github.com/sjdv1982/seamless

Documentation

http://sjdv1982.github.io/seamless

Name		Name	Last commit message	Last commit date
Latest commit History 2,786 Commits
bin		bin
conda		conda
docker-patch		docker-patch
docs		docs
examples		examples
js		js
mypy		mypy
seamless		seamless
sphinx/source		sphinx/source
tests		tests
wip		wip
.gitignore		.gitignore
.pylintrc		.pylintrc
CONFIG.md		CONFIG.md
LICENSE.txt		LICENSE.txt
README.md		README.md
TODO		TODO
build-seamless-minimal.sh		build-seamless-minimal.sh
installation.md		installation.md
seamless-minimal-dependencies.yaml		seamless-minimal-dependencies.yaml
setup.py		setup.py
zen.md		zen.md

License

sjdv1982/seamless

Folders and files

Latest commit

History

Repository files navigation