Workflow system designed for daily data analysis
Python Jupyter Notebook JavaScript Smarty Vim script Shell Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.

README.md

PyPI version DOI Join the chat at https://gitter.im/vatlab/SoS Build Status Build Status Codacy Badge Coverage Status

Script of Scripts (SoS) is a file format for the inclusion of scripts in multiple languages into a single executable script, an interactive environment and notebook tool for working with different scripts, and a workflow engine for the execution of workflows consisting with scripts in different languages. It is designed for data scienticists and bioinformatics who routinely work with scripts in different languages such as R, Python, Perl, and bash.

Please refer to the SoS homepage for more information.

We welcome and value community contributions: please post issues to provide us feedback or get our support; please send pull requests if you have helped fixing bugs or making improvements to the source code.

Installing SoS

With Python 3.6 and above intalled (anaconda python is recommended), you can install the SoS Workflow System and its extensions with command

% pip install sos

You can install the latest git version of SoS with commands

% git clone https://github.com/vatlab/SoS.git
% cd SoS
% pip install . -U

although the development version can be less stable than the released version.

You can install SoS Notebook and all language moduels, and register the sos kernel with Jupyter using the following commands

% pip install sos-notebook
% python -m sos_notebook.install

although different modules might be needed for SoS Notebook to work with other kernels. Please refer to the installation page of sos website for details.

Change Log of SoS and SoS Notebook

SoS 0.9.16.0, SoS Notebook 0.9.16.0

  • sos#991: Use a new task file format to replace multiple files for each task. This change is not backward compatible so please upgrade only after you completed and removed all existing tasks.

SoS 0.9.15.1

  • sos-notebook#89: Added templates to highlight source code using codemirror, with optional auto-generated table of contents.

SoS 0.9.14.10

  • sos#983: Allow depending on an entire process-oriented workflow using a sos_step() target that matches multiple steps.

SoS 0.9.14.3:

  • sos#975: Add option -p to generate a summary report after the completion of workflow.
  • sos#976: Much improved workflow help message (sos run script -h).

SoS Notebook 0.9.14.4:

  • sos-notebook#79: Allow auto-completion and inspection in subkernel cells.

SoS Notebook 0.9.14.1

  • sos-notebook#74: Add a %revisions magic to display revision history of the current document.

SoS 0.9.14.1

  • sos#925: Output summary of executed and ignored step, substeps, and tasks after the execution of workflows.

SoS Notebook 0.9.13.4

SoS Notebook 0.9.12.12

SoS 0.9.12.11

  • sos#922: Use user-id for docker execution (-u)
  • sos#926: Add function zap() to SoS path classes path, paths, file_target, and sos_targets

SoS Notebook 0.9.12.11

SoS 0.9.12.9

  • sos#914: Allow option active of actions and tasks to accept conditions.
  • sos#915: Automatically expand user (~) for SoS path types path, paths and file_targets.
  • sos#916: Use hashlib instead of faster xxhash under windows

SoS Notebook 0.9.12.9

SoS 0.9.12.3

  • sos#859: Introduce automatic auxiliary steps to simplify the use of makefile steps.

SoS 0.9.11.3

  • sos#879: Add action options stdout and stderr to redict output from script-executing actions.
  • sos-notebook#42: Add option --append to magic %capture .

SoS 0.9.11.2

  • sos-notebook#39: Separation installation and deployment and use command python -m sos_notebook.install to install sos kernel to Jupyter.

SoS 0.9.10.19

  • sos#874: Add input option concurrent=True to allow parallel execution of input groups.
  • sos#874: Optimize task submission of task engines to reduce status checking

SoS Notebook 0.9.10.17