Skip to content

Latest commit

 

History

History
35 lines (16 loc) · 1.79 KB

Organisation.md

File metadata and controls

35 lines (16 loc) · 1.79 KB

Data Generation and Analysis: Pipeline Design

Twinkles simulations must be reproducible - which means laying out a transparent, openly-accessible software pipeline which can be run by anyone. We can achieve this by developing several installations in parallel, for example at SLAC, NERSC and Fermilab, and maintaining various test examples that must always just run.

The flow chart below shows the initial pipeline design, color coded to show what remains to be built in red.

The initial (Feb 2015) Twinkles flow chart

This confluence-hosted pipeline design chart is evolving with time here. We'll update this page periodically with newer versions.

Pipelined Machines

Notes on the various pieces of this pipeline are below, in approximate order of their operation.

The Pointing Selector

The Ditherer

The Sprinkler

The Light Curve Sampler

The Twinkles CatSim

This machine generates the input that PhoSim needs, and can be based on the sims_catutils script, generatePhosimInput.py. At present we have a modified version of this in the twinkles module, here.

The Analyzer

Pipeline Infrastructure: the Workflow Engines

We have several general-purpose "Workflow Engines" at our disposal. We also need a very simple script that performs the same function, so that anyone can run the entire pipeline. See issue #25 for discussion of this.