Skip to content
No description, website, or topics provided.
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
SciStreams
ci
doc
.gitignore
.travis.yml
LICENSE
README.md
requirements.txt
run_pipeline.py
run_tests.py
setup.py

README.md

SciStreams for CMS pipeline.

Author : Kevin G. Yager

Ported to git by : Julien Lhermitte

This is a work in progress and a streaming version of SciAnalysis. For various notes, see internal page: http://gisaxs.com/CS/index.php/Databroker

INSTALL:

  • python setup.py develop

TODO:

  • current tests are not unit tests. Meant only for debugging
  • add databroker interface to all functions

File Structure:

  • core/ : The backbone of the streaming pipeline. This could likely be its own library.
    • StreamDoc.py : the StreamDoc extension to the streams library
    • scistreams.py : the subclassed Stream library that acts on StreamDoc objects.
  • callbacks/ : Callbacks that take event streams
  • utils/ : various utilities that don't fit anywhere else
  • streams/ : all processes involving streams go here
  • data/ : objects that handle data
  • processing/ : functions/objects that process data. The goal is to eventually move this into another analysis library.
  • interfaces/ : Everything in the code runs on StreamDoc objects. External data must be converted to this format. All routines involving this conversion are found in folders in the interfaces folder:
    • /databroker : databroker data <-> StreamDoc conversions
    • /databroker_simulator : databroker data (simulated) <-> StreamDoc conversions
    • /plotting_mpl : matploblib <-> StreamDoc conversions
    • /xml : xmls <-> StreamDoc conversions
    • /file : file <-> StreamDoc conversions
    • StreamDoc.py : the StreamDoc object.
    • streams.py : the stream handling objects.
    • dask.py : stream handling objects involving distributed computation
  • /startup : various frontend scripts to run routine batch processing
  • /tests : unit tests
You can’t perform that action at this time.