Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-883] Add caching to Transform pipeline. #884

Merged
merged 1 commit into from
Nov 19, 2015

Commits on Nov 19, 2015

  1. [ADAM-883] Add caching to Transform pipeline.

    The Transform pipeline in the CLI has several stages (e.g., sort, indel
    realignment, BQSR) that trigger recomputation. If you are running a single
    stage off of local storage/HDFS/Tachyon, this is OK. However, if you're running
    multiple stages, or you are loading data from S3/etc, this can lead to serious
    performance degradation. To address this, I've added the proper caching
    statements. Additionally, I've added a hook so that the user can specify the
    storage level to use for caching. Resolves bigdatagenomics#883.
    fnothaft committed Nov 19, 2015
    Configuration menu
    Copy the full SHA
    9962d5f View commit details
    Browse the repository at this point in the history