Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hail] Improve IR generated by VQC/SQC to avoid exponential blowup #8032

merged 1 commit into from
Feb 5, 2020


Copy link

@tpoterba tpoterba commented Feb 4, 2020

This change lets the call_rate calculation in variant_qc and sample_qc use the same IR without forking (which led to an exponential duplication of the IR, when these appeared iteratively).

We do this by introducing a new binding in MatrixMapCols and MatrixMapRows which refers to the row/col count. These metrics can be computed at essentially no marginal cost.

$ hail-bench compare ~/misc/debug/qc_before.json ~/misc/debug/qc_after.json
                                       Name      Ratio    Time 1    Time 2
                                       ----      -----    ------    ------
                                  sample_qc     102.3%    13.173    13.470
                                 variant_qc      91.4%     5.259     4.809
variant_and_sample_qc_nested_with_filters_2      29.9%    55.963    16.749
variant_and_sample_qc_nested_with_filters_4       3.2%   1183.905    38.048

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

Successfully merging this pull request may close these issues.

None yet

3 participants