Fix filter engine: OOB write, undefined shift, wrong fallbacks, debug output#2523
Merged
pd3 merged 1 commit intosamtools:developfrom Mar 28, 2026
Merged
Fix filter engine: OOB write, undefined shift, wrong fallbacks, debug output#2523pd3 merged 1 commit intosamtools:developfrom
pd3 merged 1 commit intosamtools:developfrom
Conversation
- vcffilter.c: Fix off-by-one in AC array bounds check (ial<=n_allele should be ial<n_allele since tmp_ac has n_allele-1 elements) - filter.c: Use 1ULL instead of 1 for left shift on allele bitmask to avoid undefined behavior when allele >= 32 (mask is uint64_t) - filter.c: Fix incorrect fallback functions in sample aggregation: func_smpl_median -> func_median (was func_avg), func_smpl_stddev -> func_stddev (was func_avg), func_smpl_sum -> func_sum (was func_avg), func_smpl_count -> func_count (was func_max) - filter.c: Remove debug fprintf left in production code
sirus20x6
force-pushed
the
fix/filter-engine
branch
from
0b728a6 to
9c65b17
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
set_genotypesAC bounds check:ial<=line->n_alleletoial<line->n_allele(heap buffer overflow)1<<alleleto1ULL<<allelein all 4 GT mask sites —allelecan be up to 63 but1<<alleleis undefined behavior for signed int when allele >= 31SMPL_*fallback functions:func_smpl_medianfell back tofunc_avginstead offunc_median, and similarly for stddev, sum, and countfprintf(stderr,"pass=%d...")left incmp_vector_stringsFixes #2491
Test plan
SMPL_MEDIAN,SMPL_STDEV,SMPL_SUM,SMPL_COUNTon INFO fields