v2.0
Major overhaul of underlying clustering approach and reference creation process:
- New documentation page (still needs some love):
- All processes written in Python and use JSON and JSONL inputs / outputs.
- Segmented genomes are classified automatically using a distance-based approach. The most common segment name for each segment group is applied. This process is optimized using a global minhash per segment group rather than pairwise comparisons. This was developed to address misclassified segments or sequences with no segment assigned. Specifically, this fixes the misclassification of flu B segments 1 and 2 in NCBI.
- Clusters are created using a windowed approach, leading to greater discriminatory power for large genomes. This approach was developed to address the previous creation of a single Mpox reference genome, despite significant variability in the termini.
- Clusters are created using DBSCAN instead of hierarchal clustering of complete linkage. This greatly improves over partitioning of sequences. A similar process is now also applied during reference de-duplication (condensing).
- References can be optionally created using a "centroid" approach during clustering. This returns a real sequence that has not undergone any modification. Consensus generation has likewise been made optional.
- Consensus sequences are now created using numpy instead of awk.
- Metadata handling and summarization is greatly improved and simplified.
- References are output as single JSONL files containing both metadata and the reference sequence. This greatly simplifies downstream modification of reference sets, while also facilitating rapid processing and filtering by VAPER
Full Changelog: v1.5.5...v2.0