Skip to content

Releases

v1.6.10 — ITS2 evidence and simpler profiling

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 18 Oct 04:15

Highlights

  • Add optional host/symbiont ITS2 multi-copy recovery to profiling: linked-read recruitment, observed-read graph paths, quality-aware candidate comparison, conservative phasing, and independent target/decoy and reference-span annotations.
  • Preserve unresolved candidates and report phase components, independent variant-site support and rejection reasons. Shared or duplicate evidence cannot force low-frequency haplotypes into acceptance.
  • Reduce alignment memory with rolling score rows and compact traceback while preserving full-matrix results.
  • Keep three profiling-specific options: --profile-copy-mode host|symbiont, --profile-group-map, and --profile-decoy. Use Themisto from PATH, -p for sample concurrency, and --reference-cache-dir for shared indices.
  • Fix index reuse across different output directories. Reference changes invalidate the cache.
  • Synchronize bilingual READMEs, command-line manuals, output descriptions and profiling documentation.

Migration

Removed profiling k-mer/threshold/memory, tool-path, dedicated cache and force-rebuild flags are rejected. Remove those flags; use PATH and the common cache option where needed. GM2_THEMISTO is no longer read. Integrated defaults remain k=21, threshold=0.8, relevant fraction=0.5, and index memory=2 GB. Without copy mode, profiling remains assembly-free. Experimental non-default pseudoalignment remains available through the standalone marker_profile component.

Validation and limitations

  • All 409 workspace tests, strict Clippy, formatting, distribution build and release metadata checks passed.
  • The same three complete real WGS samples were run in host and symbiont modes before and after parameter simplification. All six comparisons produced byte-identical biological reports, recruited reads and assembly graphs.
  • Combined elapsed time was 160.92 → 161.56 seconds (+0.40%). These single paired runs do not establish a stable speedup or slowdown.
  • Shared-cache reuse across output directories, invalidation after reference changes, and assembly-free profiling were also verified.
  • Low-frequency recovery is not complete: C3st remains unresolved where independent bridging evidence is insufficient. Read support does not prove all genomic copies have been recovered, and host mode lacks matched independent truth.

Installation

This is a source release, following the existing build/install workflow. No prebuilt binaries are attached.

See the changelog, profiling guide, and WGS comparison.

v1.6.9 — Reliable mitochondrial rescue and simpler defaults

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 17 Oct 18:48

Highlights

  • Preserve both mates during mitochondrial refiltering so graph and junction checks receive valid read pairs.
  • Verify rescue assemblies before replacing primary results. Empty, partial or failed rescues restore the primary assembly; only a read-supported circle replaces it.
  • Remove stale finalization outputs on reruns, report current empty/failed states, and prevent short-circle standardization from accessing bases beyond the sequence.
  • Honor one-pass --max-reads. Support cleanup previews and limit cleanup to confirmed samples from the current invocation, preserving other samples' historical intermediates.
  • Use tipseek mito -f samples.tsv -r reference.gb -o output with automatic defaults. The legacy --mito-genbank alias and expert overrides remain supported.

Validation and limitations

  • All 372 workspace tests passed; release builds, formatting and diff checks passed.
  • Real paired-read validation confirmed that the simplified reference option preserves outputs and that partial runs retain their reads without cleanup.
  • The full real-read validation remained a partial multi-contig assembly, not a verified circle. This release fixes workflow correctness; it does not claim complete mitochondrial recovery for that dataset.
  • Given identical valid reads and contigs, all six finalization output files were byte-identical before and after the fixes.

Installation

This is a source release, following the existing build/install workflow. No prebuilt binaries are attached.

See the changelog and mitochondrial guide.

v1.6.8 — Faster population processing and reliable stage reuse

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 17 Oct 18:29

Highlights

  • Validate population stage dependencies and completion receipts; reuse raw joint calls when only genotype/site filters change.
  • Stream fixmate, coordinate sorting and duplicate marking; create BAI indexes while writing final BAMs.
  • Use indexed reference intervals for coverage QC, preserving complete flagstat statistics and single-thread fallback behavior.
  • Reduce final BAM compression overhead with lossless level 1. In the four-library real UCE benchmark, this change reduced median runtime from 9.203 to 7.108 seconds, with final BAM files 5.28% larger. These are subset/host-specific measurements, not a universal speed guarantee.
  • Simplify pipeline lifecycle handling and shared tool arguments. Downstream failures terminate and reap remaining producer processes.
  • Fix empty R2 fields, normalized sample-ID collisions, same-file reference copies, executable resolution and stage-reuse validation.

Validation

  • 368 workspace tests passed, along with Clippy and formatting checks; GitHub CI passed for the implementation commit.
  • The final refactoring comparison preserved all 4,000,012 alignment records, all variant stages, 263 filtered SNPs, 9 one-SNP-per-UCE loci, full QC reports, PLINK panels and PCA results on the same real UCE subset.
  • Real-tool regression checks cover single-thread execution, skipped duplicate marking, indexed region queries, coverage edge cases and raw-call reuse. ADMIXTURE and full-library performance were not tested.

Installation and compatibility

This source release follows the existing build/install workflow. External tools remain separate dependencies; samtools sort/markdup must support --write-index (validated with samtools 1.16.1). Existing exon backends and reference-ordered SCO output remain available.

See CHANGELOG and the validation report.

v1.6.7 — Independent exon backend and read-supported SCO recovery

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 17 Oct 16:32

Highlights

  • Added exonerate-rs as an independent exon annotation backend, invoked from the current environment by command name. Miniprot remains available.
  • Added reference-coordinate SCO reconstruction with one sequence per SCO: recovered sample bases follow reference order, while missing positions and unresolved conflicts remain N.
  • Improved cross-reference evidence projection, short exon candidate screening, and multi-contig conflict handling with read support and auditable rejection reasons.
  • Preserved reads containing internal N bases for evidence evaluation and added conservative recovery of observed terminal stop codons.
  • Added bounded family scheduling and reduced redundant candidate-chain comparisons while preserving selection and ambiguity handling.
  • Fixed output-path overlap checks involving symbolic links and parent-directory components.
  • Updated bilingual documentation and generic validation tools; removed sample-specific study materials from the release tree.

Validation

  • 361 workspace tests passed; Clippy with warnings denied and Python script compilation passed.
  • GitHub CI and CodeQL passed for the integrated implementation, including the distribution build and CLI smoke check.

Usage notes

  • Install the selected annotation backend separately and make it available on PATH.
  • Reference-coordinate SCO output preserves unknown and conflicting positions; it does not imply a fully recovered biological CDS or a physically validated contig join.

v1.6.6 — Table-based UCE evidence and Apple silicon tuning

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 11 Oct 17:06

Highlights

  • Replaces FM-index extension in UCEFilter exact-match evidence with an exact run-k table when run-k ≤ 32 (default 19): each run-k window costs one lookup, and a MEM extends by comparing 32 packed bases per step against each occurrence of its run-k prefix. Larger run-k keeps the FM index.
  • Stores the run-k table compactly (two bits per reference base, one fingerprinted u32 per hash slot); peak memory on the benchmark below is about 16 MB above the FM index.
  • Detects the host processor and memory: -p auto reports performance and efficiency cores on Apple silicon, macOS measures available memory instead of falling back to a 512 MiB UCEFilter fragment bank, and compute threads ask macOS to prefer performance cores.
  • Assembler k-mer counting workers claim small blocks instead of fixed equal slices, so faster cores take more work.
  • Leaves scheduling unchanged: stage thread budgets, stage priorities and sample order still follow the GeneMiner2 dispatcher.

Performance

575,435-pair UCE benchmark used for v1.6.5 (2,211 bivalve loci; on-target pairs simulated from real DK40 UCE contigs, off-target pairs from a real WGS library; Apple M2, -p 8, default filter assemble --assembly-mode uce), two runs each:

Stage v1.6.5 v1.6.6
Total 47.3–49.8 s 32.3–33.2 s
First-pass filter 4.8–5.0 s 1.8–2.0 s
Rescue-round filter 18.8–20.0 s 5.5–6.7 s

Assembly time is unchanged. UCEFilter alone on a 675,435-pair variant of the benchmark: exact-match evidence 4.7 s → 0.85 s, peak RSS 331 MB → 347 MB.

Verification

  • All result files were byte-identical to v1.6.5 on the benchmark above (5,556 files) and on real phyluce Gallus UCE reads (5,123 files); only run logs and manifests differ.
  • New randomized equivalence tests compare the run-k table path with the FM index (seeds, occurrence counts, strand masks, MEM statistics), packed with bytewise prefix comparison, and k-mer counts across worker counts.
  • 286 workspace tests passed; cargo fmt --check and Clippy with warnings denied passed; release build completed.

v1.6.5 — Faster UCE recruitment, rescue and assembly

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 11 Oct 15:52

Highlights

  • Prunes exact-match (MEM) evidence in UCEFilter: a read start stops extending past run-k once its longest possible match can no longer reach the current best seed, and unique run-k strands come from a per-row table instead of a sampled suffix-array walk.
  • Rewrites rescue internal-gap QC on packed 15-mer keys with one pass over reads for both the before and after contigs, and lists rescue read directories once per round instead of once per restored locus.
  • Counts assembler k-mers with 64-bit keys when k ≤ 32, including the k=21 rescue assembly.
  • Leaves scheduling unchanged: stage thread budgets, stage priorities and sample order still follow the GeneMiner2 dispatcher.

Performance

575,435-pair UCE benchmark (2,211 bivalve loci; on-target pairs simulated from real DK40 UCE contigs, off-target pairs from a real WGS library; Apple M2, -p 8, default filter assemble --assembly-mode uce):

Stage v1.6.4 v1.6.5
Total 82.6 s 49.1–49.6 s
First-pass filter 8.3 s 5.2–5.4 s
Rescue round 70.7 s 39.5–40.5 s

Verification

  • All result files were byte-identical to v1.6.4 on the benchmark above and on real phyluce Gallus UCE reads; only run logs and manifests differ.
  • New randomized equivalence tests compare each optimized path with the previous implementation (MEM seeds, counts and strands; per-base gap-support profiles; u64 vs u128 k-mer counts) and were mutation-checked.
  • 283 workspace tests passed; cargo fmt --check and Clippy with warnings denied passed; release build completed.

v1.6.4 — Exon reference derivation and validated fragment joining

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 21 Sep 09:01

Highlights

  • Derives auditable protein references from coding nucleotide baits in --assembly-mode exon; external .faa files remain optional per-family overrides.
  • Simplifies complementary-fragment screening: query overlap is no longer an independent veto, unique coverage follows --gene-min-model-coverage, and a one-residue margin is used only to decide whether padded reannotation should be attempted.
  • Keeps the final safety boundary unchanged: a padded join is accepted only after unique complete-model reannotation places the entire synthetic interval inside an intron and outside every exon.
  • Validates the behavior across 56 real multi-exon Patiria pectinifera genes with 2.0–20.0-kb target introns; all 29 accepted padded CDS results match their unbroken-candidate results.
  • Adds the integrated GeneMiner2-to-TipSeek innovation schematic and harmonizes the Chinese-primary, English-switchable documentation across exon, UCE, mitochondrial, RAD, population, profiling, and repeatome workflows.

Verification

  • 280 workspace tests passed.
  • Clippy passed with warnings denied.
  • Release build completed successfully.
  • 198 local documentation links were validated.

v1.6.3 — Integrated exon annotation

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 20 Sep 18:36

Integrated exon annotation

Highlights

  • Added --assembly-mode exon as the complete protein-guided extension of the default gene-family recovery workflow.
  • Added deterministic miniprot-based exon model selection and structural QC for coverage, CDS phase, translation, frameshifts, internal stops, splice classes, and competing models.
  • Added coordinate-preserving CDS, protein, exon, intron, gene, supercontig, GFF3, model, and audit outputs.
  • Added conservative N-padded joining for one unambiguous pair of complementary terminal partial models. A derived join is accepted only after reannotation produces one complete resolve-eligible model and places the entire synthetic interval inside an intron without exon overlap.
  • Simplified the public interface: tipseek is the default gene workflow, --assembly-mode exon adds structural annotation, and --assembly-mode uce remains independent.
  • Added focused Chinese and English exon documentation and separated candidate recovery, exon annotation, and phylogenetic resolution into non-overlapping guides.

Exon-mode input

Each nucleotide family reference (.fa or .fasta) must have a matching protein reference with the same family stem and the .faa extension. Exon annotation requires miniprot 0.18 or later.

cli/tipseek --assembly-mode exon \
  -f samples.tsv \
  -r family_reference \
  --gene-protein-reference family_proteins \
  -o exon_output \
  -p 8

Validation

  • cargo test --workspace: 272 passed
  • cargo clippy --workspace --all-targets -- -D warnings: passed
  • Documentation links, code fences, deprecated invocations, and whitespace checks: passed

Full Changelog: v1.6.2...v1.6.3

v1.6.2 — TipSeek identity and evidence-bounded UCE recovery

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 15 Aug 21:50
882569c

TipSeek identity and evidence-bounded UCE recovery

Highlights

  • Renamed the public project from TStools to TipSeek and changed the primary executable from cli/geneminer2 to cli/tipseek.
  • Changed UCE defaults to k=23/step=4 automatic recruitment followed by one evidence-constrained rescue round.
  • Strengthened fallback-locus assignment and rescue eligibility with full-panel probe checks, explicit provisional-core states, long-inverted-repeat rejection, internal-gap review, and per-locus audit tables.
  • Added the TipSeek UCE manuscript and supplementary methods, including the 11-sample coral benchmark of TipSeek R0/R1/R2, GeneMiner2, and SPAdes + PHYLUCE.

Command change

The main command is now:

cli/tipseek

Existing scripts that invoke cli/geneminer2 must update that executable path. Workflow names and their task-specific options remain available through the new entry point.

UCE defaults and evidence controls

  • UCE mode now defaults to -kf 23, step 4, --uce-recruit-mode auto, and one rescue round.
  • Recruitment k is independent from exact-match verification k, allowing more sensitive recruitment without weakening the verification threshold.
  • Automatic recruitment retries only fast-pass unresolved loci, then evaluates gated fragments against the complete probe panel before locus assignment.
  • Fallback-only candidates must meet length, target coverage, identity, and panel-uniqueness gates before becoming provisional cores.
  • Exact long inverted repeats are rejected. Internal read-chain gaps are reported as review evidence; review-only cores are retained but cannot seed rescue.
  • Rescue remains reversible per locus, and a failed extension restores the previously accepted candidate.
  • The previous k=31 fast-pass workflow without rescue remains reproducible through explicit overrides.

Manuscript and benchmark

  • Restructured the UCE manuscript to follow the Bioinformatics original-paper format.
  • Retained the coral-only benchmark and full TipSeek R0/R1/R2 results, including sample–locus recovery, panel coverage, shared loci, sequence length, wall time, and peak RSS.
  • Added supplementary methods, reproducible command placeholders, and the complete benchmark table.

Validation

  • cargo test --workspace
  • git diff --check

Full Changelog: v1.6.1...v1.6.2

v1.6.1 — Automatic UCE recruitment and manuscript draft

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 04 Aug 04:14
3d198e3

Automatic UCE recruitment and manuscript foundation

Highlights

  • Added the opt-in --uce-recruit-mode auto workflow for conservative recovery of unresolved UCE loci. The default remains fast.
  • Renamed the public project to TStools while retaining repository URLs, the cli/geneminer2 entry point, legacy output identifiers, and scholarly citations for compatibility.
  • Added the Chinese TStools/UCE manuscript draft under manuscript/TStools_UCE_draft_ZH.md for collaborative revision and provenance tracking.

Conservative automatic recruitment

  • Retries only fast-pass unresolved loci with a sensitive k=21/step=1 coarse-recruitment gate.
  • Requires fallback reads to provide a 45 bp alignment at 80% identity and merges only newly recovered locus FASTQs.
  • Requires fallback-only contigs to be at least 200 bp, cover at least 80% of the target probe at 80% identity, and avoid near-tied panel loci.
  • Preserves acceptance and rejection evidence in auditable TSV sidecars and archived outputs.

Validation

  • GitHub CI passed, including MSRV checking, formatting, workspace tests, Clippy, distribution build, release metadata, and CLI entry-point validation.
  • GitHub CodeQL analysis passed.

Full Changelog: v1.6.0...v1.6.1