v1.0.0

1.0.0 (2024-05-13)

Features

• 0 threads when copying, rename group (83b4290)
• accelerate fastp (e5ba830)
• adaptors: adaptors should be explicitely written (cc12e74)
• add execution groups (2d21e86)
• add fastp to preprocess group (df0df2e)
• add groups in run_slurm (778f2a4)
• add indexes and references to cache (16cc736)
• add micro profile (6aad3bf)
• add profile (7e0fc8a)
• Add run_slurm [skip ci] (a3123b1)
• bowtie2: disable minimizer compression (dd66c46)
• bowtie2: get ram from params (9c8162d)
• bowtie2: increase compression level (d870e95)
• bowtie2: remove bottleneck (933bece)
• cache bowtie2 indexes (4ec4a55)
• cache omitting software versions (f91122c)
• clean code (2006df1)
• consistent naming (cb49e19)
• correct column names in samples.tsv (016f850)
• coverm: accept cram as input (26b601b)
• coverm: refactor aggregator function (74dcd06)
• cram to fastq in quantify. add defaults to profiles (4055a4c)
• delete all the code mentioning SE reads (57f3ac3)
• extract threads and resources to profile in preprocess (c854f80)
• extract threads and resources to profile in quantify (5846fc6)
• extract threads and resources to profile in reference/recompress (0292ca0)
• extract threads and resources to profile in report (a46a0b5)
• extract threads and resources to profile in samtools (d5653ff)
• fastp: insert extra trims to fastp (71dee60)
• fastp: manage more elegantly the adaptors and the NAs (0a81ad8)
• fastp: put an assert just in case (c3fc4a9)
• fastp: refactor functions (854d1e3)
• fastp: remove temps, raise compression level (afe88aa)
• final profiles (ca3cde5)
• folders: consistent reordering (3358c0a)
• functions: remove unused code (4a3acfa)
• gha: split into steps (269ec70)
• gha: upload logs as artifacts on failure (b52692a)
• handle big and small bowtie2 indexes (af2f956)
• humanize default profile (5a4c615)
• in bowtie2 map rules, clean past tmp bam files (8a68596)
• join heavy processes into a single group (e52712a)
• kraken2 as a service (ed02c97)
• kraken2 as in mg_assembly (201af9b)
• kraken2: print the hostname (a91735f)
• lower compression level, don't produce unpaired (60e2c67)
• mark singlem fastqs as temp (62c3943)
• mem_mb -> mem_gb (58d085f)
• nonpareil: cleaner code (345f381)
• organization: dunder all the important files (0c6b088)
• params: remove mentions to extra fields (f489d2f)
• pre-commit: add R linters (df017d8)
• pre-commit: add R linters (cd26577)
• pre-commit: raise message to the top (aa6b764)
• preprocess: run everything by default, and everything but singlem on testing (5c949c2)
• raise threads for bowtie (67ca596)
• raise threads for bowtie (8d02fd0)
• readme: Update badges (d5a4e1d)
• readme: update with new control files (670c818)
• reads: refactor functions (b8952ac)
• reads: split into subworkflows (e05481d)
• remove commented code. remove tmp of links (4bc96c1)
• remove dead code. cache correct files (b1eaa94)
• remove old scripts (87bb8e0)
• rename and relink preprocessing (3d73aef)
• rename and relink reads (d44c60c)
• rename and relink reference (c579973)
• rename and relink report (2e7dbbf)
• rename and relink stats (b206ee6)
• rename helpers (374a63e)
• rename rules and add fai to cache (486a57f)
• rename rules, create bowtie2 indexes in their place, and with multiext (9935c4b)
• reorganization: env.yml => environment.yml (cb685d3)
• reorganize config/params.yml (da9af8e)
• reorganize features.yml (4e3d07c)
• reorganize into preprocess and quantify (3ce6e4c)
• reports: double ram and use retries (7d0e06a)
• restructure helpers (87d307d)
• sample, library -> sample_id, library_id (59a4c5d)
• separator now is @ (e7d6d18)
• simplify kraken2 code (0817e6a)
• singlem: compute microbial fraction (f274288)
• singlem: reorganize (8793c65)
• singlem: update to 0.16.0 (5c3a698)
• sort env (6db2a90)
• sort environments alphabetically (6f2effb)
• sort flags in rm (136d34e)
• test kraken2 using parallel and 1 thread (0fd4f61)
• update all environments (18d5c28)
• update envs (0a99d8d)
• update minimum snakemake version (fcb1a98)
• use cram files instead of cram<->fq conversions (ef7dfc0)

Bug Fixes

• adaptors: typo (ffceca2)
• add again memory per thread in samtools (6645c4d)
• add cram to fastq rules to groups (067240a)
• back to collating fq files (9583a3b)
• back to sort -n (a1066e6)
• back to sort by name instead of collate (cf93e0d)
• bowtie2: in sort, store the temp files next to the cram file (bcff482)
• bowtie2: in sort, store the temp files next to the cram file (ff2c07d)
• cache gzi too. use samtools collate (4b2bc6f)
• collating (683a6d5)
• collating? (7af905b)
• config: right patsh , mags now with @ (6599b97)
• correct per library report (d49b2fd)
• correct placement for pig genome (1c776c6)
• correct rule name (dd8056c)
• correct rule names in grouping (9b87aee)
• coverm: do not filter out unaligned reads (9e1370a)
• coverm: forgot the exclude flags (1e9a9a1)
• do samtools sort -n so fastq are generated correctly (0e9fd3d)
• extra edge in rulegraph (4002fef)
• extract only unmapped pairs (4030f0d)
• extreme case when no magalogue is present (777ed90)
• fastp min_length, crais for coverm, more time for nonpareil (83f0fed)
• fastp: raise ram to 8GB (31f500e)
• fastp: typo (ae2fedd)
• folders: consistent naming (079efe5)
• gha: add again preprocess__kraken2 (304f99d)
• gha: add r packages for pre-commit (8d6812c)
• gha: delete kraken2 (d6b5a4a)
• gha: do not run kraken since /dev/shm is not available (bd2937f)
• gha: no comments allowed in a yaml list (1701eca)
• gha: preprocessing (29a69b7)
• gha: quantify, not stats (7ffe5a0)
• gha: use custom snakemake action (b007667)
• go back to run kraken2 db copy and assignment in a single job (dd1d550)
• improve kraken2 (869b223)
• increase memory and time resources on bowtie2 and fastp [skip ci] (8d2c09c)
• its mem_mb not memory (d7c90bf)
• kraken2 paired (b9703ff)
• kraken2: back to doing it sequentially (e855958)
• kraken2: use whoami instead of $USER (2ada7cf)
• limit fastp to 16 threads (adbcd98)
• linking reads should are localrules [skip ci] (f69bce0)
• may give incorrect files (9ff4ef1)
• nonpareil: f-string in params (10b2851)
• nonpareil: move packages between environments (8348ee0)
• nonpareil: wrong fq (4a45032)
• nonpareil: wrong variable names (aedcd1e)
• parenthesis (8d3d352)
• pre-commit: remove R stylers. pre-commit can't find the r packages (0119339)
• preprocess: add csvkit to environment (14b6c4b)
• proper number of threads in pigz (c67426c)
• put log in the correct place (c647e8c)
• put temp collate files next to the final cram (ce02591)
• raise memory in fastp fastq indexing (1c01928)
• raise memory on bowtie2 fastq indexing and mapping [skip ci] (eae48d9)
• raise memory on fastp indexing [skip ci] (44c3fa1)
• raise memory on fastq extraction for slurm (e3e394c)
• readme: update and point to new path (2f8f383)
• readme: wrong config files: tsv->yml (14de54d)
• remove the .fa in the renamed mags (91cd41a)
• report: give 4GB to kraken2 multiqc report (e62cbd3)
• report: improve readability, merge reports, remove dead code (098ae25)
• report: remove samtools idxstats in mag reports. They are massive (e8cb472)
• saner way to get unpaired from samtools [skip ci] (585ae8f)
• singlem: hardcoded output file (434acc9)
• singlem: missing f-string (dcdfcc5)
• singlem: raise runtime (a4e3951)
• singlem: raise singlem pipe runtime (eb39f00)
• singlem: use 1 CPU and 16GB of RAM (17e80d8)
• snakefmt (7e10116)
• split the collate to produce an actual bam file (2bcd77c)
• typo (baa7ffc)
• unfinished command (931dfae)
• use small index for preprocess, big index for quantify (ce91805)
• wrong singlem path (65ecfbb)

Performance Improvements

• reduce thread usage when converting (3d1893d)