GET_PHYLOMARKERS_v2.2.1_2024-04-18

This minor release (v2.2.1, 2024-04-18) contains the following improvements:

• run_get_phylomarkers_pipeline.sh v2.8.1.4_2024-04-17
  • Improved reporting at the end of run (STDOUT and file written to disk) of essential output files and figures generated by the pipeline, calling new functions rm_top_dir_prefix_from_wkdir and read_svg_figs_into_hash
• get_phylomarkers_fun_lib v2024-04-17
  • created new functions rm_top_dir_prefix_from_wkdir and read_svg_figs_into_hash
  • prepended missing "$bindir"/ to several iqtree calls in run_ASTRAL and compute_ASTRALspTree_branch_lenghts
  • improved print_start_time progress reporting from run_ASTRAL and compute_ASTRALspTree_branch_lenghts
• compiled a static binary of snp-sites. Thanks to Alfredo Hernandez from CCG-UNAM & LCG-UNAM for compiling this binary
• run_get_phylomarkers_pipeline.sh >= v2.8.1._2024-04-15
  • calls ($bindir/snp-sites-static ) a static binary of snp-sites for Linux (see: bin/linux/snp-sites-static)
  • Updated Dockerfile (v2024-04-18) to avoid installing non-required R packages, generating a significantly lighter image
  • Updated INSTALL.md and README.md
  • updated install_R_deps.R and apt-install_R_dependencies.sh v2024-04-15: replaced devtools for the much lighter remotes; removed uncessary R packages
  • updated .travis.yml v2024-04-15; does not call apt install snp-sites, as it is now provided as static binary in bin/linux/snp-sites-static

Docker image

1. The GET_PHYLOMARKERS_v2.1.1_2024-04-18 release has the corresponding Docker image v20240418 ready to pull from Docker Hub.
   To pull the latest version of the GET_PHYLOMARKERS Docker image, type: docker pull vinuesa/get_phylomarkers:latest, or docker pull vinuesa/get_phylomarkers:20240418

2. Docker image v20240418 is significantly lighter (2.0GB), compared to the previous one (v20240414; 2.09GB)

GET_PHYLOMARKERS_v2.2.0_2024-04-14

This major release (v2.2.0, 2024-04-14) contains new features, significant code improvements, binary updates, and some bug fixes

New features

1. significant extension of run mode 2 (run_get_phylomarkers_pipeline.sh -R 2) for population genetics (multiple sequences from the same species):
   • the FASTA files of non-recombinant, kdetrees-compliant, and neutral loci are saved in their directory and concatenated
   • SNP sites are extracted from the concatenated alignment with snp-sites and saved as FASTA and VCF formats
   • ML trees are estimated from the SNP supermatrix using either IQ-TREE or FastTree
2. run_get_phylomarkers_pipeline.sh now also calls the C binary WEIGHTED-ASTRAL to estimate a species tree using as input the filtered gene trees estimated by iqtree2 or FastTree from the core-genome clusters computed by get_homologues.
   • run_ASTRAL computes best-fitting model from protein or DNA concat alignments using IQT with wASTRAL species tree as a constraint
   • run_ASTRAL calls compute_ASTRALspTree_branch_lenghts from protein or DNA concatenated alignments using IQ-TREE with wASTRAL species tree as a constraint
   • run_ASTRAL now calls astral4 AND wASTRAL from the ASTER package, using the wASTRAL species tree for the downstream analyses listed above
3. The main script run_get_phylomarkers_pipeline.sh
   • Added complex protein mixture models for concatenated protein alignments
   • Prints script invocation arguments to STDOU at the beginning of the run
   • Collects results of the different filtering steps and prints an overview of the pipeline's filtering process before exiting
   • Full shellcheck compliance

Updated and new external programs

• A static binary of ASTRAL-IV is used to estimate the concatenation-free species tree
• A static binary of WEIGHTED-ASTRAL is used to estimate the concatenation-free species tree
• snp-sites is now used under run mode 2 (run_get_phylomarkers_pipeline.sh -R 2)

Updated scripts, library code, and test files

• run_get_phylomarkers_pipeline.sh
• run_test_suite.sh
• lib/get_phylomarkers_fun_lib
• install_R_deps.R omits installing R packages that are not used anymore
• test_get_phylomarkers.t now runs 24 tests

Docker image

The distribution contains a Dockerfile used to build the Docker image ready to pull from Docker Hub. On Dockerhub, you will find detailed instructions on installing and configuring the Docker client on your machine, pulling the latest image, and running the containerized instance of the GET_PHYLOMARKERS pipeline.

GET_PHYLOMARKERS_v2.1.1_2024-04-01

This minor release (v2.1.1, 2024-04-01) contains the following improvements:

• run_kdetrees.R v0.4.0_2024-04-01 checks for the number of outlier trees and quits if none are found after the basic R plots are generated.
• run_get_phylomarkers_pipeline.sh v2.7.2_2024-04-01: improved checking and handling of run_kdetrees output when no outlier trees (branch length or topology) are found.

We are refactoring the code base and expect to publish more releases in the coming weeks. Stay tuned!

Docker image

1. The GET_PHYLOMARKERS_v2.1.1_2024-04-01 release has the corresponding Docker image ready to pull from Docker Hub.
   To pull the latest version of the GET_PHYLOMARKERS Docker image, type: docker pull vinuesa/get_phylomarkers:latest.

GET_PHYLOMARKERS_v2.1.0_2024-03-31

This release (v2.1.0, 2024-03-31) contains new features, significant code improvements, binary updates, and some bug fixes

New features

1. Implemented the maximal matched-pairs tests to asses violations of the data to the Stationarity, Reversibility, and Homogeneity (SRH) assumptions made by maximum-likelihood phylogenetic models, as implemented in iqtree2.

2. run_get_phylomarkers_pipeline.sh now calls the C binary ASTRAL-IV instead of the older java ASTRAL-III jar to estimate a species tree using as input the filtered gene trees estimated by iqtree2 or FastTree from the core-genome clusters computed by get_homologues.

3. The main script run_get_phylomarkers_pipeline.sh runs under Bash's unofficial strict mode set -euo pipefail. Additionally, the code was thoroughly revised, improving the code layout and using a more idiomatic and modern bash syntax.

Updated external programs

• ASTRAL-IV replaces ASTRAL-III to estimate the concatenation-free species tree
• IQ-TREE v.2.3.1

Updated scripts and library code

• run_get_phylomarkers_pipeline.sh
• run_kdetrees.R
• lib/get_phylomarkers_fun_lib

Docker image

The distribution contains a Dockerfile used to build the Docker image ready to pull from Docker Hub. On Dockerhub, you will find detailed instructions on installing and configuring the Docker client on your machine, pulling the latest image, and running the containerized instance of the GET_PHYLOMARKERS pipeline.

GET_PHYLOMARKERS_v2.0.1_2023-01-08

GET_PHYLOMARKERS_v2.0.1_2023-01-08

This minor release (2023-01-08) contains a bug fix in add_labels2tree.pl, and some code improvements to speed up iqtree calls on the concatenated supermatrix.

Docker images

1. The distribution contains a Dockerfile used to build the Docker image ready to pull from Docker Hub.

2. This version is also bundled as a Docker image that runs GET_HOMOLOGUES + GET_PHYLOMARKERS, ready to pull from Docker Hub

Travis CI and updated test suite

• CI with 23 tests implemented in test_get_phylomarkers.t, deployed using Travis CI

GET_PHYLOMARKERS_v2.0.0_2022-11-20

GET_PHYLOMARKERS_v2.0.0_2022-11-20

This release (2022-11-20) contains new features, significant code improvements, binary updates, and some bug fixes

New features

The most salient new feature is that run_get_phylomarkers_pipeline.sh now also calls ASTRAL-III to estimate a concatenation-independent species tree, using as input the filtered gene trees estimated by iqtree2 from the core-genome clusters computed by get_homologues.

IQTree2 calls have been updated to match those of version 2.2.0 and replaced the -T AUTO with the -T $IQT_threads call, which results in a ~20-fold speedup when running the included run_test_suite.sh.

The main scripts run_get_phylomarkers_pipeline.sh and estimate_pangenome_phylogenies.sh have been thoroughly revised, fixing bugs, using a more idiomatic and modern bash syntax, and improving code layout for better legibility. Both scripts require bash v4.3 or newer, and the new function check_bash_version ensures that the host runs a proper version.

Updated external programs

• IQ-Tree (v2.2.0) 64-bit Linux binaries

Updated scripts and library code

• run_get_phylomarkers_pipeline.sh
• estimate_pangenome_phylogenies.sh
• run_kdetrees.R
• lib/get_phylomarkers_fun_lib

New scripts

install_kdetrees_from_github.R

Docker image

The distribution contains a Dockerfile used to build the Docker image ready to pull from Docker Hub.

Documentation

• Updated documentation and tutorial materials, both in Markdown and HTML formats

Travis CI and updated test suite

• CI with 23 tests implemented in test_get_phylomarkers.t, deployed using Travis CI

GET_PHYLOMARKERS_v1.3.2_2021-09-18

GET_PHYLOMARKERS_v1.3.2_2021-09-18

This release (2021-09-18) contains several important updates and bug fixes

• packages and runs the latest FastTree (v2.1.11.) and IQ-Tree (v2.1.3) 64-bit Linux binaries
• contains a Dockerfile used to build the Docker image ready to pull from Docker Hub
• the release bundles a new versions of: run_get_phylomarkers_pipeline.sh, estimate_pangenome_phylogenies.sh, hcluster_pangenome_matrix.sh, install_R_deps.R
• new scripts included in the release for convenient testing: run_test_suite.sh, run_make_tests.sh
• Updated documentation and tutorial materials, both in Markdown and HTML formats
• CI with 21 tests implemented in test_get_phylomarkers.t, deployed using Travis CI
• run_get_phylomarkers_pipeline.sh issues a warning, instead of dying if the kdetree test could not be run due to installation issues with the kdetrees R package

GET_PHYLOMARKERS_v1.3.1_02042020

• updated Dockerfile to run R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
• updated install_R_deps.R to use the bionic-cran35 repo

GET_PHYLOMARKERS_v1.3.0_02042020

• activated the GitHub integration to start testing and deploying on Travis CI
• updated IQ-TREE v2.0-rc1 binaries (64-bit) for ubuntu Linux and MacOSX
• estimate_pangenome_phylogenies.sh v1.1_10Jan20; added option -S <abayes|UFBoot|both>

GET_PHYLOMARKERS_v1.2.11_03092019

• updated PAUP* versions: paup4a166_ubuntu64 and paup4a166_osx
• updated FastTree v2.1.11 Double precision (No SSE3) for Linux
• updated IQ-TREE v1.6.12 binaries (64-bit) for ubuntu Linux and MacOSX
• tidied the ${perl_scripts[@]} in check_scripts_in_path() and changed hard-coded -e min. = 4 for $min_no_ext_branches
• updated manual: minor fixes and more info on running docker