Developer tutorial for Beast 2.7.x using IntelliJ
Create a directory for IntelliJ dev and get BEAST2 and BEASTFX v2.7.6.
mkdir ~/intellij
cd ~/intellij
wget https://github.com/CompEvol/beast2/archive/refs/tags/v2.7.6.tar.gz
wget https://github.com/CompEvol/BeastFX/archive/refs/heads/master.zip
tar -xvzf v2.7.6.tar.gz
mv beast2-2.7.6 beast2
unzip master.zip
mv BeastFX-master/ BeastFX
Azul JDK 17 (Java 17)
https://www.azul.com/downloads/?version=java-17-lts&package=jdk-fx
This tutorial was set up using zulu17.50.19-ca-fx-jdk17.0.11. Versions 17.0.4 and 17.0.5 did not work and are not recommended.
IntelliJ IDE (Community Edition)
https://www.jetbrains.com/idea/download/
Apache Ant(TM) version 1.10.12
File > New > Project
Set the JDK to Zulu 17
File > Project Structure > Global Libraries > + > Java
or Ctrl+Alt+Shift+S
Select all files under BeastFX/locallib
Rename the library to b2fx-lib
Select the beast2/lib
directory
Rename the library to b2-lib
Select the beast2/junit
directory
Rename the library to b2-junit
File > Project Structure
Import the module beast2 by selecting the directory, selecting next.
In the Libraries window, edit change 'lib' to 'beast-lib' and untick DensiTree.
Modules > + > Import Module
Import BeastFX with the same procedure
File > Project Structure > Modules > beast2 > Sources
Select test folder and mark as Tests
File > Project Structure > Modules > beast2 > Dependencies
Add the beast2 library and beast2 junit test library, setting Scope
to Compile
.
+ > 2 Library > Global Libraries
File > Project Structure > Modules > BeastFX > Sources
Select src/test folder and mark as Tests
File > Project Structure > Modules > BeastFX > Dependencies
Add a Module Dependency
for beast2
and the BeastFX library, setting Scope
to Compile
.
For MyPackage, add Module Dependency
for both the beast2
and BeastFX
.
In the Project
tab, navigate to BeastFX > src > beastfx.app > beauti
and select Beauti
.
Right click Beauti
and select Modify Run Configuration
.
To run Beauti
, Run > Debug > Beauti
.
In the Project
tab, navigate to BeastFX > src > beastfx.app > beast
and select BeastMCMC
.
Right click BeastMCMC
and select Modify Run Configuration
.
In Program Arguments
, locate the xml you wish to run, e.g. beast2/examples/testHKY.xml
To run Beast2
, Run > Debug > BeastMCMC
.
Alternatively, use the same debug configuration for BeastMain
, which will bring up a window where an xml file can be selected.
We are implementing the F84 substitution model.
Create a new Package
called beast.base.evolution.substitutionmodel
.
Right click src > New > Package
.
Right click beast.base.evolution.substitutionmodel
and create a new Java Class
called F84
.
Open the F84.java
class.
Add extends SubstitutionModel.Base
after F84
.
public class F84 extends SubstitutionModel.Base {
}
Click on
Select the error to declare F84
and click on the light bulb Show Quick Fixes
.
Select Implement Methods
and click OK
, adding in unimplemented methods.
The F84.java
file should now look like this
Add the extends SubstitutionModel.Base
after the class name.
public class F84 extends SubstitutionModel.Base{
public Input<RealParameter> kF84 = new Input<RealParameter>("kF84", "k parameter in the F84 model", Validate.REQUIRED);
To resolve the errors, Import Class
for each unimplemented method.
The following declarations should have been added.
import beast.base.core.Input;
import beast.base.inference.parameter.RealParameter;
@Override
public boolean canHandleDataType(DataType dataType) {
if (dataType instanceof Nucleotide) {
return true;
}
return false;
}
Add the unimplement method, inserting the following code.
import beast.base.evolution.datatype.Nucleotide;
Fill in the getTransitionProbabilities
method with the following:
@Override
public void getTransitionProbabilities(Node node, double fStartTime, double fEndTime, double fRate, double[] matrix) {
double[] freqs = frequencies.getFreqs();
double freqA = freqs[0];
double freqC = freqs[1];
double freqG = freqs[2];
double freqT = freqs[3];
double freqR = freqA + freqG;
double freqY = freqC + freqT;
double k = kF84.get().getValue();
double sumPiSquared = freqA*freqA + freqC*freqC + freqG*freqG + freqT*freqT;
double sumPiRatios = freqA*freqA/freqR + freqC*freqC/freqY + freqG*freqG/freqR + freqT*freqT/freqY;
double mu = (1.0 - sumPiSquared) + k*(1.0 - sumPiRatios);
double distance = (fStartTime - fEndTime) * fRate;
double expTerm = Math.exp(-mu * distance);
double expTermK = Math.exp(-mu * distance * (k + 1.0));
matrix[0] = freqA + freqA*(1.0/freqR - 1.0)*expTerm + ((freqR - freqA)/freqR)*expTermK; //AA
matrix[1] = freqC*(1.0 - expTerm); //AC
matrix[2] = freqG + freqG*(1.0/freqR - 1.0)*expTerm - (freqG/freqR)*expTermK; //AG
matrix[3] = freqT*(1.0 - expTerm); //AT
matrix[4] = freqA*(1.0 - expTerm); //CA
matrix[5] = freqC + freqC*(1.0/freqY - 1.0)*expTerm + ((freqY - freqC)/freqY)*expTermK; //CC
matrix[6] = freqG*(1.0 - expTerm); //CG
matrix[7] = freqT + freqT*(1.0/freqY - 1.0)*expTerm - (freqT/freqY)*expTermK; //CT
matrix[8] = freqA + freqA*(1.0/freqR - 1.0)*expTerm - (freqA/freqR)*expTermK; //GA
matrix[9] = freqC*(1.0 - expTerm); //GC
matrix[10] = freqG + freqG*(1.0/freqR - 1.0)*expTerm + ((freqR - freqG)/freqR)*expTermK; //GG
matrix[11] = freqT*(1.0 - expTerm); //GT
matrix[12] = freqA*(1.0 - expTerm); //TA
matrix[13] = freqC + freqC*(1.0/freqY - 1.0)*expTerm - (freqC/freqY)*expTermK; //TC
matrix[14] = freqG*(1.0 - expTerm); //TG
matrix[15] = freqT + freqT*(1.0/freqY - 1.0)*expTerm + ((freqY - freqT)/freqY)*expTermK; //TT
}
Paste the following lines before the F84
class:
@Description("F84 substitution model")
@Citation("Kishino, H., and M. Hasegawa. 1989. Evaluation of the maximum likelihood estimate of the evolutionary tree topologies from DNA sequence data, and the branching order in Hominoidea. Journal of Molecular Evolution 29:170-179.")
Import the Description
class from beast.base.core
.
Import the class for Citation
.
Create a new File
in MyPackage
called version.xml
and copy the following lines.
<package name="MyPackage" version="0.0.1">
<depends on='BEAST.base' atleast='2.7.0'/>
<depends on='BEAST.app' atleast='2.7.0'/>
<service type="beast.base.evolution.datatype.DataType">
<provider classname="beast.base.evolution.datatype.Nucleotide"/>
</service>
<service type="beast.base.core.BEASTInterface">
<provider classname="beast.base.evolution.substitutionmodel.F84"/>
<provider classname="beast.base.evolution.alignment.Alignment"/>
<provider classname="beast.base.evolution.alignment.Sequence"/>
<provider classname="beast.base.evolution.datatype.Nucleotide"/>
<provider classname="beast.base.evolution.sitemodel.SiteModel"/>
<provider classname="beast.base.evolution.tree.Tree"/>
<provider classname="beast.base.evolution.tree.TreeHeightLogger"/>
<provider classname="beast.base.evolution.tree.TreeParser"/>
<provider classname="beast.base.evolution.tree.coalescent.ConstantPopulation"/>
<provider classname="beast.base.evolution.tree.coalescent.RandomTree"/>
<provider classname="beast.base.inference.Logger"/>
<provider classname="beast.base.inference.CompoundDistribution"/>
<provider classname="beast.base.inference.MCMC"/>
<provider classname="beast.base.inference.Operator"/>
<provider classname="beast.base.inference.parameter.RealParameter"/>
<provider classname="beast.base.inference.distribution.Prior"/>
<provider classname="beast.base.inference.distribution.OneOnX"/>
<provider classname="beast.base.inference.util.ESS"/>
<provider classname="beast.base.evolution.likelihood.TreeLikelihood"/>
<provider classname="beast.base.evolution.operator.ScaleOperator"/>
<provider classname="beast.base.evolution.operator.SubtreeSlide"/>
<provider classname="beast.base.evolution.operator.Uniform"/>
<provider classname="beast.base.evolution.operator.Exchange"/>
<provider classname="beast.base.evolution.operator.WilsonBalding"/>
<provider classname="beast.base.evolution.substitutionmodel.Frequencies"/>
</service>
</package>
Create an examples
folder in the MyPackage directory.
Copy the testHKY.xml
from the examples folder from beast2
.
Right click the file and select Refactor > Rename
and rename the file to testF84.xml
Click the file and select Edit > Find > Replace in Files
, then select Scope
and set it to Current File
.
Replace all instances of HKY
with F84
, the replaces all instances of kappa
with kF84
.
Right click BeastMain
under BeastFX > src > beastfx.app > beast > BeastMain
to Modify Run Configuration
.
Set the classpath to -cp MyPackage
.
Add the following snippet to program arguments in debug: -version_file version.xml examples/testF84.xml
Create a folder called fxtemplates
in the MyPackage directory.
Inside the folder, create an xml file called F84-beauti-template.xml
copying in the following contents.
<beast version='2.0'
namespace='beast.app.beauti:beast.pkgmgmt:beast.base.core:beast.base.inference:beast.base.evolution.branchratemodel:beast.base.evolution.speciation:beast.base.evolution.tree.coalescent:beast.base.util:beast.base.math:beast.evolution.nuc:beast.base.evolution.operator:beast.base.inference.operator:beast.base.evolution.sitemodel:beast.base.evolution.substitutionmodel:beast.base.evolution.likelihood:beast.evolution:beast.base.inference.distribution'>
<mergewith point='substModelTemplates'>
<subtemplate id='F84' class='beast.base.evolution.substitutionmodel.F84' mainid='F84.s:$(n)'>
<![CDATA[
<plugin spec='F84' id='F84.s:$(n)'>
<parameter id="kF84.s:$(n)" name='kF84' value="2.0" lower="0.0" estimate='true'/>
<frequencies id='estimatedFreqs.s:$(n)' spec='Frequencies'>
<frequencies id='freqParameter.s:$(n)' spec='parameter.RealParameter' dimension='4' value='0.25' lower='0' upper='1'/>
</frequencies>
</plugin>
<operator id='kF84Scaler.s:$(n)' spec='kernel.BactrianScaleOperator' scaleFactor="0.1" weight="1" parameter="@kF84.s:$(n)"/>
<operator id='FrequenciesExchanger.s:$(n)' spec='kernel.BactrianDeltaExchangeOperator' delta="0.01" weight="0.1" parameter="@freqParameter.s:$(n)"/>
<prior id='kF84Prior.s:$(n)' x='@kF84.s:$(n)'>
<distr spec="LogNormalDistributionModel" meanInRealSpace='true'>
<parameter name='M' value="1.0" estimate='false'/>
<parameter name='S' value="1.25" estimate='false'/>
</distr>
</prior>
]]>
<connect srcID='kF84.s:$(n)' targetID='state' inputName='stateNode' if='inposterior(F84.s:$(n)) and kF84.s:$(n)/estimate=true'/>
<connect srcID='freqParameter.s:$(n)' targetID='state' inputName='stateNode' if='inposterior(F84.s:$(n)) and inposterior(freqParameter.s:$(n)) and freqParameter.s:$(n)/estimate=true'/>
<connect srcID='kF84Scaler.s:$(n)' targetID='mcmc' inputName='operator' if='inposterior(F84.s:$(n)) and kF84.s:$(n)/estimate=true'>Scale F84 transition-transversion parameter of partition $(n)</connect>
<connect srcID='FrequenciesExchanger.s:$(n)' targetID='mcmc' inputName='operator' if='inposterior(F84.s:$(n)) and inposterior(freqParameter.s:$(n)) and freqParameter.s:$(n)/estimate=true'>Exchange values of frequencies of partition $(n)</connect>
<connect srcID='kF84.s:$(n)' targetID='tracelog' inputName='log' if='inposterior(F84.s:$(n)) and kF84.s:$(n)/estimate=true'/>
<connect srcID='freqParameter.s:$(n)' targetID='tracelog' inputName='log' if='inposterior(F84.s:$(n)) and inposterior(freqParameter.s:$(n)) and freqParameter.s:$(n)/estimate=true'/>
<connect srcID='kF84Prior.s:$(n)' targetID='prior' inputName='distribution' if='inposterior(F84.s:$(n)) and kF84.s:$(n)/estimate=true'>F84 transition-transversion parameter of partition $(n)</connect>
</subtemplate>
</mergewith>
</beast>
In this next step we will run a beast2 package, treestat2, that is deployed through a a graphical user interface from the applauncher.
cd ~/intellij
git clone https://github.com/alexeid/TreeStat2
Import the TreeStat2 module and add the beast2 and BeastFX modules as depedencies.
File > Project Structure > Modules > + > Import Module > TreeStat2
Create a debug configuration for TreeStat2.
To run TreeStat2
, Run > Debug > TreeStatApp
.
To test the package for release, we use ant to compile it.
First, you need to create a build.xml file in the MyPackage directory with the following script.
<!-- Build F84 model -->
<project basedir="." default="build_jar_all_F84" name="BUILD_F84">
<description>
Build F84.
JUnit test is available for this build.
$Id: build_F84.xml $
</description>
<!-- set global properties for this build -->
<property name="srcF84" location="src" />
<property name="buildF84" location="build" />
<property name="libF84" location="lib" />
<property name="release_dir" value="release" />
<property name="distF84" location="${buildF84}/dist" />
<property name="beast2path" location="../beast2" />
<property name="libBeast2" location="${beast2path}/lib" />
<property name="srcBeast2" location="${beast2path}/src" />
<property name="beast2classpath" location="${beast2path}/build" />
<property name="Package_dir" value="${release_dir}/package" />
<import file="${beast2path}/build.xml" />
<property name="main_class_BEAST" value="beast.app.BeastMCMC" />
<property name="report" value="${buildF84}/junitreport"/>
<path id="classpath">
<pathelement path="${buildF84}"/>
<fileset dir="${libBeast2}" includes="junit-4.8.2.jar"/>
<pathelement path="${beast2classpath}"/>
</path>
<!-- start -->
<target name="initF84">
<echo message="${ant.project.name}: ${ant.file}" />
</target>
<target name="cleanF84">
<delete dir="${buildF84}" />
</target>
<!-- clean previous build, and then compile Java source code, and Juint test -->
<target name="build_all_F84" depends="cleanF84,compile-allF84,junitF84" description="Clean and Build all run-time stuff">
</target>
<!-- clean previous build, compile Java source code, and Junit test, and make the beast.jar and beauti.jar -->
<target name="build_jar_all_F84" depends="cleanF84,compile-allF84,junitF84" description="Clean and Build all run-time stuff">
</target>
<!-- No JUnit Test, clean previous build, compile Java source code, and make the F84.jar and beauti.jar -->
<target name="build_jar_all_F84_NoJUnitTest" depends="cleanF84,compile-allF84" description="Clean and Build all run-time stuff">
</target>
<!-- compile Java source code -->
<target name="compile-allF84" depends="initF84,compile-all">
<!-- Capture the path as a delimited property using the refid attribute -->
<property name="myclasspath" refid="classpath"/>
<!-- Emit the property to the ant console -->
<echo message="Classpath = ${myclasspath}"/>
<mkdir dir="${buildF84}" />
<!-- Compile the java code from ${srcF84} into ${buildF84} /bin -->
<javac srcdir="${srcF84}" destdir="${buildF84}" classpathref="classpath" fork="true" memoryinitialsize="256m" memorymaximumsize="256m">
<include name="mypackage/**/**" />
<!-- compile JUnit test classes -->
<include name="test/mypackage/**" />
</javac>
<echo message="Successfully compiled." />
</target>
<jar jarfile="${distF84}/F84.src.jar">
<fileset dir="${srcF84}">
<include name="mypackage/**/*.java" />
<include name="mypackage/**/*.png" />
<include name="mypackage/**/*.xsl" />
</fileset>
</jar>
<jar jarfile="${dist}/F84.package.jar">
<manifest>
<attribute name="Built-By" value="${user.name}" />
</manifest>
<fileset dir="${buildF84}">
<include name="mypackage/**/*.class" />
<include name="mypackage/**/*.png" />
<include name="mypackage/**/*.properties" />
</fileset>
</jar>
<!-- run beast.jar -->
<target name="run_F84">
<java jar="${distF84}/F84.jar" fork="true" />
</target>
<!-- JUnit test -->
<target name="junitF84">
<mkdir dir="${report}" />
<junit printsummary="yes"> <!--showoutput='yes'-->
<classpath>
<path refid="classpath" />
<path location="${buildF84}" />
</classpath>
<formatter type="xml" />
<batchtest fork="yes" todir="${report}">
<fileset dir="${srcF84}">
<include name="test/**/*Test.java"/>
</fileset>
<fileset dir="${srcBeast2}">
<include name="test/beast/integration/**/*Test.java"/>
<exclude name="test/beast/integration/**/ResumeTest.java"/>
</fileset>
</batchtest>
</junit>
<echo message="JUnit test finished." />
</target>
<target name="junitreport">
<junitreport todir="${report}">
<fileset dir="${report}" includes="*.xml"/>
<report format="frames" todir="${report}"/>
</junitreport>
<echo message="JUnit test report finished." />
</target>
<target name="package"
depends="build_jar_all_F84_NoJUnitTest"
description="release BEAST 2 package version of F84">
<delete dir="${Package_dir}" />
<!-- Create the release directory -->
<mkdir dir="${Package_dir}" />
<mkdir dir="${Package_dir}/lib" />
<mkdir dir="${Package_dir}/fxtemplates" />
<copy todir="${Package_dir}/lib">
<fileset dir="${dist}" includes="F84.package.jar" />
</copy>
<copy todir="${Package_dir}">
<fileset dir="${dist}" includes="F84.src.jar" />
</copy>
<copy todir="${Package_dir}/fxtemplates">
<fileset file="fxtemplates/F84-beauti-template.xml" />
</copy>
<jar jarfile="${dist}/F84.package.zip">
<fileset dir="${Package_dir}">
<include name="**/*" />
</fileset>
</jar>
<echo message="Package version release is finished." />
</target>
</project>
Now, you can compile by running the following lines in the MyPackage directory.
ant package
The package can be found in the directory: /beast2/build/dist/F84.package.zip.
Unzipping the package in ~/.beast/2.7/ allows it to be deployed in BEAST2.