Skip to content

Getting Started

Jump to bottom
Charlie Bayne edited this page Mar 30, 2026 · 2 revisions

Getting Started

Prerequisites and System Requirements

System Requirements

  • Operating System: Linux (required for Apptainer container support)
  • Hardware:
    • 64 GB+ RAM recommended
    • Multi-core processor (8+ cores recommended)
    • Substantial storage for raw data and databases (raw Astral MS files are ~7 GB each; reference databases can reach 50–200 GB depending on method)

Note: Conduit does not run on macOS. Windows is not recommended but may work without containers.

Required Software

  • Snakemake (≥7.0.0)
  • Apptainer/Singularity (≥1.1.0)

All other dependencies (R, Python, DIA-NN, MetaPhlAn, Bakta, eggNOG-mapper, etc.) are handled automatically via Apptainer containers.

Installation

1. Install Snakemake and Apptainer

conda create -n conduit
conda activate conduit
conda install -c bioconda snakemake
conda install -c bioconda apptainer

2. Clone the repository

git clone https://github.com/baynec2/conduit-ascent.git
cd conduit-ascent

Setting Up Your Experiment

3. Create an experiment directory

Create a directory under experiments/ with this structure:

experiments/your_experiment/
├── input/
│   ├── raw_files/            # Your Thermo .raw files
│   └── sample_annotation.txt # Sample metadata
└── config/
    └── snakemake.yaml        # Experiment configuration

Additional input files may be required depending on your search space method (see Search Space).

4. Prepare input files

Sample annotation file (input/sample_annotation.txt) — tab-delimited:

file	condition
sample1	control
sample2	treatment

The file column must match the .raw filenames (without extension) in input/raw_files/.

For ncbi_taxonomy_id method — input/ncbi_taxa_ids.txt:

ncbi_taxa_id
83333
9606

For uniprot_proteome_id method — input/proteome_ids.txt:

UP000000625
UP000005640

For metaphlan method — place FASTQ files in input/fastq_files/.

For MAGs method — place MAG FASTA files in input/MAG_files/.

5. Create a configuration file

Copy the template and fill in the required fields:

cp config/snakemake.yaml experiments/your_experiment/config/snakemake.yaml

At minimum, set:

experiment: your_experiment
run_name: run1
search_space_method: ncbi_taxonomy_id  # or whichever method fits your data

See the Configuration Reference in the README for all available parameters.

6. Run the workflow

snakemake --configfile experiments/your_experiment/config/snakemake.yaml --use-apptainer

Add --cores N to use multiple CPU cores (recommended).

Outputs

All outputs for a run are written to experiments/{experiment}/runs/{run_name}/:

Output Description
database_resources/database.fasta Protein FASTA used as the search space
database_resources/taxonomy.txt Taxonomy information for organisms in the database
database_resources/protein_info.txt Per-protein metadata
database_resources/database.predicted.speclib DIA-NN predicted spectral library
database_resources/README.md Summary metrics about the database
database_resources/detected_protein_resources/ Proteins detected by DIA-NN and their annotations
output_files/{experiment}_{run_name}_conduit.rds Final Conduit R object

The .rds file can be loaded into R with conduitR or opened with Conduit-GUI for interactive exploration.

Running multiple configurations

Each run is namespaced by run_name. You can run the same experiment with different search space methods or parameters by changing run_name in the config — outputs will not overwrite each other.

Running on HPC (Barnacle2 / SLURM)

These instructions are for Knight Lab members running Conduit on Barnacle2 via SLURM. They can be adapted to other SLURM-based HPC systems.

1. Login

ssh <username>@barnacle2.ucsd.edu

2. Clone the repository

git clone https://github.com/baynec2/conduit-ascent.git
cd conduit-ascent
mkdir slurm_out

3. Install Snakemake

Barnacle2 has Singularity available system-wide. Install Snakemake into the base environment — do not create a separate conda environment, as that will shadow the system Singularity.

singularity --version   # verify Singularity is available
pip install snakemake
pip install wheel
pip install datrie

4. Submit the job

Use the run_conduit_barnacle2.slurm script in the repository root. Update --mail-user with your email before submitting.

sbatch run_conduit_barnacle2.slurm
squeue --me           # check job status
cat slurm_out/*.err   # check Snakemake logs

Troubleshooting

NCBI API failures — If the ncbi_taxonomy_id method fails to fetch taxonomy, the NCBI API may be temporarily unavailable. Wait a few minutes and retry.

Configuration errors — Ensure your snakemake.yaml has values for all required fields: experiment, run_name, search_space_method, and sample_annotation.

File path errors — Verify that the file column in sample_annotation.txt matches the .raw filenames exactly (case-sensitive, no extension).

Container issues — Ensure Apptainer/Singularity is installed and accessible. On HPC systems this may differ from root-enabled personal Linux machines; see Dependency Management.

Memory issues — Increase available RAM (64 GB+ recommended) or reduce parallel jobs with --cores.

Network issues — Ensure internet access for UniProt/KEGG API access and database downloads. Check proxy settings if behind an institutional firewall.

For additional help, check the Issues page or open a new issue with your error message, configuration file, and relevant log from runs/{run_name}/logs/.

Clone this wiki locally