Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snakemake draft #41

Merged
merged 4 commits into from
Dec 16, 2022
Merged

Snakemake draft #41

merged 4 commits into from
Dec 16, 2022

Conversation

sebastian-luna-valero
Copy link
Contributor

I was playing a bit with https://github.com/c-scale-community/c-scale-tutorial-snakemake

Then decided to create this draft for a snakemake workflow for HiSea.

Regarding #40, Snakemake can also help with job submission to SLURM:
https://snakemake.readthedocs.io/en/stable/executing/cluster.html

@sebastian-luna-valero
Copy link
Contributor Author

Sorry, forgot to add instructions for testing:

# install snakemake
mkdir working-dir
cd working-dir
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -p conda-install
source conda-install/etc/profile.d/conda.sh
conda install mamba -c conda-forge --yes
mamba create -c bioconda -c conda-forge -n snakemake snakemake-minimal --yes
conda activate snakemake

# run workflow
git clone https://github.com/c-scale-community/use-case-hisea.git
cd use-case-hisea/
git checkout --track origin/snakemake
snakemake -c1 pre_processing_boundary --use-conda --conda-frontend mamba

@sebastian-luna-valero
Copy link
Contributor Author

Fixed now the snakemake rule for pre_processing_boundary

Note that I had to manually execute:

mkdir -p data/output
mkdir -p data/output/DFMWAQ_input/

@sebastian-luna-valero
Copy link
Contributor Author

@backeb @lorincmeszaros I fixed the issue with pre_processing_boundary.

I think the best next step is for us three to have a dedicated meeting to move this forward with new steps for the workflow (if you still want to go down the snakemake route)

@backeb
Copy link
Contributor

backeb commented Dec 14, 2022

I've forwarded the meeting for Friday morning, where Lőrinc and I are planning to work on this.

#make the /data/tmp directory if it does not exist
Path('/data/cmems/tmp').mkdir(parents=True, exist_ok=True)
#make the data/tmp directory if it does not exist
Path('data/cmems/tmp').mkdir(parents=True, exist_ok=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sebastian-luna-valero I think we change these it will affect running these scripts from docker...

@backeb backeb merged commit a9bb406 into main Dec 16, 2022
@backeb backeb deleted the snakemake branch December 16, 2022 09:40
@sebastian-luna-valero sebastian-luna-valero restored the snakemake branch December 16, 2022 10:04
@backeb backeb deleted the snakemake branch December 16, 2022 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants