Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cellranger vdj #289

Merged
merged 38 commits into from Mar 7, 2024
Merged

Cellranger vdj #289

merged 38 commits into from Mar 7, 2024

Conversation

mapo9
Copy link
Contributor

@mapo9 mapo9 commented Dec 7, 2023 •
edited

Hi :),

I added a new subworkflow that supports raw input from single-cell seq.
Currently, it only works for 10X genomics data but I plan to add more kits of course.

To do so, I added a new --library_generation method and a parameter to add the required reference.

I will also figure out whether the reference can be added to igenomes.
Also, I would like to add the cellranger_mkvdjref module as well, so that users can generate their own references.

I didnt yet test if the branch alters the results of the bulk workflow (what it shouldnt because it runs completely agnostic to it), but it would be great if you could use your dev dataset to test this or if you could quickly guide me to a suitable test dataset.

It would be awesome to get a review on the status of integration of the new "subway-line" of single-cell raw data analyis and to start a discussion on integrating this tools into the pipeline.

PR checklist

  • [ x] This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/airrflow branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 7, 2023

Python linting (black) is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:

  • Install black: pip install black
  • Fix formatting errors in your pipeline: black .

Once you push these changes the test should pass, and you can hide this comment 👍

We highly recommend setting up Black in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!

Thanks again for your contribution!

@mapo9 mapo9 requested a review from ggabernet December 7, 2023 15:41
@mapo9 mapo9 requested review from ggabernet and removed request for ggabernet December 15, 2023 14:14
@ggabernet
Copy link
Member

The implementation looks great 🚀 , and sorry it has taken me so long to review! There's a couple of things that make the workflow fail like the samplesheet validation, that I can also help with fixing

@ggabernet
Copy link
Member

We should add tests as well, if the cellranger VDJ nf-core module has test data, then maybe this one is already sufficient

ggabernet
ggabernet reviewed Feb 8, 2024
View reviewed changes
Copy link
Member

@ggabernet ggabernet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just got some comments

nextflow_schema.json Outdated Show resolved Hide resolved
nextflow_schema.json Outdated Show resolved Hide resolved
subworkflows/local/repertoire_analysis_reporting.nf Outdated Show resolved Hide resolved
subworkflows/local/sc_raw_input.nf Outdated Show resolved Hide resolved
workflows/airrflow.nf Outdated Show resolved Hide resolved
workflows/airrflow.nf Outdated Show resolved Hide resolved
workflows/airrflow.nf Outdated Show resolved Hide resolved
subworkflows/local/sc_raw_input.nf Outdated Show resolved Hide resolved
mapo9 and others added 8 commits February 8, 2024 15:53
@mapo9 @ggabernet
Update nextflow_schema.json
f8b6a0f 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9 @ggabernet
Update nextflow_schema.json
747d4ba 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9 @ggabernet
Update subworkflows/local/repertoire_analysis_reporting.nf
d052124 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9 @ggabernet
Update subworkflows/local/sc_raw_input.nf
1adb1a7 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9 @ggabernet
Update workflows/airrflow.nf
d8cee70 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9
single cell based on parameter
40a53d6
@mapo9
fixed samplesheet_val
75d05f7
@mapo9
linting
90a2ef4
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 9, 2024 •
edited

nf-core lint overall result: Failed ❌

Posted for pipeline commit 184d559

+| ✅ 194 tests passed       |+
#| ❔   7 tests were ignored |#
-| ❌  10 tests failed       |-

❌ Test failures:

  • files_exist - File must be removed: lib/Utils.groovy
  • files_exist - File must be removed: lib/WorkflowMain.groovy
  • files_exist - File must be removed: lib/NfcoreTemplate.groovy
  • files_exist - File must be removed: lib/WorkflowAirrflow.groovy
  • files_unchanged - .github/CONTRIBUTING.md does not match the template
  • files_unchanged - .github/PULL_REQUEST_TEMPLATE.md does not match the template
  • files_unchanged - .github/workflows/branch.yml does not match the template
  • files_unchanged - .github/workflows/linting_comment.yml does not match the template
  • files_unchanged - .github/workflows/linting.yml does not match the template
  • files_unchanged - pyproject.toml does not match the template

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 2.13.1
  • Run at 2024-03-06 09:42:15

@mapo9
Copy link
Contributor Author

mapo9 commented Feb 14, 2024 •
edited

I implemented the changes and am working on making the tests run now.
While I was able to solve the main problem, I don't really know what to make out of the remaining error.
I also cant reproduce the error locally where everything runs through fine. Also when specifying the nextflow version locally it runs through fine locally.

Maybe you can make more out of the error and point me towards where things go wrong?

ERROR ~ Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:DEFINE_CLONES_COMPUTE (Patient2)'
Caused by:
Process NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:DEFINE_CLONES_COMPUTE (Patient2) terminated with an error exit status (1)
Command executed:
Rscript -e "enchantr::enchantr_report('define_clones',
report_params=list('input'='Sample5_meta-pass_Sample5_collapse-pass.tsv,Sample6_meta-pass_Sample6_collapse-pass.tsv,Sample7_meta-pass_Sample7_collapse-pass.tsv,Sample8_meta-pass_Sample8_collapse-pass.tsv',
'imgt_db'='imgtdb_base',
'species'='auto',
'cloneby'='subject_id',
'outputby'='subject_id',
'force'=FALSE,
'threshold'=0.2370449,
'singlecell'='single_cell','outdir'=getwd(),
'nproc'=2,
'log'='Patient2_clone_command_log' ,'linkage'='single','method'='nt','min_n'=30,'model'='hierarchical','outname'='','skip_convergence'='true'))"

  cp -r enchantr Patient2_clone_report && rm -rf enchantr

  echo "NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:DEFINE_CLONES_COMPUTE": > versions.yml
  Rscript -e "cat(paste0('  enchantr: ',packageVersion('enchantr'),'

'))" >> versions.yml
Command exit status:
1
Command output:
(empty)
Command error:
Error in knit_params_get(input_lines, params) :
render params not declared in YAML: skip_convergence
Calls: ... render_cur_session -> -> knit_params_get
In addition: Warning messages:
1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr'
2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr'
3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr'
Execution halted

@mapo9 mapo9 requested a review from ggabernet February 14, 2024 14:39
@ggabernet ggabernet changed the base branch from dev to master February 15, 2024 23:20
@ggabernet ggabernet changed the base branch from master to dev February 15, 2024 23:20
@ggabernet
Copy link
Member

I see these tests that are failing are running the Immcantation container, we use this for some internal checks, but don't worry if they fail. I tried to make them optional, let's see if that worked

ggabernet
ggabernet approved these changes Feb 27, 2024
View reviewed changes
Copy link
Member

@ggabernet ggabernet left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Just had one comment left

modules/local/cellranger/mkvdjref.nf Outdated Show resolved Hide resolved
@mapo9
Copy link
Contributor Author

mapo9 commented Feb 29, 2024

I finally found time to add the docs.
Would be great if you could have a look again.

@mapo9 mapo9 requested a review from ggabernet February 29, 2024 07:56
ggabernet
ggabernet approved these changes Mar 6, 2024
View reviewed changes
Copy link
Member

@ggabernet ggabernet left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! I just had some minor comments on the docs organization. The prettier tests are also failing, but if you don't know how to fix them I'll do it in the next PR

CHANGELOG.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
mapo9 and others added 5 commits March 6, 2024 08:20
@mapo9 @ggabernet
Update README.md
cdef84f 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9 @ggabernet
Update CHANGELOG.md
ec048d9 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9 @ggabernet
Update README.md
6f0844b 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9 @ggabernet
Update README.md
575b41d 
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@mapo9
prettier
184d559
@ggabernet
Copy link
Member

we'll do a template update on the next PR :)

@ggabernet ggabernet merged commit 63b9692 into dev Mar 7, 2024
21 of 28 checks passed
@mapo9 mapo9 deleted the cellranger_vdj branch March 7, 2024 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ggabernet ggabernet ggabernet approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@mapo9 @ggabernet