Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added better request error handling #26

Merged
merged 7 commits into from
May 14, 2024
Merged

Added better request error handling #26

merged 7 commits into from
May 14, 2024

Conversation

itrujnara
Copy link
Collaborator

Closes #22

  • added a custom function to utils that throws a custom error if request fails before response
  • switched all request scripts to use that function
  • added retries to all network processes in config

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • Make sure your code lints (nf-core lint). Except some template files cause old branch
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>). It doesn't but I'll still try CI.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 13, 2024 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 1bcc0a9

+| ✅ 216 tests passed       |+
!| ❗  13 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: conf/igenomes.config
  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
  • pipeline_todos - TODO string in ci.yml: You can customise CI pipeline run tests as required
  • pipeline_todos - TODO string in ci.yml: You can customise CI pipeline run tests as required
  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline

✅ Tests passed:

Run details

  • nf-core/tools version 2.14.1
  • Run at 2024-05-14 08:47:26

@itrujnara itrujnara marked this pull request as ready for review May 13, 2024 12:34
JoseEspinosa
JoseEspinosa requested changes May 13, 2024
View reviewed changes
Copy link
Member

@JoseEspinosa JoseEspinosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small suggestion and a question 😄

bin/fetch_afdb_structures.py Outdated Show resolved Hide resolved
conf/modules.config Outdated
@@ -28,6 +28,8 @@ process {
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
errorStrategy = 'retry'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't we catch which error numbers to retry?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, can we? I have to check in the docs

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

JoseEspinosa
JoseEspinosa approved these changes May 14, 2024
View reviewed changes
Copy link
Member

@JoseEspinosa JoseEspinosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome job!
Thanks

@JoseEspinosa JoseEspinosa merged commit 22f1f7d into nf-core:dev May 14, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JoseEspinosa JoseEspinosa JoseEspinosa approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add request error handling in Python scripts
2 participants
@itrujnara @JoseEspinosa