Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print usage when no argument is given, exit with aggregated test status #55

Merged
merged 4 commits into from
Jan 9, 2024

Conversation

nwiltsie
Copy link
Member

@nwiltsie nwiltsie commented Jan 3, 2024

Description

This fixes two interface annoyances with NFTest.

First, calling nftest with no arguments throws an error due to this line:

$ nftest
Traceback (most recent call last):
  File "/hot/code/nwiltsie/venvs/nftest-dev/bin/nftest", line 8, in <module>
    sys.exit(main())
  File "/hot/code/nwiltsie/tools/tool-NFTest/nftest/__main__.py", line 119, in main
    args.func(args)
AttributeError: 'Namespace' object has no attribute 'func'

03f49e1 changes that behavior to print the usage and exit with code 1:

$ nftest
usage: nftest [-h] [-V] {init,run} ...
$ echo $?
1

Second, as per #53 nftest run always exits with code 0, regardless of any test failures. After c526bf6 it now exits with the number of tests that failed to run (i.e. Nextflow threw an error like "ERROR ~ Unable to parse config file") or failed any assertions.

$ nftest run
...
2024-01-03 18:45:18,165 - NFTest - INFO - Beginning tests...
...
2024-01-03 18:45:27,289 - NFTest - ERROR -  [ failed ]
...
2024-01-03 18:45:35,986 - NFTest - ERROR -  [ failed ]
...
2024-01-03 18:45:45,400 - NFTest - ERROR -  [ failed ]
$ echo $?
3

Closes #53

Checklist

  • This PR does NOT contain Protected Health Information (PHI). A repo may need to be deleted if such data is uploaded.
    Disclosing PHI is a major problem1 - Even a small leak can be costly2.

  • This PR does NOT contain germline genetic data3, RNA-Seq, DNA methylation, microbiome or other molecular data4.

  • This PR does NOT contain other non-plain text files, such as: compressed files, images (e.g. .png, .jpeg), .pdf, .RData, .xlsx, .doc, .ppt, or other output files.

  To automatically exclude such files using a .gitignore file, see here for example.

  • I have read the code review guidelines and the code review best practice on GitHub check-list.

  • I have set up or verified the main branch protection rule following the github standards before opening this pull request.

  • The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].

  • I have added the major changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.

Footnotes

  1. UCLA Health reaches $7.5m settlement over 2015 breach of 4.5m patient records

  2. The average healthcare data breach costs $2.2 million, despite the majority of breaches releasing fewer than 500 records.

  3. Genetic information is considered PHI.
    Forensic assays can identify patients with as few as 21 SNPs

  4. RNA-Seq, DNA methylation, microbiome, or other molecular data can be used to predict genotypes (PHI) and reveal a patient's identity.

@nwiltsie nwiltsie changed the title Nwiltsie dev Print usage when no argument is given, exit with consolidated test status Jan 3, 2024
@nwiltsie nwiltsie changed the title Print usage when no argument is given, exit with consolidated test status Print usage when no argument is given, exit with aggregated test status Jan 3, 2024
@nwiltsie
Copy link
Member Author

nwiltsie commented Jan 4, 2024

Small bugfix with 22aa4ff - I was missing positive test results, so it was always reporting that every test failed! Fixed now.

(nftest-dev) [nwiltsie@F2-prod pipeline-align-DNA]$ nftest run
LOG: .env not found in /hot/code/nwiltsie/pipelines/pipeline-align-DNA.
LOG: .env not found in /mnt/exports/shared/home/nwiltsie.
WARN: unable to find .env. Default values will be used.
=========================================================================== NFTEST STARTS ============================================================================
2024-01-04 17:44:11,171 - NFTest - INFO - Beginning tests...
2024-01-04 17:44:11,171 - NFTest - INFO - a_mini_n2-spark: test development branch with a-mini-n2 and Spark's mark duplicates
2024-01-04 17:44:11,172 - NFTest - INFO - NXF_WORK=././test/work nextflow run ./main.nf -c /hot/code/nwiltsie/pipelines/pipeline-align-DNA/test/global.config -c ./test/a_mini_n2-spark.config --output_dir a_mini_n2-spark
2024-01-04 17:44:14,294 - NextFlow - INFO - N E X T F L O W  ~  version 23.04.2
2024-01-04 17:44:20,378 - NextFlow - INFO - ERROR ~ Unable to parse config file: '/hot/code/nwiltsie/pipelines/pipeline-align-DNA/./test/a_mini_n2-spark.config'
2024-01-04 17:44:20,477 - NextFlow - INFO -
2024-01-04 17:44:20,477 - NFTest - ERROR -  [ failed ]
2024-01-04 17:44:20,621 - NFTest - INFO - a_mini_n2-picard: test development branch with a-mini-n2 and Picard's mark duplicates
2024-01-04 17:44:20,621 - NFTest - INFO -  [ skipped ]
2024-01-04 17:44:20,643 - NFTest - INFO - a_mini_n2-picard_yaml: test development branch with a-mini-n2 and Picard's mark duplicates with a YAML config file
2024-01-04 17:44:20,643 - NFTest - INFO -  [ skipped ]
(nftest-dev) [nwiltsie@F2-prod pipeline-align-DNA]$ echo $?
1

Copy link
Contributor

@yashpatel6 yashpatel6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@nwiltsie nwiltsie merged commit 5960fb0 into main Jan 9, 2024
1 check passed
@nwiltsie nwiltsie deleted the nwiltsie_dev branch January 9, 2024 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Aggregated exit code
2 participants