Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing setup #6

Merged
merged 9 commits into from
Aug 2, 2022
Merged

Testing setup #6

merged 9 commits into from
Aug 2, 2022

Conversation

susan-garry
Copy link
Contributor

This PR does a couple of things:

  • depth.py now parses command line arguments and can accept an optional argument specifying a file which in turn specifies a subset of paths in the graph. Node depth will only be computed with respect to these paths; other paths will not be incuded in the tally.
  • Add a few very simple test cases for testing node depth over a subset of paths
  • Set up testing functionality for node depth over a subset of paths. I added a subdirectory in test called subset-paths which contains .paths files. Each .path file specifies a subset of paths for some .og file. Additionally, for each {base}.paths file there is a {base}.txt file which specifies the .og file which {base}.paths corresponds to. For example, there is a file called ex1.paths, so there is a correspond file ex1.txt which simply contains the line ARGS: ../basic/ex1.og, specifying that ex1.paths lists a subset of the paths in ../basic/ex1.og. (I cannot put comments in the .paths files because of how odgi parses them. :( )
  • Wrangle with the Makefile and try to keep it updated and clean

This approach to testing, while requiring these .txt files, allows us to keep all of the .paths files in the same directory even though the .gfa files may be in different directories, and it allows us to generate multiple .paths files for the same .og files without duplicating the .og files and placing them in separate directories.

The Makefile is a bit... messy. The target make og is especially difficult for the human eye to parse. If you have time and feel like messing with Makefiles, I would especially appreciate if you could take a look and let me know how this could be done better!

@susan-garry susan-garry requested a review from sampsyo June 24, 2022 22:41
Copy link
Collaborator

@sampsyo sampsyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking very awesome!!! Here are a few comments throughout. I have a few Makefile suggestions, centered on its ability to do interesting pattern substitution with syntax that looks like $(VAR:%.foo=%.bar), which substitutes every element in the value of the variable VAR by looking for .foo and swapping that for .bar.

test/*.og
test/*.gfa
test/*.out
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checking: did you mean to ignore the .out files? The usual setup with Turnt is to keep them checked into the repository. That way, if someone else clones the repository, they can also verify that the output matches the expected result. (Of course, it's impractical if the output files are gigantic—do you know how big they are for the tests we currently have?)

Makefile Outdated Show resolved Hide resolved
Makefile Outdated
clean:
rm -rf test/*.gfa
test: og
-turnt --save --env baseline $(foreach file,$(TEST_FILES),test/$(file).gfa)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see! Possibly belay my comment above about ignoring the .out files. Used like this to automatically generate a baseline, perhaps we don't want to check in the expected output files at all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intuition was that it is more in keeping with the philosophy of differential testing to generate the .out files every time we run the tests, especially since we get some of our tests from the odgi repo, and these tests could theoretically change without us knowing about it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds great to me.

Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
depth.py Outdated Show resolved Hide resolved
depth.py Outdated Show resolved Hide resolved
depth.py Show resolved Hide resolved
parse_data.py Show resolved Hide resolved
test/subset-paths/turnt.toml Show resolved Hide resolved
@susan-garry susan-garry requested a review from sampsyo July 26, 2022 19:28
Copy link
Collaborator

@sampsyo sampsyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay; this all looks great!!! Thanks for bearing with the latency on this review, but this all looks good to go. Feel free to do the honors and push that green button!

@susan-garry susan-garry merged commit cecc6ee into main Aug 2, 2022
@susan-garry susan-garry deleted the testing-setup branch October 18, 2022 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants