Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sylvia whittle/remove scars #390

Merged
merged 55 commits into from
Jan 13, 2023
Merged

Sylvia whittle/remove scars #390

merged 55 commits into from
Jan 13, 2023

Conversation

SylviaWhittle
Copy link
Collaborator

@SylviaWhittle SylviaWhittle commented Dec 6, 2022

Closes #202
Closes #154

This pull request adds the long awaited scar removal feature to TopoStats. Configuration for the scar removal has been added in the config file, and good/useful defaults have been set for the parameters.

The scar removal is done by the new Scars class. Much like with the Filters class, its instantiation takes an image and the required parameters, then one can call the remove_scars() class method on the instance. remove_scars() is the only method intended to be used externally to the class, though it is written in a way that you could call other class methods if you really wanted to.

There is a config option to turn off the scar removal if wanted.

This scar removal has been deemed to be of good enough quality, and it has been tested on a reasonable sample of scarred images, however with more usage, bugs may still be found.

An example of what the scar removal in action:

With scars:
image

Scars removed:
image

@codecov-commenter
Copy link

codecov-commenter commented Dec 8, 2022

Codecov Report

Base: 76.88% // Head: 80.07% // Increases project coverage by +3.19% 🎉

Coverage data is based on head (f039bcd) compared to base (4ca0e43).
Patch coverage: 98.11% of modified lines in pull request are covered.

❗ Current head f039bcd differs from pull request most recent head ffca8fa. Consider uploading reports for the commit ffca8fa to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #390      +/-   ##
==========================================
+ Coverage   76.88%   80.07%   +3.19%     
==========================================
  Files          32       36       +4     
  Lines        3638     3915     +277     
==========================================
+ Hits         2797     3135     +338     
+ Misses        841      780      -61     
Impacted Files Coverage Δ
topostats/run_topostats.py 86.43% <ø> (+28.43%) ⬆️
topostats/validation.py 100.00% <ø> (ø)
topostats/filters.py 88.09% <71.42%> (-2.09%) ⬇️
topostats/scars.py 99.04% <99.04%> (ø)
tests/test_scars.py 100.00% <100.00%> (ø)
tests/tracing/test_dnacurvature.py 100.00% <100.00%> (ø)
topostats/tracing/dnacurvature.py 100.00% <100.00%> (ø)
tests/test_validation.py 100.00% <0.00%> (ø)
tests/test_run_topostats.py 100.00% <0.00%> (ø)
... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Collaborator

@MaxGamill-Sheffield MaxGamill-Sheffield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work! Results look great!

The one thing most of my comments are about is whether this should be a separate class in filters.py, just because this is going to be used in the filters workflow and it seems a unusual to have a separate config block for this where it could be incorporated into the filters block and maybe a scar_removal subsection.

topostats/default_config.yaml Outdated Show resolved Hide resolved
tests/conftest.py Outdated Show resolved Hide resolved
topostats/filters.py Outdated Show resolved Hide resolved
topostats/run_topostats.py Outdated Show resolved Hide resolved
tests/conftest.py Outdated Show resolved Hide resolved
tests/test_scars.py Outdated Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
tests/test_scars.py Outdated Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
@SylviaWhittle
Copy link
Collaborator Author

SylviaWhittle commented Jan 5, 2023

I have removed the Scars class from scars.py, and now it is just a module with a set of functions, one externally-accessible function, remove_scars, and several internal functions, not intended for external use.

I have also added a bit more documentation, though if anything isn't clear, please do say.

With regards to using the .npy files to store the test input/output arrays, as @MaxGamill-Sheffield rightfully asks in a previous comment on this PR, do we think we should prefer putting them in the code, as a large array, or as .npy files like I have done?

If I haven't adequately addressed a previous concern, please do re-raise it.

Copy link
Collaborator

@ns-rse ns-rse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @SylviaWhittle saves the overhead of instantiating a class where most methods were @staticmethod and keeps things simple.

Most of my comments pertain to where configuration is being defined and the logic to control the running of scar removal.

The configuration doesn't have to be nested, but it makes things a bit simpler in my view as we wouldn't have to subset those out from the configuration dictionary and pass it around, they're a key/value within config_filter. This removes a few lines of code from run_topostats.py (keep code bloat down is good in my view).

tests/conftest.py Outdated Show resolved Hide resolved
topostats/default_config.yaml Outdated Show resolved Hide resolved
topostats/run_topostats.py Outdated Show resolved Hide resolved
topostats/run_topostats.py Outdated Show resolved Hide resolved
topostats/filters.py Outdated Show resolved Hide resolved
topostats/scars.py Show resolved Hide resolved
topostats/scars.py Show resolved Hide resolved
topostats/scars.py Outdated Show resolved Hide resolved
tests/test_scars.py Show resolved Hide resolved
tests/test_scars.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@ns-rse ns-rse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I've understood this correctly but I think the logic here can be simplified/clearer.

topostats/scars.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@ns-rse ns-rse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor question about pylint.

topostats/filters.py Show resolved Hide resolved
Jean-Du added a commit that referenced this pull request Jan 12, 2023
Copy link
Collaborator

@ns-rse ns-rse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for not getting back to this sooner.

Looks good to go.

@SylviaWhittle SylviaWhittle dismissed MaxGamill-Sheffield’s stale review January 13, 2023 18:07

Requested changes no longer relevant due to refactor away from using the Scars class.

@SylviaWhittle SylviaWhittle merged commit 0d9a399 into main Jan 13, 2023
@SylviaWhittle SylviaWhittle deleted the SylviaWhittle/remove_scars branch January 13, 2023 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove scarring Add grain removal based on height (identify and isolate scars)
4 participants