Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the Anomaly Task to work & Add integration test for Anomaly #3007

Merged
merged 14 commits into from
Mar 13, 2024

Conversation

harimkang
Copy link
Contributor

@harimkang harimkang commented Feb 29, 2024

Summary

Modify the Anomaly Task so that the integration Test works.

  • Installation
    Add a dependency of anomalib[core] for Anomalib to work properly.
    image

This PR will be opened once anomalib1.0 is working with torchmetrics==1.3.1.

For now, install based on the hot-fix commit on the anomalib side.
Add hazelnut dataset for anomaly integration test

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@github-actions github-actions bot added TEST Any changes in tests BUILD labels Feb 29, 2024
@harimkang harimkang added the OTX 2.0 For OTX v2.0 label Feb 29, 2024
@github-actions github-actions bot removed the OTX 2.0 For OTX v2.0 label Mar 5, 2024
@github-actions github-actions bot added DEPENDENCY Any changes in any dependencies (new dep or its version) should be produced via Change Request on PM OTX 2.0 For OTX v2.0 labels Mar 7, 2024
@harimkang harimkang changed the title Add integration test for Anomaly Task Modify the Anomaly Task so that the integration Test works Mar 7, 2024
Copy link
Contributor

@ashwinvaidya17 ashwinvaidya17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.
Let's see how we can sort out TorchMetrics PrecisionRecallCurve issue.

tests/integration/cli/test_cli.py Outdated Show resolved Hide resolved
src/otx/algo/anomaly/stfpm.py Show resolved Hide resolved
@ashwinvaidya17 ashwinvaidya17 mentioned this pull request Mar 7, 2024
8 tasks
Copy link

codecov bot commented Mar 7, 2024

Codecov Report

Attention: Patch coverage is 6.25000% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 61.93%. Comparing base (50a4d17) to head (d4fc83c).

Files Patch % Lines
...otx/core/model/module/anomaly/anomaly_lightning.py 0.00% 10 Missing ⚠️
src/otx/algo/anomaly/padim.py 0.00% 1 Missing ⚠️
src/otx/algo/anomaly/stfpm.py 0.00% 1 Missing ⚠️
src/otx/core/data/entity/anomaly/classification.py 0.00% 1 Missing ⚠️
src/otx/core/data/entity/anomaly/detection.py 0.00% 1 Missing ⚠️
src/otx/core/data/entity/anomaly/segmentation.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##               v2    #3007      +/-   ##
==========================================
- Coverage   61.96%   61.93%   -0.03%     
==========================================
  Files         259      259              
  Lines       15167    15174       +7     
==========================================
  Hits         9398     9398              
- Misses       5769     5776       +7     
Flag Coverage Δ
py310 61.92% <6.25%> (-0.04%) ⬇️
py311 61.93% <6.25%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ashwinvaidya17 ashwinvaidya17 mentioned this pull request Mar 12, 2024
8 tasks
@ashwinvaidya17
Copy link
Contributor

@harimkang I checked out your branch and added the changes from #3057. Except for the OpenVINO tests, the rest of the integration tests are passing. Maybe you can merge the changes from that PR to yours

image

@harimkang harimkang marked this pull request as ready for review March 12, 2024 13:51
ashwinvaidya17
ashwinvaidya17 previously approved these changes Mar 12, 2024
Copy link
Contributor

@ashwinvaidya17 ashwinvaidya17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the efforts!

@harimkang harimkang added this to the 2.0.0 milestone Mar 13, 2024
@harimkang
Copy link
Contributor Author

harimkang commented Mar 13, 2024

OV-related features will follow in this PR (#3057), and tests will be added in that PR.

@harimkang harimkang enabled auto-merge (squash) March 13, 2024 00:23
@harimkang harimkang changed the title Modify the Anomaly Task so that the integration Test works Fix the Anomaly Task to work & Add integration test for Anomaly Mar 13, 2024
@harimkang harimkang merged commit a616ce9 into openvinotoolkit:v2 Mar 13, 2024
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BUILD DEPENDENCY Any changes in any dependencies (new dep or its version) should be produced via Change Request on PM OTX 2.0 For OTX v2.0 TEST Any changes in tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants