Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update for using locally stored pretrained weights in the CI for torch hub models #3541

Merged
merged 6 commits into from
Jun 4, 2024
Merged

Update for using locally stored pretrained weights in the CI for torch hub models #3541

merged 6 commits into from
Jun 4, 2024

Conversation

yunchu
Copy link
Contributor

@yunchu yunchu commented May 27, 2024 •
edited
Loading

Summary

While downloading a model through torch.hub, HTTP error 403 raised on e2e tests for releases/2.0.0. (https://github.com/openvinotoolkit/training_extensions/actions/runs/9247351524/job/25440736054)
To avoid this issue, update torch.hub.load() call to use local source when passed data storage path through CI_DATA_ROOT environment variable for the CI test runs.

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have ran e2e tests and there is no issues.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@yunchu yunchu marked this pull request as ready for review May 27, 2024 07:08
@sovrasov
Copy link
Contributor

Do these changes mean that DinoV2 pretrained weights downloading can be silently skipped?

@vinnamkim
Copy link
Contributor

vinnamkim commented May 27, 2024 •
edited
Loading

@yunchu

Basically this problem should be solved by introducing a model repository mirror. We must upload the pretrained weights we use into the private mirror server and OTX code should download the model weights from it if it is in the test environment, not downloading from the original source, e.g. torch.hub.

I recommend you to escalate this issue to @MarkByun and make a schedule for this improvement.

@yunchu yunchu changed the title add skip_varification=True option to torch.hub.load() Update for using locally stored pretrained weights for torch hub models May 29, 2024
@yunchu yunchu changed the title Update for using locally stored pretrained weights for torch hub models Update for using locally stored pretrained weights in the testing for torch hub models May 29, 2024
@yunchu yunchu changed the title Update for using locally stored pretrained weights in the testing for torch hub models Update for using locally stored pretrained weights in the CI for torch hub models May 29, 2024
harimkang
harimkang previously approved these changes May 30, 2024
View reviewed changes
goodsong81
goodsong81 previously approved these changes May 30, 2024
View reviewed changes
vinnamkim
vinnamkim previously approved these changes May 30, 2024
View reviewed changes
@yunchu yunchu dismissed stale reviews from goodsong81 and harimkang via 181c1db May 30, 2024 11:19
harimkang
harimkang previously approved these changes May 31, 2024
View reviewed changes
eunwoosh
eunwoosh reviewed Jun 4, 2024
View reviewed changes
Copy link
Contributor

@eunwoosh eunwoosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left minor comments. Could you check it?

src/otx/algo/classification/dino_v2.py Outdated Show resolved Hide resolved
src/otx/algo/segmentation/backbones/dinov2.py Outdated Show resolved Hide resolved
@yunchu yunchu merged commit f862ae7 into openvinotoolkit:releases/2.0.0 Jun 4, 2024
13 checks passed
harimkang added a commit that referenced this pull request Jun 19, 2024
@yunchu @harimkang
@eunwoosh @kprokofi @sungchul2 @ashwinvaidya17 @eugene123tw @vinnamkim @wonjuleee
Mergeback 2.0.0rc0 to develop (#3583)
340ea62 
* Remove LITMODULE_PER_TASK

* Disable Resnext101_ATSS model on XPU (#3514)

* diable resnext101_atss on XPU

* give detailed xpu device info to perf tag

* revert debug code

---------

Co-authored-by: kirill prokofiev <kirill.prokofiev@intel.com>

* Remove invalid perf benchmark reference history for v2.0.0 (#3517)

* Fix a bug that dino_v2 model can't be run w/ HPO (#3518)

* add reduce function to dino backbone

* add unit test

* update integration test

* change name

* Fix detection export performance degradation (#3520)

* Fix to use right index

* Align forward by adding missed parts

* Revert metrics threshold (#3528)

Revert threshold

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* Tile with full img optional (#3530)

* make tile dataset with full image optional

* Add a feature to adapt max value of HPO batch size search space (#3532)

* implement adaptive max value of bs search space

* implement unit test

* Remove duplicates in get_idx_list_per_classes (#3537)

* Remove duplicates in get_idx_list_per_classes

* Reduce time complex

* Revisit Docker image build script (#3536)

* Revisit

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Fix test

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

---------

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Bump datumaro version to 1.6.1 (#3535)

* Add F1 metric computation during detection tasks (#3539)

* add f1 metric during training for detection

* add f1 metric during training for detection

* add a new metric

* remove configure_metric in det model

* Update doctstring for MeanAveragePrecisionFMeasure

Co-authored-by: Vinnam Kim <vinnam.kim@gmail.com>

* Trigger Build

* resolve precommit error

---------

Co-authored-by: Vinnam Kim <vinnam.kim@gmail.com>

* Fix yolox export perf degradation (#3534)

* Add `Focus` export pipeline
* Update `update_ov_subset_pipeline` to update `image_color_channel`

* Fix MaskRCNN IR Accuracy Drop (#3540)

fix ir maskrcnn accuracy drop

* Hotfix/geti integration (#3543)

* Fix

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Fix

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

---------

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Fix Anomaly OV export flag (#3558)

swap_rgb to False

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* Fix accuracy drop in MaskRCNN (#3562)

* fix maskrcnn accuracy drop

* pad to square

* remove unnecessary changes

* remove unnecessary changes

* Update codeql workflow to generate a report (#3559)

* Fix & Refine HPO (#3565)

* refine engine/hpo

* implement draft test code

* change replace to shutil.copy

* use same initial weight during HPO

* implement unit test

* search_space support Path

* move Path out of TYPE_CHECKING

* Fix Optimize in Anomaly Task (#3561)

* use val_dataloader

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>

* Apply suggestions from code review

Co-authored-by: Harim Kang <harim.kang@intel.com>

---------

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>
Co-authored-by: Harim Kang <harim.kang@intel.com>

* Update dependencies (#3570)

* Bump anomalib from 1.0.1 to 1.1.0 (#3572)

Bump anomalib version to 1.1.0

* Fix F1 instance seg accuracy drop (#3578)

* add MaskRLEMeanAPFMeasureCallable

* update recipes

* format

* Update for using locally stored pretrained weights in the CI for torch hub models (#3541)

* apply changed arg of OTXModel.export() to YOLOX model

* update dinov2 pretrained weights loading

* skip yolox-tiny-tile xai test

* make semi-sl dino support pickle

---------

Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com>
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Co-authored-by: Kang, Harim <harim.kang@intel.com>
Co-authored-by: Eunwoo Shin <eunwoo.shin@intel.com>
Co-authored-by: kirill prokofiev <kirill.prokofiev@intel.com>
Co-authored-by: Kim, Sungchul <sungchul.kim@intel.com>
Co-authored-by: Ashwin Vaidya <ashwin.vaidya@intel.com>
Co-authored-by: Eugene Liu <eugene.liu@intel.com>
Co-authored-by: Vinnam Kim <vinnam.kim@intel.com>
Co-authored-by: Wonju Lee <wonju.lee@intel.com>
Co-authored-by: Vinnam Kim <vinnam.kim@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vinnamkim vinnamkim vinnamkim approved these changes

@eunwoosh eunwoosh eunwoosh approved these changes

@goodsong81 goodsong81 goodsong81 left review comments

@harimkang harimkang harimkang left review comments

@samet-akcay samet-akcay Awaiting requested review from samet-akcay samet-akcay is a code owner

@jaegukhyun jaegukhyun Awaiting requested review from jaegukhyun jaegukhyun is a code owner

@eugene123tw eugene123tw Awaiting requested review from eugene123tw eugene123tw is a code owner

@kprokofi kprokofi Awaiting requested review from kprokofi kprokofi is a code owner

@chuneuny-emily chuneuny-emily Awaiting requested review from chuneuny-emily chuneuny-emily is a code owner

@sovrasov sovrasov Awaiting requested review from sovrasov sovrasov is a code owner

@sungchul2 sungchul2 Awaiting requested review from sungchul2 sungchul2 is a code owner

@GalyaZalesskaya GalyaZalesskaya Awaiting requested review from GalyaZalesskaya GalyaZalesskaya is a code owner

@negvet negvet Awaiting requested review from negvet negvet is a code owner

@wonjuleee wonjuleee Awaiting requested review from wonjuleee wonjuleee is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@yunchu @sovrasov @vinnamkim @goodsong81 @harimkang @eunwoosh