Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add features to the YOLO model from the latest YOLO variants #817

Merged
merged 103 commits into from
May 29, 2023

Conversation

senarvi
Copy link
Contributor

@senarvi senarvi commented Mar 28, 2022

What does this PR do?

The YOLO model has been largely refactored, so that it's easy to incorporate new features from different YOLO variants, such as different algorithms for assigning targets to anchors. It also supports defining the network structure as a PyTorch module, in addition to loading a Darknet configuration file. The implementation now supports the most important features of YOLOv3, YOLOv4, YOLOv5, YOLOv7, Scaled-YOLOv4, and YOLOX.

  • Supports several new algorithms for matching targets to anchors.
  • Adds support for DIoU and CIoU losses.
  • By default the target for confidence of a detection that has been assigned a target is 1.0. This pull request adds a support for using the overlap between the target and the predicted box instead.
  • The code is refactored from one big class into model, anchor matching, and loss function classes.
  • Target class labels may be specified as a matrix of class probabilities, allowing multiple classes per object.
  • Automatic padding in convolutional and max pooling layers works now in every case.
  • Weight decay is applied only to convolutional layer weights.
  • Supports label smoothing.
  • Command line interface is now using LightningCLI.
  • Network architectures can be written as PyTorch modules. YOLOv4, YOLOv5, YOLOv7, and YOLOX architectures are included.
  • Supports deep supervision (auxiliary detection heads).
  • Calculates MAP metric using TorchMetrics.
  • Adds complete type hints for static type checking.
  • Removes iou and giou metrics and losses from Bolts, as they are available from Torchvision.

Fixes #816

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements) - issue for discussion: Upgrade the YOLO model with features from new variants #816
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

* Supports several algorithms for matching targets to anchors.
* Added support for DIoU and CIoU losses.
* Target for confidence can be selected between 1.0 and the overlap between the target and the predicted box.
* Target classes may be specified as a matrix of class probabilities, allowing multiple classes per object.
* Fixed rounding of maxpool layer size.
* Weight decay is applied only to convolutional layer weights.
* Command line interface is now using LightningCLI.
* Calculates MAP metric using TorchMetrics.
* Convolutional and max pooling layers automatically add the correct amount of padding.
* Added YOLOv4 network.
@mergify mergify bot added the has conflicts label May 4, 2022
* Added the deeper P6 variant of YOLOv4.
* Width of each backbone stage can be configured freely.
* The number of input channels in the PyTorch architectures can be changed.
* Added predict_step().
@mergify mergify bot removed the has conflicts label Jun 8, 2022
@senarvi senarvi force-pushed the yolo-update branch 2 times, most recently from 0d6f4b8 to 64a3e47 Compare April 6, 2023 10:55
@Borda
Copy link
Member

Borda commented May 18, 2023

@senarvi we have fixed master a few munites ago, so shall be 🟢 could you pls merge it here 🐿️

@senarvi
Copy link
Contributor Author

senarvi commented May 19, 2023

@senarvi we have fixed master a few munites ago, so shall be 🟢 could you pls merge it here 🐿️

Great. Seems like the tests are passing now.

@Borda Borda enabled auto-merge (squash) May 22, 2023 21:09
@Borda
Copy link
Member

Borda commented May 22, 2023

@senarvi we have fixed master a few munites ago, so shall be 🟢 could you pls merge it here 🐿️

Great. Seems like the tests are passing now.

perfect, it just sai about some conflicts, but I can't see them, mind check it?
Then I set ti on auto-merge so shall by landing any time soo 🎉

auto-merge was automatically disabled May 23, 2023 11:39

Head branch was pushed to by a user without write access

@mergify mergify bot removed the has conflicts label May 23, 2023
@mergify mergify bot added the ready label May 23, 2023
@mergify mergify bot removed the ready label May 24, 2023
mypy doesn't understand "= ... if ... else ..." and ruff doesn't accept the longer form.
@mergify mergify bot added the ready label May 25, 2023
@senarvi
Copy link
Contributor Author

senarvi commented May 25, 2023

@senarvi we have fixed master a few munites ago, so shall be 🟢 could you pls merge it here 🐿️

Great. Seems like the tests are passing now.

perfect, it just sai about some conflicts, but I can't see them, mind check it? Then I set ti on auto-merge so shall by landing any time soo 🎉

@Borda managed to fix it finally. ruff formatted the code in a way that caused mypy to fail. Formatting in a way that mypy accepts caused ruff to fail. In the end I added a "type: ignore" comment. It wasn't merged automatically, but all the tests have passed.

@@ -0,0 +1,59 @@
# YOLO

The YOLO model has evolved quite a bit, since the original publication in 2016. The original source code was written in C, using a framework called [Darknet](https://github.com/pjreddie/darknet). The final revision by the original author was called YOLOv3 and described in an [arXiv paper](https://arxiv.org/abs/1804.02767). Later various other authors have written implementations that improve different aspects of the model or the training procedure. [YOLOv4 implementation](https://github.com/AlexeyAB/darknet) was still based on Darknet and [YOLOv5](https://github.com/ultralytics/yolov5) was written using PyTorch. Most other implementations are based on these.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc: @lantiga

Copy link

@lantiga lantiga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool contribution, thank you @senarvi

@Borda Borda merged commit 5f5adaf into Lightning-Universe:master May 29, 2023
@senarvi senarvi deleted the yolo-update branch May 29, 2023 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade the YOLO model with features from new variants
6 participants