-
-
Notifications
You must be signed in to change notification settings - Fork 15.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modifying YOLOv5 model for a common backbone but "2 different heads" #13086
Comments
👋 Hello @jain-abhay, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution. If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it. If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results. RequirementsPython>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started: git clone https://github.com/ultralytics/yolov5 # clone
cd yolov5
pip install -r requirements.txt # install EnvironmentsYOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
StatusIf this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit. Introducing YOLOv8 🚀We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀! Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects. Check out our YOLOv8 Docs for details and get started with: pip install ultralytics |
@jain-abhay hello, Thank you for your question and for searching the existing issues and discussions before posting. Modifying the YOLOv5 model to include a common backbone with two different heads is an interesting task and can be quite useful for various applications. To achieve this, you'll need to make changes in several files. Here's a detailed guide to help you through the process:
Here's a simplified example to give you an idea of what changes might look like in class Model(nn.Module):
def __init__(self, cfg='yolov5s.yaml', ch=3, nc=None, anchors=None):
super(Model, self).__init__()
# Load model configuration
self.yaml = self._load_yaml(cfg)
# Define the backbone
self.backbone = self._build_backbone(self.yaml['backbone'])
# Define the two heads
self.head1 = self._build_head(self.yaml['head1'])
self.head2 = self._build_head(self.yaml['head2'])
def forward(self, x):
x = self.backbone(x)
out1 = self.head1(x)
out2 = self.head2(x)
return out1, out2
def _build_backbone(self, backbone_cfg):
# Build the backbone from the configuration
pass
def _build_head(self, head_cfg):
# Build the head from the configuration
pass
For more detailed information on the YOLOv5 architecture and how to modify it, you can refer to the YOLOv5 Architecture Description. If you encounter any specific issues or need further assistance, please provide a minimum reproducible example of your code. This will help us better understand the problem and provide more accurate guidance. You can find more information on creating a minimum reproducible example here. Best of luck with your modifications! If you have any more questions, feel free to ask. |
Hi @glenn-jocher , thanks a lot for your response. Just an example like I wish to predict 2 different classes (one for each head) in an image, then what exactly should be added in the common.py file? Also in the yolov5 model, where exactly do we have to make changes in the loss functions to handle 2 heads? |
Hi @jain-abhay, Thank you for your follow-up question! Let's dive into the specifics of modifying Modifying
|
Thanks a lot @glenn-jocher for your suggestions. I will try this out and let you know! |
Hi @jain-abhay, You're very welcome! I'm glad you found the suggestions helpful. 😊 Please do try them out and let us know how it goes. If you run into any issues or have further questions, feel free to reach out. Remember, providing a minimum reproducible example can greatly assist us in diagnosing any problems you encounter. You can find more details on how to create one here. Best of luck with your modifications, and we're here to help if you need any further assistance! Thanks and |
Hi @glenn-jocher, just wanted to confirm a few things. The file common.py won't be altered here right since I am just defining 2 heads Also in the file yolo.py, the following functions need to be changed right? :
Please can you kindly confirm this once? Thanks in anticipation |
Hi @jain-abhay, Thank you for your follow-up! Let's clarify your questions regarding the modifications needed for
|
Hi @glenn-jocher , thanks for your inputs, will work on it. Also, I came across this issue and found your comment on it : #2001 **"Yes the https://github.com/ultralytics/yolov5/tree/classifier branch adds a classify.py file that does standalone classifier training. It does not attempt to merge tasks though. There is a C5_divergent branch (https://github.com/ultralytics/yolov5/tree/C5_divergent) that examines some updated architectures that are similar to hydra nets. For example this model has one backbone and two heads: Basically the yaml files are very flexible and can allow you to define a lot of interesting shapes such as the dual-head network above, which I was using to see if there was any benefit to having different heads compute different losses (i.e. a box regression head and a obj/cls head seperately). I didn't find any benefit from doing this strangely enough, so it's possible that in the case of the detection the loss components might complement each other. "** Is this kind of similar to my task of one backbone 2 heads, also the links that you had mentioned aren't opening (https://github.com/ultralytics/yolov5/blob/C5_divergent/models/hub/yolov5l6d-640.yaml). Please, it would be really great if you could share updated links for these repositories. Thanks a lot for your help and guidance. |
Hi @jain-abhay, Thank you for your message and for sharing the context. I'm glad to hear that you're making progress with the modifications! Regarding your question, yes, the concept of having one backbone with two heads is indeed similar to what you are aiming to achieve. The I apologize for the broken links. It seems the To ensure we can assist you effectively, please verify the following:
If you need further assistance or have any specific questions, feel free to ask. We're here to help! 😊 |
Hi @glenn-jocher please can you explain how the yaml file works, specifically how did we get [[-1, 5]] in the third line of the head block and also how did we get the numbers [[16, 19, 22], in the last line of the head block? parametersnc: 2 # number of classes anchorsanchors:
YOLOv5 backbonebackbone: [from, number, module, args][[-1, 1, StemBlock, [64, 3, 2]], # 0-P1/2 YOLOv5 headhead: [-1, 1, Conv, [256, 1, 1]], [-1, 1, Conv, [256, 3, 2]], [-1, 1, Conv, [512, 3, 2]], [[16, 19, 22], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5) |
Hi @jain-abhay, Thank you for your detailed question! Let's break down the YOLOv5 YAML configuration file to understand how the Understanding the YAML ConfigurationThe YOLOv5 YAML configuration file defines the architecture of the model, including the backbone and head. Each line in the [from, number, module, args]
Explanation of the
|
👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help. For additional resources and information, please see the links below:
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed! Thank you for your contributions to YOLO 🚀 and Vision AI ⭐ |
Search before asking
Question
Hi, I was curious to know that if I wish to modify the YOLOv5 model, for a new model that has a common backbone but 2 different heads, then which all files require changes/modifications?
In my knowledge, I guess the yaml file will have 2 heads defined now and then we might have to change common.py and yolo.py
But please I would really appreciate if you give a detailed clarity about what all needs to be modified in these files. Thanks in anticipation.
Additional
No response
The text was updated successfully, but these errors were encountered: