Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace backbone #6003

Closed
1 task done
Chengcheng1998727 opened this issue Dec 16, 2021 · 27 comments
Closed
1 task done

Replace backbone #6003

Chengcheng1998727 opened this issue Dec 16, 2021 · 27 comments
Labels
question Further information is requested Stale

Comments

@Chengcheng1998727
Copy link

Search before asking

Question

sir，I recently wanted to replace the backbone of yolov5 with resnet101, but there was an error that bothered me. This is the problem:
Traceback (most recent call last):
File "G:\Chengcheng\yolov5-5.0\models\yolo.py", line 93, in init
m.stride = torch.tensor([s / x.shape[-2] for x in self.forward(torch.zeros(1, ch, s, s))]) # forward
File "G:\Chengcheng\yolov5-5.0\models\yolo.py", line 123, in forward
return self.forward_once(x, profile) # single-scale inference, train
File "G:\Chengcheng\yolov5-5.0\models\yolo.py", line 139, in forward_once
x = m(x) # run
File "E:\anaconda\anaconda\envs\cc2\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\anaconda\anaconda\envs\cc2\lib\site-packages\torch\nn\modules\container.py", line 117, in forward
input = module(input)
File "E:\anaconda\anaconda\envs\cc2\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "G:\Chengcheng\yolov5-5.0\models\common.py", line 97, in forward
x = self.conv2(x)
File "E:\anaconda\anaconda\envs\cc2\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\anaconda\anaconda\envs\cc2\lib\site-packages\torch\nn\modules\conv.py", line 423, in forward
return self._conv_forward(input, self.weight)
File "E:\anaconda\anaconda\envs\cc2\lib\site-packages\torch\nn\modules\conv.py", line 419, in _conv_forward
return F.conv2d(input, weight, self.bias, self.stride,
RuntimeError: Given groups=1, weight of size [80, 320, 3, 3], expected input[1, 80, 61, 61] to have 320 channels, but got 80 channels instead
I really hope you can help me solve this problem

Additional

No response
@Chengcheng1998727 Chengcheng1998727 added the question Further information is requested label Dec 16, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Dec 16, 2021 •
edited by glenn-jocher
Loading

👋 Hello @Chengcheng1998727, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Status

CI CPU testing

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

@glenn-jocher
Copy link
Member

@Chengcheng1998727 part of the complication with backbone replacement is that YOLO requires skip connections from the backbone to the head in various places, and of course these must meet shape/stride constraints otherwise errors like this will appear.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 16, 2022 •
edited by glenn-jocher
Loading

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

Access additional Ultralytics ⚡ resources:

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

@github-actions github-actions bot added the Stale label Jan 16, 2022
@marziyemahmoudifar
Copy link

@Chengcheng1998727 part of the complication with backbone replacement is that YOLO requires skip connections from the backbone to the head in various places, and of course these must meet shape/stride constraints otherwise errors like this will appear.

How can I use Resenet 50 as the backbone of Yolov5?

@moahaimen
Copy link

same question

@hojat-ardi
Copy link

@Chengcheng1998727 part of the complication with backbone replacement is that YOLO requires skip connections from the backbone to the head in various places, and of course these must meet shape/stride constraints otherwise errors like this will appear.

How can I use Resenet 50 as the backbone of Yolov5?

سلام وقت بخیر
من میخوام backbone yolo رو تغییر بدم.
به مشکل خوردم
امکانش هست به بنده کمک کنید؟
آدرس ایمیل بنده: hojat.ardi.aut.ac.ir

@glenn-jocher
Copy link
Member

@hojat-ardi hello! Thank you for your interest in YOLOv5 and your question about using Resnet50 as the backbone.

To use Resnet50 as the backbone in YOLOv5, you need to make modifications to the YOLOv5 code. Here are the general steps:

  1. Locate the models\models.py file in the YOLOv5 repository.
  2. In this file, you will find a section that defines the backbone architecture. Look for the CSPDarknet or CSPResNet class, depending on the YOLOv5 version you are using.
  3. Replace the existing backbone with the Resnet50 architecture. You can use the torchvision implementation of Resnet50 or any other implementation that suits your needs. Make sure the input and output shapes of the backbone match the expected input and output shapes of the YOLOv5 model.
  4. Update the forward() method of the model to account for the changes. Adjust the forward pass to include the necessary connections between the backbone and the head of the YOLOv5 model.

Keep in mind that making such changes may require additional modifications to ensure the compatibility of the model. You might need to adjust the shape/stride constraints and handle any errors or inconsistencies that arise.

Please note that while this approach can be used as a starting point, it requires careful consideration and testing to ensure that the modified model functions correctly.

I hope this helps! If you have any further questions, please let me know.

@marziyemahmoudifar
Copy link

marziyemahmoudifar commented Jul 7, 2023 via email

@glenn-jocher
Copy link
Member

@marziyemahmoudifar برای ایجاد تغییرات در backbone در YOLOv5 می‌توانید تغییرات را در فایل yolov5.yaml که در پوشه models قرار دارد، ایجاد کنید. در این فایل، می‌توانید معماری backbone را تغییر دهید و از Resnet50 یا هر معماری دیگری که برایتان مناسب است، استفاده کنید. در همین فایل، به قسمت backbone: بروید و معماری مورد نظر خود را جایگزین کنید. حتماً به این نکته توجه کنید که شکل و گام‌های داده شده توسط backbone با شکل و گام‌های مورد انتظار مدل YOLOv5 هماهنگ باشند. بعد از انجام این تغییرات، ممکن است نیاز به تنظیم مجدد constraint‌های مربوطه و رفع هرگونه خطا یا ناهماهنگی باشد.

توجه داشته باشید که با تغییر backbone، ممکن است نیاز به تغییر دیگر متدل ها و قسمت‌های مربوطه باشد تا اطمینان حاصل شود که مدل به درستی کار می‌کند.

با انجام این تغییرات بیشتر مدل YOLOv5 شما تغییر داده خواهد شد اما اگر طبق دستورالعمل بالا از ک

@moahaimen
Copy link

خواهر عزیز میشه بیشتر توضیح بدی چطور این کار رو انجام بدم من دارم برای دکترا در عراق میخونم و روی توسعه ستون فقرات یولو کار میکنم.بابت زبانم متاسفم من از گوگل برای ترجمه استفاده میکنم.

@moahaimen
Copy link

@glenn-jocher where to change to Resnet50 ? where is this choice on what path please? and what other choices i could make besides Resnet50

@marziyemahmoudifar
Copy link

marziyemahmoudifar commented Jul 7, 2023 via email

@moahaimen
Copy link

در فایل yolov5.yaml تمام شبکه های کانولوشنی و ماژول های مورد استفاده در بخش ستون فقرات و گردن و سر یولو به همراه آرگومان های ورودی قرار گرفته اند. ماژول ها و لایه های کانولوشنی که در این فایل هستند از فایل common.py فراخوانی می‌شوند. اگر میخواهید تغییراتی در ستون فقرات شبکه ایجاد کنید برای مثال از ماژول جدیدی بجای ماژول C3 در ستون فقرات استفاده کنید ابتدا باید تابع ماژول جدید را در فایل common.py بنویسید و سپس آن را در فایل yolov5.yaml قرار دهید.
…
On Fri, Jul 7, 2023 at 18:27 moahaimen @.> wrote: خواهر عزیز میشه بیشتر توضیح بدی چطور این کار رو انجام بدم من دارم برای دکترا در عراق میخونم و روی توسعه ستون فقرات یولو کار میکنم.بابت زبانم متاسفم من از گوگل برای ترجمه استفاده میکنم. — Reply to this email directly, view it on GitHub <#6003 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASMY6LZVWAFSYF3S6YFFOGDXPAPTZANCNFSM5KFVR76A . You are receiving this because you were mentioned.Message ID: @.>

خواهر گرامی ترجمه به وضوح به دست من نرسید و من چیزی نفهمیدم درخت خرما چه نسبتی با معماری یولو دارد؟

@marziyemahmoudifar
Copy link

marziyemahmoudifar commented Jul 7, 2023 via email

@moahaimen
Copy link

In the yolov5.yaml file, all the convolutional layers and modules used in the backbone,neck and head of Yolo are placed along with the input arguments. The modules and convolution layers that are in this file are called from the common.py file. If you want to make changes in the backbone of the network, for example, use a new module instead of the C3 module in the backbone, you must first write the function of the new module in the common.py file and then put it in the yolov5.yaml file.
…
On Fri, Jul 7, 2023 at 19:06 moahaimen @.> wrote: در فایل yolov5.yaml تمام شبکه های کانولوشنی و ماژول های مورد استفاده در بخش ستون فقرات و گردن و سر یولو به همراه آرگومان های ورودی قرار گرفته اند. ماژول ها و لایه های کانولوشنی که در این فایل هستند از فایل common.py فراخوانی می‌شوند. اگر میخواهید تغییراتی در ستون فقرات شبکه ایجاد کنید برای مثال از ماژول جدیدی بجای ماژول C3 در ستون فقرات استفاده کنید ابتدا باید تابع ماژول جدید را در فایل common.py بنویسید و سپس آن را در فایل yolov5.yaml قرار دهید. … <#m_-4144234384128232094_> On Fri, Jul 7, 2023 at 18:27 moahaimen @.> wrote: خواهر عزیز میشه بیشتر توضیح بدی چطور این کار رو انجام بدم من دارم برای دکترا در عراق میخونم و روی توسعه ستون فقرات یولو کار میکنم.بابت زبانم متاسفم من از گوگل برای ترجمه استفاده میکنم. — Reply to this email directly, view it on GitHub <#6003 (comment) <#6003 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASMY6LZVWAFSYF3S6YFFOGDXPAPTZANCNFSM5KFVR76A https://github.com/notifications/unsubscribe-auth/ASMY6LZVWAFSYF3S6YFFOGDXPAPTZANCNFSM5KFVR76A . You are receiving this because you were mentioned.Message ID: @.> خواهر گرامی ترجمه به وضوح به دست من نرسید و من چیزی نفهمیدم درخت خرما چه نسبتی با معماری یولو دارد؟ — Reply to this email directly, view it on GitHub <#6003 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASMY6L4KJHTN7B6ZXQAZFR3XPAUIRANCNFSM5KFVR76A . You are receiving this because you were mentioned.Message ID: @.**>

thank you for explaining,yes i build 2 blocks and added them in blocks.py, but failed to call them in yolo.yaml , and when call them several errors happens , also i tried to add or delete any layers in yolo.yaml , a tensor shape size appeared i couldn't fix, if you can help me i would really appreciate it,

@marziyemahmoudifar
Copy link

marziyemahmoudifar commented Jul 7, 2023 via email

@glenn-jocher
Copy link
Member

@marziyemahmoudifar if you're experiencing errors when trying to call your custom modules in yolo.yaml, I can help you troubleshoot the issue. Could you please provide more details about the errors you are encountering? Additionally, if you could share the specific changes you made in blocks.py and yolo.yaml, it would be helpful for understanding the problem.

@itmr97
Copy link

itmr97 commented Jul 19, 2023 •
edited
Loading

@moahaimen لكن مدا اعرف شلون اذا ممكن اتواصل وياك backbone و اريد اغير yolov8 السلام عليكم اخوية اني ايضا اشتغل على موضوع

@glenn-jocher
Copy link
Member

مرحبًا @itmr97،

للتواصل ومعرفة كيفية تغيير بنية الـ backbone من YOLOv5 إلى YOLOv8، يُنصح بإنشاء سؤالك كـ issue جديد في مستودع المشروع YOLOv5 على GitHub حتى يتمكن المطورون والمشاركون المتخصصون في المشروع من مساعدتك بشكل فعال.

شكرًا لك على استخدام YOLOv5 وحظًا موفقًا في أبحاثك الدكتوراه.

مع خالص التحية،
فريق دعم YOLOv5

@moahaimen
Copy link

moahaimen commented Jul 20, 2023 •
edited
Loading

هلو اخوية اني نجحت بالمووضوع والحمدلله طبعا ساعدني السيد @glenn-jocher كثيرا شكرا وتقدير له , لكن اني بموضوع كتابة الاطروحة الخاصة بي لشهادة الدكتوراة, عموما شتحب اساعدك بالضبط؟ moahaimen@gmail.com
translation: hello yes i succeded in that , and special thanks to mr. Glenn he helped me a lot i did the changes for my PH.D thesis, also i provided my email if you need anything

@glenn-jocher
Copy link
Member

@moahaimen hello,

Glad to hear that you were successful in your task and that Mr. Glenn was able to assist you effectively. Congratulations!

Regarding your offer to help, we appreciate your willingness to contribute. If there are any specific questions or issues related to YOLOv5 that you would like to address, please feel free to share them in this thread or open a new GitHub issue. We'll do our best to assist you.

Thank you for your support and congratulations again on your progress with your PH.D thesis.

Best regards,
YOLOv5 Support Team

@itmr97
Copy link

itmr97 commented Jul 20, 2023

@moahaimen ارسلتك ايميل استاذ مهيمن

@glenn-jocher
Copy link
Member

Hello @itmr97,

Thank you for reaching out. I have received your email and will respond to you shortly. Please be patient as I review your message and gather any necessary information to assist you.

Best,
YOLOv5 Support Team

@hojat-ardi
Copy link

@hojat-ardi hello! Thank you for your interest in YOLOv5 and your question about using Resnet50 as the backbone.

To use Resnet50 as the backbone in YOLOv5, you need to make modifications to the YOLOv5 code. Here are the general steps:

  1. Locate the models\models.py file in the YOLOv5 repository.
  2. In this file, you will find a section that defines the backbone architecture. Look for the CSPDarknet or CSPResNet class, depending on the YOLOv5 version you are using.
  3. Replace the existing backbone with the Resnet50 architecture. You can use the torchvision implementation of Resnet50 or any other implementation that suits your needs. Make sure the input and output shapes of the backbone match the expected input and output shapes of the YOLOv5 model.
  4. Update the forward() method of the model to account for the changes. Adjust the forward pass to include the necessary connections between the backbone and the head of the YOLOv5 model.

Keep in mind that making such changes may require additional modifications to ensure the compatibility of the model. You might need to adjust the shape/stride constraints and handle any errors or inconsistencies that arise.

Please note that while this approach can be used as a starting point, it requires careful consideration and testing to ensure that the modified model functions correctly.

I hope this helps! If you have any further questions, please let me know.

thank you very much
It was great

@hojat-ardi hojat-ardi mentioned this issue Jul 23, 2023
Closed
@glenn-jocher
Copy link
Member

@hojat-ardi You're welcome! We're glad that our response was helpful to you. If you have any further questions or need any more assistance, please don't hesitate to ask. We're here to help!

Thank you and have a great day!

@NasserAi
Copy link

@Chengcheng1998727 part of the complication with backbone replacement is that YOLO requires skip connections from the backbone to the head in various places, and of course these must meet shape/stride constraints otherwise errors like this will appear.

How can I use Resenet 50 as the backbone of Yolov5?

هلو اخوية اني نجحت بالمووضوع والحمدلله طبعا ساعدني السيد @glenn-jocher كثيرا شكرا وتقدير له , لكن اني بموضوع كتابة الاطروحة الخاصة بي لشهادة الدكتوراة, عموما شتحب اساعدك بالضبط؟ moahaimen@gmail.com translation: hello yes i succeded in that , and special thanks to mr. Glenn he helped me a lot i did the changes for my PH.D thesis, also i provided my email if you need anything

أهلا اخ مهيمن. ويارب دراستك ماشية تمام. حاولت اغير ال backbone الى Resnet50 لكن ماقدرت...احتاج مساعدة اذا زبطت معاك

@glenn-jocher
Copy link
Member

@NasserAi hello! 🌟

Using ResNet 50 as the backbone for YOLOv5 involves modifying the model architecture to incorporate ResNet 50. Here's a simplified approach to get you started:

  1. Import ResNet 50: Use torchvision's implementation or another one you prefer.
  2. Integrate ResNet 50 as Backbone: In your YOLOv5 model file (likely models/yolo.py), replace the current backbone segment with ResNet 50. Ensure to adjust for the input and output layers to match YOLOv5's requirements.
  3. Adjust Skip Connections: Ensure the ResNet 50 layers you're using are compatible with YOLOv5's skip connections. You might need to tweak the layers and their output features.

Remember, aligning the dimensions and feature maps from the backbone to the YOLO head is crucial. If you encounter any shape/stride errors, double-check the connections and layer outputs.

Since you've mentioned having trouble with this step, could you share any specific errors or issues you're facing? This will help in providing more targeted advice.

Good luck with your project, and don't hesitate to reach out for more assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested Stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@NasserAi @glenn-jocher @moahaimen @itmr97 @Chengcheng1998727 @marziyemahmoudifar @hojat-ardi