Fix ConvTranspose#93
Conversation
|
👋 Hello @FrzMtrsprt, thank you for submitting an
If this PR addresses a bug, please include a Minimum Reproducible Example (MRE) if not already provided:
For more guidance, please refer to our Contributing Guide at https://docs.ultralytics.com/help/contributing/. Don’t hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! 🚀🛠️✨ |
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
|
@onuralpszr can you review the math here? |
|
Of course, I am start reviewing
|
…s, grouped, and random parameters. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
|
First of all thank you for the PR @FrzMtrsprt first thing I notice and did some research to verify it. fvcore return "MACs" despite they say "FLOPs" but their actual return is "MACs" author of fvcore people also accept and opened a PR long ago but still not merge yet to fix terminology. URL1: Lyken17#208 For sanity check import torch
from ultralytics import YOLO
from thop import clever_format, profile
model_names = ["yolo11n.pt", "yolo11s.pt", "yolo11m.pt", "yolo11l.pt", "yolo11x.pt"]
input_shape = [640, 640]
dummy_input = torch.randn(1, 3, input_shape[0], input_shape[1])
for model_name in model_names:
model = YOLO(model_name)
macs, params = profile(model.model, (dummy_input,), verbose=False)
macs_readable, params_readable = clever_format([macs, params], "%.3f")
flops_readable = clever_format([macs * 2, params], "%.3f")
print(f"Total MACs for {model_name}: {macs_readable}")
print(f"Total params for {model_name}: {params_readable}")
print(f"Total FLOPs for {model_name}: {flops_readable}")
print()For ConvTranspose I include tests and did tests to verify that also compare with fvcore(since it is also MACs) New tests are All ConvTranspose2d tests use the correct formula: [input_size × (output_channels / groups) × kernel_size] For Extra Manuel Test to Verify the Math import torch
import torch.nn as nn
from fvcore.nn import FlopCountAnalysis # despite it says "FlopCountAnalysis as I said it does return MACs
import thop
def verify_convtranspose_math():
print("=" * 50)
model = nn.ConvTranspose2d(3, 1, (2, 2), stride=(2, 2))
input_tensor = torch.randn(1, 3, 2, 2)
thop_result = thop.profile(model, inputs=(input_tensor,))
thop_macs = thop_result[0]
fvcore_flops = FlopCountAnalysis(model, (input_tensor,))
fvcore_result = fvcore_flops.total()
output = model(input_tensor)
print(f"Input shape: {input_tensor.shape}")
print(f"Output shape: {output.shape}")
print(f"Kernel size: {model.kernel_size}")
print(f"Stride: {model.stride}")
print(f"Groups: {model.groups}")
print()
print("=" * 50)
print("Manual MAC calculation for ConvTranspose2d")
print("Formula: input_size × (output_channels / groups) × kernel_size")
print()
print("=" * 50)
input_size = input_tensor.numel()
output_channels = model.out_channels
groups = model.groups
kernel_h, kernel_w = model.kernel_size
kernel_size = kernel_h * kernel_w
manual_macs = input_size * (output_channels / groups) * kernel_size
print(f"input_size = {input_tensor.shape} = {input_size} elements")
print(f"output_channels / groups = {output_channels} / {groups} = {output_channels / groups}")
print(f"kernel_size = {kernel_h} × {kernel_w} = {kernel_size}")
print(f"MACs = {input_size} × {output_channels / groups} × {kernel_size} = {manual_macs}")
print(f"FLOPs = 2 × MACs = 2 × {manual_macs} = {2 * manual_macs}")
print()
print("=" * 50)
print("Results comparison:")
print(f"thop MACs: {thop_macs}")
print(f"fvcore MACs: {fvcore_result}")
print(f"Manual MACs: {manual_macs}")
print(f"Manual FLOPs: {2 * manual_macs}")
print()
return True
def demonstrate_conv_convtranspose_symmetry():
print("\n3. Conv2d vs ConvTranspose2d symmetry:")
print("-" * 42)
# Create symmetric operations
# Conv2d: downsample 4x4 -> 2x2
conv = nn.Conv2d(1, 3, kernel_size=2, stride=2)
conv_input = torch.randn(1, 1, 4, 4)
# ConvTranspose2d: upsample 2x2 -> 4x4 (reverse)
convt = nn.ConvTranspose2d(3, 1, kernel_size=2, stride=2)
convt_input = torch.randn(1, 3, 2, 2)
# Calculate MACs
conv_result = thop.profile(conv, inputs=(conv_input,))
conv_macs = conv_result[0]
conv_output = conv(conv_input)
convt_result = thop.profile(convt, inputs=(convt_input,))
convt_macs = convt_result[0]
convt_output = convt(convt_input)
print(f"Conv2d: {conv_input.shape} -> {conv_output.shape}, MACs: {conv_macs}, FLOPs: {2 * conv_macs}")
print(f"ConvTranspose2d: {convt_input.shape} -> {convt_output.shape}, MACs: {convt_macs}, FLOPs: {2 * convt_macs}")
print()
def main():
verify_convtranspose_math()
demonstrate_conv_convtranspose_symmetry()
if __name__ == "__main__":
exit(main())As an extra I fixed all of the terms in conv2d test FLOPs to MACs to reflect correct terms as well. Thank you again for PR 👍 |
|
cc @glenn-jocher I wrote a detailed analysis and checked and include tests make sure we cover new parameter. |
|
@onuralpszr awesome analysis, thanks for the detailed comparison here. Agree sometimes the two terms are used interchangeably even though they're not the same. @FrzMtrsprt merging PR and releasing a new version of the package with your fix. Thank you for your contributions! |
|
Merged and marvelous! 🎉 Huge thanks to @FrzMtrsprt for leading this, with thoughtful contributions from @glenn-jocher and @onuralpszr. As Peter Drucker said, “What gets measured gets managed.” This PR makes THOP’s measurements more trustworthy by correctly counting MACs for ConvTranspose layers, aligning terminology from FLOPs to MACs, and strengthening tests—culminating in v2.0.16. The result: clearer, accurate metrics and more reliable benchmarking for our community and tooling. Grateful for the precision and care you brought to this—awesome work, team! 🚀 |
4 participants
Fixes #92
I have read the CLA Document and I sign the CLA
🛠️ PR Summary
Made with ❤️ by Ultralytics Actions
🌟 Summary
Improves THOP’s accuracy and clarity by correctly counting MACs for ConvTranspose layers, aligning terminology to MACs (not FLOPs), expanding tests, and releasing version 2.0.16. 🚀
📊 Key Changes
🎯 Purpose & Impact