Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Inconsistency Between Training and Inference Caused by Multi-Scale Input Handling for Rectangular Inputs. #422

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Ryanshuai
Copy link

Problem:
Inconsistency issues identified between the training and inference phases related to how multi-scale inputs are handled.

Details:
During training, if the input image's height does not equal its width, our multi-scale process adjusts the image to be square. However, during inference, the inputs are maintained as rectangles. This inconsistency has led to a substantial degradation in model performance, with a decrease in mean Average Precision (mAP) by approximately 20% on our company's datasets.

Modifications:

  1. Multi-Scale Feature Interpolation: During training with multi-scale enabled, input images will be resized to match sz, whichever is larger in width or height.
  2. HybridEncoder Adjustments: In the HybridEncoder, bilinear interpolation is used for better feature alignment since the scale change is not an integer.

@Ryanshuai Ryanshuai changed the title Fix Inconsistency Between Training and Inference Caused by Multi-Scale Input Handling Fix Inconsistency Between Training and Inference Caused by Multi-Scale Input Handling for Rectangular Inputs. Aug 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant