Resolve mean/std swap for VITDet backbone

[ianstenbit]

@tirthasheshpatel does your SAM demo perform as-expected with this change?

[tirthasheshpatel]

Thanks @ianstenbit for the quick fix! Quoting from #2086 (comment):

Here's the demo I usually run to test that the model is close to the original model at facebookresearch/segment_anything.

Just one non-blocking suggestion: What do you think about keeping the normalization step an opt-in rather than always having it on? This way the users can choose the preprocessing they want on their images.

[ianstenbit]

Here's the demo

@tirthasheshpatel what is the purpose of making this optional? You mean the mean/stddev bit I assume?
Throughout KerasCV we've standardized as 0-255 being the default input format, so unless there's a compelling reason I'd like to stick with the standard.

[tirthasheshpatel]

what is the purpose of making this optional? You mean the mean/stddev bit I assume?

Yes, I was referring to the mean/stddev part. The reason for doing that is to give users more control over input preprocessing (which might be useful for training). But I am not against changing the 0-255 input precedent (having that is great!). Just something like an imagenet_rescaling=True argument that'd let the users control whether they want that or not. What do you think?

[ianstenbit]

what is the purpose of making this optional? You mean the mean/stddev bit I assume?

Yes, I was referring to the mean/stddev part. The reason for doing that is to give users more control over input preprocessing (which might be useful for training). But I am not against changing the 0-255 input precedent (having that is great!). Just something like an imagenet_rescaling=True argument that'd let the users control whether they want that or not. What do you think?

I'm not sure what the value of this would be -- is there a reason that a user would care how the model internally does rescaling, as long as the API is clear about what is expected from the caller?

[ianstenbit]

/gcbrun

[tirthasheshpatel]

I'm not sure what the value of this would be -- is there a reason that a user would care how the model internally does rescaling, as long as the API is clear about what is expected from the caller?

I just thought there would be cases where the ImageNet rescaling would mess a bit with padded inputs (making borders non-zero). Although there is a way to avoid that (using mean as the padding value), just that some users might find it unintuitive.

Thinking in the wild here :) Please feel free to make a decision based on your best judgment!

[ianstenbit]

I'm not sure what the value of this would be -- is there a reason that a user would care how the model internally does rescaling, as long as the API is clear about what is expected from the caller?

I just thought there would be cases where the ImageNet rescaling would mess a bit with padded inputs (making borders non-zero). Although there is a way to avoid that (using mean as the padding value), just that some users might find it unintuitive.

Thinking in the wild here :) Please feel free to make a decision based on your best judgment!

Ahh I see your point here -- I think in this case it's okay. Zero padding will still end up being represented as the same pixel values as all-black pixels, which is logically the same as if no rescaling were done, it's just that the values aren't literally 0.

[tirthasheshpatel]

Zero padding will still end up being represented as the same pixel values as all-black pixels, which is logically the same as if no rescaling were done, it's just that the values aren't literally 0.

Ah OK, makes sense. Thanks for clarifying! Feel free to merge!