Fix owlv2 compilation in old inference

[PawelPeczek-Roboflow]

What does this PR do?

Fix for mokney-patching of owlv2 in old inference

Type of Change

• Bug fix (non-breaking change that fixes an issue)

Testing

• gut feeling and CI on main
• I have tested this change locally
• I have added/updated tests for this change

Test details:

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code where necessary, particularly in hard-to-understand areas
• My changes generate no new warnings or errors
• I have updated the documentation accordingly (if applicable)

Additional Context