Hello @qubvel! The state of the PR is that I've gotten the forward pass to match the original implementation up to the required precision. I've marked this as a draft because I wanted to just get a first opinion on if I'm modifying the correct files in the codebase. Let me know if I'm missing anything big. Regarding tests, i've copied some from Deformable Detr but haven't tried getting them to work. Let me know if I need to add any apart from what's already there.

Hi @konstantinos-p! Thanks a lot for working on the model, super excited to see it merged! 🚀

Before diving into the implementation details, here are some general comments we need to address in the PR:

• Integration Tests
  Let's set up integration tests to ensure output consistency is maintained while refactoring the modeling code. Please use torch.testing.assert_close when comparing tensors (not use self.assertTrue(torch.allclose(...))

• Modeling Code
  It would be super nice to use modular approach! It's the way to use inheritance in transformers while keeping a one-model-one-file format, please see more information here: https://huggingface.co/docs/transformers/main/en/modular_transformers and in examples/modular-transformers folder of the repo. Also, Siglip2 and RT-DETRv2 model were added using modular.

• Consistent Naming for Classes & Modules

  • Model names and types: DinoDetr and dino_detr
  • All module names should be prefixed with the model name. For example, instead of ResidualBlock, use DinoDetrResidualBlock.

• Code Paths & Cleanup

  • No asserts in code and minimum raise statements, config params should be validated in config.

  • Remove unused conditional statements if no existing configuration requires them. Also, remove the unnecessary config parameters.

    # Instead of:
    if config.use_last_layer_norm:
        self.norm = nn.LayerNorm()
    else:
        self.norm = nn.Identity()

    # If `use_last_layer_norm = True` in all configs, simplify it to:  
    self.norm = nn.LayerNorm()

• Code Style

  • Use clear, descriptive variable names: avoid single-letter (x) or placeholder (tmp) variables.
  • Add comments for non-obvious code.
  • Use type hints for modules (e.g. in __init__ and forward).

• Conversion Script Standard
  Please follow the mllama model format for the conversion script. Define a single key mapping using regex (see rt_detr_v2, ijepa, mllama, superglue models for reference). This is our new standard for all newly added models.

Thanks again! Looking forward to the updates 🚀

Thanks for the comments! I'll start addressing them!