Skip to content

[FEATURE] add t2t_vit #2364

@holderhe

Description

@holderhe

Activity

rwightman

rwightman commented on Dec 14, 2024

@rwightman
Collaborator

@holderhe at this point I don't have the time to prioritize models like this, if someone created a fully functional and clean PR I'd consider, but it's a bit of work to pass the tests.

For reference, much closer to original vit arch (with a few tweaks), updated training recipe my explorations here appear quite a bit better than those models on in1k pretrain when you compare img/sec .... see the 82.5 top-1% little vs the 81.5 t2t_vit_14, without f.sdpa the vit_little is slightly faster in infer, quite a bit in train, and then with the F.sdpa (default0 enabled it's much faster. https://huggingface.co/timm/vit_little_patch16_reg1_gap_256.sbb_in12k

linked a pull request that will close this issue on Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @rwightman@holderhe

      Issue actions

        [FEATURE] add t2t_vit · Issue #2364 · huggingface/pytorch-image-models