Skip to content

[Codefuse开源轻训营] Support for Encoder-only models #10

@elvis-t9

Description

@elvis-t9

Add support for Encoder-only models (BERT-style architectures) in addition to the current Decoder-only LLM support. This enhancement will enable the conversion and fine-tuning of popular encoder-based models for embedding tasks, providing users with more architectural choices and leveraging the strengths of bidirectional attention for representation learning.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions