Skip to content

Add FNet Encoder Layer #40

@abheesht17

Description

@abheesht17

@mattdangerw , @chenmoneygithub and the rest of the keras-nlp team:

The FNet paper, which came out last year, uses FFTs instead of attention and achieves 95% of BERT's performance on various downstream tasks. It would be great if we could add this to the library. Simultaneously, a text classification example can be added (using a few FNet Encoder layers). Would love to take this up, if the team is interested :).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions