Comparison with perceiver-pytorch?

[xloem]

How does this repository compare with https://github.com/lucidrains/perceiver-pytorch ?

Would you have any interest in generalizing and integrating the two implementations together?

[esceptico]

How does this repository compare with https://github.com/lucidrains/perceiver-pytorch ?

Hey! They differ mainly in the structure of the code.
My version is closer to how the codebase is structured in the DeepMind source repo (link) and in the Transformers library.
There are also differences in how encoder and decoders relate. In lucidrains repo encoder and decoders are not separated. In this repository, they are separated, which allows to combine different encoder-decoder architectures.
Also lucidrains repo contains more data processing utilities for Perceiver IO (like Fourier positional encoding).

Would you have any interest in generalizing and integrating the two implementations together?

I don't think they can be combined :)
But I would like to have an implementation (no matter whose) Perceiver IO in the Transformers library

[xloem]

I don't think they can be combined :)

Well it sounds like it would mean deciding on a code architecture and adding the features offered by one project into the other project. Since they've already been implemented in one less skill would be needed to do the re-/factoring into the other.

But I would like to have an implementation (no matter whose) Perceiver IO in the Transformers library

Yeah that sounds pretty helpful. Do you think you'll be opening a PR?

[esceptico]

As mentioned here

@NielsRogge is currently working on it

so I hope perceiver io will be part of transformers library soon