LlamaFlashAttention2 __init__ is undefined.

[Luke20000429]

The LlamaFlashAttention2 has no __init__(). As a result running the system with flash-attn will crash.

LMDrive/LAVIS/lavis/models/blip2_models/modeling_llama.py

Line 415 in ae0643d

class LlamaFlashAttention2(LlamaAttention):

[deepcs233]

Hi!

Sorry for the late reply. I've been very busy lately.

It's an experimental feature, and you can simply replace the forward function and other related functions from LlamaAttention with those from LlamaFlashAttention2. I also update the code.

[Luke20000429]

I've included that in my local repo, but not a bad thing for you to include in your public repo.