update lib

[haqishen]

Check

• docker runtime
• flash-attn

[maxjeblick]

test_generation_is_the_same_as_for_causal_language_modeling is failing.
I assume that huggingface/transformers@3f60d11 is the root of the issue (this line):

causal_mask[..., :mask_length].eq(0.0) * attention_mask[:, None, None, :].eq(0.0)
Traceback (most recent call last):
  File "/home/max/.local/share/JetBrains/Toolbox/apps/pycharm-professional/plugins/python/helpers-pro/pydevd_asyncio/pydevd_asyncio_utils.py", line 114, in _exec_async_code
    result = func()
  File "<input>", line 1, in <module>
RuntimeError: The size of tensor a (64) must match the size of tensor b (136) at non-singleton dimension 3

Will try to create a minimal example that reproduces this error and potentially create an issue in transformers library.

[maxjeblick]

I've resolved the issue that originated from code changes in the transformers library that required config.max_position_embeddings = 1024 (before it was 32) in MaxJeblick/llama2-0b-unit-test.

[maxjeblick]

Thanks a lot for the upgrade!
I've tested with several models (including flash attention), looks good!