Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Save tensors in lower precision #273

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Save tensors in lower precision #273

wants to merge 9 commits into from

Commits on May 10, 2024

  1. Save attribution tensors with a lower precision. 

    inseq-team#202
Adds functionality for saving feature attributions objects and tensors in float16 or float8 format,
depending on `scores_precision` parameters.
Tensors are saved in huggingface safetensor format, and quantized using
zeropoint quantization. Because safetensors are bytes objects, they are
encoded with b64 to be saved in the output json and decoded upon
reloading.
    @LuukSuurmeijer
    LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    62298a2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'inseq-team:main' into feature/score_precision

    @LuukSuurmeijer
    LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    0e9828c View commit details
    Browse the repository at this point in the history

  3. Add support for device_map use (inseq-team#264) 

    * Add device_map support

* Fix device setter in HF model
    @gsarti @LuukSuurmeijer
    gsarti authored and LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    5c7e10f View commit details
    Browse the repository at this point in the history

  4. Support Value Zeroing for non-eager attention types (inseq-team#267)

    @gsarti @LuukSuurmeijer
    gsarti authored and LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    6133a50 View commit details
    Browse the repository at this point in the history

  5. Fix nans in attn, add pad if missing (inseq-team#269)

    @gsarti @LuukSuurmeijer
    gsarti authored and LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    9168e39 View commit details
    Browse the repository at this point in the history

  6. Add transformers v4.40 models to config, update changelog

    @gsarti @LuukSuurmeijer
    gsarti authored and LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    3c1627e View commit details
    Browse the repository at this point in the history

  7. run tests

    @LuukSuurmeijer
    LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    647c8b4 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'feature/score_precision' of github.com:LuukSuurmeijer/i… 

    …nseq into feature/score_precision
    @LuukSuurmeijer
    LuukSuurmeijer committed May 10, 2024
    Configuration menu
    Copy the full SHA
    96f9629 View commit details
    Browse the repository at this point in the history

Commits on May 11, 2024

  1. Minor fixes

    @gsarti
    gsarti committed May 11, 2024
    Configuration menu
    Copy the full SHA
    a682761 View commit details
    Browse the repository at this point in the history