Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Hard error when ignoring tensors. #27484

Merged
merged 6 commits into from
Feb 5, 2024
Merged

[WIP] Hard error when ignoring tensors. #27484

merged 6 commits into from
Feb 5, 2024

Commits on Nov 14, 2023

  1. Configuration menu
    Copy the full SHA
    ae5cadb View commit details
    Browse the repository at this point in the history
  2. Better selection/error when saving a checkpoint.

    - Find all names we should normally drop (those are in the transformers
      config)
    - Find all disjoint tensors (for those we can safely trigger a copy to
      get rid of the sharing before saving)
    - Clone those disjoint tensors getting rid of the issue
    - Find all identical names (those should be declared in the config
      but we try to find them all anyway.)
    - For all identical names:
      - If they are in the config, just ignore them everything is fine
      - If they are not, warn about them.
    - For all remainder tensors which are shared yet neither identical NOR
      disjoint. raise a hard error.
    Narsil committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    92d1715 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    88571f3 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d8e1ed1 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Apply suggestions from code review

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    Narsil and ArthurZucker committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    7ee5bd0 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2024

  1. Configuration menu
    Copy the full SHA
    6a47030 View commit details
    Browse the repository at this point in the history