Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Phi-3-mini-4k-instruct checkpoint #1341

Merged
merged 36 commits into from
Jul 1, 2024
Merged

Add Phi-3-mini-4k-instruct checkpoint #1341

merged 36 commits into from
Jul 1, 2024

Commits on Apr 23, 2024

  1. Add phi-3 checkpoint

    @rasbt
    rasbt committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    93f3024 View commit details
    Browse the repository at this point in the history

  2. progress

    @rasbt
    rasbt committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    1012eaf View commit details
    Browse the repository at this point in the history

  3. weight loading works

    @rasbt
    rasbt committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    581e27f View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Convert Phi3 qkv into an interleaved one

    @Andrei-Aksionov
    Andrei-Aksionov committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    40fe01f View commit details
    Browse the repository at this point in the history

  2. Config: Phi3 doesn't use parallel residual

    @Andrei-Aksionov
    Andrei-Aksionov committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    0322ecd View commit details
    Browse the repository at this point in the history

  3. Fix layer shapes in Phi3MLP

    @Andrei-Aksionov
    Andrei-Aksionov committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    7f33850 View commit details
    Browse the repository at this point in the history

  4. Config: update vocab size

    @Andrei-Aksionov
    Andrei-Aksionov committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    1b217ba View commit details
    Browse the repository at this point in the history

  5. Add prompt

    @Andrei-Aksionov
    Andrei-Aksionov committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    6fc4a7c View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into phi-3-checkpoint

    @Andrei-Aksionov
    Andrei-Aksionov committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    ba1c930 View commit details
    Browse the repository at this point in the history

  7. Add test for Phi3 model

    @Andrei-Aksionov
    Andrei-Aksionov committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    2ee1e0d View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. Update litgpt/prompts.py

    @rasbt
    rasbt committed May 3, 2024
    Configuration menu
    Copy the full SHA
    29760ab View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into phi-3-checkpoint

    @rasbt
    rasbt committed May 3, 2024
    Configuration menu
    Copy the full SHA
    6c4cd25 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Merge branch 'main' into phi-3-checkpoint

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    fbc45b4 View commit details
    Browse the repository at this point in the history

  2. Fix prompt

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    efb8388 View commit details
    Browse the repository at this point in the history

  3. The prompt has been changed. Update it

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    7f092fa View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. A workaround for a Phi-3 tokenizer

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    a2acd37 View commit details
    Browse the repository at this point in the history

  2. Convert in copy_weihght_phi without Phi3MLP

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    ef21d37 View commit details
    Browse the repository at this point in the history

  3. Config: Phi3MLP -> LlaMAMLP

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    aa184e7 View commit details
    Browse the repository at this point in the history

  4. test_model.py: add test for Phi-3

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    4f941bb View commit details
    Browse the repository at this point in the history

  5. model.py: drop Phi3MLP

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    3bd0692 View commit details
    Browse the repository at this point in the history

  6. convert_hf: copy_weight_llama without Phi3 related code

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    9583cd7 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into phi-3-checkpoint

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    1c661be View commit details
    Browse the repository at this point in the history

  8. test_model.py: update test for Phi-3

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    c25e533 View commit details
    Browse the repository at this point in the history

  9. test_covert_hf: add test for qkv_reassemble

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    6e484eb View commit details
    Browse the repository at this point in the history

  10. Update test_tokenzer to match AutoTokenizers

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    c8a1e03 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Merge branch 'main' into phi-3-checkpoint

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    39614e7 View commit details
    Browse the repository at this point in the history

  2. convert_lit: add Phi-3 code

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    81e56b6 View commit details
    Browse the repository at this point in the history

  3. test_convert_lit: prettify test for qkv_split

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b483ca2 View commit details
    Browse the repository at this point in the history

  4. Update test_prompts.py

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    82b4124 View commit details
    Browse the repository at this point in the history

  5. Add Phi-3-mini to the list of supported models

    @Andrei-Aksionov
    Andrei-Aksionov committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d252505 View commit details
    Browse the repository at this point in the history

  6. Update README.md

    @rasbt
    rasbt committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0eb288e View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Merge branch 'main' into phi-3-checkpoint

    @rasbt
    rasbt committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a2683d7 View commit details
    Browse the repository at this point in the history

  2. Update tutorials/download_model_weights.md

    @rasbt
    rasbt committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a923c5e View commit details
    Browse the repository at this point in the history

  3. Update tutorials/download_model_weights.md

    @rasbt
    rasbt committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    bcbddec View commit details
    Browse the repository at this point in the history

  4. Apply Sebastian's suggestion: model_name.lower()...

    @Andrei-Aksionov
    Andrei-Aksionov committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    9dc0330 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into phi-3-checkpoint

    @rasbt
    rasbt committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    ceae946 View commit details
    Browse the repository at this point in the history