Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Activation/representation based merging #199

Closed
wants to merge 42 commits into from
Closed

Activation/representation based merging #199

wants to merge 42 commits into from

Commits on Mar 11, 2024

  1. Create zipit.yml

    @shamanez
    shamanez committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    a826e48 View commit details
    Browse the repository at this point in the history

  2. Add alignment method to config

    @metric-space
    metric-space committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    83f97eb View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. First stab at activations dumper

    @metric-space
    metric-space committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    1368c47 View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2024

  1. Differentiate between activations and hidden_states

    @metric-space
    metric-space committed Mar 16, 2024
    Configuration menu
    Copy the full SHA
    ea5f816 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2024

  1. Post test run corrections

    @metric-space
    metric-space committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    a134612 View commit details
    Browse the repository at this point in the history

  2. More corrections

    @metric-space
    metric-space committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    f5668e7 View commit details
    Browse the repository at this point in the history

  3. ZipIt Similarity. (#201) 

    Modified `dump_out_features.py` and added `dump_out_sim_metrics.py`
    @shamanez
    shamanez committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    52293c1 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. Fdfmm 40 zipit metric (#205)

    @shamanez
    shamanez committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    0004eb1 View commit details
    Browse the repository at this point in the history

  2. Gpt-2 residual connection correction

    @metric-space
    metric-space committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    2bc97ab View commit details
    Browse the repository at this point in the history

  3. Fix for architecture.py

    @metric-space
    metric-space committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    d4cb463 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. Playing with subgraphs generated via zipit forward-backward propagations

    @metric-space
    metric-space committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    b998989 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Mainly adding modified M_U computation. (#249) 

    - computation of correlation matrices 
- computation mo m&u
- correlation modification 


@metric-space

---------

Co-authored-by: Luke Meyers <functor.soup@gmail.com>
    @shamanez @metric-space
    shamanez and metric-space committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    da1bc52 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Attempt to make zipit work speak the same language as rest of mergekit ( 

    #251)

Make the work more merekit style
    @metric-space
    metric-space committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    a6a2480 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. Fixes in implementation

    @metric-space
    metric-space committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    0e0ef40 View commit details
    Browse the repository at this point in the history

  2. Another default gone

    @metric-space
    metric-space committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    0e234b2 View commit details
    Browse the repository at this point in the history

  3. Delete examples/zipit.yml

    @metric-space
    metric-space committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    9f324b6 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2024

  1. Change boolean default

    @metric-space
    metric-space committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    7f673f7 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Code removal

    @metric-space
    metric-space committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    ab6d131 View commit details
    Browse the repository at this point in the history

  2. Make sure average of correlations are taken

    @metric-space
    metric-space committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    5a56d27 View commit details
    Browse the repository at this point in the history

  3. Remove on the fly GQA handling for now

    @metric-space
    metric-space committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    0e85590 View commit details
    Browse the repository at this point in the history

  4. Delete test_zipit.sh

    @metric-space
    metric-space committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    0a02343 View commit details
    Browse the repository at this point in the history

  5. Put back config source file to original state

    @metric-space
    metric-space committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    d3b5970 View commit details
    Browse the repository at this point in the history

  6. Put back config source file to original state (part 2)

    @metric-space
    metric-space committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    0b237e9 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. Code cleanup for feature extraction script

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    f0fcc6f View commit details
    Browse the repository at this point in the history

  2. Another round of refactors and getting rid of unnecessary steps

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    2ef5c62 View commit details
    Browse the repository at this point in the history

  3. variable name correction

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    6bcf6d8 View commit details
    Browse the repository at this point in the history

  4. Left over correction

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    293801c View commit details
    Browse the repository at this point in the history

  5. Cleaner logic for activations garbbing hooks

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    b7af5fb View commit details
    Browse the repository at this point in the history

  6. More refactors for feature extraction script

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    5b1344b View commit details
    Browse the repository at this point in the history

  7. More refactors and corrections

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    f92bc58 View commit details
    Browse the repository at this point in the history

  8. Make script efficient to avoid oom errors when device is set to gpu

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    10407fe View commit details
    Browse the repository at this point in the history

  9. Make final script device configurable

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    f845c50 View commit details
    Browse the repository at this point in the history

  10. Make final script device configurable

    @metric-space
    metric-space committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    fb55abe View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. Add chat template ability to feature extraction script

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    43863b7 View commit details
    Browse the repository at this point in the history

  2. Add datasets dependency to project dependencies

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    5d47dd0 View commit details
    Browse the repository at this point in the history

  3. Bug fix

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    dc53a58 View commit details
    Browse the repository at this point in the history

  4. Bug fix

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    42a9543 View commit details
    Browse the repository at this point in the history

  5. Yet another bug fix

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    2bff4d8 View commit details
    Browse the repository at this point in the history

  6. Encode connection between att_kq and attn_v

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    547c400 View commit details
    Browse the repository at this point in the history

  7. Give proper script commands

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    6327cfd View commit details
    Browse the repository at this point in the history

  8. New folder and location change

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    fe548dc View commit details
    Browse the repository at this point in the history

  9. Delete test_by_gen.py

    @metric-space
    metric-space committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    1ab4b2b View commit details
    Browse the repository at this point in the history