Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Attempt to make zipit work speak the same language as rest of mergekit #251

Merged
merged 28 commits into from
Apr 29, 2024
Merged

Attempt to make zipit work speak the same language as rest of mergekit #251

merged 28 commits into from
Apr 29, 2024

Conversation

metric-space
Copy link
Contributor

@metric-space metric-space commented Apr 7, 2024 •
edited
Loading

Untested

@metric-space metric-space changed the title Attempting for work to speak the same language as rest of mergekit Attemp for zipit work to speak the same language as rest of mergekit Apr 7, 2024
@metric-space metric-space changed the title Attemp for zipit work to speak the same language as rest of mergekit Attempt to make zipit work speak the same language as rest of mergekit Apr 7, 2024
@metric-space
Shared kq work around
f230abb
@metric-space
Correction
4b4a03b
@metric-space
More corrections
a30a78d
@metric-space
self-attention activations sizing up problem fix
dc222f7
@metric-space
Resizing surgery on k attention activations
ee2d109
@metric-space
Corrections but broken
b6dd515
@metric-space
Working post-self-attention-gqa-fix features dump
9f10410
@metric-space
Corrections in progress
8284760
@metric-space
Corrections to the dump_out_m_and_u script
01e637a
@metric-space
Hard code in specific detail for merge/unmerge script
3ac6a67
@metric-space
Corrections to merging script
fea03a1
@metric-space
Correction to rewritten repeat_kv
f58d277
@metric-space
Correction
64e7d6f
@metric-space
Experimental GQA merge/unmerge calc
a9b40f5
@metric-space
extra lines for experimental add gqa stuff
0fccc8e
@metric-space
Fix shape check
4f746d9
@metric-space
First stab at GQA handling with permutation of heads
a103d5a
@metric-space
Corrections
38a90f1
@metric-space
Introduction of shrink v to match k and then grow v from gqa, correct…
b8ada39 
…ions
@metric-space
Corrections
13a1db5
@metric-space
changes to testing scripts
edc59d3
@metric-space
Including a route for non-gqa and correction to parameter
01cc86b
@metric-space
Leftover corrections
1ada605
@metric-space
Some cleanup
5280a8b
@metric-space
Delete files
f3c57a3
@metric-space
Transfer of utils to respectives files that previously imported them
28445da
@metric-space
Correction
7e31cb6
@metric-space metric-space marked this pull request as ready for review April 29, 2024 18:38
@metric-space metric-space merged commit a6a2480 into wip-zipit Apr 29, 2024
3 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@metric-space