Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document check sums of models so that we can confirm issues are not caused by bad downloads or conversion #238

Closed
gjmulder opened this issue Mar 17, 2023 · 9 comments
Closed

Document check sums of models so that we can confirm issues are not caused by bad downloads or conversion #238

gjmulder opened this issue Mar 17, 2023 · 9 comments
Labels
documentation Improvements or additions to documentation model Model specific

Comments

@gjmulder
Copy link
Collaborator

gjmulder commented Mar 17, 2023 •
edited by j-f1
Loading

Can someone please confirm the following md5 sums are correct? I regenerated them with the latest code.

$ md5sum ./models/*/*.pth | sort -k 2,2
0804c42ca65584f50234a86d71e6916a  ./models/13B/consolidated.00.pth
016017be6040da87604f77703b92f2bc  ./models/13B/consolidated.01.pth
f856e9d99c30855d6ead4d00cc3a5573  ./models/30B/consolidated.00.pth
d9dbfbea61309dc1e087f5081e98331a  ./models/30B/consolidated.01.pth
2b2bed47912ceb828c0a37aac4b99073  ./models/30B/consolidated.02.pth
ea0405cdb5bc638fee12de614f729ebc  ./models/30B/consolidated.03.pth
9deae67e2e7b5ccfb2c738f390c00854  ./models/65B/consolidated.00.pth
0c4b00c30460c3818bd184ee949079ee  ./models/65B/consolidated.01.pth
847194df776dd38f8ae9ddcede8829a1  ./models/65B/consolidated.02.pth
3b6c8adcb5654fd36abab3206b46a0f1  ./models/65B/consolidated.03.pth
68d61d1242597ad92616ec31b8cb6b4c  ./models/65B/consolidated.04.pth
7f71259eaee2b906aa405d8edf39925f  ./models/65B/consolidated.05.pth
0574e26b6891ab2cb0df7340d773fe9b  ./models/65B/consolidated.06.pth
e5d9790df955270b836aec79462ead22  ./models/65B/consolidated.07.pth
6efc8dab194ab59e49cd24be5574d85e  ./models/7B/consolidated.00.pth
Edit: File format has changed. Don’t use these collapsed weights! 
$ md5sum ./models/*/ggml-model-f16* | sort -k 2,2
0d851faaf144ff75ff9683685cbcbedc  ./models/13B/ggml-model-f16.bin
5cde948c6a27f41dc822b1a8a0587e79  ./models/13B/ggml-model-f16.bin.1
c80e0c824c7e853c3d5be915afb37eef  ./models/30B/ggml-model-f16.bin
72da29fca244f2a64f85b2c14b20290d  ./models/30B/ggml-model-f16.bin.1
16f07b182f44116fd72a9cc174dc0db2  ./models/30B/ggml-model-f16.bin.2
2413e326c00b476e8cd13d5f1fe65854  ./models/30B/ggml-model-f16.bin.3
eb8f7835d1d7e716f96af02fefdd5c04  ./models/65B/ggml-model-f16.bin
30f08121b86fe90db2497bd87f844d3b  ./models/65B/ggml-model-f16.bin.1
98983c0e2338d2985a0d9bb8bd27efb5  ./models/65B/ggml-model-f16.bin.2
635ebf87ef9053f7facccc665a0c826a  ./models/65B/ggml-model-f16.bin.3
6ca89293e1a9c8ad96b476406739827c  ./models/65B/ggml-model-f16.bin.4
696e4afe846ddfe2a2366db927a0dffa  ./models/65B/ggml-model-f16.bin.5
39a7f52b968aa833212c027d6fd58ccf  ./models/65B/ggml-model-f16.bin.6
a8ac8b55c152565573b118b0a0109726  ./models/65B/ggml-model-f16.bin.7
0fd0234fd08a7310f93f64faff7fda15  ./models/7B/ggml-model-f16.bin

$ md5sum ./models/*/ggml-model-q4_0* | sort -k 2,2
b405d83aff658379cc8b1b59b9a39668  ./models/13B/ggml-model-q4_0.bin
b06456f82bbc9d1fd46afa635ce0eba4  ./models/13B/ggml-model-q4_0.bin.1
c8bdc3fedd676b4c30bcc61812dab84f  ./models/30B/ggml-model-q4_0.bin
aad0750e54004014b65fa65aedacdf84  ./models/30B/ggml-model-q4_0.bin.1
88876dca38cedf53ba0a915e817921ed  ./models/30B/ggml-model-q4_0.bin.2
4063e11be83d342893ba4e3e299a4436  ./models/30B/ggml-model-q4_0.bin.3
dbdd682cce80e2d6e93cefc7449df487  ./models/65B/ggml-model-q4_0.bin
5d7c7e0e30b351af5237b81852e4b01b  ./models/65B/ggml-model-q4_0.bin.1
2ca89995c8c17890b2935022aede929e  ./models/65B/ggml-model-q4_0.bin.2
88e36f69163fe09da11531332410f4d4  ./models/65B/ggml-model-q4_0.bin.3
4fe105f7d77d54d94daa33bbfd582733  ./models/65B/ggml-model-q4_0.bin.4
1106d57cdf87ecbf83540f3a0027b480  ./models/65B/ggml-model-q4_0.bin.5
c5759417ae123248bb2cecf85546680f  ./models/65B/ggml-model-q4_0.bin.6
cedfc3b77578db761f871f8c8baa8323  ./models/65B/ggml-model-q4_0.bin.7
919e4f8aee6ce4f3fbabb6cbcd7756db  ./models/7B/ggml-model-q4_0.bin
@gjmulder gjmulder added documentation Improvements or additions to documentation model Model specific labels Mar 17, 2023
@gjmulder gjmulder self-assigned this Mar 17, 2023
@j-f1
Copy link
Collaborator

j-f1 commented Mar 17, 2023 •
edited
Loading

LGTM!

$ md5sum ./models/*/*.pth | sort -k 2,2 
0804c42ca65584f50234a86d71e6916a  ./models/13B/consolidated.00.pth
016017be6040da87604f77703b92f2bc  ./models/13B/consolidated.01.pth
f856e9d99c30855d6ead4d00cc3a5573  ./models/30B/consolidated.00.pth
d9dbfbea61309dc1e087f5081e98331a  ./models/30B/consolidated.01.pth
2b2bed47912ceb828c0a37aac4b99073  ./models/30B/consolidated.02.pth
ea0405cdb5bc638fee12de614f729ebc  ./models/30B/consolidated.03.pth
9deae67e2e7b5ccfb2c738f390c00854  ./models/65B/consolidated.00.pth
0c4b00c30460c3818bd184ee949079ee  ./models/65B/consolidated.01.pth
847194df776dd38f8ae9ddcede8829a1  ./models/65B/consolidated.02.pth
3b6c8adcb5654fd36abab3206b46a0f1  ./models/65B/consolidated.03.pth
68d61d1242597ad92616ec31b8cb6b4c  ./models/65B/consolidated.04.pth
7f71259eaee2b906aa405d8edf39925f  ./models/65B/consolidated.05.pth
0574e26b6891ab2cb0df7340d773fe9b  ./models/65B/consolidated.06.pth
e5d9790df955270b836aec79462ead22  ./models/65B/consolidated.07.pth
6efc8dab194ab59e49cd24be5574d85e  ./models/7B/consolidated.00.pth
$ sha1sum ./models/*/*.pth | sort -k 2,2 
5a6bbbe7718a329133940ed48c206e3c4bbab4fb  ./models/7B/consolidated.00.pth
f089e099366453606b64bc6e2ee89da2e9b804a1  ./models/13B/consolidated.00.pth
dd439dd9d04b4134aadc05939aa9eb24c227cd27  ./models/13B/consolidated.01.pth
4e4554a983f9bbba5af70f364cf18959aa6095cd  ./models/30B/consolidated.00.pth
3509ed49fb4d5410a551f39597084ff11af341f8  ./models/30B/consolidated.01.pth
454c90decf1c455c21c03b83811f7295dde78d3d  ./models/30B/consolidated.02.pth
e204400163fe38caba8fff69ececf8d4c8bd225c  ./models/30B/consolidated.03.pth
8a409074782aa26533222159badb1851f6320ed7  ./models/65B/consolidated.00.pth
37bb7400dd507abfce4e11da76e6ad5f4adc4516  ./models/65B/consolidated.01.pth
5cab9a91954ff18a32721fa81522f220d889e20b  ./models/65B/consolidated.02.pth
db658accfe0c681b065276093cc77045793c1b08  ./models/65B/consolidated.03.pth
0b5b2f6359b4f09d4656de1ffababbb2ee8072ae  ./models/65B/consolidated.04.pth
c6b9c34e07abe5e93cb1f5c18f06b16c56a75617  ./models/65B/consolidated.05.pth
c3e67dbf79bdeeb60c0a9d641848b5b844a077da  ./models/65B/consolidated.06.pth
77b9491d386594bb528fdbb589a17af52ba221f2  ./models/65B/consolidated.07.pth

@gjmulder gjmulder mentioned this issue Mar 17, 2023
Closed
4 tasks
@Lupul
Copy link

Lupul commented Mar 17, 2023

FWIW just for 7B and 13B models (all are matching):

md5sum ./models/*/*.pth | sort -k 2,2
✓ 0804c42ca65584f50234a86d71e6916a  ./models/13B/consolidated.00.pth
✓ 016017be6040da87604f77703b92f2bc  ./models/13B/consolidated.01.pth
✓ 6efc8dab194ab59e49cd24be5574d85e  ./models/7B/consolidated.00.pth

md5sum ./models/*/ggml-model-f16* | sort -k 2,2
✓ 0d851faaf144ff75ff9683685cbcbedc  ./models/13B/ggml-model-f16.bin
✓ 5cde948c6a27f41dc822b1a8a0587e79  ./models/13B/ggml-model-f16.bin.1
✓ 0fd0234fd08a7310f93f64faff7fda15  ./models/7B/ggml-model-f16.bin

md5sum ./models/*/ggml-model-q4_0* | sort -k 2,2
✓ b405d83aff658379cc8b1b59b9a39668  ./models/13B/ggml-model-q4_0.bin
✓ b06456f82bbc9d1fd46afa635ce0eba4  ./models/13B/ggml-model-q4_0.bin.1
✓ 919e4f8aee6ce4f3fbabb6cbcd7756db  ./models/7B/ggml-model-q4_0.bin

@kaliber91
Copy link

65B all are matching.

@prusnak prusnak mentioned this issue Mar 18, 2023
Closed
@Green-Sky
Copy link
Collaborator

Green-Sky commented Mar 19, 2023 •
edited
Loading

Please either, pin this issue, or commit checklist.chk files like facebook did, or add this to a wiki page ...

Because this should be the first thing in troubleshooting and needs to be accessible. :)

edit: we also need to keep in mind, that over time the fileformat will change and so will the hashes...

This was referenced Mar 19, 2023
Closed
Merged
@Green-Sky
Copy link
Collaborator

since the fileformat changed. this needs new info.

@gjmulder gjmulder pinned this issue Mar 20, 2023
@gjmulder
Copy link
Collaborator Author

gjmulder commented Mar 20, 2023 •
edited
Loading

Can somebody please do a quick scan and verify the following sha256 sums? I downloaded ggml-alpaca-7b-q4.bin and ggml-alpaca-13b-q4.bin. Should these be placed in the 7B and 13B dirs, respectively?

llama.cpp$ git log | head -1
commit 074bea2eb1f1349a0118239c4152914aecaa1be4

llama.cpp$ for D in ./models.sha256/*B; do echo "=="; echo $D/*.sha256; cat $D/*.sha256; done
==
./models.sha256/13B/checklist.sha256
745bf4e29a4dd6f411e72976d92b452da1b49168a4f41c951cfcc8051823cf08  consolidated.00.pth
d5ccbcc465c71c0de439a5aeffebe8344c68a519bce70bc7f9f92654ee567085  consolidated.01.pth
a6bd0537c6873f36c47292df0b6f794e1135f5aafb89c3343bcc9e93264bf167  ggml-model-f16.bin
0fb0951b90f2ec46c1f2f2372af5dacb4614b27e9fb6c10c69fbec58d7dd0e36  ggml-model-f16.bin.1
1c218ba37ae61e15e35efd9949c78d6edf553b6280824c263cad56ae0b9d5a8f  ggml-model-q4_0.bin
c37a20c2ab9fa74b006b389085660269ee06110d1e45a494eb57d4602c9bcdb2  ggml-model-q4_0.bin.1
4ab77bec4d4405ccb66a97b282574c89a94417e3c32e5f68f37e2876fc21322f  params.json
==
./models.sha256/30B/checklist.sha256
e23294a58552d8cdec5b7e8abb87993b97ea6eced4178ff2697c02472539d067  consolidated.00.pth
4e077b7136c7ae2302e954860cf64930458d3076fcde9443f4d0e939e95903ff  consolidated.01.pth
24a87f01028cbd3a12de551dcedb712346c0b5cbdeff1454e0ddf2df9b675378  consolidated.02.pth
1adfcef71420886119544949767f6a56cb6339b4d5fcde755d80fe68b49de93b  consolidated.03.pth
def20ea508f4e36793719f857471e85b85f96e497a2cbffbbaa1b60e2b18202c  ggml-model-f16.bin
b37040aa67fa8608cb2d8e0719132cf3e267fd35ec1e2f0d37dbc9fa43d674f1  ggml-model-f16.bin.1
e7f263557e99069fe29003262ea5fa9ed885dbe79069083e6eb569b328cf30d3  ggml-model-f16.bin.2
2ad6a23af05eb720f202f63d130f4fc5de9b6d2efc95b921be003209a56695aa  ggml-model-f16.bin.3
7de31d005e6d02ebd9603b2cf5329ad2f832b65d08873a098c5cafc4046cb9ed  ggml-model-q4_0.bin
f91feef9f30f9a023616db2e91297ca6d5d5d7b9eb351e452a82115c46f7da9e  ggml-model-q4_0.bin.1
66f3a0916ac7a81839153eb061fa861030ed1892477c2f7af2ce4f98d2f6d06f  ggml-model-q4_0.bin.2
e3c587ba97f83d2088b001bcda3026571065649ee3090bef6743a51390b01d3b  ggml-model-q4_0.bin.3
2c07118ea98d69dbe7810d88520e30288fa994751b337f8fca02b171955f44cb  params.json
==
./models.sha256/65B/checklist.sha256
135c563f6b3938114458183afb01adc9a63bef3d8ff7cccc3977e5d3664ecafe  consolidated.00.pth
9a600b37b19d38c7e43809485f70d17d1dc12206c07efa83bc72bb498a568bde  consolidated.01.pth
e7babf7c5606f165a3756f527cb0fedc4f83e67ef1290391e52fb1cce5f26770  consolidated.02.pth
73176ffb426b40482f2aa67ae1217ef79fbbd1fff5482bae5060cdc5a24ab70e  consolidated.03.pth
882e6431d0b08a8bc66261a0d3607da21cbaeafa96a24e7e59777632dbdac225  consolidated.04.pth
a287c0dfe49081626567c7fe87f74cce5831f58e459b427b5e05567641f47b78  consolidated.05.pth
72b4eba67a1a3b18cb67a85b70f8f1640caae9b40033ea943fb166bd80a7b36b  consolidated.06.pth
d27f5b0677d7ff129ceacd73fd461c4d06910ad7787cf217b249948c3f3bc638  consolidated.07.pth
7eba2625260cd91f8de901fd9704a1aa39448425514a335a0d3878de4ab9dc77  ggml-model-f16.bin
f6aa886575df0785d4231f30cc776d499ccde18857818effc0378c65b178e0b5  ggml-model-f16.bin.1
076037141682f5d7537955058c4740ab27f285aa4588915f830874a589c0693d  ggml-model-f16.bin.2
7853d96d2903ad7de2b2a89c4acf5a33a2f8e3c24ac39c9df6b44cdb42bf530a  ggml-model-f16.bin.3
b16b7b941abb3bc03a14df1656140855e9360a5371c83e919b9da83a72362314  ggml-model-f16.bin.4
5291270216f888697695acb78ef28df0c080f9e85d3245c92fb9992d1fde6678  ggml-model-f16.bin.5
0685ee77715f34686841006f8f94d3e7eaf148b97cecc9d3eee72808b0f7989c  ggml-model-f16.bin.6
00d993d73bb21d7c29388ffe0dced008cbaa0d391831dea77d7eb8f0b5c404b9  ggml-model-f16.bin.7
4e398f05842206e08cdc5e7bb4f6c7c34b9dc373435ece6f261b14b7b4fe9b89  ggml-model-q4_0.bin
4c4e899e3b12d9f57c9dcea5a1fb41bbc72023323535551f6273582ca7d7294b  ggml-model-q4_0.bin.1
d7b4594bbbd192043b3db0e5acc2561c42e6944e1cb91cc6e61510eee89dbcd8  ggml-model-q4_0.bin.2
9a099d271648863d923d0d097391ea0bc75591f27a2ca3a327760f42e6b69af2  ggml-model-q4_0.bin.3
5ee474051e418c5732b7949190b084d9d679db447f83c1de0d2a82daaa1a0cfa  ggml-model-q4_0.bin.4
a45aa05e7212bd6782790722d68056c5419667ea6b564ccc94bbcb8111d79b8b  ggml-model-q4_0.bin.5
a58fda714b759c28ad5e4c1d8bf8fda7b158fd5e4c4a49f851f36342fa97a105  ggml-model-q4_0.bin.6
a3540cfcbcda33c223c6b0d606034adbd78f17e0e5de1582b78795e78754f7a8  ggml-model-q4_0.bin.7
999ed1659b469ccc2a941714c0a9656fa571d17c9f7c8c7589817ca90edef51b  params.json
==
./models.sha256/7B/checklist.sha256
700df0d3013b703a806d2ae7f1bfb8e59814e3d06ae78be0c66368a50059f33d  consolidated.00.pth
abe4aec2cdc297e2916011f66c7efd6fb4424e0e84315503005b5c118358cc22  ggml-model-f16.bin
f495fa02a0b5ef265e1864d9680eede7fd23a60b0a2f93edba8091e2a4ca68b9  ggml-model-q4_0.bin
7e89e242ddc0dd6f060b43ca219ce8b3e8f08959a72cb3c0855df8bb04d46265  params.json
==
./models.sha256/alpaca-13B/checklist.sha256
e17730c6b62b565b098af023ca446dcb9e3535d4222ead6369c7aae67207eb3d  ggml-alpaca-13b-q4.bin
==
./models.sha256/alpaca-7B/checklist.sha256
1f582babc2bd56bb63b33141898748657d369fd110c4358b2bc280907882bf13  ggml-alpaca-7b-q4.bin

@gjmulder gjmulder changed the title Document md5sums of models so that we can confirm issues are not caused by bad downloads or conversion Document check sums of models so that we can confirm issues are not caused by bad downloads or conversion Mar 20, 2023
gjmulder added a commit that referenced this issue Mar 20, 2023
@gjmulder
sha256 check sums to verify original and converted model data (#238)
c458447 
* Add shadow ./model.sha256 dir containing a dir for each model and a corresponding checklist.sha256 containing sha256 sums of the *.pth *bin* and *json files

* Add script chk_sha256sums.sh to walk user supplied ./models subdir and run sha256sum against above files to update checklist.sha256 for each model

* Update README.md with corresponding instructions
@gjmulder gjmulder removed their assignment Mar 20, 2023
This was referenced Mar 21, 2023
Closed
Merged
@shiipou shiipou mentioned this issue Mar 22, 2023
Closed
4 tasks
@gjmulder gjmulder unpinned this issue Mar 23, 2023
@dice10240
Copy link

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin

Algorithm Hash Path

SHA256 4E398F05842206E08CDC5E7BB4F6C7C34B9DC373435ECE6F261B14B7B4FE9B89 D:\llama\backup\ggml-model-q4...

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin.1

Algorithm Hash Path

SHA256 4C4E899E3B12D9F57C9DCEA5A1FB41BBC72023323535551F6273582CA7D7294B D:\llama\backup\ggml-model-q4...

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin.2

Algorithm Hash Path

SHA256 D7B4594BBBD192043B3DB0E5ACC2561C42E6944E1CB91CC6E61510EEE89DBCD8 D:\llama\backup\ggml-model-q4...

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin.3

Algorithm Hash Path

SHA256 9A099D271648863D923D0D097391EA0BC75591F27A2CA3A327760F42E6B69AF2 D:\llama\backup\ggml-model-q4...

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin.4

Algorithm Hash Path

SHA256 5EE474051E418C5732B7949190B084D9D679DB447F83C1DE0D2A82DAAA1A0CFA D:\llama\backup\ggml-model-q4...

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin.5

Algorithm Hash Path

SHA256 A45AA05E7212BD6782790722D68056C5419667EA6B564CCC94BBCB8111D79B8B D:\llama\backup\ggml-model-q4...

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin.6

Algorithm Hash Path

SHA256 A58FDA714B759C28AD5E4C1D8BF8FDA7B158FD5E4C4A49F851F36342FA97A105 D:\llama\backup\ggml-model-q4...

PS C:\Users\d2> Get-FileHash D:\llama\backup\ggml-model-q4_0.bin.7

Algorithm Hash Path

SHA256 A3540CFCBCDA33C223C6B0D606034ADBD78F17E0E5DE1582B78795E78754F7A8 D:\llama\backup\ggml-model-q4...

@gjmulder
Copy link
Collaborator Author

Thanks. Can you just tell us which ones don't match?

@dice10240
Copy link

All of them have same hash. Your files are verified.

@sw sw closed this as completed May 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation model Model specific
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@sw @Green-Sky @Lupul @gjmulder @dice10240 @j-f1 @kaliber91