Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save bandwidth by using a torrent to distribute more efficiently #73

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

ChristopherKing42
Copy link

No description provided.

@facebook-github-bot
Copy link

Hi @ChristopherKing42!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

Copy link

@ControlCplusControlV ControlCplusControlV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 3, 2023
@facebook-github-bot
Copy link

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

1 similar comment
@facebook-github-bot
Copy link

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@BetaDoggo
Copy link

Good catch! This will save millions in bandwidth costs.

@elwolf6
Copy link

elwolf6 commented Mar 3, 2023

my fucking sides

@mrcichon
Copy link

mrcichon commented Mar 3, 2023

i love the internet

Copy link

@kryptoklob kryptoklob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, you're hired.

@jp-x-g
Copy link

jp-x-g commented Mar 3, 2023

Big if true.

Copy link

@ionicc ionicc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

README.md Outdated Show resolved Hide resolved
@zb3
Copy link

zb3 commented Mar 3, 2023

At this point Meta should publish the original checksums so that we know if it's the real model. It appears that PyTorch models can execute arbitrary code, so this might as well be a clever attack..

@hatkidchan
Copy link

Could've helped with.. uh.. reducing bandwidth, but don't have enough space for the entire model, anyone else distributing the load?

@Fannovel16
Copy link

Fannovel16 commented Mar 3, 2023

At this point Meta should publish the original checksums so that we know if it's the real model. It appears that PyTorch models can execute arbitrary code, so this might as well be a clever attack..

Just convert them to safetensors on a VM or cloud computer.

@Disconsolance
Copy link

I kneel

@d9j
Copy link

d9j commented Mar 3, 2023

the magnet link is broken. update your pull request pls

@nukeop
Copy link

nukeop commented Mar 3, 2023

I can't sneed

Copy link

@igorbrigadir igorbrigadir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

README.md Outdated Show resolved Hide resolved
Copy link

@altryne altryne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a tiny markdown change

README.md Outdated Show resolved Hide resolved
Copy link

@KimgCheems KimgCheems left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link

@proudmuslim-dev proudmuslim-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@RobWC
Copy link

RobWC commented Mar 24, 2023

Ship it

Copy link

@RobWC RobWC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Heliumra
Copy link

Can anyone give away the 7b? I have a Mac m1 with 256gb and I can't download via the main link because the file size exceeds the size of my storage

@goryramsy
Copy link

goryramsy commented Mar 24, 2023 via email

@Heliumra
Copy link

Thank you! I would probably like to rent a server for 65b but not sure I can afford it

@wassimrkik
Copy link

no seeders on the torrent file

@yebot
Copy link

yebot commented Mar 24, 2023

LGTM

@Daviljoe193
Copy link

@wassimrkik

no seeders on the torrent file

Did you try the alt torrent, magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352&dn=LLaMA

@0xboredagain
Copy link

you deserve a bonus ser!

@CEO-Chair
Copy link

LGTM

@m3rzb0w
Copy link

m3rzb0w commented Mar 25, 2023

LGTM

@Arlodotexe
Copy link

Arlodotexe commented Mar 25, 2023

Some have been having trouble with the magnet. For preservation, I've reuploaded the original torrent content to an ipfs node.

http gateways (the links below) will be slow to retrieve until more people have the files. Use a local node like Kubo or Brave Browser if possible - it's faster and helps reseed the content for others temporarily (after downloading), or permanently (if pinned).


Full backup: ipfs://Qmb9y5GCkTG7ZzbBWMu2BXwMkzyCKcUjtEKPpgdZ7GEFKm

7B: ipfs://QmbvdJ7KgvZiyaqHw5QtQxRtUd7pCAdkWWbzuvyKusLGTw
13B: ipfs://QmPCfCEERStStjg4kfj3cmCUu1TP7pVQbxdFMwnhpuJtxk
30B: ipfs://QmSD8cxm4zvvnD35KKFu8D9VjXAavNoGWemPW1pQ3AF9ZZ
65B: ipfs://QmdWH379NQu8XoesA8AFw9nKV2MpGR4KohK7WyugadAKTh


You can download normally, or use these commands from the Kubo CLI:

# Optional: Preload the 7B model. Retrieves the content you don't have yet. Replace with another CID, as needed.
ipfs refs -r QmbvdJ7KgvZiyaqHw5QtQxRtUd7pCAdkWWbzuvyKusLGTw

# Optional: Pin the 7B model. The GC removes old content you don't use, this prevents the model from being GC'd if enabled.
ipfs pin add QmbvdJ7KgvZiyaqHw5QtQxRtUd7pCAdkWWbzuvyKusLGTw

# Download from IPFS and save to disk via CLI:
ipfs get QmbvdJ7KgvZiyaqHw5QtQxRtUd7pCAdkWWbzuvyKusLGTw --output ./7B

Bump

@herc5854
Copy link

LGTM

Copy link

@Ark-kun Ark-kun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even with smaller models (StableDiffusion) I had many issues downloading form HuggingFace via Git LFS or the client.
Torrent seems to be much more reliable. And bandwidth-saving as well.
You get my stamp of approval.

@X-Session
Copy link

@Arlodotexe hey those are your files? On ipfs will those file be there or will you delet it please let me know

@Arlodotexe
Copy link

Arlodotexe commented Mar 28, 2023

@Arlodotexe hey those are your files? On ipfs will those file be there or will you delet it please let me know

You can think of ipfs like BitTorrent + Git + Http. Files are retrievable as long as any one person has them on a local node.

This post and this video are good introductions to ipfs. To learn more, ask your favorite search engine.

Copy link

@Casperd88 Casperd88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Odyssey28
Copy link

@Arlodotexe hey those are your files? On ipfs will those file be there or will you delet it please let me know

You can think of ipfs like BitTorrent + Git + Http. Files are retrievable as long as any one person has them on a local node.

This post and this video are good introductions to ipfs. To learn more, ask your favorite search engine.

I have no idea what I'm doing, but I am good at following tutorials, and I will try to seed the smaller models. Unless I am missing something and I shouldn't do that?

@simonmahns
Copy link

Lgtm

1 similar comment
@Fabien017
Copy link

Lgtm

@marcus35000
Copy link

hope I get it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

438 participants