Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handroll bencode #46

Merged
merged 5 commits into from
Feb 13, 2024
Merged

handroll bencode #46

merged 5 commits into from
Feb 13, 2024

Conversation

mistermoe
Copy link
Member

Overview

First-pass run-through at handrolling bencode to replace usage of "github.com/anacrolix/torrent/bencode". No qualms with the lib itself it just happens to be a full fledged bittorrent client package

Note

What's in this PR at the very least fulfills the needs of did:dht

Rationale

The lib we're depending on adds 627 dependencies to our dependency graph. removing it brought the dep count down to 15. I recently had to handroll bencode in kotlin and have to do it again in dart so it was fresh on my mind. Figured i might as well knock it out here before forgetting all about it.

Warning

I'm not sure my implementation is idiomatic go. i looked at the anacrolix implementation and it looks pretty involved. I kept the API surface that we're using identical to the 3rd party dep. my variable names are kaka bc i wrote most of this during meetings. definitely open to renaming stuff if the names are jarring

func Marshal(input any) ([]byte, error) {
switch v := input.(type) {
case string:
encoded := fmt.Sprintf("%d:%s", len(v), v)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this length in bytes or in UTF-8 codepoints?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to the "spec", just string length

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disturbingly ambiguous!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the format doesn't support UTF-8 at all...


return []byte(encoded), nil
case int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64:
encoded := fmt.Sprintf("%s%d%s", string(IntegerPrefix), v, string(EndSuffix))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use %c for characters if you prefer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeah much cleaner! done: b5bec44

// Marshal encodes the given input into a Bencode formatted byte array.
// More information about Bencode can be found at:
// https://wiki.theory.org/BitTorrentSpecification#Bencoding
func Marshal(input any) ([]byte, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is not supposed to be able to encode structs? Might be good to mention that, as typically in Go, Marshal/Unmarshal can.

If you do want to do that, you'll have to use reflection, which you can probably just get away with in the default block.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the purpose we need it, no, doesn't need to do structs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call. no need for encoding structs right now so went ahead and added a comment d23f683

though would be cool to support that so i'll create an issue

return nil, err
}

b = append(b, encoded...)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how efficient you want this to be, but if you pass a singlebytes.Buffer around it will likely be significantly faster because you won't be allocating lots of tiny byte slices, instead you'll be writing to a single buffer.

If performance isn't a real concern than ignore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do do this you can then use fmt.Fprintf and fmt.Fscanf` directly on the buffer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great idea. performance isn't a big deal right now given that we're only ever marshaling a map with two fields that have primitive values.

regardless, what you're suggesting is a better approach nonetheless. going to create an issue to track your suggestion so someone can tackle it

Co-Authored-By: Alec Thomas <alec@swapoff.org>
// Marshal encodes the given input into a Bencode formatted byte array.
// More information about Bencode can be found at:
// https://wiki.theory.org/BitTorrentSpecification#Bencoding
func Marshal(input any) ([]byte, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the purpose we need it, no, doesn't need to do structs

Co-Authored-By: Alec Thomas <alec@swapoff.org>
@mistermoe mistermoe merged commit 2aff6ea into dht-resolver-fix Feb 13, 2024
@mistermoe mistermoe deleted the handroll-bencode branch February 13, 2024 18:41
mihai-chiorean added a commit that referenced this pull request Feb 14, 2024
* main:
  handroll bencode (#46)
  Fix: Added BEP44 decode to DHT resolve
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants