ed3k

[ModderMule]

As some users already saw the ed3k showing up since v0.7.0....

The goal:
We have recently seen a revive in eMule dev activity with the use of AI. We should consider this a good thing and make room for new developers adding their own ideas. However, original ed2k (from eDonkey200) was never intended to be extensible. We see these problems today:

1. Kad is "patched" next to servers without the server being aware of any Kad status. Clients handle LowID vs Firewalled inconsistently. It falls short even today as a LowID callback to a client leaving his server is abandoned "unreachable" instead of retrying via his Kad buddy. eMuleQt has patch for this in tryToConnect() . In ed3k a server is just a "big" Kad buddy that indexes everything using the same binary packet format (skipping initial Kad loop).
2. The CSafeMemFile packet format is not freely extensible. Writing data at the end (outside of a tag/multipacket section) as some mods and servers started doing breaks the reader that expects a different format and it throws or skips quietly. Using protobuf for all packets as ed3k is a freely extensible binary encoding while minimizing overhead (see integer compression).
3. There is a lot of old debt in the code confusing developers and wasting AI tokens. The multi packet vs single packets, the "less than 4GB" vs "larger than 4GB" tag handling, the old Kad 1 logic that was announced to be removed 20 years ago... https://forum.emule-project.net/index.php?showtopic=130257

Consequences of the above we see today for example in 3 different incompatible NAT-T implementations: The original NeoMule one using its own UDP streams based on eMule TCP. Then the eMuleAI ones based on uTP streams and a 2nd QUIC NAT-T implementation as spoken by aMule + eMuleQt.

We can expect this to become more cluttered as more AI devs join while official devs don't set new standards anymore. Every server operator knows over 2/3 of the network has closed ports (LowID), kad buddies can not be found anymore reliably due to max 1 buddy limit - resulting in lowID clients even losing highID sources https://forum.emule-project.net/index.php?showtopic=167269&pid=1111179&st=0

Torrent solved extensibility better due to:

1. an easy an extensible packet format bencode
2. a site where protocol extensions are listed and refined by active developers where everyone can contribute (via their Github): https://www.bittorrent.org/beps/bep_0000.html

The example that you likely never heard of BEP 50 publish & subscribe in torrent while its still perfectly valid proofs this point.

Going forward:

The README of the ed3k spec provides an overview of what's possible and allows for every dev to write their own extensions - to be then reviewed and formally added to ed3k as torrent BEPs do it: https://github.com/ed3k-protocol/ed3k-spec
The protobuf packet format is compiler-enforced for 20+ supported languages, thus generating a much more consistent base layer for every ed3k implementation. The current packet format is "only" about 1k lines: https://github.com/ed3k-protocol/ed3k-spec/blob/main/proto/ed3k.proto (including 25+ proposed extensions).

ed3k is experimental early stage and does not claim to be "the best" solution for the problems listed above. It is simply the intention and invitation to provide a way ahead.

P.S. the litmus test of ed3k shall be some "AI slop guy" implementing 3.5 inch floppy disk transfers over IPv6 without interfering other more serious devs :D

[Aokromes]

imho, server and client devs must join some place, for example telegram where to talk about follow one unique strategy to allow this world of cgnat, people with 0 tech knowledge to open ports, locked isp routers to share data more easy.

[Aokromes]

afaik amule guys are joining emule-ai way no?

[ModderMule]

I checked and he now added both, first uTP (eMule AI only) and now QUIC (all except original NeoMule)

[0x391F]

I think it's a bad idea to add "Selling" feature in ed3k, because it violates P2P sharing spirit (free, help each other), even though "private torrent" in bittorrent network is also free.

https://www.emule-project.com/home/perl/general.cgi

eMule is a free and open-source peer-to-peer file sharing client, allowing you to connect to millions of users to download and share files with them. By using the ED2K and Kademlia Network it supports semi-centralized as well as decentralized searches and operations. All of it free and without any adware or advertising.

[ModderMule]

I feel similar and it will likely not be enabled in eMuleQt GUI. It's just one protocol extension I wanted to try. Remember the above text was written before cryptocurrencies, today you could do the same already:

1. give file a random name
2. hide it behind a paywall on website
3. ask for payment before showing the (ed2k) link

On ed3k the payment itself is still not part of the network and requires external protocols - only the enforcement of the upload slot assignment. And even then we all know people can re-release the same file....

[0x391F]

About "official devs don't set new standards anymore", I think nowadays here're 2 mainstream (widely used and still in developing) ed2k client (eMule and aMule), just need they reach a consensus, other mod clients follow up, or other mods have a try, then mainstream client follow up (like uTP). Something like Bittorrent's BEP is also good.

[ModderMule]

aMule became active again under a partially new team.
For eMule:

• the competent modders left 10+ years ago
• the official devs went idle even before that (why modders likely left)
• the eMule "community edition" dev on emule-project.net repeatedly showed no interest in leading the protocol or adding any new meaningful network features such as NAT-T

Reaching consensus in a torrent BEP style is my goal here. And ensure if no consensus is reached that 2 different extensions can co-exist amicably.