Mechanism for link- or file-based pointers to shared files

[Searinox]

Following the same format as magnet links or .torrent files, it would facilitate sharing if people could point other people to shared content via links or more elaborate data similar to .torrent files, that can then be easily shared across the many media of communication.

I would like to present several proposals:

1. The simplest proposal of all is perhaps the ability to "linkify" a shared file. This would be as simple as making it possible to create a protocol://b64hash of what is available on MuWire's network that can be clicked in browser and will initiate a search in MuWire.

2. This proposal builds on 1 but instead of search it is queue for download, and it allows for more than one file, so perhaps a protocol://hash1&hash2&hash3... type of link, except in this case it would really help if you also had the ability to specify what name each file should have since a download is likely to find the same file by hash on multiple sources with different filenames and for the sake of organization, it would help to be able to pre-plan their naming.

3. This further builds on point 2 and proposes a download plan with an entire folder tree list with filenames and the hashes of each file. This is much closer to how a .torrent works and in order to facilitate launching a download that may contain hundreds/thousands of files, it is better all organized into a small file that can be opened with MuWire to initiate the download. Since this type of collection could be quite large, this should all be placed in its own folder, so perhaps like torrents have names these download plans should also have names, which are used as the default download folder name, but user-changeable just before committing to download.

I believe any level of sophistication between 1 and 3 would be a net benefit to ease of sharing. What do you think?

[zlatinb]

This has come up a few times in the past, and it comes down to defining an "URI" scheme. That in itself is not difficult, and the reason I haven't done it is because of all the plumbing required to get it working with browsers:

1. Register a scheme handler with the OS (relatively easy these days)
2. Modify the MuWire executable to check if MuWire is already running ( not sure how to do it w/o native code)

• If it's already running, pass the link on to the existing instance (really not sure how to do this)
• If it isn't, launch MuWire from fresh

I used to work at LimeWire and remember that part of the code being particularly complex. But then again other Java applications do it - namely BiglyBT - so it's probably worth looking into how they've done it.

As to the points 1-3, 2 and 3 can be solved by the "Collections" feature. The Collection metainfo is already kept in a descriptor file, so providing hash to that should be easy enough. It just needs to be distinguished from a single file, i.e. we would either need two separate schemes or use a part of the scheme to specify whether the URI is for a single file or for a collection.

[Searinox]

For problem number 2 I would propose:

On start, get the current application directory and attempt to open an "instance.lock" file from the location for writing but still allow reading, just like with a log file.

If successful, then we are the only instance running. Write the current PID to that file.
If another process has the write lock, then there is another instance of MW running. Read the instance.lock file and get the PID from it. Then, pass the link to that PID and exit.

Working with a process ID has the advantage of being compatible with both standalone binaries(muwire.exe) and java-launched packages(java.exe muwire.jar). This should also work cross-platform.

[zlatinb]

That will probably work. Instead of a pid I would have MuWire open a random port on startup then write the value of the port in the lock file. Then of course only local connections would be accepted on that port.

[zlatinb]

I think I'll have a stab at this for the next release, mainly because I'm tired of having to search for hashes people paste in chat :)

Such link would look like:

muwire:hash:base64HashOfFile?
   &length=<long>
   &pieceSizePow2=<integer>
   &fileName=url_encoded_name
   &host=base64OfPersona
   &sig=base64signature

The base64HashOfFile, pieceSizePow2 and fileName are necessary to initialize the download.
host is the person who created the link and is a seed and also serves as a tracker.
sig is a signature of the other fields, exactly which ones TBD. The goal is to make it impossible to forge links.

I will not address the plumbing issues listed in the post above right away, the user will need to paste the link in MuWire manually.

[zzzi2p]

Which params are mandatory and which are optional? Mandatory ones are better put into the base part of the URI.

If you can search now by copy/paste of hashes from chat, then that tells me all of the params are optional. An unforgeable link is also an optional feature? Hashes are by definition unforgeable so why need a sig?

magnet links are b32 to provide case-insensitivity which is a good thing for URIs, at least the base part

if you are signing, need a spec on how to canonicalize the signed data

[zlatinb]

If the search functionality is used then everything but the hash is optional. But that is a very inconvenient way of doing things.

I think I'll go in the other extreme and make all fields mandatory including the signature. The sig is there to ensure that whoever is listed as a host in the link:

1. Is the same person that created the link
2. attests to the name of the file mainly and to the other metadata in a lesser degree

The goal is to be easy for a human to look at the link and decide based on the fileName and the host fields whether it is safe to download the file because links with invalid signatures will be rejected. This means I should also add a hostShort or some such field which contains the human-readable representation of the host.

[zzzi2p]

not sure if realistic for a human to look past dozens of b32/b64 chars to validate if something looks right farther to the right. If that is part of the goal, the human-readable part should be leftmost in the URI.

While perhaps inconvenient, people are quite familiar with magnet links and searching by hash... perhaps a 2nd URI scheme, e.g. muwire:search:b32hash should also be supported? If inconvenient, why, and what could be done in the software to make it less so? Are there use cases for a search URI?

[zlatinb]

A search URI sounds very interesting. I don't immediately see any use cases but others may come up with ideas.

Until the missing plumbing described earlier is in place there's not much to be gained from a search URI support because of the necessity to copy-paste.

[zlatinb]

I have implemented a preliminary support for links to files in this CI build. There is a new option in the library right-click menu "copy link to clipboard" and if you paste that link in the search box of another MuWire instance it will start the download right away.

This is the structure of a link to a file:

muwire://base64(persona)/FILE/base64(file-hash) ?
    name=url_encoded(file_name) &
    fileSize=12345.. &
    pieceSizePow2=17-27 &
    sig=base64(signature(other fields))

The FILE path element identifies this as a link to a single file. I plan to add links to collections before the 0.8.13 release which would be structured very similarily. In the more distant future links to blogs, chat rooms etc. are all possible.

The signature covers the file hash, file name, link type and file size. MuWire will refuse to open any links that do not verify.

[zlatinb]

One bit that I'm not clear on is whether the Base64-encoded elements need to be URLEncoded as well. The character in question is the tilde ~. Wikipedia says no, but the Java URLEncoder class does percent-encode it.