Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Keystore/encryption/ipfs usecase for a file-sharing application #3866
As asked by @whyrusleeping, here is my usecase for ipfs and in particular for the keystore/encryption feature.
I'm building a file sharing application presented as a social network. The main target is friend to friend sharing (think holiday pictures) with a control on who has access to the data by using encryption. The underlying goal is to reduce the need to use solution like Facebook, Dropbox, Drive... for data exchange that should be kept private and to create a better solution.
The project is here (no readme, sorry, I should do that). The tech stack is
What is described below is a general plan and not fully figured out.
I don't use a central authority like an identity server or a blockchain. As the target is a friend-to-friend network, I chose to have identity simply being a crypto keypair (
Each user will have a contact list with the keys stored in the
Users share their contact list with their contact (with some privacy settings). That mean that once the first contact is added, the application can build a suggestion list for the user to easily add new contacts and build his social graph. The same way as before, the profile is retrieved from ipfs and stored locally.
Sharing and encryption
There is 3 kind of sharing. The
When a user share something, the data is added to the IPFS repo (using the no-copy filestore
Targeted sharing (main priority)
Users can share with a set of specific contact. In this case the data should be encrypted with a generated key that is then encrypted for each recipient public key and added as metadata of the share. As far as I can tell this is not possible with the current encryption spec (MISSING)
The root is also signed by the author (
Users can share for anyone interested and follow public identity. In this case the data don't really need to be encrypted (?), but the root should be signed (
User can duplicate their identity on several node and share files to themselves to have kind of private repository of data or synchronization tool. Not really a priority but could be nice. In this case, the data is encrypted using the own author keys and signed.
Relax the online constraint
To relax the constraint of having both user online at the same time for them to communicate, contacts could help each others (they are friends, right ?). That include:
Another way would be to use a commercial pinner.
Tracking the download progress
While I'm at it, I need a way to track the progress of a download (MISSING). That means:
This can partially done by retrieving the file graph using
I'll need some time to dissect this more thoroughly, but on a first readthrough sounds very interesting.
As for tracking download progress, we recently added a --progress option to
I'll try and get the keystore encryption apis worked on soon, but no promises on a timeline :)
What's the plan public sharing? Pubsub with the 'followers' as subscribers?
We'd like to have similar functionality but I'm not sure how well it will work if someone follows a ton of people.
There's also an issue if they come back online from an absence then we want to load the objects that were shared while they were away (easy to do if you only follow a couple nodes but won't work well with larger numbers).
I suppose we could just not load them and only display the last seen messages but there's still the issue of subscribing to many.
Well, that's the more "long term and not really figured out" part.
This way, each user could have a namespaced space in this database to publish sharing metadata. Data would have a longer lifetime and the would be replicated. That could solve both of your problems.
In summary, we are a decentralised file store (built on IPFS) with private sharing between users and we try to hide not only the file data and metadata, but also the social graph (You don't even see your friends' friends by default). Access is capability (CAP) based and can be read only or writable. We achieve, public sharing by just sharing a file/directory's CAP publicly, e.g. in a link:
We don't need a key store, because the root keys are derived from your username and password (and never writtent o disk), which gets you decentralized log-in for free. The remaining keys (every file/dir has several keys associated with it) are stored in a cryptree data structure. All the encrypted file content, including the cryptree structure is stored in a merkle-btree in IPFS.
We can share files 1-to-many in a way that the network can't deduce the social graph (in the data structures at rest, or with live traffic analysis once we use Tor). We are also are careful about quantum computers and current all files are resistant to quantum computer attacks, excluding ones that have been shared (and we're working on that).
We get tracking of file download progress for free because we have to download the fragments ourselves to decrypt them locally (each 5MiB chunk is encrypted independently, and each of those is split into 128 KiB fragments).
Incidentally, @cpacia we've been investigating getting IPFS working over Tor and tried to follow your efforts. Do you have any detailed write-ups of the process?