Datastore + Aragon Drive App

[macor161]

Request for Nest membership and funding #15

Team name: Espresso Team

Research whitepaper: Whitepaper

Burn rate: $18k/month for 8 months

Legal structure: Aragon DAO

Team and Roadmap

Proposal: Datastore + Aragon Drive App

Sharing and storing files is a fundamental necessity in every organization. The majority of services available today (Dropbox, Google Drive, Box, Onedrive...) are centralized and though they provide cheap and easy access to storage, there are also numerous consequences to this centralization: far from perfect server availability, arbitrary censorship, flawed privacy, possibility of power misuse and these are just a few examples of the drawbacks it can cause.

Of course, decentralized alternatives exist and are starting to fill the gap, but nothing is specifically designed to address the needs of DAOs. With the recent release of Aragon 0.5, creating and managing DAOs is more simple than ever but there is still a need for a service that would let people as well as apps to easily store and share files with specific permissions to other members of the organization and to the general public.

This is what we propose here: The Datastore, a set of smart contracts based on AragonOS coupled with a javascript interface that will let entities of an organization store, modify, delete and set permissions to files and other data.

To showcase the possibilities of this tool, we also propose to create Drive, an Aragon app that will use the Datastore to let members of a DAO upload files and organize them by folders.

Preview of the Drive App:

[macor161]

This is our interpretation of the Datastore, don't hesitate to propose any changes! ;o)

[izqui]

Awesome work, really excited about this idea. What does the first 12 week phase of the project involve? What are the 'basic smart contracts'? Also what does aragonOS integration entail besides the permissions?

[macor161]

Hey Jorge! Thanks! Basically, the first 12 weeks will be to create a minimum viable product of the datastore with a simulation of Keep and dummy encryption workflow. It will only support public files.

The basic smart contracts will contain the Datastore contract itself, the File smart contract and a pluggable system for storage providers so we can easily support Swarm in the future.

aragonOS integration will pretty much be about the ACL hehe! But it'll be the first time we work with aragonOS on a project of this scale so we will probably mess around a lot with it to see if we can integrate other components. I'm particularly interested in testing the concept of a forwarder for the pluggable storage part.

[mariapao]

Hi @macor161 we have been reviewing your application and have the following questions:

1. Have you talked to anybody from the keep team to make sure the timeline is feasible?

2. Is there a particular reason you want to deploy a File contract for every file instead of having the File data as a struct mapping in the Datastore smart contract?

3. Have you thought about data availability? In terms of storing on IPFS, there is no guarantee it will be there forever. Do you want to leverage Filecoin or something similar, or are you going to provide auxiliary services where you pay a monthly fee for someone to pin the file?

4. Have you thought about how you want to create the comment system?

[macor161]

Hello @mariapao, I hope the following comment will answer your questions ;) Don't hesitate if you have anything else to ask.

1 . We have naturally contacted the team before submitting the whitepaper and it's true that there is a real risk that the technology will not be ready in time. This is why we have proposed the following planning:

Since Keep is only necessary to ensure read protection for private files, it means we can work without problems on:

• Write access
• Storage
• Encryption
• Javascript interface
• The entire Drive app with public files support

And though we would ideally like to work on the Keep integration starting at the 13th week, we can push back the task up to the 7th month without real consequences.

And even then, if Keep is not yet ready, we could ship a complete Datastore and Drive app that manage public files and supports 90%+ of the features, which I think would still be useful to a lot of people.

2 . We had a few discussions on the subject me and Maxime Cote, another of our team member. He strongly favors the struct map solution as it would be more efficient and simpler to implement. But I personally find that separating the logic into multiple contracts makes the code more readable and generally easier to test. I think the File contract solution would be a more modular approach and would make it easier to add more features in the future. But it’s true that a contract for each file would be more expensive and I’m totally open to discuss the subject further and/or to quickly test both solutions to see if the tradeoffs are worth it.

3 . We will absolutely offer Filecoin and Swarm as soon as they are production ready! ;) (We have applied for the Filecoin Early Users program and will start working on it as soon as we have access) IPFS is really only used because neither Filecoin or Swarm are ready yet. In the meantime, we will set up our own network of nodes with pinned files and do periodic cleanups of everything that is not related to the Datastore. We don’t plan to use external pinning services.

Of course it’s not a perfect solution (possibility of abuse, etc) but if we take into account that the Datastore/Drive app will be released 8-9 months from now, it will be a really short time temporary solution before we move to Filecoin/Swarm.

4 . Of course we have hehe! By default, anyone with read access will have the permission to add a comment to a file. The owner of the file will be able to switch comments off if wanted.

As for the question of whether the comments should be readable by somebody who doesn’t have read access to the file, we are still thinking about it. If the comments are not considered sensitive information, then a simple smart contract will do the job. But if they are, we would use the Datastore itself to store the encrypted comments.

Public files must be treated with a special care because of the danger for spam and general abuse. Even though we suspect that for many malicious users, the small fee for posting comments will counterbalance the economic incentives of posting spam, stronger protection mechanisms still need to be put in place. After a few discussions with the team, we agreed that the easiest to use and most effective way to prevent abuse would be a simple whitelist at the app level where admins would give access to members. Other options could be added in the future, like permitting tokenholders with more than x tokens to comment.

[leftab]

2. +1 for the map of struct approach!! ;)

[onbjerg]

Thank you for your answers, it clarified a lot of things. As for the mapping approach, I think you should try it out if you have the time (without skewing your timeline, of course) - I don't think you gain a lot of benefits from doing a smart contract per file 😊

[macor161]

Well it looks like I'm the minority here haha! We will definitely try out the struct map approach ;)

[mariapao]

Hi @macor161 thanks for the answers :)

We are trying to assess the ability of the team to take on the project. This project is heavy on smart contracts and blockchain technology, and we need to have a way to evaluate if the team will be able to ship it. I was trying to look at the github profiles but there is not much information there. We encourage teams to apply with ideas as long as we are able to evaluate the skills in other ways such as github activity, collaboration in the Ethereum/Aragon dev communities, etc.

Do you think you'll be able to submit an mvp or at least a poc? Thanks!

[macor161]

Sure no problem! We will need a little bit of time though, is it alright if we show you a POC/MVP in 2-3 weeks?

[mariapao]

Yes, take your time!

[macor161]

Hi I just want to do a quick follow-up. So far the development is going great! But we will probably need a little more time, I'd say approximately 2 weeks if it's ok.

[macor161]

Hello! You can find our proof of concept of the Datastore at the following address : https://github.com/macor161/aragon-datastore

Hope you’ll like it :)

So just to put some context, the Datastore consists of 2 parts:

1. A javascript library to store files in a distributed file system, IPFS as of right now. It will also handle the encryption/decryption part.
2. A solidity smart contract that keeps the information of those files and manages permissions.

To make it as extensible as possible we used a system of providers in the js library. This is how you create an instance of the Datastore in an Aragon app as of right now:

const datastore = new Datastore({
      storageProvider: new providers.storage.Ipfs(),
      encryptionProvider: new providers.encryption.Aes(),
      rpcProvider: new providers.rpc.Aragon(aragonApp)
});

So for example, one would be able to use Filecoin by selecting a different storage provider or use web3 in a normal DApp by changing the 4th line to:

rpcProvider: new providers.rpc.Web3(...)

Here is an example of an Aragon app that uses the Datatore and also a Web3 DApp :

https://github.com/macor161/aragon-drive-app
https://github.com/macor161/datastore-web3-example

With this POC done, we have re-evaluated our roadmap and estimate that it would now take 8 months instead of 9 to complete this project, so I updated the roadmap.

And here are two small demos:

1. Permission management using web3: https://www.youtube.com/watch?v=rzi5iRr8tVA
2. General features using Aragon: https://www.youtube.com/watch?v=FG8iYRSE75I

A few notes

We first intended to make the Datastore as a separate Aragon app with no ui, that you can install with the apm. But we found a couple of drawbacks. It wouldn't be easy for different apps to use the Datastore with different parameters (e.g. one using IPFS and the other Filecoin). The other drawback is the Datastore would become a dependency that a user must manually install before being able to use an app the requires storage. We think it would not be very user friendly. So for the moment, the Datastore is a library that you import in your project.

Also, right now we use internal logic instead of the ACL for the permissions because after talking with @sohkai, it turns out that it's not possible yet to set new permissions with the ACL from the frontend of an app. For each file, we have a map of address that points to read/write permissions. It still works pretty well even without the ACL though :)

There is no readme yet, I will try to add one tonight. Feel free to ask me any questions.

[mariapao]

Hi @macor161 thanks a lot for submitting a PoC. We will start reviewing it now and will let you know if we have questions/doubts

[luisivan]

This is very cool

[mariapao]

Hi @macor161 we have been reviewing the PoC and we like it very much!

I just want to check with you, because it has been a while since we asked for the PoC, there is any information that should be updated on the application? team, roadmap? Let us know! Thanks :)

[macor161]

Hi @mariapao glad you like it :o) We have changed the roadmap from 9 to 8 months after submitting the POC but everything should be up to date right now. Feel free to ask if you have any other questions.

[bingen]

Really nice job guys, I love this app. From my review so far, I have a couple of first doubts:

• About the inheritance issue here:
  https://github.com/macor161/aragon-drive-app/blob/master/contracts/DriveApp.sol#L7
  Why are you using that empty DriveApp contract? Is it because you want to keep a non-Aragon version for the web3 Dapp?

• I see the logic is contained in just one contract which is good as we have experienced some problems linking more complex apps that comprise several contracts due to the way aragon.js works (kind of similar to what you already experienced, actually). After reading your whole proposal, I can not imagine why you would need to add more contracts to it and I guess the remaining features can be done within the current one, but if you plan to change your architecture just keep this in mind. We plan to fix those issues, but don't know when yet, so it may become a stopper for you.

[macor161]

Thanks @bingen ! :)

Yeah this empty contract is a little bit weird right now since the app is not complete yet hehe. But there is a reason for it to be there. Since the Datastore is a library that will be used with a variety of different apps, it contains a limited set of features by design. For example, it makes no assumption as to how the files should be organized. So the folder structure will be handled at the app level, in the DriveApp contract. The comments system is another example of a feature that will be implemented by the app itself.

Regarding the second point, I totally agree with you. We initially thought of separating the Datastore in multiple contracts but we changed our mind while working on the PoC. One contract is more simple to use and is also more efficient.

[mariapao]

Hi @macor161 after reveiwing the PoC and application we are pleased to let you know that your request for funding is approved. This is an app we would love to see in Aragon :)

Next steps: I'll send you an email to schedule a call this week to go over some minor details to close this process and discuss the onboarding process to the program.

Welcome to nest! :)

[macor161]

Awesome! :o) On behalf of all the team, I would like to thank the Aragon developers and @mariapao for their support. You were always there to answer our questions on Github or in the chat and this is what makes the Aragon community so vibrant. We are extremely grateful for this oportunity and to be able to take part in helping build this incredible ecosystem that is Aragon.

[Smokyish]

@mariapao Are there any blockers for merging this?