Distributed Blockchain-based File Storage in Elixir
DBFS is an experiment to implement a (very naïve) distributed file storage service using a blockchain
in Elixir. With growing interest in Blockchains and Cryptocurrencies, I wanted to understand the complex
inner workings of Blockchains and their feasibility in the context of business applications. I started
by writing a very simple Blockchain to get started, and then wrote
DBFS as a
slightly more complex version that actually did something.
Screenshots, Talk and Paper
I also wrote a basic whitepaper detailing the application as part of my semester project, and you can also see some screenshots of the application here:
The application is divided into two parts:
- Elixir Backend: The core application, responsible for storing the data, performing consensus among nodes and exposing an API for clients
The application is designed as a "Private Blockchain" after weighing the pros and cons of different approaches, and even though it should absolutely not be used in production, it is meant to be run on private infrastructure instead of being made publicly available so any node can connect.
sys.config file defines 3 nodes. You can change this to add or remove nodes, but you'll have to
perform the setup on each node (if you're running it on different machines). You'll also have to make
sure that your
hosts file points these domains to their correct IPs.
You need to have these dependencies installed at minimum:
- Erlang/OTP 20.1
- Elixir 1.5.2
- Postgres 9.4
- Node.js 9.6.1 for Assets and Yarn
- Ruby 2.3.3 for Sass
Set up the Backend:
$ git clone to.shyr.io/dbfs ~/dbfs $ cd ~/dbfs $ mix deps.get $ mix compile
Set up the Frontend:
$ git clone to.shyr.io/dbfs-web ~/dbfs-web $ cd ~/dbfs-web $ yarn install
For the first node of the backend, we need to create its instance and initialize the blockchain database. For other nodes, we can simply create their instances without creating the blockchain (They will automatically be synchronized when we start them). Assuming, you're starting all nodes on the same machine:
$ cd ~/dbfs $ NODE=newyork PRIMARY=1 mix do ecto.create, ecto.migrate, ecto.seed $ NODE=london PRIMARY=0 mix do ecto.create, ecto.migrate, ecto.seed $ NODE=singapore PRIMARY=0 mix do ecto.create, ecto.migrate, ecto.seed
Get it Running
To finally start the backend nodes, you need to pass them a port, node name and boot-up config. You can start one node or multiple, depending on the requirements.
$ NODE=newyork PORT=3000 elixir --name firstname.lastname@example.org --erl "-config sys.config" –S mix phoenix.server
If you decide to run multiple nodes together, you also need to initialize the distributed Mnesia tables which are responsible for performing consensus across the network. First enter the REPL command-line of one of the running nodes, and enter the following:
Finally, to start the Web-Client:
$ cd ~/dbfs-web $ PORT=4000 yarn start
To use the application and upload/download files, they need to be encrypted using your private key. You can
generate one using
$ ssh-keygen -b 1024 -t rsa -f key -q -N ""
Nothing. This was just an experiment and a learning experience. I'm not planning on doing any more work on this, but you're free to play around and learn from my mistakes (You should absolutely not make the same design decisions I made).
All code here is available as open source under the terms of the MIT License.