Data Storage Integration

[sofiatti]

As a person going through the demo, I want a data storage integration so that I can understand off-chain storage.

Prior completion of #4 is desired

Acceptance Criteria

• University names and rankings are saved in an external storage space

Please also note:

1. If a conversation about this issue is happening when it is being worked on (i.e., has an assignee),
   the outcomes should be documented here (not in the comments).

Resources:

Add image here.

[ceteri]

How would the query results look? For example, this describes two columns [ "name", "ranking"] so could a simple version of the results be a JSON list of tuples with those two columns? As string data? Or (string, int)?

Future:

• ostensibly, we'll have error codes, etc., added later
• also other mime types supported for the responses

@ceteri, I'll be addressing these questions in #4! Check out that issue later and let me know if you have additional comments -Caroline

[ceteri]

Got a prototype service to demo ...

The request, after authentication:

http://35.197.28.157:5000/auth/get?file=univs.json

Sample results:

{
  "results": [
    { "rank": 1, "name": "Deep Springs College" },
    { "rank": 2, "name": "Leland Stanford Junior University" },
    { "rank": 3, "name": "Bard College at Simon's Rock" },
    { "rank": 4, "name": "Gupton-Jones College of Funeral Service" },
    { "rank": 5, "name": "University of Coimbra" }
  ]
}

[ceteri]

FWIW, this is example code -- no effect on Computable.io code base -- except to show a simple, extensible example for what @rbharath proposed last Monday in the team meeting. I would suggest call this "storage" instead of "database", since the former is more general and pluggable, while the latter has strong connotations which will likely confuse people who are outside of our discussions.

Anywho, here's the sample code -- less the nginx scripts: https://github.com/DerwenAI/stor
Though it was built for running as WSGI with nginx fronting it. I'll add those quickly, e.g., we'd want to run HTTPS-only/TLS/HSTS, though that will require:

1. a subdomain from computable (or other domain)
2. a certificate for the TLS (shared/cheap is fine)

@ceteri: Changed the name of the issue. Looking forward to trying this out! @SUICIDALcow, take a peak! -Caroline

[ceteri]

Also, the "key management" and user model is super trivial at first: it's a JSON with user tuples as ("email", "pwd-hash") based on bcrypt_sha256

However, the libraries used provide for secure authentication, API keys, etc. That all depends on product specs :)

@paco, I'll add this to #6!

[geoffkinder]

This task has been completed using IPFS and storing a “block” of data, i.e. a byte array of a stringified json blob. This allows complete data flexibility, but will probably have to be revisited when we investigate our data query requirements/strategy.

After persistence to IPFS, we can store the IPFS CID (content identifier) on the data attribute of a registry listing and use it for data retrieval later.