Feature/keyfile #739

jcnelson · 2017-09-10T22:23:06Z

Requires:

blockstack-storage.js 0.3.0 (in the develop-multiplayer-storage branch)
blockstack-core (rc-0.14.3-with-tokenfile branch)
blockstack.js (develop-keyfile branch)

…JSON string or a JWT. Handle both cases.

…d signing key, which will be used to generate key files for profiles.

…profile with a keyfile instead.

…rofile's keyfile)

…red in the keyfile

…der datastore for the application for the given account, and (2) will re-generate the account's profile to add/update the key file with the datastore's routing information.

…r has a blockchain ID, create a multi-reader datastore and update the user's profile's keyfile with the datastore's routing information. Upload the profile to the user's storage providers, and update the locally-cached profile state to reflect the newly-created profile.

CLAassistant · 2017-09-10T22:23:12Z

Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 4 committers have signed the CLA.

✅ kkomaz
❌ jcnelson
❌ larrysalibra
❌ kantai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

jcnelson · 2017-09-10T22:24:05Z

The failed check is due to the fact that a required dependency blockstack-storage is not in npm yet (need 0.3.0, available in the develop-multiplayer-storage branch).

larrysalibra · 2017-09-13T12:41:34Z

I tried testing this with the existing todos app to make sure it doesn't break existing apps and it doesn't work.

To make sure we are on the same page, my understanding is

existing apps should continue to function without update because all of the interfaces are the same.
we need a migration tool is because the on disk format has changed.

This being the case, I did not update the todos app to use the newer version of blockstack.js.

api_endpoint.log output

 seconds
[2017-09-13 19:36:46,703] [DEBUG] [rpc:1415] (22463.123145418272768) Failed to get datastore: Failed to load public datastore record
127.0.0.1 - - [13/Sep/2017 19:36:46] "GET /v1/stores/16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM?device_ids=0&blockchain_id=a50d01b4478039d12113e8b74d02dc83878cf72b8991161b31734cb37732e2c1 HTTP/1.1" 404 -
[2017-09-13 19:36:46,887] [DEBUG] [registrar:949] (22463.123145409859584) Registrar sleeping for 1
[2017-09-13 19:36:47,054] [DEBUG] [rpc:4106] (22463.123145418272768) 
full path: /v1/stores
method: POST
path: /v1/stores
qs: {}
headers:
origin: http://localhost:8080
content-length: 1868
accept-language: en-US,en;q=0.8
accept-encoding: gzip, deflate, br
connection: keep-alive
accept: */*
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
host: localhost:6270
referer: http://localhost:8080/
content-type: application/json
authorization: bearer eyJhbGciOiJFUzI1NksiLCJ0eXAiOiJKV1QifQ.eyJhcHBfZG9tYWluIjoiaHR0cDovL2xvY2FsaG9zdDo4MDgwIiwibWV0aG9kcyI6WyJzdG9yZV93cml0ZSJdLCJ0aW1lc3RhbXAiOjE1MDUzMDU4MTMsImV4cGlyZXMiOjE1MDc5ODQyMTMsImFwcF91c2VyX2lkIjoiMTZta1lmOGVGc3hOaVk0OHFSdExNTFE3Q2ZiSER1cVJ4TSIsImRldmljZV9pZCI6IjAiLCJibG9ja2NoYWluX2lkIjpudWxsLCJzdG9yYWdlIjp7ImNsYXNzZXMiOnsid3JpdGVfcHJpdmF0ZSI6WyJkaXNrIiwiZHJvcGJveCJdLCJyZWFkX2xvY2FsIjpbImRpc2siXSwicmVhZF9wdWJsaWMiOlsiZHJvcGJveCJdLCJ3cml0ZV9sb2NhbCI6WyJkaXNrIl0sIndyaXRlX3B1YmxpYyI6W10sInJlYWRfcHJpdmF0ZSI6WyJkaXNrIl19LCJwcmVmZXJlbmNlcyI6e319LCJhcGlfZW5kcG9pbnQiOiJsb2NhbGhvc3Q6NjI3MCIsImFwcF9wdWJsaWNfa2V5cyI6W3sicHVibGljX2tleSI6IjAyMDA1N2MxMGI3YzlmODc0YWQzM2IxN2FhMGQzZjc2MDI2MTQyYTNmZmMyMzJlY2RkOTkxMzA2OTkzNjhmN2NiNyIsImRldmljZV9pZCI6IjAifV0sInZlcnNpb24iOjF9._QMWlWs9pNCtDXAgm9ucmW2Q7gbXhCbTTehDqEanQPq98mWFs7q0FGB8tYRhG-wGbRrsdc0T8Nq-dx2YQnekRw

[2017-09-13 19:36:47,127] [DEBUG] [app:177] (22463.123145418272768) In test mode; verifying that sessions are not stale
[2017-09-13 19:36:47,128] [DEBUG] [rpc:4149] (22463.123145418272768) localhost:8080 is not a valid app domain
[2017-09-13 19:36:47,129] [DEBUG] [rpc:4212] (22463.123145418272768) Authenticated with session
[2017-09-13 19:36:47,131] [ERROR] [rpc:1442] (22463.123145418272768) Additional properties are not allowed (u'root_blob_idata', u'root_blob_header' were unexpected)

Failed validating 'additionalProperties' in schema['properties']['datastore_info']:
    {'additionalProperties': False,
     'properties': {'datastore_blob': {'type': 'string'},
                    'root_blob': {'type': 'string'}},
     'required': ['datastore_blob', 'root_blob'],
     'type': 'object'}

On instance['datastore_info']:
    {u'datastore_blob': u'{"data":"{\\"device_ids\\":[\\"0\\"],\\"drivers\\":[\\"dropbox\\",\\"disk\\"],\\"pubkey\\":\\"040057c10b7c9f874ad33b17aa0d3f76026142a3ffc232ecdd99130699368f7cb7a1e0f1071d97f69ce3668c3abd36cb63b7c069df400822726dd3a01165572438\\",\\"root_uuid\\":\\"393607f7-b5fa-4976-a907-f9129b94e0e7\\",\\"type\\":\\"datastore\\"}","fq_data_id":"0%3A16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM.datastore","timestamp":1505306207028,"version":1}',
     u'root_blob_header': u'{"data":"{\\"data_hash\\":\\"6c7908a92b3b8b5635be4d1e172db068d2a5e071be3abe826806523451e65e8f\\",\\"owner\\":\\"16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM\\",\\"proto_version\\":1,\\"readers\\":[],\\"type\\":2,\\"uuid\\":\\"393607f7-b5fa-4976-a907-f9129b94e0e7\\",\\"version\\":1}","fq_data_id":"0%3A16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM.393607f7-b5fa-4976-a907-f9129b94e0e7.hdr","timestamp":1505306207028,"version":1}',
     u'root_blob_idata': u'{"children":{},"header":{"data_hash":"0000000000000000000000000000000000000000000000000000000000000000","owner":"16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM","proto_version":1,"readers":[],"type":2,"uuid":"393607f7-b5fa-4976-a907-f9129b94e0e7","version":1}}'}
Traceback (most recent call last):
  File "/Users/larry/git/blockstack-testing/tokenfile/blockstack-venv/lib/python2.7/site-packages/blockstack-0.14.5.0-py2.7.egg/blockstack_client/rpc.py", line 1439, in _store_signed_datastore
    jsonschema.validate(datastore_sigs_info, CREATE_DATASTORE_REQUEST_SCHEMA)
  File "/Users/larry/git/blockstack-testing/tokenfile/blockstack-venv/lib/python2.7/site-packages/jsonschema/validators.py", line 541, in validate
    cls(schema, *args, **kwargs).validate(instance)
  File "/Users/larry/git/blockstack-testing/tokenfile/blockstack-venv/lib/python2.7/site-packages/jsonschema/validators.py", line 130, in validate
    raise error
ValidationError: Additional properties are not allowed (u'root_blob_idata', u'root_blob_header' were unexpected)

Failed validating 'additionalProperties' in schema['properties']['datastore_info']:
    {'additionalProperties': False,
     'properties': {'datastore_blob': {'type': 'string'},
                    'root_blob': {'type': 'string'}},
     'required': ['datastore_blob', 'root_blob'],
     'type': 'object'}

On instance['datastore_info']:
    {u'datastore_blob': u'{"data":"{\\"device_ids\\":[\\"0\\"],\\"drivers\\":[\\"dropbox\\",\\"disk\\"],\\"pubkey\\":\\"040057c10b7c9f874ad33b17aa0d3f76026142a3ffc232ecdd99130699368f7cb7a1e0f1071d97f69ce3668c3abd36cb63b7c069df400822726dd3a01165572438\\",\\"root_uuid\\":\\"393607f7-b5fa-4976-a907-f9129b94e0e7\\",\\"type\\":\\"datastore\\"}","fq_data_id":"0%3A16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM.datastore","timestamp":1505306207028,"version":1}',
     u'root_blob_header': u'{"data":"{\\"data_hash\\":\\"6c7908a92b3b8b5635be4d1e172db068d2a5e071be3abe826806523451e65e8f\\",\\"owner\\":\\"16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM\\",\\"proto_version\\":1,\\"readers\\":[],\\"type\\":2,\\"uuid\\":\\"393607f7-b5fa-4976-a907-f9129b94e0e7\\",\\"version\\":1}","fq_data_id":"0%3A16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM.393607f7-b5fa-4976-a907-f9129b94e0e7.hdr","timestamp":1505306207028,"version":1}',
     u'root_blob_idata': u'{"children":{},"header":{"data_hash":"0000000000000000000000000000000000000000000000000000000000000000","owner":"16mkYf8eFsxNiY48qRtLMLQ7CfbHDuqRxM","proto_version":1,"readers":[],"type":2,"uuid":"393607f7-b5fa-4976-a907-f9129b94e0e7","version":1}}'}
[2017-09-13 19:36:47,133] [ERROR] [rpc:1444] (22463.123145418272768) Invalid datastore and sig info

I'm going to move on to trying to test this after updating blockstack.js to the patched version on my local copy of todos

jcnelson · 2017-09-13T15:23:04Z

You will not only need the develop-keyfile branch of blockstack.js, but also the rc-0.14.3-with-tokenfile branch of blockstack-core.

larrysalibra · 2017-09-14T13:31:09Z

@jcnelson

I verified that I am using the rc-0.14.3-with-tokenfile branch of core and am still getting the above error with the currently shipping version of blockstack-todos:

Specifically, I'm using the blockstack-venv.tar.gz artifact from this build: https://circleci.com/gh/blockstack/blockstack-core/618#artifacts/containers/0

muneeb-ali · 2017-09-15T18:34:02Z

Looks like this one needs more work in the current sprint.

@jcnelson were we able to figure out the issue that Larry was having?

jcnelson · 2017-09-15T18:39:09Z

@muneeb-ali Yes--blockstack.js does not break application-facing interfaces, but it does break the application/Core interface. This means that either we need to ship both Gaia v1 and Gaia v2 code paths in Core to support legacy blockstack.js, or we need to get everyone upgraded to the new blockstack.js and have it redirect the user to upgrade their Core node if they try to use it with a legacy Core node.

v0.16.0

Fix Search Button styles and refactor button vs. link logic

…p local node_modules out of the picture

…owser into feature/keyfile

kantai · 2017-09-26T19:53:39Z

Okay -- here's some instructions for getting this running with blockstack-core, browser, and blockstack todos using docker containers:

Get these repos:

git clone https://github.com/blockstack/blockstack-browser -b feature/keyfile+gaiahub
git clone https://github.com/blockstack/blockstack-core -b rc-0.14.3-with-tokenfile
git clone https://github.com/kantai/blockstack-todos -b multi-reader-storage

Now -- build your containers:

cd blockstack-browser && docker build . -f Dockerfile.keyfile.sources --tag browser:keyfile
cd ../blockstack-core && docker build . -f integration_tests/deployment/docker/Dockerfile.tests --tag blockstack-core:keyfile
cd ../blockstack-todos && docker build . --tag todos:keyfile

Now, run your containers:

docker run -dt --name core-keyfile -p 6270:6270 -e BLOCKSTACK_TEST_CLIENT_RPC_PORT=6270 -e BLOCKSTACK_TEST_CLIENT_BIND=0.0.0.0 blockstack-core:keyfile blockstack-test-scenario --interactive 2 blockstack_integration_tests.scenarios.portal_test_env
docker run -d --name todos-keyfile -p 5000:5000 -e BSK_TODOS_PORT=5000 -e BSK_TODOS_BINDADDR=0.0.0.0 todos:keyfile npm run start
docker run -d --name browser-keyfile -p 8888:8888 browser:keyfile blockstack-browser
docker run -d --name browser-cors -e CORSPROXY_HOST="0.0.0.0" -p 1337:1337 browser:keyfile blockstack-cors-proxy

Open your browser to localhost:8888, make a profile, etc. and use blockstack-todos from localhost:5000

This works for me in regtest -- would be good to get other people testing this out as well.

kantai · 2017-09-26T21:42:12Z

I've added another branch for the combo of feature/keyfile+gaiahub

kantai · 2017-09-28T14:17:09Z

Moved over to PR #833

jcnelson added 8 commits September 10, 2017 17:56

When resolving a zone file to a profile, the profile can either be a …

0c42553

…JSON string or a JWT. Handle both cases.

update deriveIdentityKeypair() to include the (base58-encoded) key an…

51cbcd8

…d signing key, which will be used to generate key files for profiles.

When signing a profile for upload while registering a name, create a …

563fa70

…profile with a keyfile instead.

registerName() now takes the key index (which will be stored to the p…

f4f8bad

…rofile's keyfile)

when registering a name, include the identity address index to be sto…

857ad0e

…red in the keyfile

add a method setupAppDatastore(), which will (1) create a multi-rea…

6c9557c

…der datastore for the application for the given account, and (2) will re-generate the account's profile to add/update the key file with the datastore's routing information.

require blockstack-storage 0.3.0

e7e4d5e

no longer need a blockchain ID to create a keyfile

9d9aa87

larrysalibra self-requested a review September 13, 2017 12:41

larrysalibra mentioned this pull request Sep 13, 2017

Story: multi-reader storage #752

Closed

larrysalibra and others added 7 commits September 21, 2017 23:58

Merge tag 'v0.16.0' into develop

9219ee4

v0.16.0

add windows + linux instructions for the release checklist

6143324

Fix Search button styles and refactor button vs. link logic

83a5ec2

Revert navigation back to Links

d7ddd3f

Merge pull request #823 from kkomaz/search-color-fix-part-2

176cf00

Fix Search Button styles and refactor button vs. link logic

add note to rename files

f4a639e

Merge branch 'feature/checklist-win+linux' into develop

62db6fc

larrysalibra added the backlog label Sep 22, 2017

kantai and others added 4 commits September 22, 2017 18:48

Merge branch 'master' into feature/keyfile

5eb8f74

Dockerfile to build keyfile js, plus including a .dockerignore to kee…

6d9d10b

…p local node_modules out of the picture

blockstack-storage-js -> aaron's fork

e19f130

start enforcing lint-free code as part of automated tests #829

1a0d0c0

larrysalibra and others added 6 commits September 26, 2017 21:33

values should be boolean

401bc9f

revert Dockerfile + add a keyfile specific one

4b68914

revert Dockerfile + add a keyfile specific one

f8f1a60

Merge branch 'feature/keyfile' of github.com:blockstack/blockstack-br…

234b98c

…owser into feature/keyfile

Merge branch 'develop' into feature/keyfile

cecaa04

kantai/blockstack-storage-js -> blockstack/blockstack-storage-js

375a1b6

fixes weirdness with the docker filesystem operations

ef6f742

kantai mentioned this pull request Sep 28, 2017

Add support to browser for gaiahub and keyfiles #833

Closed

kantai closed this Sep 28, 2017

jcnelson mentioned this pull request Nov 7, 2017

putFile should return a public link to the file updated hirosystems/stacks.js#288

Closed

timstackblock pushed a commit that referenced this pull request Sep 7, 2021

fix: cursor on settings popover, fixes #739

8d58259

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/keyfile #739

Feature/keyfile #739

jcnelson commented Sep 10, 2017

CLAassistant commented Sep 10, 2017 •

edited

jcnelson commented Sep 10, 2017

larrysalibra commented Sep 13, 2017

jcnelson commented Sep 13, 2017

larrysalibra commented Sep 14, 2017

muneeb-ali commented Sep 15, 2017

jcnelson commented Sep 15, 2017

kantai commented Sep 26, 2017 •

edited by jackzampolin

kantai commented Sep 26, 2017

kantai commented Sep 28, 2017

Feature/keyfile #739

Feature/keyfile #739

Conversation

jcnelson commented Sep 10, 2017

CLAassistant commented Sep 10, 2017 • edited

jcnelson commented Sep 10, 2017

larrysalibra commented Sep 13, 2017

jcnelson commented Sep 13, 2017

larrysalibra commented Sep 14, 2017

muneeb-ali commented Sep 15, 2017

jcnelson commented Sep 15, 2017

kantai commented Sep 26, 2017 • edited by jackzampolin

kantai commented Sep 26, 2017

kantai commented Sep 28, 2017

CLAassistant commented Sep 10, 2017 •

edited

kantai commented Sep 26, 2017 •

edited by jackzampolin