Self-hosted S3 doesn't work

[larrysalibra]

The self-hosted S3 option for user profiles is currently broken because a valid date header isn't being sent with PUT requests:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>AWS authentication requires a valid Date or x-amz-date header</Message><RequestId>55953331BD193D2A</RequestId><HostId>F0x0cYWfrVu/HJQKtZdrc5RtSntE1SSS9DOPuoV5ai95oWdG7S/59JqRP+AI+fd2F7W5FS5Bpzk=</HostId></Error>

This issue on knox indicates that browserify strips the Date header as unsafe: Automattic/knox#299

knox prevents us from using the alternative x-amz-date (which doesn't get stripped by browserify).

I've forked knox and spent some time modifying it to use x-amz-date instead, however I'm having trouble getting signatures to match.

---

Additionally, since we've moved from electron to in-browser, users wishing to use the self-hosted S3 must add a CORS policy to their S3 bucket (assuming we can resolve the above issue). I'm currently using the following:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>Authorization</AllowedHeader>
    </CORSRule>
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>PUT</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>POST</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>Authorization</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

[larrysalibra]

@shea256 If we're going to be storing user self-hosted assets in a cloud provider (like dropbox) connected to a local instance of blockstack-core, does it make sense to spend time working around these issues?

[shea256]

@larrysalibra I feel like S3 should still be an option for users. They might want to use that instead of Dropbox. That said, if we can get Dropbox support up sooner rather than later, we could always focus on that one.

Why don't we create an issue for adding dropbox support? We can add a "login with dropbox" button and we can get an app-specific API key for GETs and PUTs.

[larrysalibra]

@shea256 I'm a bit confused. I was under the impression that once the blockstack-core REST API is ready, profile and profile assets (along with things that apps want to store) would be stored in a user store accessible via blockstack-core's REST API and backed by various cloud providers.

If that's the case, both this web app and blockstack-core will need to talk to cloud storage providers? Wouldn't it be better to have this app talk to the local blockstack-core api and blockstack-core talk to the cloud providers (since it needs to do this anyways)?

We can have this app provide an interface for acquiring credentials for various cloud providers and pass them back to the local blockstack-core instance.

--

I agree S3 & Dropbox (among others) should options for users. Just want to make sure we're putting functionality in the right place.

[larrysalibra]

Can consider using: https://www.npmjs.com/package/aws-signature-v4 to generate a signature and upload directly using:
http://docs.aws.amazon.com/AmazonS3/latest/dev/HTTPPOSTExamples.html

[larrysalibra]

temporarily removed s3 self hosting until we can get it working. 9bf2a89

[larrysalibra]

Since we now have Dropbox self-hosting #141 , I'm pushing this off for the next release so that we can focus on other issues.