Skip to content

Conversation

@bertrandg
Copy link

No description provided.

@AidasK
Copy link
Member

AidasK commented Feb 4, 2021

Nice job! @drzraf @evilaliv3 Can you think of any additions to this?

@drzraf
Copy link
Collaborator

drzraf commented Feb 5, 2021

An asymmetric StreamEncryptor?

class StreamEncryptor {
    constructor(gpgKeys) {
        this.gpgKeys = gpgKeys;
        this._reader = [];
    }

    async init(flowObj) {
        const { message } = await openpgp.encrypt({
            message: openpgp.message.fromBinary(flowObj.file.stream(), flowObj.file.name),
            publicKeys: this.gpgKeys
        });

        this._reader[flowObj.uniqueIdentifier] = openpgp.stream.getReader(message.packets.write());
        flowObj.size = flowObj.file.size + compute_pgp_overhead(this.gpgKeys, flowObj.file.name);
    }

    async read(flowObj, startByte, endByte, fileType, chunk) {
        const buffer = await this._reader[flowObj.uniqueIdentifier].readBytes(flowObj.chunkSize);
        if (buffer && buffer.length) {
            return new Blob([buffer], {type: 'application/octet-stream'});
        }
    }
}

var encryptor = new StreamEncryptor(gpgKeys);
new Flow({
    // ...
    asyncReadFileFn: encryptor.read.bind(encryptor),
    initFileFn: encryptor.init.bind(encryptor),
    forceChunkSize: true,
});

@bertrandg
Copy link
Author

@drzraf nice! :)
I've never used openpgp and curious about it, I see it encrypts message (so file bytes here) with recipient public key.
So using stream you need to upload chunks one after one? (no chunk upload parallelization)

In my case, I use AES symmetric keys (to encrypt/decrypt files) managed with RSA asymmetric key pairs.

We can modify the page and just put our 2 file encryption methods.

@AidasK
Copy link
Member

AidasK commented Feb 8, 2021

@bertrandg that would be awesome, can you add it please? 👍

@drzraf
Copy link
Collaborator

drzraf commented Feb 8, 2021

@drzraf nice! :)
So using stream you need to upload chunks one after one? (no chunk upload parallelization)

The trick is that my query() {} uploads to a ${chunk.offset}-derivated URLs in order to create OpenStack Swift DLO (which take care of reassembling itself).

Since each chunk has it's own URL I can upload in parallel : Thanks to v3 stream support and respect for simultaneousUploads and since encryption is quick in comparison to upload, it ends up being parallel uploads of sequential (and non-racy, thanks to readStreamGuard / readStreamChunk) stream read.

@bertrandg
Copy link
Author

bertrandg commented Feb 23, 2021

I've updated the doc file with your example @drzraf

@AidasK
Copy link
Member

AidasK commented Feb 23, 2021

Sweet, can we merge it @drzraf ?

@drzraf drzraf merged commit 218b2bf into flowjs:v3 Feb 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants