Bad encryption performance

[vitonsky]

Current implementation of encryption takes ~3 seconds to complete encryption integration test if buffer size is 4400000 (4.19Mb) that is quite real size of database.

Postgres data takes 3mb for empty DB.

To reproduce test, modify text sample to 'Hello world! This is encryption example text'.repeat(100000).

The performance is ridiculous low. The interesting fact is that disabling ciphers does not improve performance.

We have to figure out the problem and think how to fix it.

In theory, we could move encryption to WASM if would have proofs it may improves performance.

[vitonsky]

Recent tests from PR #129

• no compressed database 29.3 MB
• total sync time that includes database dump creation + encryption + write buffer at FS
• sync was triggered by notes opening and random editing

No encryption:
ManagedDatabase.ts:89 Sync is done 398.2999999821186
ManagedDatabase.ts:89 Sync is done 394.19999998807907
ManagedDatabase.ts:89 Sync is done 403.69999998807907
ManagedDatabase.ts:89 Sync is done 396.10000002384186
ManagedDatabase.ts:89 Sync is done 402
ManagedDatabase.ts:89 Sync is done 384.90000000596046
ManagedDatabase.ts:89 Sync is done 350.10000002384186
ManagedDatabase.ts:89 Sync is done 398.19999998807907
ManagedDatabase.ts:89 Sync is done 329.5


AES encryption (with hardware acceleration):
ManagedDatabase.ts:89 Sync is done 636.9000000059605
ManagedDatabase.ts:89 Sync is done 757.9000000059605
ManagedDatabase.ts:89 Sync is done 773
ManagedDatabase.ts:89 Sync is done 634.7999999821186
ManagedDatabase.ts:89 Sync is done 523.2999999821186
ManagedDatabase.ts:89 Sync is done 586.1000000238419
ManagedDatabase.ts:89 Sync is done 680.4000000059605
ManagedDatabase.ts:89 Sync is done 571.5

Combined encryption:
ManagedDatabase.ts:89 Sync is done 16136.59999999404
ManagedDatabase.ts:89 Sync is done 6673.600000023842
ManagedDatabase.ts:89 Sync is done 11104.59999999404
ManagedDatabase.ts:89 Sync is done 10803.40000000596
ManagedDatabase.ts:89 Sync is done 7711.5999999940395

AES encryption show acceptable performance overhead (avg about 200ms) while combined encryption Twofish-AES perform very slow (overhead is a many seconds).

Patch to apply at ccf211a to reproduce

diff --git a/src/core/storage/database/ManagedDatabase.ts b/src/core/storage/database/ManagedDatabase.ts
index 9094d92..7e65535 100644
--- a/src/core/storage/database/ManagedDatabase.ts
+++ b/src/core/storage/database/ManagedDatabase.ts
@@ -78,12 +78,15 @@ export class ManagedDatabase<T> implements IManagedDatabase<T> {
 
 		this.onSyncStateUpdated('pending');
 
+		const start = performance.now();
 		return new Promise<void>((resolve, reject) => {
 			// Add task
 			this.syncRequests.push({ resolve, reject });
 
 			// Run worker if not started
 			this.syncWorker();
+		}).then(() => {
+			console.log("Sync is done", performance.now() - start);
 		});
 	};