Built-in Diagnostic Tool (formerly known as Sanitizer)

[tobihagemann]

Summary

With a diagnostic tool, Cryptomator will be able to check the "health" of a vault. This works similarly to the Sanitizer project but will be built-in for more convenience.

Motivation

The location of a Cryptomator vault is not a controlled environment. Basically, "anything" could happen to the encrypted files. Files and/or folders could be renamed due to sync conflicts. Even file contents could be malformed without Cryptomator's control that make encrypted contents unreadable. Of course, if users deliberately changes these files, they will expect that the decryption is erroneous. But in most cases, changes happen without the users' awareness.

A diagnostic tool should be able to guide users to detect or even solve problems with the vault. If users are unable to solve these problems on their own, they at least would have a diagnostic report at hand so that others could have a look at it.

Proposal

To keep things easy as a first step, the diagnostic tool should only be able to perform a check on the vault without solving any problems or other specific commands. Therefore, the feature set is smaller than Sanitizer in the initial implementation.

Check Vault

This command should detect problems in Cryptomator vaults that are related to the directory structure and filenames.

1. Check the existence of masterkey.cryptomator and confirm the password before proceeding.
2. Make a directory listing of the vault root and list all names that do not match d and masterkey.cryptomator(.*). Check the existence of d before proceeding.
   • Reasoning is to find directories like d (1) that could exist due to sync conflicts.
3. Recursively traverse through d from a cleartext perspective. Begin with cleartext root /.
   1. Resolve ciphertext path from cleartext path.
   2. Check the existence of the ciphertext path.
   3. Make a directory listing of the ciphertext path and check each item name against the expected pattern.
      • If it matches the expected pattern:
        • Try to decrypt ciphertext name. If it's a shortened filename, resolve it first.
        • If it's a .c9r directory, make a directory listing, log any unexpected items, and check if it's only a directory or only a symlink.
        • If it's a .c9s directory, check its consistency as well.
      • If it doesn't match the expected pattern:
        • Log ciphertext filenames because these files would be ignored by Cryptomator. (Do not consider the ciphertext filenames that would potentially be resolved by Cryptomator via the conflict resolver. Or maybe do if it's feasible.)
   4. Collect the ciphertext path in a list for step 4.
   5. Repeat from step 3.i for all cleartext directories that have been found.
4. Recursively traverse through d from a ciphertext perspective. Log the folders that have not been checked in step 3.
   • It's debatable if it would make sense to check these files even further. It's probably enough to log the folder names that they are unreachable from a cleartext perspective.

The purpose of step 3 is to check if everything is consistent based on what Cryptomator would actually "see".

The purpose of step 4 is to check if there are any other items inside the vault that would be ignored by Cryptomator but may still contain encrypted data that a user would like to access.

Check Vault (Deep)

This command should extend the previous command in step 3.3 by also checking the file contents' integrity.

[overheadhunter]

I'd split this up, as we don't want a monolithic vault check. Instead I'd propose a way to hook up a variety of checks into a traverser. Each of them being a separate issue.

[infeo]

Related to this FR is the quite old issue #312.

[ysmolski]

I see that this closed issue mentions:

Check Vault (Deep)
This command should extend the previous command in step 3.3 by also checking the file contents' integrity.

But yet I do not see in the current version of Cryptomator on desktop that feature. Health Check does not check files. I have one file broken:

11:36:34.000 [Thread-6140] ERROR o.c.frontend.fuse.ReadOnlyAdapter - read /a failed.
java.io.IOException: Unauthentic ciphertext in chunk 0
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.fh.ChunkCache.loadChunk(ChunkCache.java:41)
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.fh.ChunkCache.lambda$get$0(ChunkCache.java:56)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4868)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3533)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2282)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2159)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2049)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache.get(LocalCache.java:3966)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4863)
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.fh.ChunkCache.get(ChunkCache.java:56)
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.ch.CleartextFileChannel.readLocked(CleartextFileChannel.java:108)
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.ch.AbstractFileChannel.read(AbstractFileChannel.java:155)
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.ch.AbstractFileChannel.read(AbstractFileChannel.java:139)
	at org.cryptomator.frontend.fuse@1.3.3/org.cryptomator.frontend.fuse.OpenFile.readNext(OpenFile.java:98)
	at org.cryptomator.frontend.fuse@1.3.3/org.cryptomator.frontend.fuse.OpenFile.read(OpenFile.java:55)
	at org.cryptomator.frontend.fuse@1.3.3/org.cryptomator.frontend.fuse.ReadOnlyFileHandler.read(ReadOnlyFileHandler.java:70)
	at org.cryptomator.frontend.fuse@1.3.3/org.cryptomator.frontend.fuse.ReadOnlyAdapter.read(ReadOnlyAdapter.java:230)
	at ru.serce.jnrfuse.AbstractFuseFS.lambda$init$3(AbstractFuseFS.java:138)
	at jnr.ffi.provider.jffi.NativeClosureProxy$$impl$$11.invoke(Unknown Source)
Caused by: org.cryptomator.cryptolib.api.AuthenticationFailedException: Authentication of chunk 0 failed.
	at org.cryptomator.cryptolib@2.0.3/org.cryptomator.cryptolib.v1.FileContentCryptorImpl.decryptChunk(FileContentCryptorImpl.java:98)
	at org.cryptomator.cryptolib@2.0.3/org.cryptomator.cryptolib.v1.FileContentCryptorImpl.decryptChunk(FileContentCryptorImpl.java:83)
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.fh.ChunkLoader.load(ChunkLoader.java:39)
	at org.cryptomator.cryptofs@2.3.1/org.cryptomator.cryptofs.fh.ChunkCache.loadChunk(ChunkCache.java:38)
	... 18 common frames omitted

And I can discover it by reading plaintext file and get this error:

user@mbp:/Volumes/matorcipher$ cat a
cat: a: Input/output error

Are there any plans to add files check into the "Health Check" feature? And especially the ability to repair the file when one of the blocks is broken?

[overheadhunter]

Are there any plans to add files check into the "Health Check" feature? And especially the ability to repair the file when one of the blocks is broken?

Yes, we'll add further checks over time. Feel free to create a feature request to add a Health Check for checking the integrity of file contents. Note that reparing might be impossible depending on what exactly is broken in the chunk.