Fix: issues with v4 tempering and obfuscation

[phanshiyu]

Why

We have the following problems:

1. Document can be tempered with by inserting empty [] or {} while passing verification

const SIGN_WRAPPED_DOCUMENT = document;
// malicious attacker can add a new empty array property anywhere
SIGN_WRAPPED_DOCUMENT.credentialSubject.children = []; 

verify(SIGN_WRAPPED_DOCUMENT) // still good

Leaf node

We define leaf node as a value in an object that does not have any children:
string | number | boolean | null | empty objects | empty arrays

The way we compute our hash (>v3) has always been over leaf nodes, but did not include empty objects | empty arrays. This allow [] or {} to be inserted into the document while still passing validation.

2. Document can be tempered with by changing the values from one type to the other provided they are string coerced to the same value. ie, true can be changed to "true"

This is how we hash each leaf node, as you can see, no type info is being accounted

keccak256(JSON.stringify({ [salt.path]: `${salt.value}:${get(document, salt.path)}` }))

3. Obfuscation does not error when it leaves behind new leaf nodes

How we perform obfuscation and allow the document to still verify as valid hinges on the fact that the obfuscation does not create a new leaf node; but you can imagine taking out all keys in an object would essentially have that effect, ie y: { x: 1 } is not leaf node, since it has a child, but now if we obfuscate x, we y now becomes a leaf node (see above on definition of leaf node).

The case with removing an item from an array can be illustrated below:

in js land

removing the first item from [0,1] will yield

[<empty item>, 1]

all good, empty items are not considered leaf node, so it does not affect the verification.. RIGHT!?

in JSON land

[<empty item>, 1]

becomes

[null, 1]

wait... null is a new leaf node; verification fails

What

1. traverseAndFlatten our method to find leaf nodes will not include empty arrays and objects
2. Hashing of each value now includes type information
3. Assertions are done whether a new empty object or array is being introduced during obfuscation
4. Fix obfuscation tests(still incomplete) and other tests that broke from this change
5. Some renaming and light refactoring and type fixes/improvements