web multibatch has constant merkel proof

[yancyribbens]

There's a bug that when batching certs into a single transaction, the proof that's created for each cert is the same when using the web api implementation.

[ghost]

I'm a bit confused by how this is an issue.

You have 3 certificates in a batch, issuing them gives something like root: X, left: Y, right: Z.

Then you issue them via the web api method, and receive the same tree: root: X, left: Y, right: Z?

A certificate always resolves to the same hash, and if they are included in the same order, then it's batched the same and thus the merkle tree is the same.

Am I understanding that correctly or is there a different issue?

[yancyribbens]

@AnthonyRonning thanks for the reply. The Merkle proof for each certificate is different because it only needs to include the required hash to create the proof from the leaf node up to the root. Because there wasn't already a test for how the current "file" version works I added a test for that. I then created a test to show the web api creates the same proof as the "file" version. Note I didn't change any existing functionality of the file version, only added a test to assert what it creates. Does that make sense?

[kimdhamilton]

I think I'm confused as well. Are you saying the web multibatch code paths weren't actually looking at the file content? Can you give an example?

[yancyribbens]

@kimdhamilton i'm saying that when submiting a single certificate the web API works as it should, however, if submitting a batch it doesn't work properly because each certificate needs a different proof, and the code wasn't supporting that. My apologies not better describing the issue.

For example, let's say there are 3 certificates:

    { "badge": "badge data" },
    { "badge2": "more badge data"},
    { "badge3": "even more badge data"}

by placing the files at the location /etc/cert-issuer/data/unsigned_certificates/ , three seperate certs are created at /etc/cert-issuer/data/blockchain_certificates:

file1
{"badge": "badge data", "signature": {"type": ["MerkleProof2017", "Extension"], "merkleRoot": "a59e147aa340e4a9d990cbbbe737aee694d88ae674112f4d4f6d32187c70800f", "targetHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "proof": [{"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}, {"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}], "anchors": [{"sourceId": "c9d033808d883066d57832bb7dab18a076f0db0a8154ff4151c4de07590bfc1d", "type": "BTCOpReturn", "chain": "bitcoinRegtest"}]}}

file2
{"badge2": "more badge data", "signature": {"type": ["MerkleProof2017", "Extension"], "merkleRoot": "a59e147aa340e4a9d990cbbbe737aee694d88ae674112f4d4f6d32187c70800f", "targetHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "proof": [{"left": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}, {"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}], "anchors": [{"sourceId": "c9d033808d883066d57832bb7dab18a076f0db0a8154ff4151c4de07590bfc1d", "type": "BTCOpReturn", "chain": "bitcoinRegtest"}]}}

file3:
{"badge3": "even more badge data", "signature": {"type": ["MerkleProof2017", "Extension"], "merkleRoot": "a59e147aa340e4a9d990cbbbe737aee694d88ae674112f4d4f6d32187c70800f", "targetHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "proof": [{"left": "2dba5dbc339e7316aea2683faf839c1b7b1ee2313db792112588118df066aa35"}], "anchors": [{"sourceId": "c9d033808d883066d57832bb7dab18a076f0db0a8154ff4151c4de07590bfc1d", "type": "BTCOpReturn", "chain": "bitcoinRegtest"}]}}

as you can see, the proofs are not the same, and by contrast the current web api returns only one proof which is the last proof when in fact the web api should return an array of proofs. This PR does that by building an array in memory and returning the results:

[{"badge": "badge data", "signature": {"type": ["MerkleProof2017", "Extension"], "merkleRoot": "a59e147aa340e4a9d990cbbbe737aee694d88ae674112f4d4f6d32187c70800f", "targetHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "proof": [{"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}, {"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}], "anchors": [{"sourceId": "64894112516b02b2570ac7716292322a98036d3f7f2a0249a12dc79aade21e2f", "type": "BTCOpReturn", "chain": "bitcoinRegtest"}]}}, {"badge2": "more badge data", "signature": {"type": ["MerkleProof2017", "Extension"], "merkleRoot": "a59e147aa340e4a9d990cbbbe737aee694d88ae674112f4d4f6d32187c70800f", "targetHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "proof": [{"left": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}, {"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}], "anchors": [{"sourceId": "64894112516b02b2570ac7716292322a98036d3f7f2a0249a12dc79aade21e2f", "type": "BTCOpReturn", "chain": "bitcoinRegtest"}]}}, {"badge3": "even more badge data", "signature": {"type": ["MerkleProof2017", "Extension"], "merkleRoot": "a59e147aa340e4a9d990cbbbe737aee694d88ae674112f4d4f6d32187c70800f", "targetHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "proof": [{"left": "2dba5dbc339e7316aea2683faf839c1b7b1ee2313db792112588118df066aa35"}], "anchors": [{"sourceId": "64894112516b02b2570ac7716292322a98036d3f7f2a0249a12dc79aade21e2f", "type": "BTCOpReturn", "chain": "bitcoinRegtest"}]}}]

the current broken web ap implementation for 3 badges returns:
{"type": ["MerkleProof2017", "Extension"], "merkleRoot": "a59e147aa340e4a9d990cbbbe737aee694d88ae674112f4d4f6d32187c70800f", "targetHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "proof": [{"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}, {"right": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}], "anchors": [{"sourceId": "da6a2f15ae55eaab512ae64104dc9ecf1478c59cb6bf94c172280c7e5b57cc4c", "type": "BTCOpReturn", "chain": "bitcoinRegtest"}]}

[kimdhamilton]

ok, I gotcha. This makes sense

[ghost]

Looks like the PR was merged, closing. Thanks for opening up the issue and fixing it!