TAP 15: misc improvements and fixes - #155
Merged
Merged
Conversation
added 8 commits
June 9, 2022 14:00
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
The spec suggests that the hyphen is implicitly added
(`NAME_PREFIX-COUNT`), and the example suggests that it has to be
added explicitly ("alice.hbd-").
This commit makes it clear that the former is the case.
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
The motivational section mentions 32-bit values for keys, and how repeating them leads to big metadata overheads, however the example code uses 'abc123' place holders for keyids. This commit adds realistic 32-bit keyids to the examples, to better bridge between text and example and to put emphasis on the vast redundancy. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Use the same number of bins, i.e. 2048, used for overhead calcs in both example snippets for a coherent story. Further improvements include in the traditional hash bin delegation example include: - list the first two and the last bin, instead of the first 4. This has the advantage that the bin index in the bin name, and the path hash prefixes can be clearly identified as hex numbers, it also shows better how many entries there are in total. - spell out the values in `path_hash_prefixes` Previously they were abbreviated with with an asterisk, which was confusing as the TUF spec also has fields that support glob patterns, but this is not one of them. Spelling them out also stresses emphasis on the metadata overhead that can be saved with succinct hash bin delegation. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Intended improvements include: - Remove redundancy - Better tie text to metadata snippet (mentions number of bins) - Change argument for using only one key - Don't focus on keyid only, but all the redundant information in they entries. - Remove special focus for name prefix. In reality this is just redundant information. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Traditional hash bin delegation does seem to solve the issue of a large number of delegations in a targets metadata file, so we shouldn't mention it in the opener of the Motivation section. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
- reformat metadata snippets with black - re-indent a few lines Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
lukpueh
commented
Jun 9, 2022
Contributor
|
Need anyone else to review here? |
Contributor
Yes, we need one more maintainer review (non-maintainer reviews also welcome, but not required) |
Merged
lukpueh
pushed a commit
to lukpueh/taps
that referenced
this pull request
Aug 3, 2022
Prior to this commit we used different numbers of bins for traditional and succinct hbd, to compare their minimal metadata overheads, for a fictional number of targets, and showing absolute numbers. This commit uses the same number of targets and bins for both schemes, bases them on the real world example from PEP 458, and focuses on the size of the delegating metadata and the relative difference between the overheads. This should better underline the advantage of succinct hash bin delegation. See theupdateframework#155 (comment) for discussion. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
lukpueh
pushed a commit
to lukpueh/taps
that referenced
this pull request
Aug 4, 2022
Prior to this change we compared the overall metadata overhead of traditional and succinct hash bin delegation (hbd), using different numbers of bins optimized for each hashing scheme for a "fair" comparison. However, using different numbers makes it harder to understand the advantage of succinct hbd, i.e. no growth of the delegating metadata vs. linear growth with number of bins. This change focuses on the different sizes of the delegating targets metadata for the same number of bins to underline the advantage of succinct hbd. Background info: The optimum number of bins for a given number of target files varies between the two schemes, because in traditional hbd the positive effect of many bins is undermined by the huge size of the delegating metadata. Other changes include: - use a real world number of bins (PEP 458) instead of fictional - compare metadata overhead percentages instead of absolute numbers - perform keyid calcs with 64 instead of 32 bytes to match reference implementation - remove mention of inaccurate (32-bit) and not so relevant keyid size See theupdateframework#155 (comment) for initial discussion. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
lukpueh
pushed a commit
to lukpueh/taps
that referenced
this pull request
Aug 9, 2022
Prior to this change we compared the overall metadata overhead of traditional and succinct hash bin delegation (hbd), using different numbers of bins optimized for each hashing scheme for a "fair" comparison. However, using different numbers makes it harder to understand the advantage of succinct hbd, i.e. no growth of the delegating metadata vs. linear growth with number of bins. This change focuses on the different sizes of the delegating targets metadata for the same number of bins to underline the advantage of succinct hbd. Background info: The optimum number of bins for a given number of target files varies between the two schemes, because in traditional hbd the positive effect of many bins is undermined by the huge size of the delegating metadata. Other changes include: - use a real world number of bins (PEP 458) instead of fictional - compare metadata overhead percentages instead of absolute numbers - perform keyid calcs with 64 instead of 32 bytes to match reference implementation - remove mention of inaccurate (32-bit) and not so relevant keyid size See theupdateframework#155 (comment) for initial discussion. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu> Co-authored-by: Joshua Lock <jlock@vmware.com>
mnm678
pushed a commit
that referenced
this pull request
Aug 9, 2022
Prior to this change we compared the overall metadata overhead of traditional and succinct hash bin delegation (hbd), using different numbers of bins optimized for each hashing scheme for a "fair" comparison. However, using different numbers makes it harder to understand the advantage of succinct hbd, i.e. no growth of the delegating metadata vs. linear growth with number of bins. This change focuses on the different sizes of the delegating targets metadata for the same number of bins to underline the advantage of succinct hbd. Background info: The optimum number of bins for a given number of target files varies between the two schemes, because in traditional hbd the positive effect of many bins is undermined by the huge size of the delegating metadata. Other changes include: - use a real world number of bins (PEP 458) instead of fictional - compare metadata overhead percentages instead of absolute numbers - perform keyid calcs with 64 instead of 32 bytes to match reference implementation - remove mention of inaccurate (32-bit) and not so relevant keyid size See #155 (comment) for initial discussion. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu> Co-authored-by: Joshua Lock <jlock@vmware.com> Co-authored-by: Joshua Lock <jlock@vmware.com>
mnm678
pushed a commit
to mnm678/taps
that referenced
this pull request
Oct 4, 2022
…k#159) Prior to this change we compared the overall metadata overhead of traditional and succinct hash bin delegation (hbd), using different numbers of bins optimized for each hashing scheme for a "fair" comparison. However, using different numbers makes it harder to understand the advantage of succinct hbd, i.e. no growth of the delegating metadata vs. linear growth with number of bins. This change focuses on the different sizes of the delegating targets metadata for the same number of bins to underline the advantage of succinct hbd. Background info: The optimum number of bins for a given number of target files varies between the two schemes, because in traditional hbd the positive effect of many bins is undermined by the huge size of the delegating metadata. Other changes include: - use a real world number of bins (PEP 458) instead of fictional - compare metadata overhead percentages instead of absolute numbers - perform keyid calcs with 64 instead of 32 bytes to match reference implementation - remove mention of inaccurate (32-bit) and not so relevant keyid size See theupdateframework#155 (comment) for initial discussion. Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu> Co-authored-by: Joshua Lock <jlock@vmware.com> Co-authored-by: Joshua Lock <jlock@vmware.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains a few random improvements and fixes of things that I noticed in my last reading of the document. The commit log has details about the changes. When reviewing the diff I recommend disabling whitespace changes.