Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OTIP (OpenTimestaps Improvement Proposal) to support human readable .JSON to ease the adoption #43

Closed
marsrobertson opened this issue May 4, 2019 · 7 comments

Comments

Projects
None yet
2 participants
@marsrobertson
Copy link

commented May 4, 2019

I thought about outputting .ots content in <pre> element on the website, together with archive.org and IPFS:

image


Reading about the format on Wikipedia

https://en.wikipedia.org/wiki/OpenTimestamps#Show_timestamp_information

The timestamp is saved in a binary file to save space and avoid problems of interpretation, encoding and compatibility between systems.

371 bytes: https://github.com/genesisdotre/legalscreenshot.com/blob/master/bug-report/saved-from-OpenTimestamps.ots

927 bytes: https://github.com/genesisdotre/legalscreenshot.com/blob/master/bug-report/saved-as-text-file-on-Mac-SublimeText.ots

cat saved-from-OpenTimestamps.ots
epenTimestampsProof���� L}H��
m+��P���6����T5%�'��y
.�               ���V�jؿ���\�3��F��b�l���
.��.-https://alice.btc.calendar.opentimestamps.org��e��p�m��p��\�3�.0cd�:G���
.��,+https://bob.btc.calendar.opentimestamps.org��vչ�w�Jۛ2������.K�U�\�3�d�$���Q���
  �)(https://finney.calendar.eternitywall.com
cat saved-as-text-file-on-Mac-SublimeText.ots
004f 7065 6e54 696d 6573 7461 6d70 7300
0050 726f 6f66 00bf 89e2 e884 e892 9401
0819 4c7d 48a3 800d 6504 0b54 2e7d fc5a
4dbd b5cc 67a0 0843 6ba8 242c 1836 7208
27f0 10f8 790d 6d2b fbee 50eb e09b f236
afc3 5808 fff0 1054 3525 a90b c2c9 cb56
956a d8bf b8e0 3408 f104 5ccc 33de f008
8a46 abf7 6217 dd6c 0083 dfe3 0d2e f90c
8e2e 2d68 7474 7073 3a2f 2f61 6c69 6365
2e62 7463 2e63 616c 656e 6461 722e 6f70
656e 7469 6d65 7374 616d 7073 2e6f 7267
fff0 1065 ee19 a115 1214 7098 6d8f fa81
70a3 6c08 f104 5ccc 33de f008 2e30 0763
64e0 3a47 0083 dfe3 0d2e f90c 8e2c 2b68
7474 7073 3a2f 2f62 6f62 2e62 7463 2e63
616c 656e 6461 722e 6f70 656e 7469 6d65
7374 616d 7073 2e6f 7267 f010 a976 d5b9
f5a4 7784 4a16 db9b 3203 1389 08f0 10c9
cffa ffaa fb0e 082e 4b00 f785 8455 b808
f104 5ccc 33de f008 6484 241e 9fae f651
0083 dfe3 0d2e f90c 8e29 2868 7474 7073
3a2f 2f66 696e 6e65 792e 6361 6c65 6e64
6172 2e65 7465 726e 6974 7977 616c 6c2e
636f 6d

https://youtu.be/P2NwhjvD8Ps

image

I genuinely believe there is a fair balance between usability / user adoption and trying to save each and every byte.

On that basis I recommend extending the standard to support both:

  • Binary format
  • Human readable JSON

To create such proof, it turns out that it is not necessary to publish d on the blockchain, which would be expensive, but it is enough to commit d to the blockchain. Such commitment proves that d existed prior to a certain block, in the sense that if d changes, then the proof becomes invalid and hence useless.

We are already saving data, not publishing any data on the blockchain, even if the .JSON file is 4KB it is still the minimal filesize: https://stackoverflow.com/questions/26002504/whats-the-smallest-possible-file-size-on-disk


UPDATE: on the other hand, if we acknowledge the existence of other blockchains: https://solidity.readthedocs.io/en/v0.5.8/miscellaneous.html

EVM operates on 32 bytes at a time

If reducing size is not possible, then hash or multihash of .ots can be stored on-chain.

@petertodd

This comment has been minimized.

Copy link
Member

commented May 7, 2019

OpenTimestamps doesn't use binary to reduce size; it uses binary to reduce risk.

Binary parsing is ideal for that precisely because it isn't forgiving: unless you can parse the proof completely you probably can't parse it at all. That's a good thing for security software! If your OTS implementation has bugs in it, those bugs might be exploitable, so we'd rather those bugs lead to the code not working at all rather than it appearing to work.

Secondly, OTS proofs aren't really human readable in any form, because humans can't do cryptographic hashes in their head. Sure, you can gain some insight into what the proof is doing - good for education. But you can't understand a proof fully without code. So you might as well accept that and default to binary, with the exception of informational tools like ots info.

For things like HTML where pure binary simply can't be embedded, I'd suggest just encoding the binary in whatever is the most efficient encoding scheme you have available. Base64 is probably fine.

@petertodd petertodd closed this May 7, 2019

@marsrobertson

This comment has been minimized.

Copy link
Author

commented May 8, 2019

Fragility is a feature.

Thank you for explaining.

For browsers - there are simple workarounds, for example: https://stackoverflow.com/questions/23451726/saving-binary-data-as-file-using-javascript-from-a-browser


https://en.wikipedia.org/wiki/OpenTimestamps#cite_note-ots1-11

image

@petertodd

This comment has been minimized.

Copy link
Member

commented May 8, 2019

Yup, that's a a perfectly good work around.

You know, another way to think about this is we already do this for cryptographic things like signatures and hashes all the time: a signature or even a hash digest could be represented a "human readable" form consisting of whatever individual numbers the algorithm put out. But that'd be useless as humans can't do anything useful with those numbers, so even in JSON encodings standards tend to just hex-encode such data.

@marsrobertson

This comment has been minimized.

Copy link
Author

commented May 12, 2019

Random:

I'm taking part in a bug bounty.

I would like to send the report privately and update the issue on GitHub and paste the content of the .ots file.

Another use case I can think of - compiling a report in .PDF and including the .ots is some shape or form.

Binary data is not suited for displaying on GitHub, email or .PDF.

In my realm of web technologies, JSON is pretty much everywhere and I think there is a place for human-readable representation. Even if the verification (calculating hashes) will require a computer, even if fragility is a feature (pretty clever) there is still value if it can be displayed in plain text. I guess the ots info is used for that.

Analogy to PGP - verifying signatures requires some software and yet the -----BEGIN PGP SIGNATURE----- is in plain text, not binary.

Here is the representation in the current form:

$ ots stamp filename.ext
Submitting to remote calendar https://a.pool.opentimestamps.org
Submitting to remote calendar https://b.pool.opentimestamps.org
Submitting to remote calendar https://a.pool.eternitywall.com
Submitting to remote calendar https://ots.btc.catallaxy.com
$ ots info filename.ext.ots
File sha256 hash: 71c9476d9b21d6d79746227d53c2f45425259293d2018d10f40f91d750171f05
Timestamp:
append 33ea0c2a2aa4c36d308054ece01f07ca
sha256
 -> append 6c65a4bd091de9e1e47b8019660e991e
    sha256
    prepend 0c9b386d9b52a6966f21f4dd71ee5f426a723e68297156236a0c95a68faf237b
    sha256
    prepend 5cd83092
    append e94d36ba65809755
    verify PendingAttestation('https://alice.btc.calendar.opentimestamps.org')
 -> append 838d82d4bbfd06566b78dd060eea9c24
    sha256
    append 8c5869183ee6ef5fff092a58490bb43c
    sha256
    append f81b373d109d8de94cddb665bdaf4ae8e55f86872414305aedf5416d19d73745
    sha256
    prepend 5cd83092
    append 96dd3e21939e843d
    verify PendingAttestation('https://finney.calendar.eternitywall.com')
 -> append 9ddb329f49487125fd09a7892da189e4
    sha256
    prepend 3b9af151ab63d5b491a00ac44fd6ee55d4b0a05c3cca83ecc06649919bfc3b1c
    sha256
    prepend 5cd83092
    append ba4fef614630d691
    verify PendingAttestation('https://bob.btc.calendar.opentimestamps.org')
 -> append c2be91ecaedd18f71b9beb38c2f209f8
    sha256
    prepend 5cd83092
    append 3c002209d2c08b77
    verify PendingAttestation('https://btc.calendar.catallaxy.com')
@petertodd

This comment has been minimized.

Copy link
Member

commented May 19, 2019

So I wouldn't call clearsigned PGP plain text. Rather it's a plain text encoding of binary data.

You can do that with OTS too, with the very simplest way to be just to base64-encode it. I'd also be willing to accept a "clear-stamped" extension format for OTS, that basically worked similarly to how PGP clear-signing works. Main thing there is you want to ensure that you can timestamp a clearsigned document, with the timestamp applying to the PGP signature as well as the document text. This also means you'd want to follow exactly how the OpenPGP standard does clear-signing: https://tools.ietf.org/html/rfc4880#section-7

@marsrobertson

This comment has been minimized.

Copy link
Author

commented May 21, 2019

Something like that?

-----BEGIN OPEN TIMESTAMP-----
Version: v0.6.0




-----END OPEN TIMESTAMP-----

It looks familiar, similar to PGP and -----BEGIN BITCOIN SIGNED MESSAGE-----

It does the job.

It can be represented in print.


Main thing there is you want to ensure that you can timestamp a clearsigned document, with the timestamp applying to the PGP signature as well as the document text.

Can you explain?

Can I opentimestamp (used as verb) a document without opentimestamping the PGP signature?

@petertodd

This comment has been minimized.

Copy link
Member

commented May 22, 2019

Something like that?

Yup, kinda like that.

Can you explain? Can I opentimestamp (used as verb) a document without opentimestamping the PGP signature?

So basically, it's best if PGP can directly verify the signature on the clearsigned text without any fuss, and OpenTimestamps can also verify it directly, while also verifying that the signature itself is timestamped.

Notably, GPG does allow multiple PGP signatures to be attached to one clearsigned piece of text! So you can re-use that flexibility for opentimestamps by sticking the timestamp at the very end.

Play around with it a bit by making up some clearsigned things with fake timestamps at the end and you'll see what I mean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.