Verifying HTTPS page in smart contract | Bounty: USD $500 in NEAR #15
Comments
jakestutzman
changed the title
|
I'm interested in this bounty issue. Are there any specific requirements of the contract? What interface functions do I need to implement? |
|
just google https://www.ssl.com/article/ssl-tls-handshake-overview/ picture , shared secret key stuff |
|
If I am understanding correctly, then I'm not sure if what you are asking for is possible. coinbase.com's certificate is used to verify that the server the client is communicating with is really coinbase.com, but it is not used to encrypt the content of the page. The content is encrypted using a symmetric encryption method, and a shared key. During the initial stages of communication, the server sends its certificate to the client which includes a public key. The client uses the public key to encrypt a "pre-key" which it sends back to the server. The server decrypts the pre-key, and both the client and server use the pre-key to generate the same symmetric encryption key. The symmetric encryption key is used to encrypt/decrypt all further communication, including the client's request for the page and the content of the page. Because of this, I can't see how the smart contract would be able to verify that the content given to it from the command-line tool came from coinbase.com. Perhaps the smart-contract could verify the certificate by doing something like this (not necessarily in this order):
References: |
|
Hi Benjamin,
You are correct, I didn't realize that symmetric encryption has been used
after certificate handshake was established.
Just to check, what about next logic:
- Submit the certificate to the smart contract
- It generates a pre-master key just by sourcing a random number from the
chain.
- It's visible to everyone, and anyone can actually submit it to the
server. Servers respond with encrypted full symmetric key.
- The page is required from the server encrypted with this full key and
receives it's response.
- Submit that to a smart contract: <encrypted full key>, <request>,
<response> <-- contract can verify that server signed full key with public
key in the certificate, the request and response can be decrypted by this
key. This symmetric encryption key can only be submitted once obviously.
Everyone can read pre-master key and use it to interact with the website
potentially (e.g. shared session). But submitted data on the chain is
verified based on the full key that is given back from the server.
What do you think of this?
…On Sun, Mar 14, 2021 at 11:02 AM benjamingreenberg ***@***.***> wrote:
If I am understanding correctly, then I'm not sure if what you are asking
for is possible.
coinbase.com's certificate is used to verify that the server the client
is communicating with is really coinbase.com, but it is not used to
encrypt the content of the page. The content is encrypted using a symmetric
encryption method, and a shared key. During the initial stages of
communication, the server sends its certificate to the client which
includes a public key. The client uses the public key to encrypt a
"pre-key" which it sends back to the server. The server decrypts the
pre-key, and both the client and server use the pre-key to generate the
same symmetric encryption key. The symmetric encryption key is used to
encrypt/decrypt all further communication, including the client's request
for the page and the content of the page. Because of this, I can't see how
the smart contract would be able to verify that the content given to it
from the command-line tool came from coinbase.com.
Perhaps the smart-contract could verify the certificate by doing something
like this (not necessarily in this order):
- Receive the certificate from the command-line tool, along with the
string "coinbase.com"
- Check that the certificate is for the domain coinbase.com
- Verify the signature on the certificate
- Verify that the certificate is still valid (e.g. has not expired,
been revoked, etc)
- Check that the certificate path contains a certificate signed by a
certificate authority that the smart-contract trusts ("trust anchor")
- Verify that, starting from the certificate signed by the trust
anchor, each certificate's private key was used to issue *the next
certificate* in the path, all the way down to the coinbase.com
certificate.
- Reply back to the command-line tool with the result, which would
include the reason why a certificate failed validation, if applicable
References:
Browsers and Certificate Validation
<https://www.ssl.com/article/browsers-and-certificate-validation/>
The SSL/TLS Handshake: an Overview
<https://www.ssl.com/article/ssl-tls-handshake-overview/>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#15 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABK27X2QMFACSQHMP74KH3TDT24DANCNFSM4OKHXWNQ>
.
--
Best regards,
Illia Polosukhin
|
|
I think there are still some issues. The server does not send the master/session key back to the client. The server and client each use the pre-master key to generate the session key independently. Another issue is that by making the pre-master key visible to everyone, you introduce the possibility of a man-in-the-middle attack;
By the way, it's possible that you introduce a vulnerability by using info from the certificate chain to generate the pre-master key. By limiting how the pre-master key is generated in this way, it may reduce the number of possible keys so that a computer could try them all fast enough to do a man-in-the-middle attack. If you want multiple clients to communicate with the server using the same session key, then you will need to store additional data on the smart contract, like the encryption protocol the client/server agreed to use. It also needs to store whatever the server requires the client to use to identify itself, like a session-id, access token, cookie, etc. That's assuming the server even keeps/manages sessions. If the server doesn't, then every connection to the server would require negotiating and generating a new pre-master key and master key. I think it might help if you explained what your ultimate goal is, and/or what a real-world use-case looks like, and/or what problem you are trying to solve. |
|
The goal was to provide smart contracts on-chain some data and these
contracts can trust that it was indeed generated by the declared source
(e.g. coinbase.com).
Currently even if one wants prices from coinbase.com - they still need to
either trust some subset of other parties that they will provide correct
information from this website to the on-chain smart contract.
…On Mon, Mar 15, 2021 at 11:33 PM benjamingreenberg ***@***.***> wrote:
I think there are still some issues.
The server does not send the master/session key back to the client. The
server and client each use the pre-master key to generate the session key
independently.
Another issue is that by making the pre-master key visible to everyone,
you introduce the possibility of a man-in-the-middle attack;
1. Attacker gets pre-master key, uses it to generate the session key
2. Attacker intercepts the data from the server to the client
3. Attacker decrypts the data and changes it
4. Attacker encrypts the manipulated version, and sends it to the
client
By the way, it's possible that you introduce a vulnerability by using info
from the certificate chain to generate the pre-master key. By limiting how
the pre-master key is generated in this way, it may reduce the number of
possible keys so that a computer could try them all fast enough to do a
man-in-the-middle attack.
If you want multiple clients to communicate with the server using the same
session key, then you will need to store additional data on the smart
contract, like the encryption protocol the client/server agreed to use. It
also needs to store whatever the server requires the client to use to
identify itself, like a session-id, access token, cookie, etc. That's
assuming the server even keeps/manages sessions. If the server doesn't,
then every connection to the server would require negotiating and
generating a new pre-master key and master key.
I think it might help if you explained what your ultimate goal is, and/or
what a real-world use-case looks like, and/or what problem you are trying
to solve.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#15 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABK27TJQCD443U7VNCB7ALTD33THANCNFSM4OKHXWNQ>
.
--
Best regards,
Illia Polosukhin
|
|
Pretty much the alternative to this, is to be called "off-chain workers" -
something that runs on every validator, queries data from off-chain and
puts it on-chain.
The problem with this approach is that if this data doesn't agree (because
for example they didn't query at exact same time) - now you need to do
conflict resolution. Either way it becomes more and more complicated for a
smart contract to interact with external world.
On Tue, Mar 16, 2021 at 3:55 AM Illia Polosukhin ***@***.***>
wrote:
… The goal was to provide smart contracts on-chain some data and these
contracts can trust that it was indeed generated by the declared source
(e.g. coinbase.com).
Currently even if one wants prices from coinbase.com - they still need to
either trust some subset of other parties that they will provide correct
information from this website to the on-chain smart contract.
On Mon, Mar 15, 2021 at 11:33 PM benjamingreenberg <
***@***.***> wrote:
> I think there are still some issues.
>
> The server does not send the master/session key back to the client. The
> server and client each use the pre-master key to generate the session key
> independently.
>
> Another issue is that by making the pre-master key visible to everyone,
> you introduce the possibility of a man-in-the-middle attack;
>
> 1. Attacker gets pre-master key, uses it to generate the session key
> 2. Attacker intercepts the data from the server to the client
> 3. Attacker decrypts the data and changes it
> 4. Attacker encrypts the manipulated version, and sends it to the
> client
>
> By the way, it's possible that you introduce a vulnerability by using
> info from the certificate chain to generate the pre-master key. By limiting
> how the pre-master key is generated in this way, it may reduce the number
> of possible keys so that a computer could try them all fast enough to do a
> man-in-the-middle attack.
>
> If you want multiple clients to communicate with the server using the
> same session key, then you will need to store additional data on the smart
> contract, like the encryption protocol the client/server agreed to use. It
> also needs to store whatever the server requires the client to use to
> identify itself, like a session-id, access token, cookie, etc. That's
> assuming the server even keeps/manages sessions. If the server doesn't,
> then every connection to the server would require negotiating and
> generating a new pre-master key and master key.
>
> I think it might help if you explained what your ultimate goal is, and/or
> what a real-world use-case looks like, and/or what problem you are trying
> to solve.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#15 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AABK27TJQCD443U7VNCB7ALTD33THANCNFSM4OKHXWNQ>
> .
>
--
Best regards,
Illia Polosukhin
--
Best regards,
Illia Polosukhin
|
|
I'm brand new to blockchains, smart contracts, etc, so I am not sure if something like this is possible, and if it is possible, if it can be done using one smart-contract or you would need multiple smart-contracts for the different tasks.
It's important that the pre-master key and session key are kept private/secret/secure until the end of the session, because this helps ensure that the only entities that can encrypt messages are the smart-contract(s) and coinbase.com. Well, technically anyone with access to the private key whose public key the smart-contract used to encrypt the pre-master key can encrypt messages. Also, anyone with access to wherever the smart-contract stores the pre-master key and session-key will be able to encrypt messages, so this is a potential weak spot that needs to be addressed (a way to be reasonably certain that only the smart-contract can read the pre-master key and session key before the session ends and they are published). There is one other major issue to overcome, and which is alluded to in steps 23 & 24. Content on a webpage is often added dynamically after the initial page load. A good example of this is https://coinbase.com/price/bitcoin
The left part of the screenshot is the fully rendered page in chrome. The part on the right is the content received after the initial request for the page. As you can see, the most important info for that page is not included in the initial content received. The browser needs to request more content from the server (and often other servers too) to fully render the page. If your goal is to eventually publish or store a fully rendered page, or at least a part that needs to be rendered dynamically, then the App will need to make a series of requests, not just one. This makes things more complicated, but not impossible. |
|
Hey @ilblackdragon, did you get a chance to read @benjamingreenberg's approach to completing the bounty? If it makes sense, i'm happy to assign the bounty to them. @benjamingreenberg - are you still interested in completing this bounty? |
|
@cameron-NEAR This approach won't work, because there is no way a smart contract can store or work with secret data. In general, AFAIK there is currently no feasible way to verifiably get a HTTPS page from a smart contract (unless the server supports SXG, which is a tiny minority of the servers). For this reason, I think that this bounty is misguided. |
|
@abacabadabacaba if I understand correctly we don't need secret data here. We can establish a private connection with coinbase.com, but ultimately we can publish all the secret data generated publicly to be verified on-chain, given that none of this secret data actually compromises the client (i.e it is not logged in or any personal information is used). The reason I think this is doable disregarding how the protocol works is the following: I can connect with a clean browser (or python script) to coinbase and fetch some information securely that I can trust (without providing any credentials). All the validation done in my end by the browser/script can be as well done in the blockchain. |
How can the smart contract independently validate that the data you gave it actually came from coinbase.com? |
I haven't look in all technical details of the HTTPS protocol, but the idea is that the browser is only aware of the public key from the root authority, so this key is as well added to the blockchain (this step is only required once). Other public information that is known ahead of time needs to be added on chain. I expect this information doesn't change often so it only needs to be submitted once. When we connect to coinbase and fetch some information we record all data that is sent and received, and all data that is generated locally (even private keys or random numbers used). Once the communication ended we close the channel, and all the recorded information is submitted on-chain. This information should be enough to validate and deserialise the final data. The previous approach is described in an inefficient way (regarding the amount of information and computation required on-chain), since I'm only trying to say that it is possible. With more in-depth knowledge about how HTTPS actually work, we can optimise which information needs to be recorded and submitted on-chain. |
I completely agree that the previous approach is inefficient. That was my way of saying it wasn't something I thought I could do :) I hope you do come up with a solution! The only advice I would give you, is to work under the assumption that the system that is running your script has been compromised, and a malicious 3rd party can make any changes it wants to incoming data before your script sees it, and to outgoing data your script sends to the smart contract. |
|
@mfornet You probably assume that the data coming over HTTPS is signed. It is not. Instead, its authenticity is protected using a scheme called Message Authentication Code (MAC). MAC is more efficient to compute and verify. However, unlike digital signatures, it uses the same key for generation and verification. Thus, anyone who can verify a MAC can also forge it. When HTTPS is used normally, only the client and the server know the key, so the data is secure. However, the client can not prove this to anyone, because the client can generate a valid MAC for any data. |
|
It's possible in theory (by submitting the whole raw TLS encrypted bytes exchanged to the contract), but it's exceptionally difficult. The solution would require a customized web browser. More in the $500k price range in my opinion. |
|
Hi @boris-kolar, the customized browser doesn't really make sense as far as I understand, as the requirement here is to run on public blockchain and allow smart contracts to verify the content of retrieved information. As @abacabadabacaba mentioned, the actual session encryption used is established after a handshake, which means that wouldn't work with public blockchain. If there was a standard where web browsers sign with public/private cryptography the outgoing message - this would allow for a public blockchain to verify that indeed information received was signed via one of such agents. Which means, we should close this bounty at this point. The alternative approach to this problem is to extend NEAR Protocol to have off-consensus work by validators: the smart contract can request validators to do some work, which they will run outside of consensus and publish back on-chain. Similar to Substrate off-chain workers. |
|
Another option suggested by @abacabadabacaba (also complex), is to use secure multi-party computation to run the exchange protocol in such a way that several nodes runs as a client, and none of them has access to the secret key. If this is possible, then only the server will be able to sign the messages and it can be used as a proof that can be verified on-chain. Some problems with this idea:
|
|
Just for extra reference, another discussion arrived to the same conclusions that it is impossible to do by design. |
Description
HTTPS pages are signed with SSL certificates.
It would be great to have a Rust tool that will be able to verify that some payload was signed by certificate and decrypt it to be able to parse the content of a page.
Acceptance Criteria
Bounty
USD $500 in NEAR
The text was updated successfully, but these errors were encountered: