fetch ohttp keys through CONNECT tunnel #194

DanGould · 2024-02-12T16:33:34Z

Previously we were supplying the OHTTP Gateway's Key Configuration OhttpKeys manually with configuration.

Now that the Rust ohttp-relay supports fetching it via https-in-https CONNECT tunnel, we can fetch it from the server while revealing our IP address only to an ohttp relay.

While ureq and reqwest advertise https-in-https CONNECT functionality, in practice neither properly tunnels TLS in TLS, though I'm trying to change this. For this reason this version uses https-in-http CONNECT which still gets an authenticated response from the gateway (payjoin directory)

Now we only e2e test v1 payjoin cli, and this is complex behavior we probably want to test for which means standing up an ohttp-relay, payjoin directory, and provisioning https certs for them in testing.

refactored and built on #198

handle errors
test
suppress warnings

Regarding Configuration

This PR makes it possible to provide an optional default ohttp-keys config to override fetching one.

In the future, ohttp-keys may be cached per-directory and only refreshed when they fail.

DanGould

While the feature seems to be able to work in production as-is, fetching keys via relay is not yet included in integration tests. The Proxy agent does not support the danger_local_https feature either.

jbesraa · 2024-03-08T09:03:17Z

payjoin-cli/src/app/v2.rs

@@ -130,27 +139,20 @@ impl AppTrait for App {
 }

 impl App {
-    fn construct_payjoin_uri(
+    async fn construct_payjoin_uri(


why this need to be async?

Whoops nice catch. It doesn't.

I actually think the Enrolled struct has all of the information necessary to construct a Uri and should become available from one of its methods.

jbesraa · 2024-03-08T09:06:28Z

payjoin-cli/src/app/v2.rs

@@ -300,6 +305,43 @@ impl App {
    }
 }

+async fn get_ohttp_keys(config: &AppConfig) -> Result<payjoin::OhttpKeys> {


I feel that get_ohttp_keys, fetch_ohttp_keys and normalize_proxy_url should be part of the payjoin crate.
the fetch.. could require the http client the user is using as input.

I agree that there should be library code for fetching ohttp keys, however, the strong typing is already part of payjoin::OhttpKeys::decode(bytes). That leaves the i/o. Since payjoin is a no-IO crate, this makes the functionality, including normalize_proxy_url as candidates for a payjoin-io crate. normalize_proxy_url is a hack for a strange ureq bug I'm hoping can be fixed. I'm happy to begin work on the new crate but would prefer to leave that to a new PR. It would be simple to create a dependency on a local payjoin-io crate as a follow up.

get_ohttp_keys is an app method that takes config as input, so I'm not sure that can be included in a common library.

jbesraa · 2024-03-08T09:10:51Z

payjoin-cli/src/app/v2.rs

+        let relay_port = find_free_port();
+        let relay_url = Url::parse(&format!("http://0.0.0.0:{}", relay_port)).unwrap();
+        let pj_endpoint = Url::parse("https://payjo.in:443").unwrap();
+        tokio::select! {


I saw this tokio::select! few times already and always get confused about it. can please explain whats the motivation of using it in this kind of scenarios?

Select is a way to call concurrent async methods without starting new tasks. It's less boilerplate than tasks when multiple async calls, like multiple servers, need to run in e.g. tests. It also cancels all of the functions as soon as one of them returns.

https://tokio.rs/tokio/tutorial/select

jbesraa · 2024-03-08T09:12:39Z

payjoin-cli/src/app/v2.rs

@@ -162,7 +165,7 @@ impl App {
            let http = http_agent()?;
            let response = spawn_blocking(move || {
                http.post(req.url.as_ref())
-                    .set("Content-Type", "text/plain")
+                    .set("content-type", "message/ohttp-req")


the content-type is quite important in payjoin world, maybe we should consider add RequestContentType enum to the main crate with the available options. I believe these are text/plain and message/ohttp-req.

I see two ergonomic TODOs in this comment

define header key/value constants so these can't be misinterpreted during implementation

include headers in req from which to set them rather than set them manually

Beyond these TODOs, these are exactly the confusing implementation details that a payjoin-io crate could abstract.

edit: since these are ergonomic in nature and don't affect the logic of the fetch I'd rather leave them to another PR

payjoin-cli/src/main.rs

DanGould · 2024-03-08T17:10:54Z

@jbesraa wonderful review. You raised some great points which I've addressed. It's looking like the time for the IO crate to be developed is upon us.

jbesraa · 2024-03-11T13:10:23Z

payjoin/src/receive/v2.rs

@@ -28,7 +29,7 @@ pub struct Request {
 #[derive(Debug, Clone)]
 pub struct V2Context {


We should document the properties here, probably for a different PR

jbesraa · 2024-03-11T13:10:40Z

payjoin/src/receive/v2.rs

@@ -37,20 +38,14 @@ pub struct V2Context {
 #[derive(Debug, Clone)]
 pub struct Enroller {


We should document the properties here, probably for a different PR

jbesraa · 2024-03-11T13:10:53Z

payjoin/src/receive/v2.rs

@@ -108,39 +103,24 @@ fn subdirectory(pubkey: &bitcoin::secp256k1::PublicKey) -> String {
    base64::encode_config(pubkey, b64_config)
 }

-#[derive(Debug, Clone)]
+#[derive(Debug, Clone, PartialEq, Eq)]
 pub struct Enrolled {


We should document the properties here, probably for a different PR

DanGould · 2024-03-12T04:08:24Z

@jbesraa I'm leaving a note as to why that hasn't been merged yet. Thanks for all the reviews.

I'd like to merge, but not without that danger-local-https feature support and basic tests. In order to do that testing I've been working on #207 to refactor tests to run payjoin-directory as a library but that ran into MSRV issues. I may have been able to make a hack to get MSRV to 1.63.0 with sqlx but it seemed better to use the same time to kill two birds with one stone by switching to redis in #210. Our tests are getting seriously robust now.

DanGould · 2024-03-14T05:38:10Z

This PR now tests the CONNECT bootstrap mechanism by introducing an ohttp_relay dependency integration and duplicating the fetch mechanism found in payjoin-cli. @jbesraa Assuming you agree with the implementation, the duplicate code should make a great starting point for #211. The tests appear not to pass in CI because of a DNS issue I will try to fix in the morning. They seem to pass in my local environment.

DanGould · 2024-03-16T19:49:55Z

Testing the networking between local environment and CI has been frustrating. This branch seems to pass v2 tests: https://github.com/DanGould/rust-payjoin/tree/fetch-ohttp-keys-ci

but making requests to 0.0.0.0 seems to work exclusively in ci but to localhost exclusively in local v2 tests. There are even some discrepancies between rust versions: the first time v2 tests ran on that commit, test_bad_ohttp_keys failed, but the second run passed. Confusing. Perhaps there is some sort of race in addition to what appears to be a discrepancy in DNS resolution.

DanGould · 2024-03-16T20:57:18Z

The most recent test suggests that ohttp-relay is resolving localhost to ipv6 [::1]:$PORT which the directory server is not listening for. Aha!

This required a wrapper struct to make OhttpKeys de/serializable

DanGould · 2024-03-17T17:50:56Z

@jbesraa I took your feedback into account and e2e tested the results. Our test framework is way better after this as a side effect.

DanGould force-pushed the fetch-ohttp-keys branch 4 times, most recently from fa83e06 to 1ba02c0 Compare February 21, 2024 21:11

DanGould force-pushed the fetch-ohttp-keys branch 3 times, most recently from 4a82716 to d22bee4 Compare March 4, 2024 17:00

DanGould marked this pull request as ready for review March 4, 2024 17:27

DanGould requested a review from jbesraa March 4, 2024 17:28

DanGould marked this pull request as draft March 4, 2024 21:48

DanGould force-pushed the fetch-ohttp-keys branch from d22bee4 to 7957a8f Compare March 4, 2024 22:10

DanGould marked this pull request as ready for review March 4, 2024 22:12

DanGould force-pushed the fetch-ohttp-keys branch 2 times, most recently from 26029ec to b245008 Compare March 7, 2024 19:59

DanGould commented Mar 7, 2024

View reviewed changes

jbesraa reviewed Mar 8, 2024

View reviewed changes

DanGould force-pushed the fetch-ohttp-keys branch 2 times, most recently from c9f69a9 to 4a3dd68 Compare March 8, 2024 17:08

DanGould requested review from jbesraa and removed request for jbesraa March 8, 2024 17:15

jbesraa approved these changes Mar 11, 2024

View reviewed changes

DanGould force-pushed the fetch-ohttp-keys branch from 4a3dd68 to 9dd5e0d Compare March 14, 2024 03:14

DanGould requested a review from jbesraa March 14, 2024 03:14

DanGould force-pushed the fetch-ohttp-keys branch from 9dd5e0d to 52e2691 Compare March 14, 2024 03:58

DanGould removed the request for review from jbesraa March 14, 2024 05:37

DanGould force-pushed the fetch-ohttp-keys branch 2 times, most recently from 8ff3497 to ba69abc Compare March 14, 2024 21:11

DanGould force-pushed the fetch-ohttp-keys branch 10 times, most recently from 72a0fc8 to 8c6b360 Compare March 16, 2024 19:45

DanGould force-pushed the fetch-ohttp-keys branch from 8c6b360 to a055d62 Compare March 16, 2024 20:51

DanGould force-pushed the fetch-ohttp-keys branch 4 times, most recently from fbb4e7e to 1ab0f8e Compare March 17, 2024 17:02

DanGould added 9 commits March 17, 2024 13:03

Fetch ohttp-keys through CONNECT tunnel

50c9d6a

Replace ohttp_config str with OhttpKeys

6bb28b5

This required a wrapper struct to make OhttpKeys de/serializable

Use serde traits to ser_de Enrolled

f8259fc

Have directory listen on IPv4 and IPv6

bb2ff35

Test ohttp-keys CONNECT bootstrap integration

2feaeed

Deduplicate http_agent_builder

15eb868

Add integration tracing

cd68696

Test 0.0.0.0 rcgen host

0e70be6

Triple duration to improve reliability

a422b9f

DanGould force-pushed the fetch-ohttp-keys branch from 1ab0f8e to a422b9f Compare March 17, 2024 17:04

DanGould merged commit 2913646 into payjoin:master Mar 17, 2024
5 checks passed

DanGould deleted the fetch-ohttp-keys branch March 17, 2024 17:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fetch ohttp keys through CONNECT tunnel #194

fetch ohttp keys through CONNECT tunnel #194

DanGould commented Feb 12, 2024 •

edited

Loading

DanGould left a comment •

edited

Loading

jbesraa Mar 8, 2024

DanGould Mar 8, 2024

jbesraa Mar 8, 2024

DanGould Mar 8, 2024 •

edited

Loading

jbesraa Mar 8, 2024

DanGould Mar 8, 2024 •

edited

Loading

jbesraa Mar 8, 2024

DanGould Mar 8, 2024 •

edited

Loading

DanGould commented Mar 8, 2024

jbesraa Mar 11, 2024

jbesraa Mar 11, 2024

jbesraa Mar 11, 2024

DanGould commented Mar 12, 2024

DanGould commented Mar 14, 2024

DanGould commented Mar 16, 2024 •

edited

Loading

DanGould commented Mar 16, 2024

DanGould commented Mar 17, 2024

		@@ -28,7 +29,7 @@ pub struct Request {
		#[derive(Debug, Clone)]
		pub struct V2Context {

		@@ -37,20 +38,14 @@ pub struct V2Context {
		#[derive(Debug, Clone)]
		pub struct Enroller {

fetch ohttp keys through CONNECT tunnel #194

fetch ohttp keys through CONNECT tunnel #194

Conversation

DanGould commented Feb 12, 2024 • edited Loading

Regarding Configuration

DanGould left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DanGould Mar 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DanGould Mar 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DanGould Mar 8, 2024 • edited Loading

Choose a reason for hiding this comment

DanGould commented Mar 8, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DanGould commented Mar 12, 2024

DanGould commented Mar 14, 2024

DanGould commented Mar 16, 2024 • edited Loading

DanGould commented Mar 16, 2024

DanGould commented Mar 17, 2024

DanGould commented Feb 12, 2024 •

edited

Loading

DanGould left a comment •

edited

Loading

DanGould Mar 8, 2024 •

edited

Loading

DanGould Mar 8, 2024 •

edited

Loading

DanGould Mar 8, 2024 •

edited

Loading

DanGould commented Mar 16, 2024 •

edited

Loading