Upstream minurl::Url type for url parsing
#467
Conversation
tnull
force-pushed
the
2026-01-upstream-minurl
branch
2 times, most recently
from
fd71a06 to
0f95668
Compare
tnull
force-pushed
the
2026-01-upstream-minurl
branch
from
fbe008a to
d56d850
Compare
|
@tcharding Hmm, to have the |
tnull
force-pushed
the
2026-01-upstream-minurl
branch
from
d56d850 to
13bf05d
Compare
| use proptest::prelude::*; | ||
| use url::Url as MaxUrl; | ||
|
|
||
| proptest! { |
There was a problem hiding this comment.
Really feels like a better job for a fuzzer, but if we do stick with proptest maybe the simplest solution to the MSRV question is just just do the testing in a separate non-workspace crate?
There was a problem hiding this comment.
Well, the MSRV question is really orthogonal to whether or not to proptest. I don't want to get into that discussion again - in this case we already have a bunch of proptests (which actually already uncovered a few edge cases during development), so we can just use them.
The MSRV issue is really just about url's MSRV, which we can easily fix by pinning-back idna_adapter (as usual), but it's mostly a question on how this repo wants to handle that.
There was a problem hiding this comment.
The MSRV issue is really just about url's MSRV, which we can easily fix by pinning-back idna_adapter (as usual), but it's mostly a question on how this repo wants to handle that.
Right, my point was that it seems simpler to move the logic to a separate non-workspace crate in the bitreq repo? That way url and proptest also aren't even dev-deps and we don't have MSRV questions at all cause that test repo can have a stable msrv.
There was a problem hiding this comment.
Hmm, I'd really like to get input from @tcharding as the workspace owner on this: Tobin, would you prefer to either pin-back the required dev-dependency or move the test code to a separate sub-crate that doesn't run in the MSRV CI.
|
Mind adding a test that this fixes #468? |
tnull
force-pushed
the
2026-01-upstream-minurl
branch
2 times, most recently
from
c35d3ea to
f792086
Compare
tnull
force-pushed
the
2026-01-upstream-minurl
branch
2 times, most recently
from
24e1c26 to
eb3e3b1
Compare
Move the minurl URL parsing code to `bitreq/src/url.rs` and expose it as `bitreq::Url` and `bitreq::UrlParseError`. Co-Authored-By: HAL 9000 Signed-off-by: Elias Rohrer <dev@tnull.de>
Migrate the property-based tests from minurl to bitreq: - `url_properties.rs`: Tests that URL parsing properties hold - `url_parity.rs`: Tests parity with the `url` crate - `common/mod.rs`: Shared test utilities and strategies Also add the required dev-dependencies (`proptest` and `url`). Co-Authored-By: HAL 9000 Signed-off-by: Elias Rohrer <dev@tnull.de>
This commit replaces the internal `HttpUrl` type with the newly-migrated `Url` type. The change includes: - Add `pub(crate)` helper methods to `Url`: `is_https()`, `path_and_query()`, `append_query_params()`, `preserve_fragment_from()`, `write_base_url_to()`, and `write_resource_to()` - Extract field-parsing logic into `parse_inner` method for use by mutation methods, making them simpler and more robust - Update `ParsedRequest` to use `Url` instead of `HttpUrl` - Update `ConnectionParams` to use `u16` for port instead of `Port` enum - Simplify `http_url.rs` to only contain percent encoding functions - Update `Response::new()` to accept url as `String` instead of `HttpUrl` - Simplify WASM `build_full_url()` to use `Url::as_str()` Co-Authored-By: HAL 9000 Signed-off-by: Elias Rohrer <dev@tnull.de>
Add `append_query_param` method to `Url` that handles percent encoding internally, rather than having external code use the encoding functions directly. This encapsulates the encoding logic within the `Url` type. This ensures that existing percent-encoded query parameters in URLs are not double-encoded when new parameters are added, fixing issue rust-bitcoin#468. Co-Authored-By: HAL 9000 Signed-off-by: Elias Rohrer <dev@tnull.de>
tnull
force-pushed
the
2026-01-upstream-minurl
branch
from
eb3e3b1 to
668eb49
Compare
|
Now clippy should also be happy. |
|
I pulled down What if we bump the MSRV of |
3 participants
Fixes #468.
The
urlcrate has a notoriously large dependency tree which is why we want to avoid it as far as possible. However, we found us then re-implementing several aspects of URL parsing in different places. To this end, I recently (mostly vibe-)coded the 0-dependencyminurlcrate (https://github.com/tnull/minurl) which is meant as a drop-in replacement for the popularurlcrate.To this end, we kept the
UrlAPI completely compatible, and even added parity tests ensuring both APIs return exactly the same output given the same input.While I'm generally fine maintaining this as a separate crate, it makes a lot of sense to have this live as part of
bitreqand hence have it available everywhere in the ecosystem. Here I propose to upstream theminurl::Urltype (and ofc the corresponding test code). This also allows us to replace the ~half-donehttp_url::HttpUrltype.