net/http: add methods and path variables to ServeMux patterns #60227
Replies: 40 comments 119 replies
-
I welcome a proposal meant to beef up the capabilities of
This change would introduce a new way of restricting methods: within the pattern (in addition to within the handler itself). That could cause some confusion. Besides, imagine I want to configure
Or would I also have to register the I think you could argue that the concern of restricting methods is best left out of the pattern. |
Beta Was this translation helpful? Give feedback.
-
I like it! If |
Beta Was this translation helpful? Give feedback.
This comment has been hidden.
This comment has been hidden.
-
Great idea! I'm a bit confused with the Changes Example 4 Since |
Beta Was this translation helpful? Give feedback.
-
Good Idea My question why not add an additional parameter to the handle function to implement the first part "distinguishing requests based on HTTP method (GET, POST, ...)" How its at the moment: mux.Handle("/api/", apiHandler{}) How it would look with an additional parameter mux.Handle("GET","/api/", apiHandler{}) |
Beta Was this translation helpful? Give feedback.
-
In general this looks really nice, thanks, particularly the order-independence invariant. One thing though:
I might well be missing a fundamental issue with this, but this seems overly restrictive to me.
So I don't really understand why something similar to the "longest literal prefix wins" rule couldn't apply |
Beta Was this translation helpful? Give feedback.
-
Thank you so much @jba! ❤️ Finally someone cares about I just finished reading the thread and have some ideas/questions that I hope will be helpful in finalizing the details before they become a proposal. 1. What data structure will the new
|
Beta Was this translation helpful? Give feedback.
-
Isn't that a compatibility breaking change?
|
Beta Was this translation helpful? Give feedback.
This comment has been minimized.
This comment has been minimized.
-
If there were an initial draft of this API, even one that was mostly stubbed out in term of actual implementation, I could try porting our existing routing code to it to discover what does and does not work. (It is important that it actually be something that I can send to the compiler, though, otherwise subtle issues tend not to get noticed.) Are there any plans to post such a rough or draft implementation for people to try out? |
Beta Was this translation helpful? Give feedback.
-
I am in favor of this proposal. I'd like to ask to rename the proposed Generally speaking, I'm in favor of short names. However, I feel adding the This would be an 8 character field name. But for comparison, the mean and median length of existing field names on the |
Beta Was this translation helpful? Give feedback.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
-
Should this be introduced as a golang.org/x package and then moved into Go proper once stabilized? |
Beta Was this translation helpful? Give feedback.
-
I don't understand the problem this is meant to solve. By way of contrast, I understand that the problem to be solved for log/slog was that there were a lot of third party structured loggers, but if you made a stand alone library, you couldn't count on which one of the many loggers your consumers would want you to interface with. The new slog package provides a common denominator, so that whatever consumer facing log API you use or whatever log sink backend you use, it can all interface with log/slog and everyone is happy. There was a clear problem statement. What's the problem statement for http.ServeMuxt? So, obviously, there are lots of other HTTP routers for Go. Is the problem to be solved that they're incompatible? In that case, then I guess Request.Vars is good to have, although it's unfortunate that it has to go on HTTP client requests as well. In any event, existing third party routers have mostly settled on either wrapping the standard library with their own (e.g. gin.Context) or passing things through context.Context (which is not ideal in terms of type safety, but Request.Vars doesn't help here either). I don't see how this makes them significantly more compatible. In the short run, things will be worse because a new crop of third party routers will use Vars, while the old ones will all continue to exist and not use Vars. Maybe the problem to be solved is that the standard ServeMux is just slightly underpowered? That might be true, but it's not clear why that problem needs to be solved and where to draw the line in solving it. I guess the urgency is because gorilla/mux is archived? It's hard to understand why the line is being drawn where it is and what the advantage of doing so is. It just feels a bit arbitrary to include methods and path parameters but not something else. |
Beta Was this translation helpful? Give feedback.
-
This is impressive work. I confess though that I still don't understand the precedence algorithm. You write
and you give the ordering of elements. But I don't see how to use that to figure out the precedence of whole paths. Do I go left to right to find the first element where the order differs, and declare the winner from that? Or is it something else? Also, what do I do if there is no corresponding element, that is, when one pattern has more elements than the other? Regarding implementation, we may or may not use a trie. That depends on whether our current simpler algorithm actually makes any real-world servers slower. (I'm skeptical but would love to be proven wrong.) Either way, the trie should be an implementation detail and shouldn't affect the statement of the precedence rules. |
Beta Was this translation helpful? Give feedback.
-
W/out thinking about the specific rules proposed, I'd like to point out that busy programmers on tight schedules will appreciate the option to obtain all registered patterns in order for debugging purposes. I'm assuming the mux can in principle be implemented w/a slice of pattern-handler pairs sorted in an order that allows selecting the first pattern that matches the request. If this is the case, it's also an easy mental model for the programmer to adopt - whatever the actual implementation happens to be/ |
Beta Was this translation helpful? Give feedback.
-
Hi I have written a very experimental prototype of the ideas discussed here. It is not optimized and I have only done some little testing. You can find it at https://github.com/ulikunitz/mux . Here is the documentation for the Handle function: // Handle registers the provided handler for the given pattern.
//
// Examples are:
//
// m.Handle("GET example.org/a/{id}/c", handler1)
// m.Handle("{method} {host}/foo", handler2)
// m.Handle("/simple", handler3)
// m.Handle("{host}/{$}")
//
// Following patterns will be supported:
//
// m.Handle("/a/{a2}/a", h2a)
// m.Handle("/a/{a2}/b", h2b)
// m.Handle("/a/{a1}/a", h1)
//
// A request /a/foo/a will always resolve to h1, because a1 is lexicographic
// before a2. The request /a/foo/b however will resolve to h2b, because b cannot
// be satisfied by the wildcard {a1}.
//
// Note we are not supporting multiple suffix variables at the same position. So
// following code leads to a panic in the second call.
//
// m.Handle("/b/{b2...}")
// m.Handle("/b/{b1...}")
//
func (mux *Mux) Handle(pattern string, handler http.Handler) |
Beta Was this translation helpful? Give feedback.
-
A few thoughts about performance. I looked at github.com/julienschmidt/go-http-routing-benchmark and improved my implementation until I was getting numbers in the same ballpark. (I haven't submitted those improvements yet.) I see no reason why the subset precedence rule needs to be any slower than the left-to-right rules in practice. It requires backtracking on some patterns that overlap, but the patterns in that benchmark are disjoint, and so are most in the wild, based on my limited research. I'm still not convinced that router optimization is more than a game. One bit of evidence is that gorilla/mux, the most popular router, is something like 40x slower than most other routers, even That said, I did notice that once I got my implementation to be fast, creating a map from variables to their values doubled its time. So I think we should hide the variable bindings behind a method, perhaps something like
The name and signature go with the existing |
Beta Was this translation helpful? Give feedback.
-
EDITED:
PathValue
method onhttp.Request
.This is a discussion that we hope will lead to a proposal.
We would like to expand the standard HTTP mux's capabilities by adding two features: distinguishing requests based on HTTP method (GET, POST, ...) and support for wildcards in the matched paths. Both features are particularly important to REST API servers.
Background
The current mux has a few important properties that we want to preserve:
The semantics of the mux do not depend on the order of the Handle or HandleFunc calls. Being order-independent makes it not matter what order packages are initialized (for init-time registrations) and allows easier refactoring of code. This is why the tie breakers are not based on registration order and why duplicate registrations panic (only order could possibly distinguish them). It remains a key design goal to avoid any semantics that depend on registration order.
The mux is fairly simple and straightforward to understand. It is not a goal to add every last possible bell and whistle. Other custom or more full-featured muxes should remain easy to write and a well-supported part of the Go web ecosystem.
As a refresher, today's patterns used in Handle and HandleFunc take the form
[host]/path[/]
. The rules are:In the (likely) event that multiple registered patterns match a request, ties are broken as follows (in order):
Potential Changes
First, a pattern can start with an optional method followed by a space, as in
GET /codesearch
orGET codesearch.google.com/
. A pattern with a method is only used to match requests with that method. So it is possible to have the same same path pattern registered with different methods:Second, a pattern can include a wildcard path element of the form
{name}
or{name...}
. For example,/b/{bucket}/o/{objectname...}
. The name must be a valid Go identifier; that is, it must fully match the regular expression[_\pL][_\pL\p{Nd}]*
.These wildcards must be path elements, meaning they must be preceded by a slash and then be followed by either a slash or the end of the string. For example,
/b_{bucket}
is not a valid pattern. (It is not a goal to support every possible URL schema with these patterns; for special cases, using other routers will continue to be a good choice.)Normally a wildcard matches only a single path element, ending at the next literal slash (not %2F) in the request URL. If the
...
is present, then the wildcard matches the remainder of the URL, including slashes. (Therefore it is invalid for a...
wildcard to appear anywhere but at the end of a pattern.)Precedence Rules
The tie-breaking rules change to:
One pattern is more specific than another if it matches a subset of methods and paths. Put another way, a pattern p1 is more specific than p2 if p2 matches all the (method, path) pairs that p1 matches, and more.
(We wish we could get down to a single "more specific" rule that includes host, method, and path, but that would break backwards compatibility: it would say that "example.com/" and "/foo" conflict, but the current rule requires that the first win.)
If two patterns overlap but neither is more specific than the other, they conflict. For example, it is OK to register both of these:
because the third path element keeps them from ever matching the same URL.
But it is not OK to register both of these:
Both of these match, for example,
/b/_/o/_
, but the first also matches/b/_/o/path/to/object
while the second does not, and the second matches/b/_/v/n
, which the first doesn't. Since neither pattern is more specific than the other, the second registration will panic during mux.Handle / mux.HandleFunc. It can be hard to tell at a glance why two patterns conflict, so the panic message will help by providing specific paths that exhibit the conflict, as I have done in this paragraph.In contrast, these two are OK, because the second is more specific than the first:
Methods also figure into rule 2, so the two patterns
are OK because the second is more specific, but the patterns
conflict because neither is more specific than the other: the first matches a POST to /foo, which the second doesn't, and the second matches a GET to /bar, which the first doesn't.
There is one last, special wildcard:
{$}
matches only the end of the URL, allowing writing a pattern that ends in slash but does not match all extensions of that path. For example, the pattern/{$}
matches the root page/
but (unlike the pattern/
today) does not match a request for/anythingelse
.Examples
Say the following patterns are registered:
In the examples that follow, the host in the request is example.com and the method is GET unless otherwise specified.
API
To support this API, the net/http package adds a new method to Request:
It returns the part of the path associated with the wildcard in the matching pattern, or the empty string if there was no such wildcard in the matching pattern. (Note that a successful match can also be empty, for a "..." wildcard.)
Beta Was this translation helpful? Give feedback.
All reactions