Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/code: implement module proxy protocol #24

Merged
merged 3 commits into from May 7, 2019

Conversation

1 participant
@dmitshur
Copy link
Member

commented May 6, 2019

This change adds an initial version of a Go module server
that implements the module proxy protocol, as described
at https://golang.org/cmd/go/#hdr-Module_proxy_protocol.

This enables more efficient serving of Go modules, especially in cases
where only the list, .info and .mod endpoints are requested (not .zip).
That happens often: whenever the module isn't actually used in a build,
rather it just happens to be in the module requirement graph.

The module server isn't fully featured and has some known limitations,
but it's enough to cover my current needs. It serves all current module
versions at dmitri.shuralyov.com/... with identical checksums as the
canonical go mod download algorithm. Its feature set may get expanded
as my needs change.

Disclaimer: There are many wildly different ways to implement a server
that speaks the module proxy protocol. Most are much simpler than what
I've done. There are trade-offs, and I've optimized the design for my own
needs. It's also an initial version which may change over time.

internal/code: implement module proxy protocol
This change adds an initial version of a Go module server
that implements the module proxy protocol, as described
at https://golang.org/cmd/go/#hdr-Module_proxy_protocol.

This enables more efficient serving of Go modules, especially in cases
where only the list, .info and .mod endpoints are requested (not .zip).
That happens often: whenever the module isn't actually used in a build,
rather it just happens to be in the module requirement graph.

The module server isn't fully featured and has some known limitations,
but it's enough to cover my current needs. It serves all current module
versions at dmitri.shuralyov.com/... with identical checksums as the
canonical go mod download algorithm. Its feature set may get expanded
as my needs change.

dmitshur added some commits May 6, 2019

internal/code: replace unreachable case with a panic
It can never be reached because we already parse the request type
via parseModuleProxyRequest, which makes sure that it's one of the
supported ones ("list", "info", "mod", or "zip").
internal/code: adjust ModuleHandler documentation
Re-arrange it in a better way, improve formatting.

@dmitshur dmitshur merged commit df9d62c into master May 7, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@dmitshur dmitshur deleted the module-server branch May 7, 2019

dmitshur added a commit that referenced this pull request May 18, 2019

change module proxy URL to contain a non-empty host
Previously, the URL of the module proxy used to serve modules beginning
with the path dmitri.shuralyov.com contained an empty host value.
This worked successfully in Go 1.11 and Go 1.12, leading to GET requests
without stuttering. For example, to fetch a list of versions in the
dmitri.shuralyov.com/state module, the request was:

	GET https://dmitri.shuralyov.com/state/@v/list

However, a module proxy URL with an empty host no longer works in Go tip
as of https://golang.org/cl/170879, and evidence suggests that is the
better behavior to continue to enforce.

There are plans to document the requirement of a module proxy URL to
have a non-empty host. It's also planned to improve the error reporting
when the go command encounters an invalid module proxy URLs, so users
will have a better idea of what went wrong. See issue golang/go#32006.

To be compatible with Go 1.11, 1.12, and the future Go 1.13, this commit
changes the module proxy URL used to serve own modules from "https://"
to "https://dmitri.shuralyov.com/api/module". Future module proxy
requests will look like:

	GET https://dmitri.shuralyov.com/api/module/dmitri.shuralyov.com/state/@v/list

The module proxy protocol is meant primarily for machines rather than
humans, so the longer path that stutters is an acceptable compromise.

A benefit of the new proxy URL is that we no longer need to check
if a request matches. Instead, all HTTP requests with a "/api/module/"
path prefix can be immediately recognized as requests to the module
proxy, and handled as such.

As part of embracing the machine-focused aspect of these URLs,
also remove the optional human-friendly behavior of redirecting
when the module path and version are not encoded in the request URL.
Instead, serve a 400 Bad Request error. A correct module proxy client
will not make the mistake of forgetting to encode those values.

Updates #24
Updates golang/go#32006
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.