Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/pkgsite: support viewing a package @master #36811

Closed
myitcv opened this issue Jan 27, 2020 · 32 comments
Closed

x/pkgsite: support viewing a package @master #36811

myitcv opened this issue Jan 27, 2020 · 32 comments

Comments

@myitcv
Copy link
Member

@myitcv myitcv commented Jan 27, 2020

What is the URL of the page with the issue?

https://pkg.go.dev/github.com/moby/moby/client?tab=doc

What is your user agent?

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.66 Safari/537.36

Screenshot

Screen Shot 2020-01-27 at 19 52 45

What did you do?

Visited the aforementioned link.

What did you expect to see?

The current docs for github.com/moby/moby/client, like on godoc:

https://godoc.org/github.com/docker/docker/client

(notice https://github.com/docker/docker redirects to https://github.com/moby/moby)

What did you see instead?

Per the screenshot, at the time of writing, docs for v1.13.1, which was tagged on Feb 8, 2017.

The godoc version is looking at ~master at any given moment in time.

The current advice from the project maintainers is to use master of this package/module:

Hence only godoc.org works as a means of viewing "current" docs for github.com/moby/moby/client.

@julieqiu julieqiu removed the Documentation label Jan 28, 2020
@julieqiu julieqiu removed the Documentation label Jan 28, 2020
@julieqiu julieqiu changed the title go.dev: github.com/moby/moby/client docs are not current go.dev: support viewing <import-path>@master Jan 28, 2020
@julieqiu julieqiu changed the title go.dev: support viewing <import-path>@master go.dev: support viewing a package @master Jan 28, 2020
@julieqiu
Copy link
Contributor

@julieqiu julieqiu commented Jan 28, 2020

We're actively working on designing a better flow, but the workaround for this right now is to do the following:

  1. Get the semantic version from proxy.golang.org using the .info endpoint. For example:
    https://proxy.golang.org/github.com/moby/moby/@v/master.info currently shows:
{
    "Version": "v1.4.2-0.20200128034134-2ebaeef943cc",
    "Time": "2020-01-28T03:41:34Z"
}

Note. This will also automatically fetch the module for the proxy and populate it on pkg.go.dev, if it does not already exist (per instructions at https://go.dev/about#adding-a-package). It may take a minute or two for the package to appear on pkg.go.dev, if it is not already there.

  1. Visit pkg.go.dev/<import-path>@<semantic-version> for the package documentation, in this case:
    https://pkg.go.dev/github.com/moby/moby/client@v1.4.2-0.20200128034134-2ebaeef943cc.

We know that this isn't the best user experience, and are working on a better workflow.

@gopherbot gopherbot added this to the Unreleased milestone Feb 6, 2020
@mewmew
Copy link
Contributor

@mewmew mewmew commented Feb 24, 2020

We know that this isn't the best user experience, and are working on a better workflow.

How about a "Fetch latest version from master" link added to the bottom of pages on pkg.go.dev in similar fashion to the "Refresh now" link of godoc.org?

screenshot_2020-02-24_23:32:24

This is issue #37106

myitcv added a commit to cuelang/cuelang.org that referenced this issue Apr 6, 2020
This is more relevant for tip.cuelang.org, because the @master docs on
pkg.go.dev are not currently easily accessible. Hence the advice is:

golang/go#36811 (comment)

Therefore we code generate a redirect file based on the required version
of cuelang.org/go (which is automatically updated for tip)
myitcv added a commit to cuelang/cuelang.org that referenced this issue Apr 6, 2020
This is more relevant for tip.cuelang.org, because the master docs on
pkg.go.dev are not currently easily accessible. Hence the advice is:

golang/go#36811 (comment)

Therefore we code generate a redirect file based on the required version
of cuelang.org/go (which is automatically updated for tip)
myitcv added a commit to cuelang/cuelang.org that referenced this issue Apr 6, 2020
…ion (#90)

This is more relevant for tip.cuelang.org, because the master docs on
pkg.go.dev are not currently easily accessible. Hence the advice is:

golang/go#36811 (comment)

Therefore we code generate a redirect file based on the required version
of cuelang.org/go (which is automatically updated for tip)
@hugelgupf
Copy link
Contributor

@hugelgupf hugelgupf commented May 2, 2020

Hey Julie, do you have any updates on the progress? This is a major blocker for us (u-root) to go.dev :( Thanks!

@julieqiu
Copy link
Contributor

@julieqiu julieqiu commented May 2, 2020

@hugelgupf thanks for letting us know! We deprioritized this feature temporarily to work on open sourcing (#36747) and with our product/UX teams to address UX issues. I don't have an exact timeline on when this feature will be ready, but we are aiming for within the next month or so.

Out of curiosity, does the workaround in #36811 (comment) work for your use case (aside from the suboptimal user experience)?

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
The logic for GetVersionMap is changed so that an explicit module path
is now required. All logic is also now the same for all
requestedVersions (as opposed to a special case for latest).

This function is only used by the frontend to check if a version exists
as part of a frontend fetch, and this behavior makes more sense for that
use case.

Updates golang/go#36811
Updates golang/go#37002
Updates golang/go#37106

Change-Id: I415a2730daa6edc023f80c0c615521047311f35b
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/744833
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
The frontend server is not initiated with a queue. The frontend task
queue will be used to support frontend fetches.

frontend.FetchAndUpdateState is added, which is a copy of
worker.FetchAndUpdateState for use in testing and locally.

Updates golang/go#36811
Updates golang/go#37002
Updates golang/go#37106

Change-Id: I41922d30462d2623a061aa1f207bb2b39f7b54e2
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/743102
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
This column will be used to display the canonical module path when a
frontend fetch results in fetching an alternative module path.

Updates golang/go#36811
Updates golang/go#37002
Updates golang/go#37106

Change-Id: Ie8ee6ba64ea799d264c1079c345e37d2e073da38
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/751268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
The version_map.go_mod_path column is now populated in
UpsertVersionMap.

Updates golang/go#36811
Updates golang/go#37002
Updates golang/go#37106

Change-Id: I4e474d28bee67caf7625045082df46f93394f3f6
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/751269
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
FetchAndUpdateState in internal/frontend and internal/worker now
insert the GoModPath field on VersionMap.

Updates golang/go#36811
Updates golang/go#37002
Updates golang/go#37106

Change-Id: Idfce6d685d24d915608acc1748a84b9db5312ae4
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/751270
CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
A fetch endpoint is added to the frontend, which will queue a module to
be enqueued by the worker, if it doesn't already exist in the database.

After enqueuing, the fetch handler will poll the version_map table at a
constant rate, until the path returns or the request times out.

If the request fails, a corresponding statusCode and responseText will
be returned to be displayed to the user.

Updates golang/go#36811
Updates golang/go#37002
Updates golang/go#37106

Change-Id: Ic2e20146dc626bf296db05bc2abbfb50d6fd7991
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/743103
CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
isSupportedVersion is added. master is now a supported version if the
feature flag ExperimentFrontendPackageAtMaster is on.

Updates golang/go#36811

Change-Id: Ib1c885a5e75cb697b8e99fd8e8c9a347e2b344ed
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/753605
CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 15, 2020
When a path page 404s, it will now render the notfound.tmpl page (once
the frontend-fetch feature flag is on), which provides a button for the
user to make a request to fetch the package.

Updates golang/go#36811
Updates golang/go#37002
Updates golang/go#37106

Change-Id: I17fedd018435e8d3e51e2a2a4a972d3cf673df56
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/753606
CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 28, 2020

Change https://golang.org/cl/245020 mentions this issue: internal/frontend: use StatusNotFoundInVersionMap for frontend fetch

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 28, 2020
s.fetchModule is removed. The logic from that function is moved into
checkPossibleModulePaths, which is renamed from checkPossiblePaths for
clarity.

For golang/go#36811
For golang/go#37002

Change-Id: I8ac26ac13fcddc19582682bd5351b0b7a6a2858f
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245021
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 28, 2020
Previously, if the context deadline exceeded when calling GetVersionMap
in checkForPath, a 500 was returned. This is fixed so that a 408 is
returned instead.

For golang/go#36811
For golang/go#37002

Change-Id: I60d78fa0e76d5f3e4bb4dab34d66d73a45121955
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245121
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 28, 2020
Previously, we used http.StatusProcessing and http.StatusNoContent to
indicate when a row did not exist in the version_map table.

Neither of these status codes fit well, so our own error code is defined
for the case when a module path has not been processed yet:
statusNotFoundInVersionMap.

For golang/go#36811
For golang/go#37002

Change-Id: Ia55972bfbfb1950ad82254d8f1082dda052bf376
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245020
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 28, 2020
When a fetch request results in a 404, it's possible that the module
path will appear later in the index. Similarly, when a fetch request
results in a 480 or 500, it's possible that the error was transient or
has been fixed but not yet reprocessed.

In these cases, we should allow the user to refetch the path after the
taskIDChangeInterval has passed.

For golang/go#36811
For golang/go#37002

Change-Id: Ia4813140ca793b7b91d80bdc196f61af075766ca
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/244601
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 29, 2020

Change https://golang.org/cl/245437 mentions this issue: content: alphabetize classes with prefix Fetch

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 29, 2020

Change https://golang.org/cl/245402 mentions this issue: internal/worker: don't delete 40x results from version_map

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 29, 2020

Change https://golang.org/cl/245401 mentions this issue: content: center text in h3.FetchMessage

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 29, 2020
Class names with the prefix "Fetch" are alphabetized relative
to each other. No changes are made to the CSS.

For golang/go#36811
For golang/go#37002

Change-Id: I662829fd50f1e1634cc891ba50c4bf1d4f9ac0bd
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245437
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 29, 2020
At the moment, when module is fetched with a status 40x, it is deleted
from the database. However, we don't want to delete that module from
version_map, otherwise the response will never be returned to the user
for a frontend fetch.

Modules with status 40x are now deleted first, and then the result is
inserted into version_map and module_version_states.

For golang/go#36811
For golang/go#37002

Change-Id: I5eb9414c065bdc8364445edb293f21bbe870fa43
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245402
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 29, 2020
For golang/go#36811
For golang/go#37002

Change-Id: I3438a8d72617f3c7af88310469b4799a5539616e
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245401
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 29, 2020

Change https://golang.org/cl/245640 mentions this issue: internal/frontend: tweak response text for fetch requests

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 29, 2020
The response text for different frontend fetch outcomes is
updated.

The code in fetchRequestStatusAndResponseText is also
refactored.

For golang/go#36811
For golang/go#37002

Change-Id: I175921acde5edc5cbf677eb1d1f23171b40318b9
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245640
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 29, 2020

Change https://golang.org/cl/245646 mentions this issue: content: update text in fetch.tmpl for 404s

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 30, 2020
For golang/go#36811
For golang/go#37002

Change-Id: I1853ecf2798e717589bb81ad0d749a3f814b55ec
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245646
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@julieqiu
Copy link
Contributor

@julieqiu julieqiu commented Jul 30, 2020

You can now view a package at master using the URL, https://pkg.go.dev/<path>@master. For example, https://pkg.go.dev/github.com/moby/moby/client@master.

@julieqiu julieqiu closed this Jul 30, 2020
@SamWhited
Copy link
Member

@SamWhited SamWhited commented Jul 30, 2020

Are there limits to this? With the exception of the example you posted, none of the common packages I use appear to work. They all use vanity URLs, so maybe this has something to do with it?

@julieqiu
Copy link
Contributor

@julieqiu julieqiu commented Jul 30, 2020

@SamWhited - mind sharing what they are? I can look into it.

@SamWhited
Copy link
Member

@SamWhited SamWhited commented Jul 30, 2020

@julieqiu here are two of mine that appear to be exhibiting this behavior: https://pkg.go.dev/code.soquee.net/otp@master or https://pkg.go.dev/mellium.im/xmpp@master

@SamWhited
Copy link
Member

@SamWhited SamWhited commented Jul 30, 2020

Oh, apparently I have to "request" them or it 404s? I hadn't visited the actual page to click the link. I don't know what that means, but it's confusing

@julieqiu
Copy link
Contributor

@julieqiu julieqiu commented Jul 30, 2020

The reason is because code.soquee.net/otp@master and mellium.im/xmpp@master weren't in our database yet. In the future, visiting these pages will automatically refresh master.

Feel free to file a new issue with any UI suggestions.

@hugelgupf
Copy link
Contributor

@hugelgupf hugelgupf commented Sep 10, 2020

Hi @julieqiu,

a lot of repositories are now moving to renaming their default branches to main. It seems that master is hard-coded in pkgsite. Would it be possible to support some kind of magic version string that gets the default branch, whatever it may be named? This may end up being the reason that I move some of my repos back to master, which I'd like to avoid.

Let me know if I should open another issue for this.

Thanks!

@myitcv
Copy link
Member Author

@myitcv myitcv commented Sep 10, 2020

@hugelgupf - this issue is closed. Please open a new issue.

FWIW the means by which git can determine the default remote (origin in this case) branch looks like this:

git remote set-head origin -a

which then updates the following:

$ cat .git/refs/remotes/origin/HEAD
ref: refs/remotes/origin/main
@anacrolix
Copy link
Contributor

@anacrolix anacrolix commented Nov 11, 2020

In my experience, this only seems fixed for vanity URLs. More general module names don't work, like https://pkg.go.dev/github.com/anacrolix/torrent@master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants