Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/pkgsite: packages with go 1.21.0 in go line can't be fetch by pkg.go.dev #62031

Closed
matthewmueller opened this issue Aug 15, 2023 · 23 comments
Closed
Assignees
Labels
NeedsFix The path to resolution is known, but the work has not been done. pkgsite
Milestone

Comments

@matthewmueller
Copy link

matthewmueller commented Aug 15, 2023

What is the URL of the page with the issue?

What is your user agent?

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36

Screenshot

CleanShot 2023-08-14 at 22 35 31

What did you do?

I've been releasing quite a few open source Go libraries lately, but I'm having trouble getting docs generated for them.

I'm also having trouble diagnosing if I'm doing something wrong or if it takes ~days to show up on pkg.go.dev.

They all present in the proxy pretty much immediately, so I can download them:

The steps I take are:

  1. Push up a v0.0.1 tag to Github
  2. Request an index
  3. Come back later

What did you expect to see?

I'd expect when I request an index for it to show up pretty quickly. Within minutes, ideally seconds.

If there was an error, I'd expect an error message with some information on how to resolve it.

What did you see instead?

"github.com/livebud/mod" could not be found.


Thanks for your help!

@gopherbot gopherbot added this to the Unreleased milestone Aug 15, 2023
@matthewmueller matthewmueller changed the title x/pkgsite: Having trouble generating godoc x/pkgsite: Having trouble getting docs to show up on pkg.go.dev Aug 15, 2023
@findleyr
Copy link
Contributor

Hi, I've looked into this and it looks like there's a cached negative response for these packages that is preventing them from being fetched. I'll need more time to dig into it, but it looks like a pkgsite bug.

@findleyr findleyr added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 17, 2023
@jroimartin
Copy link
Contributor

Just in case it can help debugging the problem, I'm having similar issues with the following module:

https://pkg.go.dev/github.com/jroimartin/proxy

It took quite a while and several requests for v1.0.0 and v1.1.0 to show up in pkg.go.dev. After that, v.1.1.1 and v1.2.0 have not shown up yet. However, both are present in proxy.golang.org:

https://proxy.golang.org/github.com/jroimartin/proxy/@v/v1.1.1.info
https://proxy.golang.org/github.com/jroimartin/proxy/@v/v1.2.0.info

I hope it helps.

@jroimartin
Copy link
Contributor

jroimartin commented Aug 19, 2023

BTW the versions that are not showing up have the following go directive in their go.mod:

go 1.21.0

@Sven-Seyfert
Copy link

Maybe related to #62188 🤔 .

@ajanata
Copy link

ajanata commented Aug 21, 2023

Also having problems with v0.2.1 of https://pkg.go.dev/github.com/gametimesf/testy, which is also showing up in the proxy: https://proxy.golang.org/github.com/gametimesf/testy/@v/v0.2.1.info

The new version here also introduces go 1.21.0 to go.mod.

@findleyr
Copy link
Contributor

@jroimartin thanks for pointing that out. go 1.21.0 (or go 1.21rc3) is almost certainly the problem!

@findleyr
Copy link
Contributor

I see the problem: the x/mod dependency of pkgsite is quite old, and doesn't have the updated go directive syntax.

Fix incoming. We may need to manually reset broken modules; I'll handle all those listed here, and may write a script to scrape and purge recent modules from the proxy.

@findleyr findleyr added NeedsFix The path to resolution is known, but the work has not been done. Soon This needs to be done soon. (regressions, serious bugs, outages) labels Aug 21, 2023
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 21, 2023
@findleyr findleyr self-assigned this Aug 21, 2023
@gopherbot
Copy link

Change https://go.dev/cl/521125 mentions this issue: internal/worker: upgrade x/mod to allow fetching 1.21.0 modules

gopherbot pushed a commit to golang/pkgsite that referenced this issue Aug 22, 2023
Upgrade x/mod so that modfile.Parse accepts the newly relaxed go
directive syntax. Add a test.

For golang/go#62031

Change-Id: I63a8eb73ff7428e67d80446b493c624a52ad2e96
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/521125
TryBot-Result: Gopher Robot <gobot@golang.org>
kokoro-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
@findleyr findleyr changed the title x/pkgsite: Having trouble getting docs to show up on pkg.go.dev x/pkgsite: packages with go 1.21.0 in go line can't be fetch by pkg.go.dev Aug 22, 2023
@findleyr
Copy link
Contributor

Ok I believe I've re-fetched all versions mentioned in this issue. @Sven-Seyfert I'm still having trouble with yours, presumably because of the capitalization. Needs more investigation.

@jroimartin
Copy link
Contributor

@findleyr I can confirm github.com/jroimartin/proxy versions are now shown in pkg.go.dev. Thanks!

@findleyr findleyr removed the Soon This needs to be done soon. (regressions, serious bugs, outages) label Aug 22, 2023
@findleyr
Copy link
Contributor

I'm going to leave this open for a few days before devoting resources to finding and re-fetching broken versions. Unfortunately for now the process is painstakingly manual.

@hugowetterberg
Copy link

So, should we report broken versions in this issue? In that case, could you fix github.com/ttab/darknut@v0.1.0? Thanks!

@matthewmueller
Copy link
Author

Thanks for working on this @findleyr! All the packages above are now working 🎉

Got a few more for ya if you don't mind:

@findleyr
Copy link
Contributor

Yes, I'll manually fix anything folks report here. It's unfortunately non-trivial to automate.

@hugowetterberg @matthewmueller done.

@david-littlefarmer
Copy link

Hello, i have the same issue with new package, if it can be added, it would be great.

@findleyr
Copy link
Contributor

@david-littlefarmer done

@Sven-Seyfert
Copy link

Sven-Seyfert commented Aug 24, 2023

Hi @findleyr,

like I wrote in issue #62188 which was closed by you, I am grateful.

Unfortunately it seems to work only for that versions which you handled manually (two days ago).
I thought the fix was released? Because any new version of the same package (of mine) will not be displayed on https://pkg.go.dev.

I am looking for v0.6.0, but only v0.4.0 is displayed. Do you have to maintain each version on you own (manually)? I thought the fix will handle this?

https://proxy.golang.org/github.com/sven-seyfert/gomisc/@v/v0.6.0.info
https://proxy.golang.org/github.com/sven-seyfert/gomisc/@v/list

Best regards
Sven

@findleyr
Copy link
Contributor

Hi @Sven-Seyfert, the fix was deployed, and I just clicked on the button to fetch and got a response (i.e., I didn't do anything behind the scenes).

You should be able to request v0.5.0 here, and it should work: https://pkg.go.dev/github.com/sven-seyfert/gomisc@v0.5.0

I'll investigate why versions aren't flowing.

@Sven-Seyfert
Copy link

I'll investigate why versions aren't flowing.

Thanks again @findleyr 👍 .

@Sven-Seyfert
Copy link

Hi @findleyr ,

it seems to be not working. My new v0.7.0 (see links above) is in the proxy available, but will not be displayed on pkg.go.dev. I don't want to mention you or the team each time when I release a new tag. I am really sorry about it.

Thanks again for the investigation why versions are not flowing.

Best regards and a nice weekend for you
Sven

@jroimartin
Copy link
Contributor

jroimartin commented Aug 26, 2023

@Sven-Seyfert it seems pkg.go.dev is not detecting new versions automatically right now. However, you can visit https://pkg.go.dev/github.com/sven-seyfert/gomisc@v0.7.0 (or whatever it is the new version) and request it using the button. I've just done it for you and it worked.

@findleyr
Copy link
Contributor

We did figure out the root cause of the backlogged module processing queue: it was a bug in the proxy that was mitigated by https://go.dev/cl/522183, and should be fixed in the proxy ~today or tomorrow.

It looks like we caught up with backlogged versions over the weekend.

@findleyr
Copy link
Contributor

findleyr commented Sep 7, 2023

Closing as this has been working for a while now.

@findleyr findleyr closed this as completed Sep 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done. pkgsite
Projects
None yet
Development

No branches or pull requests

8 participants