Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/cmd/gitmirror: not mirroring some subrepos at this time #32931

Closed
dmitshur opened this issue Jul 3, 2019 · 8 comments

Comments

Projects
None yet
4 participants
@dmitshur
Copy link
Member

commented Jul 3, 2019

The gitmirror service was redeployed recently and it's currently mirroring the following repositories:

arch - ok
benchmarks - ok
blog - ok
crypto - ok
debug - ok
exp - ok
go - ok
image - ok
mobile - ok
net - ok
oauth2 - ok
perf - ok
review - ok
sync - ok
sys - ok
talks - ok
term - ok
text - ok
time - ok
tools - ok
tour - ok

(I got that list by port forwarding its HTTP port to my machine via kubectl port-forward <gitmirror pod name> 8585:8585.)

There are a few smaller subrepos missing, for example, mod, xerrors, website, build, playground.

The issue seems to be that the build dashboard is serving a subset of subrepos at its https://build.golang.org/packages?kind=subrepo endpoint, and gitmirror uses that endpoint to find a list of all subrepos to consider mirroring (see here). So we need to resolve the issue in the build dashboard, then redeploy gitmirror so it picks up the change.

/cc @toothrot @andybons

@cherrymui

This comment has been minimized.

Copy link
Contributor

commented Jul 3, 2019

Seems gofrontend repo mirroring also stopped.

@dmitshur

This comment has been minimized.

Copy link
Member Author

commented Jul 3, 2019

Yes, that's one of the affected subrepos as well.

The earlier analysis overlooked that there's another code path to mirror repos that talks to Gerrit API directly via instead of the build dashboard:

https://github.com/golang/build/blob/54405f243e45b89d4b1e4e1f0716387c58890f6f/cmd/gitmirror/gitmirror.go#L228

The problem seems to be there. I added a log statement to see the value of err:

        meta, err := gerritClient.GetProjects(ctx, "master")
        if err != nil {
+               log.Println("gerritClient.GetProjects:", err)
                return nil
        }

And got an 404 error. That's unexpected and I suspect is the source of the problem.

2019/07/03 19:08:57 gerritClient.GetProjects: HTTP status 404 Not Found; <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><title>Repository is not found - Git at Google</title><link rel="stylesheet" type="text/css" href="/+static/base.vsKBklzePi_Td7VvkjGVKw.cache.css"><!-- default customHeadTagPart --></head><body class="Site"><header class="Site-header"><div class="Header"><!-- default customHeader --><div class="Header-title">Git at Google</div><div class="Header-menu"> <a class="Header-menuItem" href="https://go-review.googlesource.com/">Code Review</a> <a class="Header-menuItem" href="https://accounts.google.com/o/oauth2/auth?response_type=code&amp;access_type=offline&amp;approval_prompt=force&amp;client_id=413937457453.apps.googleusercontent.com&amp;scope=https://www.googleapis.com/auth/gerritcodereview&amp;redirect_uri=https://www.googlesource.com/new-password&amp;state=go">Generate Password</a> <a class="Header-menuItem" href="https://security.google.com/settings/security/permissions">Revoke Passwords</a> <a class="Header-menuItem" href="https://accounts.google.com/AccountChooser?service=gerritcodereview&amp;continue=https://go.googlesource.com/login/projects/?b%3Dmaster%26format%3DJSON">Sign in</a> </div></div></header><div class="Site-content"><div class="Container "><h1>Repository is not found</h1></div> <!-- Container --></div> <!-- Site-content --><!-- default customFooter --><footer class="Site-footer"><div class="Footer"><span class="Footer-poweredBy">Powered by <a href="https://gerrit.googlesource.com/gitiles/">Gitiles</a></span><span class="Footer-formats"><a class="u-monospace Footer-formatsItem" href="?format=TEXT">txt</a> <a class="u-monospace Footer-formatsItem" href="?format=JSON">json</a></span></div></footer></body></html>

I think there's been some change with Gerrit and/or Gitilies and/or the frontend that routes traffic in between them. While https://go.googlesource.com/projects/?b=master&format=JSON worked before, it no longer does. But https://go-review.googlesource.com/projects/?b=master&format=JSON does work now.

@dmitshur

This comment has been minimized.

Copy link
Member Author

commented Jul 3, 2019

I'll update the Gerrit client in gitmirror to use https://go-review.googlesource.com/ as the URL, which should fix the immediate problem.

We can investigate and decide on follow up actions after.

@gopherbot

This comment has been minimized.

Copy link

commented Jul 3, 2019

Change https://golang.org/cl/184922 mentions this issue: cmd/gitmirror: update Gerrit API URL to include "-review"

@dmitshur

This comment has been minimized.

Copy link
Member Author

commented Jul 3, 2019

I've deployed CL 184922 to test it and it fixed this issue. All other subrepos are being mirrored now:

Repo List
arch - ok
benchmarks - ok
blog - ok
build - ok
crypto - ok
debug - ok
dl - ok
example - ok
exp - ok
gddo - ok
go - ok
gofrontend - ok
gollvm - ok
grpc-review - ok
image - ok
lint - ok
mobile - ok
mod - ok
net - ok
oauth2 - ok
perf - ok
playground - ok
proposal - ok
protobuf - ok
review - ok
scratch - ok
sublime-build - ok
sublime-config - ok
sync - ok
sys - ok
talks - ok
term - ok
text - ok
time - ok
tools - ok
tour - ok
vgo - ok
website - ok
xerrors - ok

As another followup item here, we should check if any other services are depending on a Gerrit API URL without the "-review" part, as they're likely to be affected too. I'm not aware of any at the moment.

@dmitshur dmitshur added NeedsFix and removed NeedsInvestigation labels Jul 3, 2019

@cherrymui

This comment has been minimized.

Copy link
Contributor

commented Jul 4, 2019

Thanks, @dmitshur

@dmitshur

This comment has been minimized.

Copy link
Member Author

commented Jul 4, 2019

Thanks to @toothrot as well for helping investigate this issue.

@dmitshur

This comment has been minimized.

Copy link
Member Author

commented Jul 4, 2019

As another followup item here, we should check if any other services are depending on a Gerrit API URL without the "-review" part, as they're likely to be affected too. I'm not aware of any at the moment.

I've checked the importers of golang.org/x/build/gerrit package, and all of the relevant ones are contained in golang.org/x/build. Other than gitmirror, all others already use https://go-review.googlesource.com as the Gerrit API URL.

I also checked the importers of github.com/andygrunwald/go-gerrit and didn't spot any issues in any of the relevant packages there either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.