x/build/cmd/gopherbot: keeps repeating some actions without effect

[dmitshur]

I noticed this in the gopherbot logs recently, and it's possible to reproduce locally in dry-run mode. Gopherbot keeps on repeating some actions every iteration. These actions don't have an effect, and so gopherbot never stops repeating them.

2018/10/22 07:44:15 got corpus update after 252.395017ms
2018/10/22 07:44:15 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 07:44:15 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 07:44:15 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 07:44:15 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/71730 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/71850 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72090 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72091 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72110 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72131 ...
2018/10/22 07:44:16 gopherbot ran in 1.335854405s

Reproduce Steps

$ go get -u golang.org/x/build/cmd/gopherbot
$ go run golang.org/x/build/cmd/gopherbot -dry-run -daemon
2018/10/22 14:27:47 Loading data from log *maintner.netMutSource ...
2018/10/22 14:27:47 Downloading 39043 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:27:47 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:27:55 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:27:56 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:27:56 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:27:56 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:27:56 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:27:56 gopherbot ran in 1.434313646s
2018/10/22 14:27:56 Updating data from log *maintner.netMutSource ...
2018/10/22 14:29:22 Downloading 1242 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:29:22 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:29:22 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:29:22 got corpus update after 1m25.647922592s
2018/10/22 14:29:23 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:29:23 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:29:23 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:29:23 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:29:23 gopherbot ran in 860.347827ms
2018/10/22 14:29:23 Updating data from log *maintner.netMutSource ...
2018/10/22 14:29:23 Downloading 1317 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:29:23 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:29:23 gerrit code.googlesource.com/gocloud: Ref {CLNumber:34491 Version:0} => 4702bd58135b78127c759a94658b7606e5445e46
2018/10/22 14:29:23 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:29:23 got corpus update after 195.853575ms
2018/10/22 14:29:24 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:29:24 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:29:24 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:29:24 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:29:24 gopherbot ran in 726.996791ms
2018/10/22 14:29:24 Updating data from log *maintner.netMutSource ...
2018/10/22 14:29:41 Downloading 37 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:29:41 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:29:41 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:29:41 got corpus update after 17.501948429s
2018/10/22 14:29:42 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:29:42 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:29:42 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:29:42 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:29:42 gopherbot ran in 723.348371ms
2018/10/22 14:29:42 Updating data from log *maintner.netMutSource ...

It's pretty harmless, but would be nice to fix.

The underlying issues causing this are:

• x/build/maintner: Gerrit CL and GitHub issue deletions are not reflected in model #22635–x/build/maintner: Gerrit CL deletions are not reflected in model
• x/build/maintner: GitHub issue becoming 404 (e.g., due to being deleted) is not reflected in model #30184–x/build/maintner: GitHub issue becoming 404 is not reflected in model
• x/build/maintner: reports incorrect Hashtags for some CLs #28318–x/build/maintner: reports incorrect Hashtags for some CLs

Fixing those issues should resolve this issue.

/cc @bradfitz @andybons

[gopherbot]

Change https://golang.org/cl/152779 mentions this issue: maintner: fix GerritMeta.Hashtags to look at earlier meta parents for answer

[dmitshur]

There's a new addition, as of #30182:

2019/02/12 01:45:58 got corpus update after 24.897577657s
freeze
	https://golang.org/issue/23772  x/build/cmd/gerritbot: <fill this in>
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/71730 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/71850 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72090 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72091 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72110 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72131 ...
2019/02/12 01:45:59 gopherbot ran in 1.894136882s

To fix the endless freezing of an issue that's 404, we'll need to update maintner so that when an issue goes 404, its NotExist field gets set to true. I've opened #30184 for tracking that.

[dmitshur]

The current state as of September 2019:

Details

2019/09/12 13:33:28 Downloading 68 bytes of https://maintner.golang.org/logs/56 ...
2019/09/12 13:33:28 wrote /cache/golang-maintner/0056.growing.mutlog
2019/09/12 13:33:28 Reloaded data from log *maintner.netMutSource.
2019/09/12 13:33:28 got corpus update after 2m24.51314801s
freeze
	https://golang.org/issue/23772  x/build/cmd/gerritbot: <fill this in>
	https://golang.org/issue/27223  x/mobile: 
2019/09/12 13:33:30 Issue 32620 already has label "Go2"; no need to send request to add it
2019/09/12 13:33:30 Issue 32620 already has label "LanguageChange"; no need to send request to add it
2019/09/12 13:33:30 Issue 33171 already has label "Go2"; no need to send request to add it
2019/09/12 13:33:31 No reviewers or cc: https://go-review.googlesource.com/c/website/+/192418
2019/09/12 13:33:31 Setting review on https://go-review.googlesource.com/c/website/+/192418: {Message: Labels:map[] Comments:map[] Reviewers:[{Reviewer:dmitshur@golang.org State:} {Reviewer:channing@golang.org State:CC} {Reviewer:365204+andybons@users.noreply.github.com State:CC}]}
2019/09/12 13:33:31 Could not set review for change "website~192418": HTTP status 400 Bad Request; )]}'
{
  "reviewers": {
    "365204+andybons@users.noreply.github.com": {
      "input": "365204+andybons@users.noreply.github.com",
      "error": "Account \u0027365204+andybons@users.noreply.github.com\u0027 not found\n365204+andybons@users.noreply.github.com does not identify a registered user or group"
    },
    "dmitshur@golang.org": {
      "input": "dmitshur@golang.org"
    },
    "channing@golang.org": {
      "input": "channing@golang.org"
    }
  },
  "error": "error adding reviewer"
}
2019/09/12 13:33:32 No reviewers or cc: https://go-review.googlesource.com/c/sys/+/178400
2019/09/12 13:33:32 Setting review on https://go-review.googlesource.com/c/sys/+/178400: {Message: Labels:map[] Comments:map[] Reviewers:[{Reviewer:iant@golang.org State:} {Reviewer:bradfitz@golang.org State:} {Reviewer:tklauser@distanz.ch State:}]}
2019/09/12 13:33:33 Could not set review for change "sys~178400": HTTP status 400 Bad Request; )]}'
{
  "reviewers": {
    "tklauser@distanz.ch": {
      "input": "tklauser@distanz.ch",
      "error": "Account \u0027tklauser@distanz.ch\u0027 not found\ntklauser@distanz.ch does not identify a registered user or group"
    },
    "iant@golang.org": {
      "input": "iant@golang.org"
    },
    "bradfitz@golang.org": {
      "input": "bradfitz@golang.org"
    }
  },
  "error": "error adding reviewer"
}
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/71730 ...
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/71850 ...
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/72090 ...
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/72091 ...
2019/09/12 13:33:34 closing scratch CL https://golang.org/cl/72110 ...
2019/09/12 13:33:35 closing scratch CL https://golang.org/cl/72131 ...
2019/09/12 13:33:36 gopherbot ran in 7.460417149s

[gopherbot]

Change https://golang.org/cl/195062 mentions this issue: internal/gophers: restore valid Gerrit emails (again)

[gopherbot]

Change https://golang.org/cl/176038 mentions this issue: cmd/gopherbot: update map of deleted CLs to include some from scratch

[gopherbot]

Change https://golang.org/cl/308790 mentions this issue: cmd/gopherbot: add more deleted issues to deletedIssues map