Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

graphql api: panic when trying to lookup contents of file matches returned by zoekt #2221

Closed
ggilmore opened this issue Feb 8, 2019 · 7 comments
Assignees
Labels
api Sourcegraph GraphQL API bug An error, flaw or fault that produces an incorrect or unexpected result, or behavior.
Milestone

Comments

@ggilmore
Copy link
Contributor

ggilmore commented Feb 8, 2019

Steps to reproduce:

  1. Start the dev server with indexed search enabled:

    INDEXED_SEARCH=1 ./enterprise/dev/start.sh
    
  2. Add http://github.com/gorilla/mux to your instance

  3. Go the GraphQL API console (http://localhost:3080/api/console#%7B%22variables%22%3A%22%7B%5Cn%20%20%5C%22query%5C%22%3A%20%5C%22repo%3Agorilla%2Fmux%20travis%20count%3A1%5C%22%5Cn%7D%22%7D) and run the following:

    query Search($query: String) {
      search(query: $query) {
        results {
          results {
            ... on FileMatch {
              file {
                commit {
                  oid
                }
                path
                content
              }
            }
          }
        }
      }
    }
    
    # Query Variables
    {
      "query": "repo:gorilla/mux travis count:1"
    }

Expected behavior:

The results succesfully return with the requested file contents:

{
  "data": {
    "search": {
      "results": {
        "results": [
          {
            "file": {
              "commit": {
                "oid": "a7962380ca08b5a188038c69871b8d3fbdf31e89"
              },
              "path": ".travis.yml",
              "content": "language: go\nsudo: false\n\nmatrix:\n  include:\n    - go: 1.7.x\n    - go: 1.8.x\n    - go: 1.9.x\n    - go: 1.10.x\n    - go: 1.11.x\n    - go: 1.x\n      env: LATEST=true\n    - go: tip\n  allow_failures:\n    - go: tip\n\ninstall:\n  - # Skip\n\nscript:\n  - go get -t -v ./...\n  - diff -u <(echo -n) <(gofmt -d .)\n  - if [[ \"$LATEST\" = true ]]; then go tool vet .; fi\n  - go test -v -race ./...\n"
            }
          }
        ]
      }
    }
  }
}

Actual behavior:

The results have null contents and contain errors:

{
  "data": {
    "search": {
      "results": {
        "results": [
          {
            "file": {
              "commit": {
                "oid": ""
              },
              "path": ".travis.yml",
              "content": null
            }
          }
        ]
      }
    }
  },
  "errors": [
    {
      "message": "graphql: panic occurred: non-absolute commit ID: \"\"",
      "path": [
        "search",
        "results",
        "results",
        0,
        "file",
        "content"
      ]
    }
  ]
}

Also see these logs:

13:50:57               frontend | graphql: panic occurred: non-absolute commit ID: ""
13:50:57               frontend | goroutine 28043 [running]:
13:50:57               frontend | github.com/graph-gophers/graphql-go/log.(*DefaultLogger).LogPanic(0x30083f0, 0x23bd740, 0xc0006220f0, 0x1f79ce0, 0xc00080aa70)
13:50:57               frontend | 	/Users/ggilmore/dev/go/pkg/mod/github.com/sourcegraph/graphql-go@v0.0.0-20180929065141-c790ffc3c46a/log/log.go:21 +0x77
13:50:57               frontend | github.com/graph-gophers/graphql-go/internal/exec.execFieldSelection.func2.1(0xc00062a240, 0x23bd740, 0xc0006220f0, 0xc00121def8, 0xc000485760)
13:50:57               frontend | 	/Users/ggilmore/dev/go/pkg/mod/github.com/sourcegraph/graphql-go@v0.0.0-20180929065141-c790ffc3c46a/internal/exec/exec.go:161 +0x87
13:50:57               frontend | panic(0x1f79ce0, 0xc00080aa70)
13:50:57               frontend | 	/usr/local/Cellar/go/1.11.5/libexec/src/runtime/panic.go:513 +0x1b9
13:50:57               frontend | github.com/sourcegraph/sourcegraph/pkg/vcs/git.ensureAbsCommit(0x0, 0x0)
13:50:57               frontend | 	/Users/ggilmore/dev/go/src/github.com/sourcegraph/sourcegraph/pkg/vcs/git/revisions.go:38 +0xed
13:50:57               frontend | github.com/sourcegraph/sourcegraph/pkg/vcs/git.readFileBytes(0x23bd740, 0xc0006435f0, 0xc000654620, 0x16, 0x0, 0x0, 0x0, 0x0, 0xc000c9ab50, 0xd, ...)
13:50:57               frontend | 	/Users/ggilmore/dev/go/src/github.com/sourcegraph/sourcegraph/pkg/vcs/git/blob.go:35 +0x50
13:50:57               frontend | github.com/sourcegraph/sourcegraph/pkg/vcs/git.ReadFile(0x23bd740, 0xc0006435f0, 0xc000654620, 0x16, 0x0, 0x0, 0x0, 0x0, 0xc000c9ab50, 0xd, ...)
13:50:57               frontend | 	/Users/ggilmore/dev/go/src/github.com/sourcegraph/sourcegraph/pkg/vcs/git/blob.go:27 +0x24d
13:50:57               frontend | github.com/sourcegraph/sourcegraph/cmd/frontend/graphqlbackend.(*gitTreeEntryResolver).Content(0xc000622240, 0x23bd700, 0xc0006009c0, 0x0, 0x0, 0x0, 0x0)
13:50:57               frontend | 	/Users/ggilmore/dev/go/src/github.com/sourcegraph/sourcegraph/cmd/frontend/graphqlbackend/file.go:26 +0x158
13:50:57               frontend | reflect.Value.call(0x2182600, 0xc0006114a8, 0x1293, 0x21b6de0, 0x4, 0xc000485800, 0x1, 0x1, 0x2182600, 0x17, ...)
13:50:57               frontend | 	/usr/local/Cellar/go/1.11.5/libexec/src/reflect/value.go:447 +0x454
13:50:57               frontend | reflect.Value.Call(0x2182600, 0xc0006114a8, 0x1293, 0xc000485800, 0x1, 0x1, 0x1293, 0x1, 0xbe073701fa)
13:50:57               frontend | 	/usr/local/Cellar/go/1.11.5/libexec/src/reflect/value.go:308 +0xa4
13:50:57               frontend | github.com/graph-gophers/graphql-go/internal/exec.execFieldSelection.func2(0xc00062a240, 0x23bd740, 0xc0006220f0, 0xc000485760, 0xc0016bfbc0, 0xc00121df40, 0x23bd740, 0xc0006222a0, 0x0)
13:50:57               frontend | 	/Users/ggilmore/dev/go/pkg/mod/github.com/sourcegraph/graphql-go@v0.0.0-20180929065141-c790ffc3c46a/internal/exec/exec.go:183 +0x3b9
13:50:57               frontend | github.com/graph-gophers/graphql-go/internal/exec.execFieldSelection(0x23bd740, 0xc0006220f0, 0xc00062a240, 0xc0016bfbc0, 0xc000485760, 0xc000642801)
13:50:57               frontend | 	/Users/ggilmore/dev/go/pkg/mod/github.com/sourcegraph/graphql-go@v0.0.0-20180929065141-c790ffc3c46a/internal/exec/exec.go:193 +0x1aa
13:50:57               frontend | github.com/graph-gophers/graphql-go/internal/exec.(*Request).execSelections.func1(0xc000c9acd0, 0xc00062a240, 0x23bd740, 0xc0006220f0, 0xc000485640, 0xc0016bfbc0)
13:50:57               frontend | 	/Users/ggilmore/dev/go/pkg/mod/github.com/sourcegraph/graphql-go@v0.0.0-20180929065141-c790ffc3c46a/internal/exec/exec.go:74 +0x161
13:50:57               frontend | created by github.com/graph-gophers/graphql-go/internal/exec.(*Request).execSelections
13:50:57               frontend | 	/Users/ggilmore/dev/go/pkg/mod/github.com/sourcegraph/graphql-go@v0.0.0-20180929065141-c790ffc3c46a/internal/exec/exec.go:70 +0x478

The GraphQL results sucessfully return if you disable indexed search by adding index:no to your search query:

Query:

query Search($query: String) {
  search(query: $query) {
    results {
      results {
        ... on FileMatch {
          file {
            commit {
              oid
            }
            path
            content
          }
        }
      }
    }
  }
}

# Query Variables
{
  "query": "repo:gorilla/mux index:no travis count:1"
}

Results:

{
  "data": {
    "search": {
      "results": {
        "results": [
          {
            "file": {
              "commit": {
                "oid": "a7962380ca08b5a188038c69871b8d3fbdf31e89"
              },
              "path": ".travis.yml",
              "content": "language: go\nsudo: false\n\nmatrix:\n  include:\n    - go: 1.7.x\n    - go: 1.8.x\n    - go: 1.9.x\n    - go: 1.10.x\n    - go: 1.11.x\n    - go: 1.x\n      env: LATEST=true\n    - go: tip\n  allow_failures:\n    - go: tip\n\ninstall:\n  - # Skip\n\nscript:\n  - go get -t -v ./...\n  - diff -u <(echo -n) <(gofmt -d .)\n  - if [[ \"$LATEST\" = true ]]; then go tool vet .; fi\n  - go test -v -race ./...\n"
            }
          }
        ]
      }
    }
  }
}

Specifiying a branch in your query e.g. r:gorilla/mux@master also works.

It seems like the indexed-search code path specifically provides an empty commit field to the fileMatchResolver:

commitID: "", // zoekt only searches default branch

However, this is an issue for the fileMatchResolver's Content() method:

contents, err := git.ReadFile(ctx, *cachedRepo, api.CommitID(r.commit.oid), r.path)

Which panics because the commit that's passed to its methods isn't an absolute one:

ensureAbsCommit(commit)

@ggilmore ggilmore added bug An error, flaw or fault that produces an incorrect or unexpected result, or behavior. search api Sourcegraph GraphQL API labels Feb 8, 2019
@ggilmore
Copy link
Contributor Author

ggilmore commented Feb 8, 2019

cc @slimsag. He mentioned that setting the correct revision is non-trivial since it runs the risk of harming search performance.

@ggilmore ggilmore changed the title graphql api: panic when trying to lookup contents of files returned by zoekt graphql api: panic when trying to lookup contents of file matches returned by zoekt Feb 8, 2019
@slimsag
Copy link
Member

slimsag commented Feb 8, 2019

I mentioned this over Slack but:

  1. The bug is definitely here:

commitID: "", // zoekt only searches default branch

  1. The reason we do this is intentional I believe: looking up the default branch of a repository is expensive and if we did it here just to resolve the default branch, it would hurt search perf and place additional load on gitserver when plain search doesn't need the commitID field. (we also do this in search suggestions in multiple locations unfortunately)

  2. The correct fix would be ensuring that r.commit is initialized when it is accessed, i.e. only if the graphql caller asks for it. One way to do this would be refactoring fileMatchResolver.commit to be a sync.Once function which returns the commit ID or something.

@ggilmore ggilmore added this to the 3.1 milestone Feb 8, 2019
@nicksnyder
Copy link
Contributor

@ijsnow @attfarhan Can you decide if this fits in the 3.1 milestone given the other work assigned to you?

@ggilmore
Copy link
Contributor Author

ggilmore commented Feb 8, 2019

@slimsag

Would the following query work in your proposed solution (caller doesn't specifically ask for the commit)?

query Search($query: String) {
  search(query: $query) {
    results {
      results {
        ... on FileMatch {
          file {
            path
            content
          }
        }
      }
    }
  }
}

# Query variables
{
  "query": "repo:gorilla/mux travis count:1"
}

Results:

{
  "data": {
    "search": {
      "results": {
        "results": [
          {
            "file": {
              "path": ".travis.yml",
              "content": "language: go\nsudo: false\n\nmatrix:\n  include:\n    - go: 1.7.x\n    - go: 1.8.x\n    - go: 1.9.x\n    - go: 1.10.x\n    - go: 1.11.x\n    - go: 1.x\n      env: LATEST=true\n    - go: tip\n  allow_failures:\n    - go: tip\n\ninstall:\n  - # Skip\n\nscript:\n  - go get -t -v ./...\n  - diff -u <(echo -n) <(gofmt -d .)\n  - if [[ \"$LATEST\" = true ]]; then go tool vet .; fi\n  - go test -v -race ./...\n"
            }
          }
        ]
      }
    }
  }
}

@slimsag
Copy link
Member

slimsag commented Feb 8, 2019

Yes. As you mentioned in the description, the content field makes use of commit:

contents, err := git.ReadFile(ctx, *cachedRepo, api.CommitID(r.commit.oid), r.path)

Effectively, that would become a call to r.commit().oid and, assuming it was the first call, it would be resolved on-the-fly.

@attfarhan
Copy link
Contributor

@ijsnow I moved this to 3.2 since that's when it will ship

@attfarhan attfarhan assigned ijsnow and unassigned ijsnow and attfarhan Feb 19, 2019
@felixfbecker
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Sourcegraph GraphQL API bug An error, flaw or fault that produces an incorrect or unexpected result, or behavior.
Projects
None yet
Development

No branches or pull requests

6 participants