dgraph 1.0.12 rc5 crashes with panic

[makitka2007]

If you suspect this could be a bug, follow the template.

• What version of Dgraph are you using?
  1.0.12 rc5

• Have you tried reproducing the issue with latest release?
  yes

please read it from bottom to top:

|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|created by google.golang.org/grpc.(*Server).serveStreams.func1|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/google.golang.org/grpc/server.go:680 +0x9f|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc1993d5f90, 0xc000130380, 0x1595340, 0xc054d94e00, 0xc0fc793900)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/google.golang.org/grpc/server.go:1249 +0x1308|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|google.golang.org/grpc.(*Server).handleStream(0xc000130380, 0x1595340, 0xc054d94e00, 0xc0fc793900, 0x0)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/google.golang.org/grpc/server.go:1011 +0x4cd|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|google.golang.org/grpc.(*Server).processUnaryRPC(0xc000130380, 0x1595340, 0xc054d94e00, 0xc0fc793900, 0xc00047f1d0, 0x1ed05b8, 0x0, 0x0, 0x0)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgo/protos/api/api.pb.go:1749 +0x23e|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler(0x13c1c20, 0x1f9a2d0, 0x158fba0, 0xc1460dc960, 0xc168324e00, 0x0, 0x0, 0x0, 0xc0fc793900, 0x1)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgraph/edgraph/server.go:495 +0x4bf|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgraph/edgraph.(*Server).Query(0x1f9a2d0, 0x158fba0, 0xc1460dc9c0, 0xc16d46a410, 0xc0f1df2540, 0x0, 0x0)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgraph/query/query.go:2614 +0x5e|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgraph/query.(*QueryRequest).Process(0xc0149a5a30, 0x158fba0, 0xc1460dc9c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgraph/query/query.go:2520 +0x61d|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgraph/query.(*QueryRequest).ProcessQuery(0xc0149a5a30, 0x158fba0, 0xc1460dc9c0, 0x1, 0x1)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgraph/query/query.go:1557 +0x39|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgraph/query.(*SubGraph).recursiveFillVars(0xc0d5250840, 0xc1460dd9b0, 0xc0f1df3da0, 0x1)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgraph/query/query.go:1617 +0x249|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgraph/query.(*SubGraph).fillVars(0xc0d5250840, 0xc1460dd9b0, 0x1, 0xc0000e3b77)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgraph/query/query.go:1646 +0x3e9|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgraph/query.(*SubGraph).replaceVarInFunc(0xc0d5250840, 0xc1460dd9b0, 0xc0000e3b77)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|/ext-go/1/src/github.com/dgraph-io/dgraph/types/conversion.go:349 +0x108a|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|github.com/dgraph-io/dgraph/types.Marshal(0x0, 0x0, 0x0, 0xc0149a5320, 0xc0000e3b34, 0xa)|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|goroutine 28820 [running]:|
|Feb 05 22:58:01.826|i-0dd4493d3f656b465|dgraph-alpha|panic: interface conversion: interface {} is nil, not string|
|Feb 05 22:58:01.816|i-0dd4493d3f656b465|dgraph-alpha|Lastcommit 30161 > current 30074. This would cause some commits to be lost.|
|Feb 05 22:58:01.816|i-0dd4493d3f656b465|dgraph-alpha|Lastcommit 30161 > current 30074. This would cause some commits to be lost.|```

[danielmai]

Can you share the steps needed to result in this panic?

• Minimal sample data set
• Queries or mutations or other steps that lead to the issue.

[srfrog]

@makitka2007 how can we reproduce it? Thanks.

[makitka2007]

unfortunately i can't share steps to reproduce it, it's too complicated, we didn't run 1 mutation and got a panic. we were sending data to dgraph in ~5.000 triples mutation batches based on kafka messages + performed lookups. on 1.0.10 it worked. on 1.0.12-rc5 it crashed after few minutes of work.

may be you have some clue based on stack trace..

[srfrog]

No worries, I understand. Yes I will use the stack trace.

[srfrog]

@makitka2007 I have PR that I believe is a fix for the issue but I haven't found a query that will trigger the crash. Do you mind checking on your end if that branch fixes it?
#2995

[makitka2007]

thank you @srfrog. we have complex deployment process, so we can test only if there will be some tag on docker like rc6/whatever, so will wait for it to test

[srfrog]

Merged fix to master. Please reopen if you still having issues.