Skip to content

Proto: Illegal wiretype 6 crash on alpha #5789

@jarifibrahim

Description

@jarifibrahim

What version of Dgraph are you using?

60b1cd0

Have you tried reproducing the issue with the latest release?

No, but it should be reproducible.

Steps to reproduce the issue (command/config used to run Dgraph).

I was using live loader to insert 7 billion rdfs (95G file). The rdfs are generated by copy-pasting the 21million.rdf.gz file multiple times.
Hers's what I did

  1. Start zero dgraph zero
  2. Start Alpha dgraph alpha
  3. Live load the file dgraph live -f ...

Expected behavior and actual result.

The live loader ran fine for about 840 million rdfs (which took around 15 hours) and then alpha crashed with the following error

I0701 16:45:44.954318   23491 log.go:34] LOG Compact 2->3, del 35 tables, add 35 tables, took 22.954189288s
I0701 16:45:44.954355   23491 log.go:34] Compaction for level: 2 DONE
[Sentry] 2020/07/01 16:46:27 Sending fatal event [419c114b94bd472392f3fb1b49f101be] to o318308.ingest.sentry.io project: 5208688
2020/07/01 16:46:27 proto: illegal wireType 6

github.com/dgraph-io/dgraph/x.Check
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/x/error.go:42
github.com/dgraph-io/dgraph/posting.ReadPostingList.func2
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/posting/mvcc.go:320
github.com/dgraph-io/badger/v2.(*Item).Value
        /home/ibrahim/Projects/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200609141616-14386ac9b764/iterator.go:111
github.com/dgraph-io/dgraph/posting.ReadPostingList
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/posting/mvcc.go:318
github.com/dgraph-io/dgraph/posting.getNew
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/posting/mvcc.go:364
github.com/dgraph-io/dgraph/posting.(*LocalCache).getInternal
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/posting/lists.go:219
github.com/dgraph-io/dgraph/posting.(*LocalCache).Get
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/posting/lists.go:242
github.com/dgraph-io/dgraph/posting.(*Txn).Get
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/posting/oracle.go:77
github.com/dgraph-io/dgraph/worker.runMutation
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/worker/mutation.go:108
github.com/dgraph-io/dgraph/worker.(*node).applyMutations.func3
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/worker/draft.go:453
github.com/dgraph-io/dgraph/worker.(*node).applyMutations.func4
        /home/ibrahim/Projects/go/src/github.com/dgraph-io/dgraph/worker/draft.go:482
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1357

Here are the alpha and zero logs (the files are huge)
alpha.log
zero.log

I have the directory on which this issue is reproducible (it always crashes) but it's 125G directory and I'm trying to figure out how to share it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/crashDgraph issues that cause an operation to fail, or the whole server to crash.kind/bugSomething is broken.status/acceptedWe accept to investigate/work on it.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions