Divergent data on Alpha nodes when deleting predicates

[lukaszlenart]

What version of Dgraph are you using?

Started on 1.2.0, then tested with 1.2.3 and now using 20.03.1

Have you tried reproducing the issue with the latest release?

Yes

What is the hardware spec (RAM, OS)?

K8s cluster on AWS, 3xAlpha, 3xZero, 1 group

Steps to reproduce the issue (command/config used to run Dgraph).

I didn't try to duplicate this behaviour on some other data set, yet I'm planning to do so

Expected behaviour and actual result.

I'm running Dgraph 20.03.1 and struggling with divergent data on Alpha nodes, I was able narrow the problem to deleted predicates, I mean, when we perform a copy operation in our system, we set assetState to locked and at the end we remove the assetState edge. Yet I was able to filter out those affected nodes in Dgraph, using the same query on two different Alphas, here is the query:

{
  q(func: eq(staticId, "99822edc-febd-472a-8882-f78a0fe88092")) {
    uid
    parentOf @filter (not has(assetState)) {
      uid
      assetState
    }
  }
}

result on one Alpha is

{
  "data": {
    "q": [
      {
        "uid": "0x5",
        "parentOf": [
          {
            "uid": "0xc378"
          },
          {
            "uid": "0xc8b0"
          },
          {
            "uid": "0x50af6"
          },
          {
            "uid": "0x50b46"
          }
        ]
      }
    ]
  },
  "extensions": {
...

and on another

{
  "data": {
    "q": [
      {
        "uid": "0x5",
        "parentOf": [
          {
            "uid": "0xc378"
          },
          {
            "uid": "0xc8b0"
          },
          {
            "uid": "0xcbeb"
          },
          {
            "uid": "0xdad0"
          },
          {
            "uid": "0x50967"
          },
          {
            "uid": "0x5096a"
          },
          {
            "uid": "0x50af6"
          },
          {
            "uid": "0x50b46"
          },
          {
            "uid": "0x50b49"
          }
        ]
      }
    ]
  },
  "extensions": {
...

if we do not remove the assetState but just changing it to submitted, everything works fine.

[minhaj-shakeel]

Github issues have been deprecated.
This issue has been moved to discuss. You can follow the conversation there and also subscribe to updates by changing your notification preferences.