Live Loader: Issue with dgraph's default addr

[MichelDiz]

What version of Dgraph are you using?

1.1.0 (ibrahim:badger-update - but it's not related to that branch)

Have you tried reproducing the issue with the latest release?

Yes

What is the hardware spec (RAM, OS)?

Tested on Windows and Darwin

Steps to reproduce the issue (command/config used to run Dgraph).

Start live load with defaults settings. Or set the addr (flag -a)

Usage:
  dgraph live [flags]

Flags:
  -a, --alpha string   Comma-separated list of Dgraph alpha gRPC server 
addresses (default "127.0.0.1:9080")

This issue also happened in Dgo. As Live Loader uses Dgo, I think Dgo should be checked too.

The error logs

Running transaction with dgraph endpoint: 127.0.0.1:9080
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...

[ashish-goswami]

Caused by #4020.
Closing this.

[MichelDiz]

This still happening on master - I've build it again in Darwin and tested.

➜  dgraph git:(master) git rev-parse HEAD
a5dbfc80e029c096c29024d2c0b2497fff265ffd
➜  dgraph git:(master) git describe
v1.1.0-45-ga5dbfc80

git commit => a5dbfc8

➜  master(18-09-19) dgraph live -f 21million.rdf.gz -s release.schema.txt                  
[Decoder]: Using assembly version of decoder
I0918 14:24:07.178355   18010 init.go:98] 

Dgraph version   : 
Dgraph SHA-256   : 22cac961c0e975b29f6f41f1255cf769eba0cca67879d8788dbcea7a929e0898
Commit SHA-1     : 
Commit timestamp : 
Branch           : 
Go version       : go1.12.9

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.



Running transaction with dgraph endpoint: 127.0.0.1:9080
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
^C

[MichelDiz]

I'll leave it opened to keep tracking it. And test it again after #4020 is solved.

[MichelDiz]

I did more tests around this

Same error with:
v1.1.0
don't work ➜ dgraph-darwin-amd64 ./dgraph live -f rdf.rdf -s schema.schema
works ➜ dgraph-darwin-amd64 ./dgraph live -f rdf.rdf -s schema.schema -a localhost:9080

v1.0.15
don't work ➜ Dgraph v1.0.15 ./dgraph live -r 21million.rdf.gz -s release.schema.txt
works ➜ ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080

v1.0.13
don't work ➜ Dgraph v1.0.13 ./dgraph live -r 21million.rdf.gz -s release.schema.txt
works ➜ ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080

v1.0.6
don't work ➜ v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema
works ➜ v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080

I do not understand why this happens, because 127.0.0.1 is Localhost on any operating system.

All the Logs

➜  dgraph-darwin-amd64 ./dgraph live -f rdf.rdf -s schema.schema          
[Decoder]: Using assembly version of decoder
I0920 12:56:58.569629   49642 init.go:98] 

Dgraph version   : v1.1.0
Dgraph SHA-256   : 98db2956f6dd8b7b9b88e02962d2036845b057fe5fe953190eaafac0a83dfcce
Commit SHA-1     : ef7cdb28
Commit timestamp : 2019-09-04 00:12:51 -0700
Branch           : HEAD
Go version       : go1.12.7

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.



Running transaction with dgraph endpoint: 127.0.0.1:9080
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
^C

Dgraph v1.0.15 ./dgraph live -r 21million.rdf.gz -s release.schema.txt                  
I0920 13:04:56.625526   49918 init.go:88] 

Dgraph version   : v1.0.15
Commit SHA-1     : ff5ee1e2
Commit timestamp : 2019-05-30 15:46:55 -0700
Branch           : HEAD
Go version       : go1.12.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x133341549
badger 2019/09/20 13:04:56 INFO: All 0 tables opened in 0s

Processing release.schema
Error while processing schema file "release.schema": rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>
badger 2019/09/20 13:04:56 INFO: Got compaction priority: {level:0 score:1.73 dropPrefix:[]}

./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080
I0920 13:05:38.323393   49940 init.go:88] 

Dgraph version   : v1.0.15
Commit SHA-1     : ff5ee1e2
Commit timestamp : 2019-05-30 15:46:55 -0700
Branch           : HEAD
Go version       : go1.12.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x573696955
badger 2019/09/20 13:05:38 INFO: All 0 tables opened in 0s

Processing release.schema.txt
Processed schema file "release.schema.txt"

Processing 21million.rdf.gz
[    2s] Txns: 103 RDFs: 103000 RDFs/sec: 49882 Aborts: 1
[    4s] Txns: 204 RDFs: 204000 RDFs/sec: 50149 Aborts: 2
[    6s] Txns: 284 RDFs: 284000 RDFs/sec: 46803 Aborts: 3

Dgraph v1.0.13 ./dgraph live -r 21million.rdf.gz -s release.schema.txt
I0920 13:08:30.621463   50037 init.go:88] 

Dgraph version   : v1.0.13
Commit SHA-1     : 691b3b35
Commit timestamp : 2019-03-09 19:33:59 -0800
Branch           : HEAD
Go version       : go1.11.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x764103728
badger 2019/09/20 13:08:30 INFO: All 0 tables opened in 0s

Processing release.schema
Error while processing schema file "release.schema": rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>
badger 2019/09/20 13:08:30 WARNING: While forcing compaction on level 0: Unable to fill tables
E0920 13:08:30.626790   50037 xidmap.go:120] Error while getting lease: rpc error: code = Unavailable desc = transport is closing

./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080
I0920 13:08:59.961881   50046 init.go:88] 

Dgraph version   : v1.0.13
Commit SHA-1     : 691b3b35
Commit timestamp : 2019-03-09 19:33:59 -0800
Branch           : HEAD
Go version       : go1.11.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x959542565
badger 2019/09/20 13:08:59 INFO: All 0 tables opened in 0s

Processing release.schema.txt
Processed schema file "release.schema.txt"

Processing 21million.rdf.gz
[    2s] Txns: 56 RDFs: 56000 RDFs/sec: 27866 Aborts: 0
[    4s] Txns: 125 RDFs: 125000 RDFs/sec: 31179 Aborts: 1
[    6s] Txns: 175 RDFs: 175000 RDFs/sec: 29123 Aborts: 2
[    8s] Txns: 216 RDFs: 216000 RDFs/sec: 26967 Aborts: 2
[   10s] Txns: 258 RDFs: 258000 RDFs/sec: 25776 Aborts: 3

➜  v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema                      
2019/09/20 13:14:43 run.go:337: Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x141447596

Processing release.schema
2019/09/20 13:14:43 rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>

➜  v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080    
2019/09/20 13:18:02 run.go:337: Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x524737574

Processing release.schema.txt
2019/09/20 13:18:02 run.go:354: Processed schema file

Processing 21million.rdf.gz
Total Txns done:       72 RDFs per second:   35360 Time Elapsed: 2s, Aborts: 25
Total Txns done:      131 RDFs per second:   32457 Time Elapsed: 4s, Aborts: 42
Total Txns done:      182 RDFs per second:   30152 Time Elapsed: 6s, Aborts: 66
Total Txns done:      237 RDFs per second:   29492 Time Elapsed: 8s, Aborts: 84

➜  v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d 127.0.0.1:9080 
2019/09/20 13:18:30 run.go:337: Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x764043649

Processing release.schema.txt
2019/09/20 13:18:30 rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>

[danielmai]

Seems like a machine config issue, @MichelDiz. I tried the same commands you shared on Mac and they work with the default --alpha flag value 127.0.0.1:9080.

with Dgraph v1.1.0:

dgraph live -f 21million.rdf.gz -s release.schema.txt

[MichelDiz]

I’ve created a VM with Windows 10 and tested it there. It works fine.

So it is indeed something else very odd. I can't figure out what is it. So in case, this issue happens to other users, please set the addr as localhost.