java.lang.NullPointerException while loading/running workload on MongoDB #73

Closed
nick0nyx opened this Issue Apr 18, 2012 · 11 comments

Projects

None yet

4 participants

@nick0nyx

Hello,
i'm having problems benchmarking MongoDB with YCSB.

This is what i've done:
1)installed MongoDB (2.0.4) and started bin/mongod (i can connect with shell to this instance)
2)git clone .../YCSB.git and then "mvn clean package". Build was successful.

Loading workload ends with this:
[root@serv1 YCSB]# bin/ycsb load mongodb -P workloads/workloada -p mongodb.url=mongodb://localhost:27017 -p mongodb.database=mytestdb -p debug=true -s > s1.txt (s1.txt content is here: http://pastebin.com/QNLLB1K4)
Loading workload...
Starting test.
0 sec: 0 operations;
java.lang.NullPointerException
0 sec: 0 operations; [INSERT AverageLatency(us)=28605]

Starting workload ends with this:
[root@serv1 YCSB]# bin/ycsb run mongodb -P workloads/workloada -p mongodb.url=mongodb://localhost:27017 -p mongodb.database=mytestdb -p debug=true -s > s2.txt (http://pastebin.com/7G0iHUQr)
Loading workload...
Starting test.
0 sec: 0 operations;
java.lang.NullPointerException
java.lang.NullPointerException
java.lang.NullPointerException
...
...
java.lang.NullPointerException
java.lang.NullPointerException
1 sec: 1000 operations; 759.3 current ops/sec; [UPDATE AverageLatency(us)=609.74] [READ AverageLatency(us)=1300.77]

I got benchmark running on cassandra, redis, hbase so far. I spent countless hours trying to figure out what do i do wrong with MongoDB.

@m1ch1
Collaborator
m1ch1 commented Apr 18, 2012

Hi nick0nyx,

Let me try it myself, and see if I can reproduce this issue.

Thanks!
--Michi

@nick0nyx

Hello Michi,
just wanted to add that YCSB connects to MongoDB instance (i can see it in MongoDB status) tho it disconnects right after.

nick

@m1ch1
Collaborator
m1ch1 commented Apr 18, 2012

Hi nick0nyx,

It looks like there is a required parameter called mongodb.writeConcern. It should be one of these 3 values:

-p mongodb.writeConcern=none
-p mongodb.writeConcern=strict
-p mongodb.writeConcern=normal

There are 3 problems here.

  1. It should be documented better.
  2. The error log messages should be clearer.
  3. I think it's reasonable to make this an optional parameter with a reasonable default (maybe "normal"?).

For now, you can pass this parameter to get going. I'll try to address these 3 issues when I find time.

Thanks!
--Michi

@nick0nyx

Hello Michi,
cheers for solving the connection problem. Opening MongoDBClient.java kind of opened my eyes now seeing it's written at the beginning, my bad.

However, now it connects but still doesn't import (load) any data:
[root@serv1 YCSB]# bin/ycsb load mongodb -P workloads/workloada -p mongodb.url=mongodb://localhost:27017 -p mongodb.database=mytestdb -p mongodb.writeConcern=normal -p debug=true -s > s1.txt
Loading workload...
Starting test.
0 sec: 0 operations;
0 sec: 0 operations; [INSERT AverageLatency(us)=52989]
[root@serv1 YCSB]#

workloada is set to 1000 records (default).
in s1.txt (output file) i can see:
new database url = localhost:27017/mytestdb
mongo connection created with localhost:27017/mytestdb
[OVERALL], RunTime(ms), 189.0
[OVERALL], Throughput(ops/sec), 0.0
[INSERT], Operations, 1
[INSERT], AverageLatency(us), 51544.0
[INSERT], MinLatency(us), 51544
[INSERT], MaxLatency(us), 51544
[INSERT], 95thPercentileLatency(ms), 51
[INSERT], 99thPercentileLatency(ms), 51
[INSERT], Return=1, 1
[INSERT], 0, 0 ==> up till 1000

Opening the DB in YCSB shell there's another strange behavior:

insert test4 test6=SomeValueToTryIt
Return code: 0
3 ms
read test4
Return code: 0
_id=test4
test6=[B@8c1dd9 -> looks like a pointer in memory not actual value
2 ms

@m1ch1 m1ch1 pushed a commit that referenced this issue Apr 19, 2012
Michi Mutsuzaki gh-73. added readme. added defaults for all the config parameters. e1de9c6
@m1ch1
Collaborator
m1ch1 commented Apr 19, 2012

Hmm strange. It's working for me. Could you try the steps described here and see if it works for you?

https://github.com/brianfrankcooper/YCSB/blob/master/mongodb/README.md

Thanks!
--Michi

@nick0nyx

I did exactly what the guide describes except of these:
downloaded i686 mongo package instead of x86_64 (because i'm vmwared linux is 32bit)
built YCSB with only those modules: core, distribution, mongodb

then i tried:
[root@serv1 YCSB]# ./bin/ycsb load mongodb -s -P workloads/workloada > s1.txt
Loading workload...
Starting test.
0 sec: 0 operations;
com.mongodb.MongoException$DuplicateKey: E11000 duplicate key error index: ycsb.usertable.$id dup key: { : "user6284781860667377211" }
at com.mongodb.CommandResult.getException(CommandResult.java:85)
at com.mongodb.CommandResult.throwOnError(CommandResult.java:121)
at com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:131)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:155)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:138)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:261)
at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:211)
at com.mongodb.DBCollection.insert(DBCollection.java:57)
at com.mongodb.DBCollection.insert(DBCollection.java:87)
at com.yahoo.ycsb.db.MongoDbClient.insert(MongoDbClient.java:153)
at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:145)
at com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461)
at com.yahoo.ycsb.ClientThread.run(Client.java:269)
0 sec: 0 operations; [INSERT AverageLatency(us)=36066]

and in mongod i can see:
Fri Apr 20 00:27:13 [initandlisten] connection accepted from 192.168.32.129:39783 #1
Fri Apr 20 00:27:13 [FileAllocator] allocating new datafile /tmp/mongodb/ycsb.ns, filling with zeroes...
Fri Apr 20 00:27:13 [FileAllocator] done allocating datafile /tmp/mongodb/ycsb.ns, size: 16MB, took 0 secs
Fri Apr 20 00:27:13 [FileAllocator] allocating new datafile /tmp/mongodb/ycsb.0, filling with zeroes...
Fri Apr 20 00:27:13 [FileAllocator] done allocating datafile /tmp/mongodb/ycsb.0, size: 16MB, took 0 secs
Fri Apr 20 00:27:13 [FileAllocator] allocating new datafile /tmp/mongodb/ycsb.1, filling with zeroes...
Fri Apr 20 00:27:13 [FileAllocator] done allocating datafile /tmp/mongodb/ycsb.1, size: 32MB, took 0 secs
Fri Apr 20 00:27:13 [conn1] building new index on { _id: 1 } for ycsb.usertable
Fri Apr 20 00:27:13 [conn1] done for 0 records 0secs
Fri Apr 20 00:27:14 [conn1] end connection 192.168.32.129:39783

database was empty from the begining (did wipe in /tmp/mongo before starting mongod)

nick

@m1ch1
Collaborator
m1ch1 commented Apr 23, 2012

Hi Nick,

Hmm strange. It does look like a record already existed in the database, but you mentioned you emptied the directory. Just to make sure, can you use another database and see if you still get the same error?

./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.database=newdb

Thanks!
--Michi

@nick0nyx

Hello, i solved all my issues with mongodb layer on sunday tho i didn't have time to write it here. You are right, --datadir was the issue in my case :)

Thank you very much for your time.

@nick0nyx nick0nyx closed this Apr 23, 2012
@m1ch1
Collaborator
m1ch1 commented Apr 23, 2012

Great to hear you solved the issue!

--Michi

@rushmila

hi,
I'm facing the same problem... can u please help me regarding this.

root@ubuntu:/home/rushmila/test-ycsb/ycsb-0.1.4# bin/ycsb load mongodb -P workloads/workloada -p mongodb.url=mongodb://localhost:27017 -p mongodb.database=test -p mongodb.writeConcern=normal -p debug=true -s > s1.txt

Loading workload...
Starting test.
0 sec: 0 operations;
0 sec: 0 operations; [INSERT AverageLatency(us)=309728]

@avgovind

I fixed this problem using solution by Michi to specify the parameter mongodb.writeConcern=normal. Thank you Michi.

@jaricftw jaricftw pushed a commit to jaricftw/YCSB that referenced this issue Jul 19, 2016
Michi Mutsuzaki gh-73. added readme. added defaults for all the config parameters. b282790
@jaricftw jaricftw pushed a commit to jaricftw/YCSB that referenced this issue Jul 19, 2016
Michi Mutsuzaki gh-73. added readme. added defaults for all the config parameters. 2565ed0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment