Exception in thread "main" java.lang.NoSuchMethodError:

[FuchengDum]

Hello, there is issue when I run benchmark. And I follow your example, It failed with : Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;

dums@machine:~/jNVMf$ java -cp target/jnvmf-1.6-jar-with-dependencies.jar:target/jnvmf-1.6-tests.jar -Djnvmf.legacy=true com.ibm.jnvmf.benchmark.NvmfClientBenchmark -a 192.168.215.143 -p 4420 -g 4096 -i 3 -m RANDOM -n 10 -nqn nqn.2014-08.org.nvmexpress:uuid:8048289a-acf2-4b84-b113-e71024f24740 -qd 1 -rw read -s 4096 -qs 64 -H -I

---

read 4096bytes with QD = 1, time[s] = 3, pattern = RANDOM, runs = 10
log4j:WARN No appenders could be found for logger (com.ibm.disni).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at com.ibm.disni.util.NetUtils.getIntIPFromInetAddress(NetUtils.java:51)
at com.ibm.disni.verbs.impl.RdmaCmNat.resolveAddr(RdmaCmNat.java:157)
at com.ibm.disni.verbs.RdmaCmId.resolveAddr(RdmaCmId.java:143)
at com.ibm.disni.RdmaEndpoint.connect(RdmaEndpoint.java:100)
at com.ibm.jnvmf.NvmfRdmaEndpoint.connect(NvmfRdmaEndpoint.java:59)
at com.ibm.jnvmf.QueuePair.connect(QueuePair.java:79)
at com.ibm.jnvmf.AdminQueuePair.connect(AdminQueuePair.java:36)
at com.ibm.jnvmf.QueuePair.(QueuePair.java:195)
at com.ibm.jnvmf.QueuePair.(QueuePair.java:134)
at com.ibm.jnvmf.AdminQueuePair.(AdminQueuePair.java:31)
at com.ibm.jnvmf.Controller.(Controller.java:66)
at com.ibm.jnvmf.Nvme.connect(Nvme.java:50)
at com.ibm.jnvmf.Nvme.connect(Nvme.java:44)
at com.ibm.jnvmf.benchmark.NvmfClientBenchmark.connect(NvmfClientBenchmark.java:216)
at com.ibm.jnvmf.benchmark.NvmfClientBenchmark.(NvmfClientBenchmark.java:203)
at com.ibm.jnvmf.benchmark.NvmfClientBenchmark.main(NvmfClientBenchmark.java:551)

Is there some problems about classpath? Could you please specify me detail?

Thanks

[FuchengDum]

hello. I follow the step in the issue [https://github.com//issues/14]. And I create a new subsystems with valid NQN. However, It occurs the same problem:

dums@machine:/jNVMf$ echo $LD_LIBRARY_PATH
:/usr/local/lib
dums@machine:/jNVMf$ java -cp target/jnvmf-1.6-jar-with-dependencies.jar:target/jnvmf-1.6-tests.jar -Djnvmf.legacy=true com.ibm.jnvmf.benchmark.NvmfClientBenchmark -a 192.168.215.144 -p 4420 -g 4096 -i 3 -m RANDOM -n 10 -nqn nqn.2019-08.com.dumnet:test -qd 1 -rw read -s 4096 -qs 64 -H -I
read 4096bytes with QD = 1, time[s] = 3, pattern = RANDOM, runs = 10
log4j:WARN No appenders could be found for logger (com.ibm.disni).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at com.ibm.disni.util.NetUtils.getIntIPFromInetAddress(NetUtils.java:51)
at com.ibm.disni.verbs.impl.RdmaCmNat.resolveAddr(RdmaCmNat.java:157)
at com.ibm.disni.verbs.RdmaCmId.resolveAddr(RdmaCmId.java:143)
at com.ibm.disni.RdmaEndpoint.connect(RdmaEndpoint.java:100)
at com.ibm.jnvmf.NvmfRdmaEndpoint.connect(NvmfRdmaEndpoint.java:59)
at com.ibm.jnvmf.QueuePair.connect(QueuePair.java:79)
at com.ibm.jnvmf.AdminQueuePair.connect(AdminQueuePair.java:36)
at com.ibm.jnvmf.QueuePair.(QueuePair.java:195)
at com.ibm.jnvmf.QueuePair.(QueuePair.java:134)
at com.ibm.jnvmf.AdminQueuePair.(AdminQueuePair.java:31)
at com.ibm.jnvmf.Controller.(Controller.java:66)
at com.ibm.jnvmf.Nvme.connect(Nvme.java:50)
at com.ibm.jnvmf.Nvme.connect(Nvme.java:44)
at com.ibm.jnvmf.benchmark.NvmfClientBenchmark.connect(NvmfClientBenchmark.java:216)
at com.ibm.jnvmf.benchmark.NvmfClientBenchmark.(NvmfClientBenchmark.java:203)
at com.ibm.jnvmf.benchmark.NvmfClientBenchmark.main(NvmfClientBenchmark.java:551)

---

Here is the output of ip addr or ifconfig. And the output of find /sys/kernel/config/nvmet/subsystems
dums@machine:~$ sudo ifconfig
[sudo] password for dums:
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.215.144 netmask 255.255.255.0 broadcast 192.168.215.255
inet6 fe80::8afd:a181:2e0a:47d1 prefixlen 64 scopeid 0x20
ether 00:0c:29:b4:8a:90 txqueuelen 1000 (Ethernet)
RX packets 452 bytes 333346 (333.3 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 242 bytes 46145 (46.1 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

dums@machine:~$ find /sys/kernel/config/nvmet/subsystems
/sys/kernel/config/nvmet/subsystems
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/allowed_hosts
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces/1
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces/1/buffered_io
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces/1/enable
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces/1/ana_grpid
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces/1/device_uuid
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces/1/device_nguid
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/namespaces/1/device_path
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/attr_serial
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/attr_version
/sys/kernel/config/nvmet/subsystems/nqn.2019-08.com.dumnet:test/attr_allow_any_host

o- / .................................................................... [...]
o- hosts .............................................................. [...]
o- ports .............................................................. [...]
| o- 1 [trtype=rdma, traddr=192.168.215.144, trsvcid=4420, inline_data_size=4096]
| o- ana_groups ..................................................... [...]
| | o- 1 ................................................ [state=optimized]
| o- referrals ...................................................... [...]
| o- subsystems ..................................................... [...]
| o- nqn.2019-08.com.dumnet:test .................................. [...]
o- subsystems ......................................................... [...]
o- nqn.2019-08.com.dumnet:test [version=1.3, allow_any=1, serial=c4302c4712965973]
o- allowed_hosts .................................................. [...]
o- namespaces ..................................................... [...]
o- 1 [path=/dev/nvme0n1, uuid=318b5c49-69c5-4288-b6bf-1de5f5248648, grpid=1, enabled]

Thanks

[PepperJo]

It looks like you are trying to run jNVMf with Java 9. The interface of ByteBuffer.clear() has changed that is why it cannot find the method. Try using a Java 8 JRE/JDK.

[FuchengDum]

Thanks for your quick respond. The version is Java 8.
After rebuild the jNVMf , I figure it out. However, I met new question.
I run the test, it looks like hang there for long time, no exception error any more. no console print too. Just like the problem of #10 . But the target IP address is right. Can you help me ?

java -cp target/jnvmf-1.6-jar-with-dependencies.jar:target/jnvmf-1.6-tests.jar -Djnvmf.legacy=true com.ibm.jnvmf.benchmark.NvmfClientBenchmark -a 192.168.215.144 -p 4420 -g 4096 -i 3 -m RANDOM -n 10 -nqn nqn.2019-08.com.dumnet:test -qd 1 -rw read -s 4096 -qs 64 -H -I

read 4096bytes with QD = 1, time[s] = 3, pattern = RANDOM, runs = 10
log4j:WARN No appenders could be found for logger (com.ibm.disni).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

[PepperJo]

It looks like it is unable to connect to the ip/port provided. Can you provide a stacktrace via jstack you can get the pid of the process with jps.

[PepperJo]

Also what RDMA device are you using? You can check with ibv_devices. I strongly advise against the use of rxe/SoftRoCE since it has known issues of dropping completions or double creation (see other issue).

[FuchengDum]

Hello , here is a stacktrace and RDMA device.
dums@machine:~/jNVMf$ jps
4225 NvmfClientBenchmark
4102 NvmfClientBenchmark
4072 NvmfClientBenchmark
4456 NvmfClientBenchmark
4168 NvmfClientBenchmark
4844 Jps
4204 NvmfClientBenchmark
4524 NvmfClientBenchmark
4142 NvmfClientBenchmark
3895 NvmfClientBenchmark
3864 NvmfClientBenchmark
4186 NvmfClientBenchmark
4122 NvmfClientBenchmark
4543 NvmfClientBenchmark

dums@machine:~/jNVMf$ jstack 4204
4204: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding

dums@machine:~/jNVMf$ ibv_devices
device node GUID
------ ----------------
rxe0 020c29fffeb48a90

[PepperJo]

It looks like there are a lot of orphaned processes try killing all those left over NVMfClientBenchmarks. There should only be one when you run the benchmark.
Again, I strongly advice against using rxe since it is not stable (The problem can easily be reproduced with e.g. rdma_send_bw). If you need a software RDMA device I recommend installing the latest 5.3 kernel RC which includes SoftiWARP.

[FuchengDum]

Thank you so much. I finally figure it out by rebuilding disni and jnvmf.

[PepperJo]

No problem.