Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in adding entries to Bookie #277

Closed
krumo opened this issue Mar 6, 2017 · 5 comments
Closed

Error in adding entries to Bookie #277

krumo opened this issue Mar 6, 2017 · 5 comments

Comments

@krumo
Copy link
Contributor

krumo commented Mar 6, 2017 •
edited
Loading

Expected behavior

The bookie could be started correctly and pass the bookiesanity test.

Actual behavior

When I start the bookie in my cluster, everything seems Okay. However, when I call bookkeeper shell bookiesanity to add some entries to my bookie for testing whether the bookie run correctly, I came across the following error in my log file:

2017-03-06 14:58:40,857 - INFO  [bookie-io-15-3:AuthHandler$ServerSideHandler@59] - Channel open [id: 0x08e77371, L:/172.18.11.160:3183 - R:/172.18.11.160:54690]
2017-03-06 14:58:40,862 - ERROR [bookie-io-15-3:BookieRequestHandler@69] - Unhandled exception occurred in I/O thread or handler
java.lang.NoSuchMethodError: io.netty.util.internal.MathUtil.safeFindNextPositivePowerOfTwo(I)I
        at io.netty.buffer.PoolThreadCache$MemoryRegionCache.<init>(PoolThreadCache.java:372)
        at io.netty.buffer.PoolThreadCache$SubPageMemoryRegionCache.<init>(PoolThreadCache.java:340)
        at io.netty.buffer.PoolThreadCache.createSubPageCaches(PoolThreadCache.java:135)
        at io.netty.buffer.PoolThreadCache.<init>(PoolThreadCache.java:86)
        at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.initialValue(PooledByteBufAllocator.java:361)
        at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.initialValue(PooledByteBufAllocator.java:354)
        at io.netty.util.concurrent.FastThreadLocal.initialize(FastThreadLocal.java:155)
        at io.netty.util.concurrent.FastThreadLocal.get(FastThreadLocal.java:149)
        at io.netty.util.concurrent.FastThreadLocal.get(FastThreadLocal.java:135)
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:266)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:177)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:168)
        at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:129)
        at io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:873)
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:389)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:305)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:745)

Steps to reproduce

Setup a cluster according to the instructions in Pulsar Cluster Setup. Start a bookie on a machine with command bin/pulsar-daemon start bookie and use command bin/bookkeeper shell bookiesanity to test the bookie.

System configuration

Pulsar version: 1.16.1
@zaa
Copy link

zaa commented Mar 23, 2017

We experience similar behaviour when starting Yahoo Pulsar 1.16.2. Logs contain the following lines

2017-03-23 08:53:53,207 - INFO  - [main:EntryCacheManager@72] - Initialized managed-ledger entry cache of 1024.0 Mb
2017-03-23 08:53:53,243 - ERROR - [main:PulsarBrokerStarter@48] - Uncaught exception in thread main: io.netty.util.internal.MathUtil.safeFindNextPositivePowerOfTwo(I)I
java.lang.NoSuchMethodError: io.netty.util.internal.MathUtil.safeFindNextPositivePowerOfTwo(I)I
	at io.netty.buffer.PoolThreadCache$MemoryRegionCache.<init>(PoolThreadCache.java:372)
	at io.netty.buffer.PoolThreadCache$SubPageMemoryRegionCache.<init>(PoolThreadCache.java:340)
	at io.netty.buffer.PoolThreadCache.createSubPageCaches(PoolThreadCache.java:135)
	at io.netty.buffer.PoolThreadCache.<init>(PoolThreadCache.java:86)
	at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.initialValue(PooledByteBufAllocator.java:361)
	at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.initialValue(PooledByteBufAllocator.java:354)
	at io.netty.util.concurrent.FastThreadLocal.initialize(FastThreadLocal.java:155)
	at io.netty.util.concurrent.FastThreadLocal.get(FastThreadLocal.java:149)
	at io.netty.util.concurrent.FastThreadLocal.get(FastThreadLocal.java:135)
	at io.netty.buffer.PooledByteBufAllocator.newHeapBuffer(PooledByteBufAllocator.java:251)
	at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:158)
	at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:149)
	at com.yahoo.pulsar.broker.service.PulsarStats.<init>(PulsarStats.java:59)
	at com.yahoo.pulsar.broker.service.BrokerService.<init>(BrokerService.java:146)
	at com.yahoo.pulsar.broker.PulsarService.start(PulsarService.java:235)
	at com.yahoo.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:59)

In our case, the broker appears to be running, but it does not listen to clients traffic

ping @merlimat

@saandrews
Copy link
Contributor

saandrews commented Mar 23, 2017 via email

@rdhabalia
Copy link
Contributor

Please check #125.
There was a similar kind of issue with async-http-client that brings different version of io.netty.util.internal.MathUtil which is not compatible with netty-all apis. So, can you verify if you can see any netty artifacts other than netty-all in the classpath.

@merlimat
Copy link
Contributor

@rdhabalia It seems like it might be the case. If you look at the jars included in the binary tgz, there are multiple netty packages with different versions that are being pulled in:

$ ls lib/ | grep netty
async-http-client-netty-utils-2.0.19.jar
netty-all-4.0.40.Final.jar
netty-buffer-4.0.42.Final.jar
netty-codec-4.0.42.Final.jar
netty-codec-dns-2.0.19.jar
netty-codec-http-4.0.42.Final.jar
netty-common-4.0.42.Final.jar
netty-handler-4.0.42.Final.jar
netty-reactive-streams-1.0.8.jar
netty-resolver-2.0.19.jar
netty-resolver-dns-2.0.19.jar
netty-transport-4.0.42.Final.jar
netty-transport-native-epoll-4.0.42.Final-linux-x86_64.jar

@merlimat
Copy link
Contributor

I think this is probably due to maven assembly ignoring the dependency exclusion.

If you look at mvn dependency:tree these jars are not included, but they are anyway added to the tgz

@merlimat merlimat mentioned this issue Mar 23, 2017
Merged
sijie pushed a commit to sijie/pulsar that referenced this issue Mar 4, 2018
@srkukarni @sijie
Better variable safety (apache#277)
433eb8b
hrsakai pushed a commit to hrsakai/pulsar that referenced this issue Dec 10, 2020
@merlimat
Increase writeRequestsCh channel buffer size (apache#277)
90307f2 
* Increase writeRequestsCh channel buffer size

* Fixed indentation
hangc0276 pushed a commit to hangc0276/pulsar that referenced this issue May 26, 2021
@dockerzhang
using the first entry, index 0 as base offset (apache#279)
576fd97 
Fix apache#277, using the first entry, index 0 as the base offset
@xiaotongwang1 xiaotongwang1 mentioned this issue Aug 4, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@zaa @merlimat @rdhabalia @krumo @saandrews