Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to split namespace bundle #172

Closed
sschepens opened this issue Jan 25, 2017 · 0 comments · Fixed by #183
Closed

Failed to split namespace bundle #172

sschepens opened this issue Jan 25, 2017 · 0 comments · Fixed by #183
Labels
type/bug The PR fixed a bug or issue reported a bug
Milestone
1.16

Comments

@sschepens
Copy link
Contributor

We had a couple of this errors last night, here are the logs from broker.

2017-01-25 08:53:54,522 - INFO  - [pool-1-thread-1:SimpleLoadManagerImpl@1381] - Will split hot namespace bundle fury/global/cbead41f2a5849b5a5be64df0f3521d2-orders-orders/0xb333332f_0xbffffffb, topics 2, producers+consumers 610, msgRate in+out 1000.1632182840386, bandwidth in+out 126300.59782161229

2017-01-25 08:53:54,523 - INFO  - [pulsar-web-44-25:Namespaces@781] - [null] Split namespace bundle fury/global/cbead41f2a5849b5a5be64df0f3521d2-orders-orders/0xbffffffb_0xccccccc8

2017-01-25 08:53:54,523 - ERROR - [pulsar-web-44-25:PulsarWebResource@364] - [null] Failed to validate namespace bundle fury/global/cbead41f2a5849b5a5be64df0f3521d2-orders-orders/0xbffffffb_0xccccccc

java.lang.IllegalArgumentException: Cannot find bundle in the bundles list
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:93)
	at com.yahoo.pulsar.common.naming.NamespaceBundles.validateBundle(NamespaceBundles.java:101)
	at com.yahoo.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleRange(PulsarWebResource.java:361)
	at com.yahoo.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleOwnership(PulsarWebResource.java:372)
	at com.yahoo.pulsar.broker.admin.Namespaces.splitNamespaceBundle(Namespaces.java:793)
	at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:143)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
	at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
	at org.eclipse.jetty.server.Server.handle(Server.java:524)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:745)

com.yahoo.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: Some error occourred on the server
	at com.yahoo.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:160)
	at com.yahoo.pulsar.client.admin.internal.NamespacesImpl.splitNamespaceBundle(NamespacesImpl.java:349)
	at com.yahoo.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl.doNamespaceBundleSplit(SimpleLoadManagerImpl.java:1396)
	at com.yahoo.pulsar.broker.loadbalance.NamespaceBundleSplitTask.run(NamespaceBundleSplitTask.java:37)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error
	at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:1032)
	at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:819)
	at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:92)
	at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:701)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
	at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:697)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:448)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.put(JerseyInvocation.java:332)
	at com.yahoo.pulsar.client.admin.internal.NamespacesImpl.splitNamespaceBundle(NamespacesImpl.java:347)
	... 9 more


2017-01-25 08:53:54,527 - ERROR - [pool-1-thread-1:SimpleLoadManagerImpl@1400] - Failed to split namespace bundle fury/global/cbead41f2a5849b5a5be64df0f3521d2-orders-orders/0xbffffffb_0xccccccc8


2017-01-25 08:53:54,527 - ERROR - [pulsar-web-44-10:PulsarWebResource@364] - [null] Failed to validate namespace bundle fury/global/cbead41f2a5849b5a5be64df0f3521d2-orders-orders/0xb333332f_0xbffffffb


2017-01-25 08:53:54,527 - INFO  - [pulsar-web-44-10:Namespaces@781] - [null] Split namespace bundle fury/global/cbead41f2a5849b5a5be64df0f3521d2-orders-orders/0xb333332f_0xbffffffb


java.lang.IllegalArgumentException: Invalid upper boundary for bundle
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:93)
	at com.yahoo.pulsar.common.naming.NamespaceBundles.validateBundle(NamespaceBundles.java:102)
	at com.yahoo.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleRange(PulsarWebResource.java:361)
	at com.yahoo.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleOwnership(PulsarWebResource.java:372)
	at com.yahoo.pulsar.broker.admin.Namespaces.splitNamespaceBundle(Namespaces.java:793)
	at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:143)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
	at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
	at org.eclipse.jetty.server.Server.handle(Server.java:524)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:745)


2017-01-25 08:53:54,528 - ERROR - [pool-1-thread-1:SimpleLoadManagerImpl@1400] - Failed to split namespace bundle fury/global/cbead41f2a5849b5a5be64df0f3521d2-orders-orders/0xb333332f_0xbffffffb


com.yahoo.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: Some error occourred on the server
	at com.yahoo.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:160)
	at com.yahoo.pulsar.client.admin.internal.NamespacesImpl.splitNamespaceBundle(NamespacesImpl.java:349)
	at com.yahoo.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl.doNamespaceBundleSplit(SimpleLoadManagerImpl.java:1396)
	at com.yahoo.pulsar.broker.loadbalance.NamespaceBundleSplitTask.run(NamespaceBundleSplitTask.java:37)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error
	at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:1032)
	at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:819)
	at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:92)
	at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:701)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
	at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:697)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:448)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.put(JerseyInvocation.java:332)
	at com.yahoo.pulsar.client.admin.internal.NamespacesImpl.splitNamespaceBundle(NamespacesImpl.java:347)
	... 9 more
@rdhabalia rdhabalia mentioned this issue Jan 31, 2017
Merged
@merlimat merlimat added this to the 1.16 milestone Jan 31, 2017
@merlimat merlimat added the type/bug The PR fixed a bug or issue reported a bug label Jan 31, 2017
sijie pushed a commit to sijie/pulsar that referenced this issue Mar 4, 2018
@srkukarni @sijie
Infer Missing Arguments (apache#172)
3424041 
* Infer Missing Arguments

* Added unittest
hrsakai pushed a commit to hrsakai/pulsar that referenced this issue Dec 10, 2020
@freeznet
[Issue 172] Add key based batcher (apache#400)
154bff0 
Fixes apache#172

### Motivation

Add a new batch message container named `keyBasedBatchContainer` to support batching message in key_shared subscription mode.

### Modifications

- add `BatchBuilder` interface, add `FlushBatches` and `IsMultiBatches` func
- change old `BatchBuilder` struct to `batchContainer`
- add `keyBasedBatchContainer`
- add tests

### Verifying this change

This change added tests and can be verified as follows:

  - *Added integration tests for key based batch producer with multiple consumer in KeyShared mode*
  - *Added integration tests for message ordering with key based batch producer and KeyShared consumer*
hangc0276 pushed a commit to hangc0276/pulsar that referenced this issue May 26, 2021
@aloyszhang
add publish time for kafka records to enable seek offset by time (apa…
d40a1c3 
…che#172)

add publish time for kafka records to enable seek offset by time
Fix apache#171

* add publish time for kafka records to enable seek offset by time

* fix check style
@xiaotongwang1 xiaotongwang1 mentioned this issue Aug 4, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Milestone
1.16
3 participants
@merlimat @rdhabalia @sschepens