Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

STORM-1999 Update docs with OutputCollector's threadsafety #1622

Merged
merged 1 commit into from Aug 12, 2016
Merged

STORM-1999 Update docs with OutputCollector's threadsafety #1622

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
68 changes: 0 additions & 68 deletions docs/Troubleshooting.md
View file
Open in desktop
Expand Up @@ -112,71 +112,3 @@ Caused by: java.util.ConcurrentModificationException
Solution:

* This means that you're emitting a mutable object as an output tuple. Everything you emit into the output collector must be immutable. What's happening is that your bolt is modifying the object while it is being serialized to be sent over the network.


### NullPointerException from deep inside Storm

Symptoms:

* You get a NullPointerException that looks something like:

```
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:55)
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:56)
at org.apache.storm.disruptor$consume_loop_STAR_$fn__1596.invoke(disruptor.clj:67)
at org.apache.storm.util$async_loop$fn__465.invoke(util.clj:377)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:24)
at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126$fn__4130.invoke(worker.clj:99)
at org.apache.storm.util$fast_list_map.invoke(util.clj:771)
at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126.invoke(worker.clj:99)
at org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3904.invoke(executor.clj:205)
at org.apache.storm.disruptor$clojure_handler$reify__1584.onEvent(disruptor.clj:43)
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:81)
... 6 more
```

or

```
java.lang.RuntimeException: java.lang.NullPointerException
at
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
~[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
~[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
~[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.disruptor$consume_loop_STAR_$fn__759.invoke(disruptor.clj:94)
~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.util$async_loop$fn__458.invoke(util.clj:463)
~[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
Caused by: java.lang.NullPointerException: null
at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
at
org.apache.storm.daemon.worker$mk_transfer_fn$fn__3548.invoke(worker.clj:129)
~[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3282.invoke(executor.clj:258)
~[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.disruptor$clojure_handler$reify__746.onEvent(disruptor.clj:58)
~[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
~[storm-core-0.9.3.jar:0.9.3]
... 6 common frames omitted
```

Solution:

* This is caused by having multiple threads issue methods on the `OutputCollector`. All emits, acks, and fails must happen on the same thread. One subtle way this can happen is if you make a `IBasicBolt` that emits on a separate thread. `IBasicBolt`'s automatically ack after execute is called, so this would cause multiple threads to use the `OutputCollector` leading to this exception. When using a basic bolt, all emits must happen in the same thread that runs `execute`.