Skip to content
This repository has been archived by the owner on Apr 18, 2018. It is now read-only.

Is pyleus 0.2.4 compatible with Storm 0.9.3 ? #86

Closed
William-Sang opened this issue Dec 12, 2014 · 16 comments
Closed

Is pyleus 0.2.4 compatible with Storm 0.9.3 ? #86

William-Sang opened this issue Dec 12, 2014 · 16 comments
Labels
question

Comments

@William-Sang
Copy link

Today I set up a storm 0.9.3 cluster, and run

$ git clone https://github.com/Yelp/pyleus.git
$ pyleus build pyleus/examples/exclamation_topology/pyleus_topology.yaml
$ pyleus --verbose local exclamation_topology.jar

Than it occurs:

......
9491 [Thread-11-exclaim2] INFO  backtype.storm.task.ShellBolt - Start checking heartbeat...
9492 [Thread-11-exclaim2] INFO  backtype.storm.daemon.executor - Prepared bolt exclaim2:(3)
10502 [Thread-22] ERROR backtype.storm.daemon.executor -
java.lang.Exception: Shell Process Exception: Traceback (most recent call last):
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/pyleus_venv/lib/python2.7/site-packages/pyleus/storm/component.py", line 233, in run
    self.run_component()
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/pyleus_venv/lib/python2.7/site-packages/pyleus/storm/bolt.py", line 46, in run_component
    self._process_tuple(tup)
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/pyleus_venv/lib/python2.7/site-packages/pyleus/storm/bolt.py", line 164, in _process_tuple
    self.process_tuple(tup)
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/exclamation_topology/exclamation_bolt.py", line 18, in process_tuple
    word = tup.values[0] + "!!!"
IndexError: list index out of range

    at backtype.storm.task.ShellBolt.handleError(ShellBolt.java:188) [storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt.access$1100(ShellBolt.java:69) [storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:331) [storm-core-0.9.3.jar:0.9.3]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
10503 [Thread-20] ERROR backtype.storm.daemon.executor -
java.lang.Exception: Shell Process Exception: Traceback (most recent call last):
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/pyleus_venv/lib/python2.7/site-packages/pyleus/storm/component.py", line 233, in run
    self.run_component()
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/pyleus_venv/lib/python2.7/site-packages/pyleus/storm/bolt.py", line 46, in run_component
    self._process_tuple(tup)
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/pyleus_venv/lib/python2.7/site-packages/pyleus/storm/bolt.py", line 164, in _process_tuple
    self.process_tuple(tup)
  File "/tmp/d05ae596-2b00-4eaa-9c4f-17c7f6ee5667/supervisor/stormdist/exclamation_topology-1-1418406574/resources/exclamation_topology/exclamation_bolt.py", line 18, in process_tuple
    word = tup.values[0] + "!!!"
IndexError: list index out of range

    at backtype.storm.task.ShellBolt.handleError(ShellBolt.java:188) [storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt.access$1100(ShellBolt.java:69) [storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:331) [storm-core-0.9.3.jar:0.9.3]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
10509 [ProcessThread(sid:0 cport:-1):] INFO  org.apache.storm.zookeeper.server.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x14a3f9ea9e1000b type:create cxid:0x33 zxid:0x22 txntype:-1 reqpath:n/a Error Path:/storm/errors/exclamation_topology-1-1418406574 Error:KeeperErrorCode = NodeExists for /storm/errors/exclamation_topology-1-1418406574
10519 [Thread-22] ERROR backtype.storm.task.ShellBolt - Halting process: ShellBolt died.
java.io.EOFException: null
    at org.msgpack.io.StreamInput.readByte(StreamInput.java:60) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.getHeadByte(MessagePackUnpacker.java:66) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.trySkipNil(MessagePackUnpacker.java:396) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:59) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:27) ~[exclamation_topology.jar:na]
    at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:527) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:496) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readMessage(MessagePackSerializer.java:198) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readShellMsg(MessagePackSerializer.java:74) ~[exclamation_topology.jar:na]
    at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:99) ~[storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:318) ~[storm-core-0.9.3.jar:0.9.3]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
10519 [Thread-22] ERROR backtype.storm.daemon.executor -
java.io.EOFException: null
    at org.msgpack.io.StreamInput.readByte(StreamInput.java:60) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.getHeadByte(MessagePackUnpacker.java:66) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.trySkipNil(MessagePackUnpacker.java:396) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:59) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:27) ~[exclamation_topology.jar:na]
    at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:527) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:496) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readMessage(MessagePackSerializer.java:198) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readShellMsg(MessagePackSerializer.java:74) ~[exclamation_topology.jar:na]
    at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:99) ~[storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:318) ~[storm-core-0.9.3.jar:0.9.3]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
10520 [Thread-20] ERROR backtype.storm.task.ShellBolt - Halting process: ShellBolt died.
java.io.EOFException: null
    at org.msgpack.io.StreamInput.readByte(StreamInput.java:60) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.getHeadByte(MessagePackUnpacker.java:66) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.trySkipNil(MessagePackUnpacker.java:396) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:59) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:27) ~[exclamation_topology.jar:na]
    at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:527) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:496) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readMessage(MessagePackSerializer.java:198) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readShellMsg(MessagePackSerializer.java:74) ~[exclamation_topology.jar:na]
    at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:99) ~[storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:318) ~[storm-core-0.9.3.jar:0.9.3]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
10521 [Thread-20] ERROR backtype.storm.daemon.executor -
java.io.EOFException: null
    at org.msgpack.io.StreamInput.readByte(StreamInput.java:60) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.getHeadByte(MessagePackUnpacker.java:66) ~[exclamation_topology.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.trySkipNil(MessagePackUnpacker.java:396) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:59) ~[exclamation_topology.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:27) ~[exclamation_topology.jar:na]
    at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:527) ~[exclamation_topology.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:496) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readMessage(MessagePackSerializer.java:198) ~[exclamation_topology.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readShellMsg(MessagePackSerializer.java:74) ~[exclamation_topology.jar:na]
    at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:99) ~[storm-core-0.9.3.jar:0.9.3]
    at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:318) ~[storm-core-0.9.3.jar:0.9.3]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
10526 [Thread-7] INFO  backtype.storm.daemon.nimbus - Shutting down master

......
@ecanzonieri
Copy link
Contributor

It should come soon, there is a pending pull request to make it compatible #9. But currently we do not support storm 0.9.3.

@William-Sang
Copy link
Author

@ecanzonieri We really appreciate your effort.

@msmakhlouf msmakhlouf mentioned this issue Dec 15, 2014
Closed
@poros poros added the question label Jan 29, 2015
@moandcompany
Copy link

Hi,

I've been able to run Pyleus successfully with Storm 0.9.3.2.2.0.0 (Hortonworks HDP 2.2 Build).

@kayrus
Copy link

kayrus commented Mar 11, 2015

Any update on Storm 0.9.3 support?

@poros
Copy link
Contributor

poros commented Mar 11, 2015

We have support for heartbeats (see #97), but we still lack a pull request for #10 and manual testing with Storm 0.9.3

@jwestboston
Copy link

Is #10 required for releasing a version (latest -develop) that supports Storm 0.9.3?

I am using Pyleus 0.2.4-develop with Storm 0.9.3 and can confirm heartbeats are working.

@poros
Copy link
Contributor

poros commented Mar 25, 2015

As far as I know, it should be required. But I didn't try myself

@hellp
Copy link
Contributor

hellp commented Mar 25, 2015

For the record, I'm also using the latest develop branch and just changed the Storm dependency in pom.xml to "0.9.3", built is custom release of pyleus, and use this to build, deploy and run a topology on a 0.9.3 Storm. Works fine.

@jwestboston
Copy link

@poros Is the lack of a PR for #10 holding up the next release of Pyleus?

@kayrus
Copy link

kayrus commented Mar 25, 2015

I've just tried to compile pyleus-base.jar with storm 0.9.3, now I can build and submit topologies. But they don't work. Here is 0.9.3 storm-supervisors' log: http://pastebin.com/0LvrNizZ

The same code works fine on 0.9.2

@kayrus
Copy link

kayrus commented Mar 25, 2015

Here is 0.9.2 storm-supervisor log:
http://pastebin.com/kn0Bwbxt

@jwestboston
Copy link

@kayrus are you sure you're running the latest -develop of Pyleus? I saw similar errors to that when running Pyleus 0.2.4 ... it was heartbeats that was choking things up, since Pyleus 0.2.4 doesn't know how to handle or unpack the heartbeat tuples.

@kayrus
Copy link

kayrus commented Mar 26, 2015

Yes, I use latest develop branch. Topology works, but very, very slow. And it produces error messages. Here is the full log from 0.9.3:
http://pastebin.com/ZhW3xDZM

I've tried to reproduce the same behavior with 0.9.2 - it works well.

@poros
Copy link
Contributor

poros commented Mar 26, 2015

@jwestboston As I said in another thread, I considered #10 as a blocking issue together with extensive testing with Storm 0.9.3. Storm is a complex system and it is not trivial to ensure that Pyleus does not trigger exotic bugs or degrades performance (as it seems to happen with 0.9.3).

However, I also feel that it is time for a new release. I'll coordinate with @patricklucas

@poros poros mentioned this issue Mar 30, 2015
Open
@poros
Copy link
Contributor

poros commented Mar 30, 2015

Update: I opened a pull request for #10, but I noticed we also need #107 for properly supporting Storm 0.9.3

@poros
Copy link
Contributor

poros commented Jun 8, 2015

We switched directly to Storm 0.9.4. Please upgrade pyleus to version 0.3.0

@poros poros closed this as completed Jun 8, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@hellp @moandcompany @poros @ecanzonieri @jwestboston @William-Sang @kayrus