STORM-207: Add storm-starter under examples directory #44

ptgoetz · 2014-03-05T22:30:30Z

This PR merges the storm-starter project into an examples directory (preserving commit history) and incorporates it into the maven build.

see https://github.com/BertrandDechoux/storm-starter/blob/java/pom.xml see http://groups.google.com/group/storm-user/browse_thread/thread/0faa0f7f42d7afe3

2.2.5 is required for Twitter (Kurtis remark)

…jects

…rm dependencies.

Add Maven 2 POM to build storm-starter with maven

…t.clj, update README

Align storm-starter with upstream Storm 0.9.0 (release).

…mit 8012cd7)

This addresses ConcurrentModificationException that might be thrown in the ranker bolts, when the Rankings object would be modified (by the emitting bolt) after sending it downstream.

ptgoetz · 2014-03-05T22:40:12Z

I will volunteer to be the initial sponsor/maintainer for this module.

xumingming · 2014-03-06T07:26:51Z

regarding examples, maybe it is worthwhile to put DemoScheduler.java and TestingApiDemo.java into storm example folder too.

revans2 · 2014-03-10T19:20:41Z

pom.xml

The spacing appears to be off here.

revans2 · 2014-03-10T19:26:05Z

I looked at the diffs from the original storm starter and it looks good to me. One minor comment about the spacing. I would like us to think a bit about how we want to package this. Do we want to include the jar in the resulting package, just not in a place that is loaded by storm by default?

@xumingming I think adding in the examples you mentioned would be great, but we can probably do them on a separate pull request.

ptgoetz · 2014-03-12T17:11:38Z

@revans2 I fixed the spacing issue.

Regarding packaging, I think for a binary release the source code for anything in the examples directory should be included, in addition to a the compiled jars. I'll update the pull request with a configuration for that.

I also agree that the scheduler and testing examples should also be considered, albeit in a separate pull request.

xumingming · 2014-03-12T17:13:01Z

Yeah, adding TestApiDemo and DemoScheduler can be in a separate PR.

ptgoetz · 2014-03-12T18:00:05Z

I just updated the build for the binary distribution to include the /examples/** source code. The deployable (fat) storm-starter jar ends up in the storm-starter directory and gets named storm-starter-topologies-${project.version}.jar.

nathanmarz · 2014-03-17T21:36:38Z

+1

clockfly · 2014-03-20T13:10:48Z

I got compilation error, related with shaded thrift?

Compiling storm.starter.clj.word-count to /mnt/vda/storm-test/storm-0.9.0-Intel/storm-starter/target/classes
Exception in thread "main" java.lang.ExceptionInInitializerError, compiling:(word_count.clj:1)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3387)
    at clojure.lang.Compiler.compile1(Compiler.java:7035)
    at clojure.lang.Compiler.compile1(Compiler.java:7025)
    at clojure.lang.Compiler.compile(Compiler.java:7097)
    at clojure.lang.RT.compile(RT.java:387)
    at clojure.lang.RT.load(RT.java:427)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$compile$fn__4895.invoke(core.clj:5426)
    at clojure.core$compile.invoke(core.clj:5425)
    at clojure.lang.Var.invoke(Var.java:415)
    at clojure.lang.Compile.main(Compile.java:81)
Caused by: java.lang.ExceptionInInitializerError
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:186)
    at storm.starter.clj.word_count$loading__4784__auto__.invoke(word_count.clj:1)
    at clojure.lang.AFn.applyToHelper(AFn.java:159)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3382)
    ... 14 more
Caused by: java.lang.ClassNotFoundException: org.apache.thrift.server.THsHaServer, compiling:(drpc.clj:16)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3387)
    at clojure.lang.Compiler.compile1(Compiler.java:7035)
    at clojure.lang.Compiler.compile1(Compiler.java:7025)
    at clojure.lang.Compiler.compile(Compiler.java:7097)
    at clojure.lang.RT.compile(RT.java:387)
    at clojure.lang.RT.load(RT.java:427)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.LocalDRPC$loading__4784__auto__.invoke(LocalDRPC.clj:16)
    at clojure.lang.AFn.applyToHelper(AFn.java:159)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3382)
    at clojure.lang.Compiler.compile1(Compiler.java:7035)
    at clojure.lang.Compiler.compile1(Compiler.java:7025)
    at clojure.lang.Compiler.compile(Compiler.java:7097)
    at clojure.lang.RT.compile(RT.java:387)
    at clojure.lang.RT.load(RT.java:427)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5298)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.testing$loading__4784__auto__.invoke(testing.clj:16)
    at clojure.lang.AFn.applyToHelper(AFn.java:159)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3382)
    at clojure.lang.Compiler.compile1(Compiler.java:7035)
    at clojure.lang.Compiler.compile1(Compiler.java:7025)
    at clojure.lang.Compiler.compile(Compiler.java:7097)
    at clojure.lang.RT.compile(RT.java:387)
    at clojure.lang.RT.load(RT.java:427)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:605)
    at clojure.core$use.doInvoke(core.clj:5392)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.LocalCluster$loading__4784__auto__.invoke(LocalCluster.clj:16)
    at clojure.lang.AFn.applyToHelper(AFn.java:159)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3382)
    at clojure.lang.Compiler.compile1(Compiler.java:7035)
    at clojure.lang.Compiler.compile1(Compiler.java:7025)
    at clojure.lang.Compiler.compile(Compiler.java:7097)
    at clojure.lang.RT.compile(RT.java:387)
    at clojure.lang.RT.load(RT.java:427)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.lang.Var.invoke(Var.java:415)
    at backtype.storm.LocalCluster.<clinit>(Unknown Source)
    ... 20 more
Caused by: java.lang.ClassNotFoundException: org.apache.thrift.server.THsHaServer
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at clojure.lang.DynamicClassLoader.findClass(DynamicClassLoader.java:61)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:186)
    at backtype.storm.daemon.drpc$loading__4784__auto__.invoke(drpc.clj:16)
    at clojure.lang.AFn.applyToHelper(AFn.java:159)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3382)

clockfly · 2014-03-20T16:14:39Z

Sorry, the above is not a real issue. the latest code "storm-buildtools/maven-shade-clojure-transformer" have solved this problem.

revans2 · 2014-03-20T16:22:59Z

+1

d2r · 2014-03-24T20:15:03Z

It seems the comment from

 Michael G. Noll    Merge pull request #50 from ptgoetz/master

showed up in an unrelated issue #50 as a reference from by me since I pulled in these commits before issuing a new pull request (#51).

So github is getting confused seeing storm-starter issues in commit messages and thinks they are incubator-storm issues. #50 looks to be the last of these, though.

Simple resource limiting

BUG-44565: Add a topology for hive bolt that reads from file to test …

Nathan Marz and others added 30 commits September 15, 2011 22:20

storm starter

9edcf17

add license

c80b088

add clojure dependencies so people can try out storm-starter from repl

e1a6615

upgrade to 0.5.1 API

433caaa

bump storm dependency for no zmq local mode

7027ea9

added example of clojure DSL

7fa873d

update storm dependency

3c4f437

formatting

4aa082d

example of doing scalable streaming rolling top N

ef3020a

tuple() -> new Values()

9d267e8

fix TwitterSampleSpout

d353e6b

make wordcounttopology and exclamationtopology run in remote mode

a17ae04

2.2.5 is required for Twitter (Kurtis remark)

a1ac4b4

see https://github.com/BertrandDechoux/storm-starter/blob/java/pom.xml see http://groups.google.com/group/storm-user/browse_thread/thread/0faa0f7f42d7afe3

added drpc examples

c2919ad

cleanup a topology

b4d90cb

Merge pull request apache#2 from BertrandDechoux/patch-1

085dd8f

2.2.5 is required for Twitter (Kurtis remark)

fix TwitterSampleSpout by relying on Java serialization for Status ob…

6b57766

…jects

Added maven2 pom for users who may not want to use lein to build.

1891618

Updated maven2 pom to create a storm-deployable jar that excludes sto…

9155031

…rm dependencies.

null check so easier to run topologies from clojure REPL

4edef6a

added example of doing single join

65bfaf3

update .gitignore

82b347c

Merge pull request apache#4 from ptgoetz/master

2a3ef25

Add Maven 2 POM to build storm-starter with maven

refresh README

e7adbdb

fix README

1f5cb7f

fix README

de9955f

fix README

3b815c3

update storm-starter for storm 0.6.0

e31f045

remove debug from reachtopology

b0805c8

improve rolling top words to take advantage of improved serialization

98915c3

Michael G. Noll and others added 12 commits August 29, 2013 12:57

SlidingWindowCounter: update Java docs

8b40120

align with version 0.9.0.1, clean up m2-pom.xml and align with projec…

8012cd7

…t.clj, update README

Merge pull request apache#50 from ptgoetz/master

7a688ae

Align storm-starter with upstream Storm 0.9.0 (release).

m2-pom.xml: fix indentation / whitespace

4c40384

README: Add instructions for IntelliJ IDEA

1bcefbb

Re-add Google Guava to maven deps config (accidentally removed in com…

74f01ec

…mit 8012cd7)

AbstractRankerBolt: Emit defensive copy of Rankings object

651ccfe

This addresses ConcurrentModificationException that might be thrown in the ranker bolts, when the Rankings object would be modified (by the emitting bolt) after sending it downstream.

README: Fix instructions for IntelliJ IDEA

8364b51

merge storm-starter into examples

f1d7fca

add examples/storm-starter to build

8a2760d

add Apache license headers

a99d9c1

update README to reflect incorporation into storm source tree

651df24

revans2 reviewed Mar 10, 2014
View reviewed changes

pom.xml Outdated

Copy link

Contributor

revans2 Mar 10, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spacing appears to be off here.

fix spacing issue

bb13cce

add examples source code to binary distribution

82a754a

asfgit merged commit 82a754a into apache:master Mar 20, 2014

knusbaum pushed a commit to knusbaum/incubator-storm that referenced this pull request Feb 11, 2015

Merge pull request apache#44 from knusbaum/resource-limits

ba0e264

Simple resource limiting

Parth-Brahmbhatt pushed a commit to Parth-Brahmbhatt/incubator-storm that referenced this pull request Mar 7, 2016

Merge pull request apache#44 from priyank5485/BUG-44565

9508d07

BUG-44565: Add a topology for hive bolt that reads from file to test …

Parth-Brahmbhatt pushed a commit to Parth-Brahmbhatt/incubator-storm that referenced this pull request Mar 7, 2016

Merge pull request apache#44 from priyank5485/BUG-44565

fce7e2d

BUG-44565: Add a topology for hive bolt that reads from file to test …

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

STORM-207: Add storm-starter under examples directory #44

STORM-207: Add storm-starter under examples directory #44

Uh oh!

ptgoetz commented Mar 5, 2014

Uh oh!

ptgoetz commented Mar 5, 2014

Uh oh!

xumingming commented Mar 6, 2014

Uh oh!

revans2 Mar 10, 2014

Uh oh!

revans2 commented Mar 10, 2014

Uh oh!

ptgoetz commented Mar 12, 2014

Uh oh!

xumingming commented Mar 12, 2014

Uh oh!

ptgoetz commented Mar 12, 2014

Uh oh!

nathanmarz commented Mar 17, 2014

Uh oh!

clockfly commented Mar 20, 2014

Uh oh!

clockfly commented Mar 20, 2014

Uh oh!

revans2 commented Mar 20, 2014

Uh oh!

d2r commented Mar 24, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

STORM-207: Add storm-starter under examples directory #44

STORM-207: Add storm-starter under examples directory #44

Uh oh!

Conversation

ptgoetz commented Mar 5, 2014

Uh oh!

ptgoetz commented Mar 5, 2014

Uh oh!

xumingming commented Mar 6, 2014

Uh oh!

revans2 Mar 10, 2014

Choose a reason for hiding this comment

Uh oh!

revans2 commented Mar 10, 2014

Uh oh!

ptgoetz commented Mar 12, 2014

Uh oh!

xumingming commented Mar 12, 2014

Uh oh!

ptgoetz commented Mar 12, 2014

Uh oh!

nathanmarz commented Mar 17, 2014

Uh oh!

clockfly commented Mar 20, 2014

Uh oh!

clockfly commented Mar 20, 2014

Uh oh!

revans2 commented Mar 20, 2014

Uh oh!

d2r commented Mar 24, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants