Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STORM-2249: Make Distribution Scripts Put Examples to the Correct Locations #1831

Closed

Conversation

hmcl
Copy link
Contributor

@hmcl hmcl commented Dec 17, 2016

  • Move all examples under STORM_HOME/examples/storm-COMPONENT-NAME-examples/COMPONENT-NAME-examples-x.y.x.jar

@hmcl
Copy link
Contributor Author

hmcl commented Dec 17, 2016

Whit these changes the examples distribution looks like this:

screen shot 2016-12-17 at 2 17 26 am

…ations

  - Move all examples under  STORM_HOME/examples/storm-COMPONENT-NAME-examples/COMPONENT-NAME-examples-x.y.x.jar
@hmcl hmcl force-pushed the Apache_master_STORM-2249_Fix_Dist_Examples branch from 2787d7e to 397e3ce Compare December 22, 2016 01:33
@HeartSaVioR
Copy link
Contributor

@hmcl
The change looks great, but we need to think why shaded jar of examples are necessary to be included to binary distribution.
AFAIK the examples modules (except storm-starter) are for referring the working example source code rather than executing the topology. Do we expect end-user would run the example topology? If we expect, we should document about topologies to README.md for each modules.

@harshach
Copy link
Contributor

@HeartSaVioR I think it makes sense to have the shaded jars. It will be easier to run the code for the users who not familiar with storm packaging and gives insight into building a topology jar as well.

@HeartSaVioR
Copy link
Contributor

@harshach @hmcl
Then we should add available topologies in README for each modules to since users don't know which topologies are available.

@hmcl
Copy link
Contributor Author

hmcl commented Feb 4, 2017

@HeartSaVioR do you mean each README of each module should have a section on how to run the JAR, and the main for that JAR? I agree with that. Perhaps I could file a follow-up JIRA for that.

@HeartSaVioR
Copy link
Contributor

@hmcl
Right. But I changed my mind that all examples should contain the code, like storm-starter in binary dist. We're extracting examples to show example codes for each modules, so the most important thing is code.
@harshach What do you think?

@HeartSaVioR
Copy link
Contributor

Also could you check how big binary dist (tar.gz / zip) will be? It's already over 200MB so IMHO it would be better to reduce/restrict the size of binary dist.

@HeartSaVioR
Copy link
Contributor

HeartSaVioR commented Mar 2, 2017

Before:

-rw-r--r--   1 jlim  staff   3.3K  3  2 19:41 apache-storm-2.0.0-SNAPSHOT.pom
-rw-r--r--   1 jlim  staff   473B  3  2 19:41 apache-storm-2.0.0-SNAPSHOT.pom.asc
-rw-r--r--   1 jlim  staff   264M  3  2 19:41 apache-storm-2.0.0-SNAPSHOT.tar.gz
-rw-r--r--   1 jlim  staff   473B  3  2 19:41 apache-storm-2.0.0-SNAPSHOT.tar.gz.asc
-rw-r--r--   1 jlim  staff   264M  3  2 19:41 apache-storm-2.0.0-SNAPSHOT.zip
-rw-r--r--   1 jlim  staff   473B  3  2 19:41 apache-storm-2.0.0-SNAPSHOT.zip.asc

After:

-rw-r--r--   1 jlim  staff   3.3K  3  2 19:46 apache-storm-2.0.0-SNAPSHOT.pom
-rw-r--r--   1 jlim  staff   473B  3  2 19:46 apache-storm-2.0.0-SNAPSHOT.pom.asc
-rw-r--r--   1 jlim  staff   564M  3  2 19:46 apache-storm-2.0.0-SNAPSHOT.tar.gz
-rw-r--r--   1 jlim  staff   473B  3  2 19:46 apache-storm-2.0.0-SNAPSHOT.tar.gz.asc
-rw-r--r--   1 jlim  staff   565M  3  2 19:46 apache-storm-2.0.0-SNAPSHOT.zip
-rw-r--r--   1 jlim  staff   473B  3  2 19:47 apache-storm-2.0.0-SNAPSHOT.zip.asc

Sorry but for me the final size (exceeding 500M) is hard to accept. Could we re-think about this?

@HeartSaVioR
Copy link
Contributor

@hmcl
We applied opposite approach for this. Could you close this PR?

@hmcl hmcl closed this Mar 31, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants