Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STORM-2090: Add integration test for storm windowing #1691

Closed
wants to merge 6 commits into from

Conversation

raghavgautam
Copy link
Contributor

No description provided.

Copy link
Contributor

@harshach harshach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great Work @raghavgautam . This will help a lot. Minor nits. +1.

import org.slf4j.LoggerFactory;

import javax.annotation.Nullable;
import java.util.*;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you avoid wildcard imports

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

drpc.servers:
- "node1"

supervisor.slots.ports: [6700, 6701, 6702, 6703, 6704, 6705, 6706, 6707, 6708, 6709]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason to have these many worker slots?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are running tests in parallel and I have put this for worst case scenario.

@HeartSaVioR
Copy link
Contributor

@raghavgautam

  1. Travis CI fails with RAT issue. Could you run your branch via mvn clean install -Prat?
  2. Could we apply fail-fast? I think integration tests don't need to be run when test is failing.
    Please refer https://travis-ci.org/apache/storm/jobs/161473780. Test is failing with RAT, and script ran integration tests.

Btw, do you plan to introduce more integration tests?
Since it seems additional 20 mins to setup and run the new integration tests so I feel it a bit heavy to test just a windowing cases.

@raghavgautam
Copy link
Contributor Author

@HeartSaVioR I have fixed the rat issue and I have put fast fail.

Yes, we will be contributing more tests to apache storm going forward.

@HeartSaVioR
Copy link
Contributor

@raghavgautam Thanks. It looks great to me. +1

@HeartSaVioR
Copy link
Contributor

I squashed commits before merging (b779ca4) but forgot to write 'Closes #1691' to commit log.

@raghavgautam Could you close this? Thanks!

@raghavgautam
Copy link
Contributor Author

Thanks @harshach @HeartSaVioR for reviewing.

@ptgoetz
Copy link
Member

ptgoetz commented Sep 23, 2016

This a fairly large commit that seemingly includes code from other projects. That's fine as long as you can document what code was copied, and what the license for that code was.

@raghavgautam
Copy link
Contributor Author

I had mention this on the jira.
A good part of the vagrant setup has been picked up from:
https://github.com/ptgoetz/storm-vagrant
https://github.com/harshach/storm-vagrant

https://issues.apache.org/jira/browse/STORM-2090

Both of them have apache license:
https://github.com/ptgoetz/storm-vagrant/blob/master/LICENSE
https://github.com/harshach/storm-vagrant/blob/master/LICENSE

Rest has been authored by me (contributed by Hortonworks).
This includes TestngListner.java which was originally contributed to apache falcon.
https://github.com/apache/falcon/blob/master/falcon-regression/merlin/src/test/java/org/apache/falcon/regression/TestngListener.java

@ptgoetz
Copy link
Member

ptgoetz commented Sep 23, 2016

Just to be clear, that wasn't a criticism. I just wanted to point out that it is important that we know the provenance and license of all code that enters our repository.

@raghavgautam
Copy link
Contributor Author

@ptgoetz Do we need to document any other place or just mentioning on the jira is sufficient ?

@ptgoetz
Copy link
Member

ptgoetz commented Sep 23, 2016

@raghavgautam Nope. All is good.

--------------------------
To run all tests:
```sh
mvn clean package -DskipTests && mvn test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mvn test will make these integration tests run in the maven test phase. I believe it would be more appropriate to have these integration tests run in the maven integration-test or verify phase.

Please refer here for relevant info.

@hmcl
Copy link
Contributor

hmcl commented Sep 28, 2016

@raghavgautam I am +1 overall. I believe it makes sense to have these integration tests run in the mvn integration-test phase.

Can you please also squash all the commits into one commit. It will make the git log much cleaner and easy to follow. Besides that i am

+1


Running tests end-to-end
------------------------
Assumption:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rename Assumption to Requirements, as it is a requirement to have this in order to run the tests

@raghavgautam raghavgautam deleted the apache_master branch January 9, 2017 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants