ITKafkaIndexingServiceTest fixes #3872
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ying send fixed number of events more fixes
|
@b-slim regarding your interrupted exception comment I think it is not necessary for integration test but let me know if you strongly think otherwise |
|
@b-slim nevermind changed the code to handle InterruptedException |
gianm
reviewed
Jan 21, 2017
|
|
||
| // these are used to compute the expected aggregations | ||
| int added = 0; | ||
| int num_events = 0; | ||
|
|
||
| // send data to kafka | ||
| while (dt.compareTo(dtStop) < 0) { // as long as we're within the time span | ||
| while (num_events < NUM_EVENTS_TO_SEND) { // as long as we're within the time span |
There was a problem hiding this comment.
The comment is no longer accurate since the event count is fixed now.
| private static final String INDEXER_FILE = "/indexer/kafka_supervisor_spec.json"; | ||
| private static final String QUERIES_FILE = "/indexer/kafka_index_queries.json"; | ||
| private static final String DATASOURCE = "kafka_indexing_service_test"; | ||
| private static final String TOPIC_NAME = "kafka_indexing_service_topic"; | ||
| private static final int MINUTES_TO_SEND = 4; | ||
| private static final int NUM_EVENTS_TO_SEND = 60; | ||
| private static final long WAIT_TIME_MILIIS = 2 * 60 * 1000L; |
There was a problem hiding this comment.
Misspelled, should be WAIT_TIME_MILLIS
| dtLast = dt; | ||
| dt = new DateTime(zone); | ||
| } | ||
|
|
||
| producer.close(); | ||
|
|
||
| LOG.info("Waiting for [%s] millis for Kafka indexing tasks to consume events", WAIT_TIME_MILIIS); |
There was a problem hiding this comment.
Would be nicer to wait up to WAIT_TIME_MILLIS so if the tasks load faster than that, the test can exit earlier. But, this is fine for now, as ITs are not expected to be the fastest things in the world.
|
@gianm took care of your comments |
dgolitsyn
pushed a commit
to metamx/druid
that referenced
this pull request
Feb 14, 2017
* remove wait between sends, wait for ingestion to complete before querying send fixed number of events more fixes * handle interrupted exception * remove while * review comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
loadstatusdoes not consider realtime segments