Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resolve #30 Embed ES as part of the build and #27 Broken tests #31

Closed
wants to merge 7 commits into from
Closed

Resolve #30 Embed ES as part of the build and #27 Broken tests #31

wants to merge 7 commits into from

Conversation

tzolov
Copy link

@tzolov tzolov commented Apr 11, 2013

https://github.com/tzolov/elasticsearch-hadoop/tree/master/src/test/java/org/elasticsearch/hadoop/crunch

Sample usage in test:

    @BeforeClass
    public static void beforeClass() {
      esServer = new EmbeddedElasticsearchServer();
    }

    @AfterClass
    public static void afterClass() {
      esServer.shutdown();
    }

There are several helper utility methods for counting, indexing and searching that I found useful for testing (check the CurnchEndToEndTest)

@tzolov
Resolve elastic#30 : Embed ES as part of the build
923573b
@tzolov
add elasticserach dependency for testing
3d94d37
@tzolov
fix the test execution order (related to elastic#27) and add Embedded…
bd389e3 
…ElasticSerach in before/after class
@tzolov
Resolve a failing CascadingLocalTest. The testReadFromES() fails beca…
28964c2 
…use the testWriteToES() never writes to ES. Solution: set the ES_BATCH_SIZE_ENTRIES to 1 in ESLocalScheme.initTargetUrl to make it flush the buffer
@tzolov
The HadoopFlowStep.initFromSink() creates tempSink and resets the ESO…
18351bd 
…utputFormat to to null output format if no output path (mapred.output.dir) is found! That means the output is never send to ES. Solution: Inside ESHadoopScheme.sinkConfInit() set a dummy output path to configuration configuration.
@tzolov
filter-out the null-fields input entries to resolve the (org.codehaus…
fa62b72 
….jackson.map.JsonMappingException: No serializer found for class org.apache.hadoop.io.NullWritable) exception thrown BufferedRestClient.addToIndex()
@tzolov
merge upstream/master 1eb0d37,98c2a07 and 957424b commits
4ccfb17
@tzolov tzolov closed this Apr 14, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
v1.3.0.M1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@tzolov