New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GORA-664 Add datastore for Elasticsearch #234
Conversation
- Create ElasticsearchStore class with connection initialization - Create basic Elasticsearch types mapping - Implement the necessary files for mapping representation (ElasticsearchMapping, ElasticsearchMappingBuilder) - Read schema from mapping file - Cover initialization with test
- Created gora.properties file with configuration properties - Loaded connection parameters from configuration - Implemented connection to Elasticsearch cluster with ElasticsearchParameters - Covered ElasticsearchParameters with tests - Added javadoc descriptions
- Added testing dependencies - Added GoraElasticsearchTestDriver with Elasticsearch container - Added javadoc descriptions to GoraElasticsearchTestDriver class - Fixed two existing tests in accordance to Elasticsearch container
Implemented schemaExists, createSchema, deleteSchema and flush methods
- Relocated gora-elasticsearch.xsd file to main resources - Covered XSD validation with test - Added gora-elasticsearch-mapping-invalid.xml file for test
- Set up password to Elasticsearch container properly - Set default Elasticsearch container server’s username in gora.properties - Added exceptions for missing arguments in authentication
- Defined a parameter for the XSD validation - Added a test case for the parameter - Made ElasticsearchStore read mapping file from properties, not configuration
- Implemented delete, get and put methods - Implemented newInstance and getUnionSchema utility methods - Implemented basic serialization/deserialization for primitive AVRO types
- Added mappings while creating an Elasticsearch index - Added getter and setter to Datatype enum
- Implemented serializeFieldValue and deserializeFieldValue methods for ARRAY, BOOLEAN, BYTES and FIXED Avro data types - Fixed deserialization for STRING Avro data type - Added javadoc descriptions
break; | ||
} | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can remove extra spaces.
public float getProgress() { | ||
if (persistentObjects.size() == 0) { | ||
return 1; | ||
} else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need to write else
. It can be:
if (persistentObjects.size() == 0) {
return 1;
}
return offset / (float) persistentObjects.size();
} catch (IOException ex) { | ||
throw new GoraException(ex); | ||
} | ||
assert response != null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider logging and throwing an error without using assert.
Congrats @podorvanova for the PR! |
Thanks @kamaci for your feedback! I have updated the PR. |
[Outreachy Winter 2020-2021]
This PR implements an Apache Elasticsearch datastore for Apache Gora.
Your feedback would be much appreciated.