Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GORA-665] - Add support for the Ignite datastore in GoraExplorer #228

Merged
merged 3 commits into from Oct 30, 2020
Merged

Conversation

mgov88
Copy link
Contributor

@mgov88 mgov88 commented Oct 25, 2020

To add support for Apache Ignite in Gora Explorer the mapping XML must be read from gora.mapping in the same way as is implemented in the Kudu datastore.

Copy link
Contributor

@carlosrmng carlosrmng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alfonsonishikawa
Copy link
Member

@mgov88 I find we will need the metadata analyzer for Ignite too to have fully support of the features related to GoraExplorer.
Thank you!

@mgov88
Copy link
Contributor Author

mgov88 commented Oct 28, 2020

@alfonsonishikawa Hi, I took a look of HBase metadata analyzer https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStoreMetadataAnalyzer.java I there are two methods to be implemented:
getTablesNames and getTableInfo.

getTablesNames is just a list of the tables
and getTableInfo is a list of the columns of a table

is it correct?

@alfonsonishikawa
Copy link
Member

alfonsonishikawa commented Oct 28, 2020

@alfonsonishikawa Hi, I took a look of HBase metadata analyzer https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStoreMetadataAnalyzer.java I there are two methods to be implemented:
getTablesNames and getTableInfo.

getTablesNames is just a list of the tables
and getTableInfo is a list of the columns of a table

is it correct?

It is correct but that is specific for HBase.

The first method, getTablesNames is needed for the type of mapping HBase has in the tag <table>. The signature for this method returns List<String> because I expected all datastores to have some "table" defined in their mappings.

The second method, getTableInfo returns specific data of the table and for HBase is an instance of HBaseTableMetadata, so in Ignite should be something like IgniteTableMetadata. The class IgniteTableMetadata that you have to create should have the needed information you can recover from the native store to help creating the mapping for that table.

The rule of thumb is that IgniteTableMetadata should return metadata about a table in Ignite. In the case of HBase are things called "Column Families", but that is absolutely related to HBase, not Ignite.

It could be, columns and types of each column, for example. Or columns, type, sizes, etc...
As I see from https://github.com/apache/gora/blob/master/gora-ignite/src/test/resources/gora-ignite-mapping.xml it looks Ignite needs the column name and it's type. So, that should be the minimum metadata returned in a (still not implemented) IgniteTableMetadata.java

@alfonsonishikawa
Copy link
Member

Oh! And the primary key!

@mgov88
Copy link
Contributor Author

mgov88 commented Oct 29, 2020

@alfonsonishikawa I developed the Metadata Analyzer feature and a test, I had to update the ignite version, I can not find the schema information of the tables in the older version.

@alfonsonishikawa alfonsonishikawa merged commit cd5ec39 into apache:master Oct 30, 2020
@alfonsonishikawa
Copy link
Member

Merged to master. Thank you, @mgov88 !

public static String createSelectAllColumnsOfTable(String schemaIgnite, String table) {
DbSpec spec = new DbSpec();
DbSchema schema = spec.addDefaultSchema();
DbTable aTable = schema.addTable("INFORMATION_SCHEMA.COLUMNS");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't spare variables names. Long are the days when each byte was worth and today memory and disk are cheap. Use descriptive meaningful variables to enhance readability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants