Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

if ES field names are in Pascal Case, Camel case or in the uppercase then presto is not able to map #12562

Closed
shubham-agarwal-ngi opened this issue Apr 1, 2019 · 7 comments · Fixed by trinodb/trino#887

Comments

@shubham-agarwal-ngi
Copy link

If we are giving the value in the camel case, pascal case or in the uppercase of the name or in jsonPath fields then the presto is not able to map and showing the NULL values for that particular fields.

Please advice.

@raghavsethi
Copy link
Contributor

By ES, do you mean Elasticsearch?

@shubham-agarwal-ngi
Copy link
Author

hi raghavsethi,

yes, ES means Elastic search

@raghavsethi
Copy link
Contributor

And can you clarify what you mean by map? If you do DESCRIBE, does Presto show the columns? Are you confident that the values are non null?

@TempleZhou @zhenxiao could one of you take a look at this?

@shubham-agarwal-ngi
Copy link
Author

Hi Raghav,

Let me tell you with an example,

Case 1:
This is the table definition in case of elastic search.
{
"tableName": "dummy",
"schemaName": "es",
"host": "0.0.0.0",
"port": 0000,
"clusterName": "clustername",
"index": "dummy",
"type": "user",
"columns": [{
"name": "Gender",
"type": "varchar",
"jsonPath": "Gender",
"jsonType": "varchar",
"ordinaryPosition": "0"
}]
}

In Elastic search index, the mapping also with the same name "Gender" and let say I have 10 records in the elastic search.
If I hit the select query using the presto for elastic search and then we get the 10 records with the NULL values of "Gender" fields.

Case 2:
This is the table definition in case of elastic search.
{
"tableName": "dummy",
"schemaName": "es",
"host": "0.0.0.0",
"port": 0000,
"clusterName": "clustername",
"index": "dummy",
"type": "user",
"columns": [{
"name": "gender",
"type": "varchar",
"jsonPath": "gender",
"jsonType": "varchar",
"ordinaryPosition": "0"
}]
}
In Elastic search index, the mapping also with the same name "gender" and let say I have 10 records in the elastic search.
In case 2, we get the 10 number of records with the valid values for gender fields.

Now my question is,
If we are giving the value in the camel case, pascal case or in the uppercase of the name or in jsonPath fields then why the presto is showing the NULL values for that particular fields.

please advice.

@jwfcps
Copy link

jwfcps commented Apr 17, 2019

The code forces field names to lowercase, the diff I posted in #12642 includes fixing this, as well as "fixing" nested fields.

@zhenxiao
Copy link
Collaborator

zhenxiao commented May 9, 2019

@shubham-agarwal-ngi sorry for the late reply
Seems we have a bug in resolving Elasticsearch upper case field names.
@jwfcps could you please submit separate PRs, or separate commits? One for resolving upper case field names, another one for fixing nested fields? I will take a review

@stale
Copy link

stale bot commented Jun 2, 2021

This issue has been automatically marked as stale because it has not had any activity in the last 2 years. If you feel that this issue is important, just comment and the stale tag will be removed; otherwise it will be closed in 7 days. This is an attempt to ensure that our open issues remain valuable and relevant so that we can keep track of what needs to be done and prioritize the right things.

@stale stale bot added the stale label Jun 2, 2021
@stale stale bot closed this as completed Jun 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants