You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I can add an Athena column (see #1448) to normalize my data structure, but there appears to be an odd limit of the number of attributes added and then no data will come back.
To Reproduce
Steps to reproduce the behavior:
Create a complex struct under "Athena"
Run a query SELECT athena.organization_id, athena
FROM "lambda:dev-elasticsearch"."esdev"."invoices" as esInvoice
LIMIT 1
No Data returned
remove one column from the items struct and data will show up.
You add that column back in and remove one that came in and data will show up, indicating that it's not the columns but the number of columns that is the issue.
Expected behavior
That all columns would come back and the data associated.
Connector Details (please complete the following information):
Version: 2023.30.1
Name elasticsearch connector.
Athena Query IDs
Additional context
The schema for the Athena column
[
{
"Name": "athena",
"Type": "struct<id:string,dob:string,organization_id:string,account_id:string,first_name:string,last_name:string,mobile_number:array<struct<phone_number:string,label:string>>,email:array<structemail:string,label:string>,oldest_unpaid_invoice:struct<due_date:string,invoice_id:string>,last_paper_statement:string,provider:struct<provider_id:bigint,name:string,account_id:string,phone_number:string,address:string,address2:string,state:string,city:string,postcode:string,stat:string,create_date:string,deleted_date:string,user_id:string,organization_id:string,type:string,remit_address2:string,remit_city:string,remit_name:string,remit_postcode:string,remit_state:string>,invoice:struct<invoice_id:string,due_date:string,paid_date:string,created_date:string,invoice_date:string,invoice_number:string,stat:string,amount:double,discount_amount:double,notes:string,integration_data:struct<original_billed_amount:double>,items:array<struct<item_id:string,quantity:int>>>>"
}
]
If I move id, dob as a sibling that will allow me to run queries and get data. If 1448 can be solved I think this issue will go completely away and reduce the amount of data I have to store in ElasticSearch.
I can also just remove id and dob and it will start having data show up.
The text was updated successfully, but these errors were encountered:
Describe the bug
I can add an Athena column (see #1448) to normalize my data structure, but there appears to be an odd limit of the number of attributes added and then no data will come back.
To Reproduce
Steps to reproduce the behavior:
FROM "lambda:dev-elasticsearch"."esdev"."invoices" as esInvoice
LIMIT 1
remove one column from the items struct and data will show up.
You add that column back in and remove one that came in and data will show up, indicating that it's not the columns but the number of columns that is the issue.
Expected behavior
That all columns would come back and the data associated.
Screenshots / Exceptions / Errors
Athena structure.
Connector Details (please complete the following information):
Additional context
The schema for the Athena column
[
{
"Name": "athena",
"Type": "struct<id:string,dob:string,organization_id:string,account_id:string,first_name:string,last_name:string,mobile_number:array<struct<phone_number:string,label:string>>,email:array<structemail:string,label:string>,oldest_unpaid_invoice:struct<due_date:string,invoice_id:string>,last_paper_statement:string,provider:struct<provider_id:bigint,name:string,account_id:string,phone_number:string,address:string,address2:string,state:string,city:string,postcode:string,stat:string,create_date:string,deleted_date:string,user_id:string,organization_id:string,type:string,remit_address2:string,remit_city:string,remit_name:string,remit_postcode:string,remit_state:string>,invoice:struct<invoice_id:string,due_date:string,paid_date:string,created_date:string,invoice_date:string,invoice_number:string,stat:string,amount:double,discount_amount:double,notes:string,integration_data:struct<original_billed_amount:double>,items:array<struct<item_id:string,quantity:int>>>>"
}
]
If I move id, dob as a sibling that will allow me to run queries and get data. If 1448 can be solved I think this issue will go completely away and reduce the amount of data I have to store in ElasticSearch.
I can also just remove id and dob and it will start having data show up.
The text was updated successfully, but these errors were encountered: