Seemingly non-deterministic behaviour, query fails sometimes, works other times

[Brendonk13]

Describe the bug
A clear and concise description of what the bug is.
Query initially gives (SQLSTATE HV000) error, then I'll remove a random WHERE clause, run that successfully, then I go back to run the original query and it works all of a sudden. Did not change my aws config or anything also, the queries ran in the image below were from one session and entered back to back.

Steampipe version (steampipe -v)
steampipe version 0.14.4

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).
$ steampipe query

> select record.name, record.type, records, record.account_id
from
aws_route53_record as record,
jsonb_array_elements_text(record.records) as records
where record.zone_id = 'XXXXXXXXX'
and record.type in ('CNAME', 'A')

Error Message:
status code: 400, request id: 054c56d0-ef4f-4883-b9ac-86e9c16ce9c5 (SQLSTATE HV000)

First command initially fails, then works (bottom query) when I run it again after some other queries.

Expected behavior
A clear and concise description of what you expected to happen.

The query to have worked both times

Additional context
Add any other context about the problem here.

Failing under single AWS account with 1 matching record.zone_id

logs:

  71   │ 2022-06-05 18:07:40.346 UTC [ERROR] steampipe-plugin-aws.plugin: [ERROR] 1654452460993: streamRows error chan select: InvalidInput:
  72   │ 2022-06-05 18:07:40.346 UTC [WARN]  steampipe-plugin-aws.plugin: [WARN]  1654452460993: Execute call failed - cancelling pending item in cachequery: 

  73   │ 2022-06-05 18:07:40.359 UTC [ERROR] steampipe-plugin-aws.plugin: [ERROR] 1654452460993: error chan select: InvalidInput:

OS: 5.12.19-1-MANJARO

[bigdatasourav]

@Brendonk13 Thank you for using Steampipe; apologies to see that you are facing this issue. We will look into it.

[bigdatasourav]

Hey @Brendonk13, The API is not working properly due to which we are facing the optional column support issue, we have raised an issue in AWS GO SDK for that. We are also engaging AWS with a support request.

While we are waiting to get a response from the above support requests, after testing and working to provide an alternative solution, for the time being, we have this draft [PR] (#972) fix. Could you please try it out and let us know your feedback.