duplicate rows/data showing up in aws_route53_record queries

[tinder-tder]

Describe the bug
There seems to be a bug when doing lookup with an 'in' clause on the 'aws_route53_record' table. Duplicate rows are shown when more than 1 qualifier is listed.

Steampipe version (steampipe -v)
v0.13.6

Plugin version (steampipe plugin list)
v0.57.0

To reproduce

> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name in ('back.example.com.','www.example.com.')
+--------------------+-------+-----+-----------+
| name               | type  | ttl | failover  |
+--------------------+-------+-----+-----------+
| www.example.com.  | CNAME | 5   | <null>    |
| www.example.com.  | CNAME | 5   | <null>    |
| back.example.com. | A     | 10  | PRIMARY   |
| back.example.com. | A     | 10  | SECONDARY |
| www.example.com.  | CNAME | 5   | <null>    |
| www.example.com.  | CNAME | 5   | <null>    |
+--------------------+-------+-----+-----------+
> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name in ('pr.example.com.','www.example.com.')
+-------------------+-------+-----+-----------+
| name              | type  | ttl | failover  |
+-------------------+-------+-----+-----------+
| www.example.com. | CNAME | 5   | <null>    |
| www.example.com. | CNAME | 5   | <null>    |
| pr.example.com.  | A     | 10  | PRIMARY   |
| pr.example.com.  | A     | 10  | SECONDARY |
| www.example.com. | CNAME | 5   | <null>    |
| www.example.com. | CNAME | 5   | <null>    |
+-------------------+-------+-----+-----------+
> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name in ('www.example.com.','back.example.com.')
+--------------------+-------+-----+-----------+
| name               | type  | ttl | failover  |
+--------------------+-------+-----+-----------+
| www.example.com.  | CNAME | 5   | <null>    |
| www.example.com.  | CNAME | 5   | <null>    |
| back.example.com. | A     | 10  | PRIMARY   |
| back.example.com. | A     | 10  | SECONDARY |
| www.example.com.  | CNAME | 5   | <null>    |
| www.example.com.  | CNAME | 5   | <null>    |
+--------------------+-------+-----+-----------+
> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name in ('www.example.com.','pr.example.com.')
+-------------------+-------+-----+-----------+
| name              | type  | ttl | failover  |
+-------------------+-------+-----+-----------+
| www.example.com. | CNAME | 5   | <null>    |
| www.example.com. | CNAME | 5   | <null>    |
| pr.example.com.  | A     | 10  | PRIMARY   |
| pr.example.com.  | A     | 10  | SECONDARY |
| www.example.com. | CNAME | 5   | <null>    |
| www.example.com. | CNAME | 5   | <null>    |
+-------------------+-------+-----+-----------+
> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name in ('back.example.com.','pr.example.com.')
+--------------------+------+-----+-----------+
| name               | type | ttl | failover  |
+--------------------+------+-----+-----------+
| back.example.com. | A    | 10  | PRIMARY   |
| back.example.com. | A    | 10  | SECONDARY |
| pr.example.com.   | A    | 10  | PRIMARY   |
| pr.example.com.   | A    | 10  | SECONDARY |
| back.example.com. | A    | 10  | PRIMARY   |
| back.example.com. | A    | 10  | SECONDARY |
+--------------------+------+-----+-----------+
> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name in ('back.example.com.','pr.example.com.','www.example.com.')
+--------------------+-------+-----+-----------+
| name               | type  | ttl | failover  |
+--------------------+-------+-----+-----------+
| www.example.com.  | CNAME | 5   | <null>    |
| back.example.com. | A     | 10  | PRIMARY   |
| back.example.com. | A     | 10  | SECONDARY |
| www.example.com.  | CNAME | 5   | <null>    |
| www.example.com.  | CNAME | 5   | <null>    |
| www.example.com.  | CNAME | 5   | <null>    |
| pr.example.com.   | A     | 10  | PRIMARY   |
| www.example.com.  | CNAME | 5   | <null>    |
| pr.example.com.   | A     | 10  | SECONDARY |
| www.example.com.  | CNAME | 5   | <null>    |
| back.example.com. | A     | 10  | PRIMARY   |
| back.example.com. | A     | 10  | SECONDARY |
+--------------------+-------+-----+-----------+
> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name in ('back.example.com.')
+--------------------+------+-----+-----------+
| name               | type | ttl | failover  |
+--------------------+------+-----+-----------+
| back.example.com. | A    | 10  | PRIMARY   |
| back.example.com. | A    | 10  | SECONDARY |
+--------------------+------+-----+-----------+
> select name,type,ttl,failover from aws_prod.aws_route53_record where zone_id = 'SOMEZONEID' and name = 'www.example.com.'
+-------------------+-------+-----+----------+
| name              | type  | ttl | failover |
+-------------------+-------+-----+----------+
| www.example.com. | CNAME | 5   | <null>   |
| www.example.com. | CNAME | 5   | <null>   |
+-------------------+-------+-----+----------+
> 

Expected behavior
rows should not be duplicated

[rajlearner17]

@tinder-tder Appreciate continuous feedback! Thank you

It is disappointing to see this happening; we could replicate the issue at our end, we are looking into it.

[bigdatasourav]

Hey @tinder-tder, The API is not working properly due to which we are facing the duplicate values and optional column support issue, we have raised an issue in AWS GO SDK for that. We are also engaging AWS with a support request.

While we are waiting to get a response from the above support requests, after testing and working to provide an alternative solution, for the time being, we have this draft [PR] (#972) fix. Could you please try it out and let us know your feedback.

[tinder-tder]

@bigdatasourav I just tested the PR and it does look like its working as expected with no dupes, thanks!