Powerpipe dashboard displaying incomplete results

[aditivarade]

Hi,

I utilized your Steampipe Docker image tailored for AWS Cloud to generate several reports. Below is a screenshot showcasing the outcomes derived from the HTML report.

Command: docker run --rm -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} -e AWS_REGION=${AWS_REGION}
-e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} -e AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}
--name steampipe-compliance --mount type=bind,source="${pwd}",target=/output
steampipe-aws-compliance check all --export /output/AWS_Report.html

I've delved into the functionalities of steampipe-mod-aws-compliance and went through the process of manually installing Steampipe and Powerpipe.

When executing the "powerpipe benchmark run aws_compliance.benchmark.all_controls" command, it indicates 158 errors detected. However, I'm unable to view any additional results or the successfully passed benchmarks (OK) on the dashboard, unlike when I conduct the analysis using the docker image.


Furthermore, I haven't observed any details regarding the resources upon which the errors have been detected.

Steps I have followed to set up steampipe with powerpipe

1. I followed the documentation provided for steampipe-mod-aws-compliance
   Below are the versions installed on my Ubuntu VM
   Powerpipe version (powerpipe -v)
   Powerpipe v0.1.3
   Steampipe version (steampipe -v)
   Steampipe v0.22.1
2. For AWS credentails I tried both the approaches
   a) Exported environment variables on my VM
   b) Created the aws credentials file
3. Ran the steampipe and powerpipe commands
   steampipe service start
   powerpipe server
4. Ran the powerpipe benchmark command
   powerpipe benchmark run aws_compliance.benchmark.all_controls

For your informartion I have used the same AWS user and its credentials in both the cases(steampipe docker image and steampipe with powerpipe)

Expected behavior
The PowerPipe dashboard should showcase comprehensive outcomes, including thorough details regarding the resources it has scanned through.

[e-gineer]

The error in the dashboard indicates that Steampipe cannot find / resolve your AWS credentials. Are you sure it's picking up the steampipe configuration properly?

[aditivarade]

Hi @e-gineer,

I tried the following steps for setting up the credentials

1. Exported AWS environment variables on my VM
2. Created the aws credentials file
3. In the aws.spc file I mentioned the profile specified in the aws credentials file
   connection "aws_account_a" {
   plugin = "aws"
   profile = "account_extensure"
   }
4. And in the aws credentials file I have mentioned the following details
   [account_extensure]
   aws_access_key_id = ${aws_access_key_id}
   aws_secret_access_key = ${aws_secret_access_key}
   aws_session_token = ${aws_session_token}
5. Created a user called steampipe and ran the following commands using the created steampipe user
   steampipe service start
   powerpipe benchmark run aws_compliance.benchmark.all_controls

Please let me know if I am missing anything else.

[e-gineer]

How is docker involved in your setup? You mention that at the start, but not in the latest info.

Are you able to run basic queries against the steampipe instance you have created? (i.e. before using powerpipe with it)

[aditivarade]

Earlier, I utilized Docker to explore Steampipe, but currently I have install both Steampipe and Powerpipe directly on my local virtual machine (VM).
Answering to you question, I'm currently unable to execute queries. To troubleshoot, I verified the functionality of my AWS credentials by employing the command "aws sts get-caller-identity" and configured them for Steampipe as per the steps I previously outlined. However, I am not sure why Steampipe fails to recognize these credentials.
Do let me know if I have missed anything as per the steps I had listed earlier.

[aditivarade]

Hi @e-gineer,

Any update?

[pskrbasu]

Hi @aditivarade,

I'm currently unable to execute queries

What is the error you're seeing when executing queries?

[aditivarade]

Hi @pskrbasu,

I tried to execute a simple query("select name from aws_iam_role"). Below is the error I get inspite of configuring the credentials correctly.

Error: aws_account_a: operation error IAM: ListRoles, exceeded maximum number of attempts, 9, failed to sign request: failed to retrieve credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, exceeded maximum number of attempts, 3, request send failed, Get "http://169.254.169.254/latest/meta-data/iam/security-credentials/": net/http: timeout awaiting response headers (SQLSTATE HV000)

[kaidaguerre]

@aditivarade it is worth ensuring there are no stray processes left over from previous failed runs. You can list staempipe processes with ps aux | grep steampipe, and kil any stray processes with pkill -f steampipe

Please could you give that a go then retry.

[aditivarade]

@kaidaguerre tried listing the processes and killing them, but no luck.

[kaidaguerre]

So to be clear - even a simple steampipe query is failing:
steampipe query "select * from aws_acount" ?

[aditivarade]

@kaidaguerre - The issue is resolved. Appreciate the responses.

[IgnatG]

@aditivarade can I check how this was resolved? I'm having similar issues