AWS - support sourcing credentials from a different profile

[AndyTitu]

Platform or tool

AWS

Desired or expected behavior

AWS shell plugin should allow for sourcing a certain profile's credentials from another profile.

Current behavior

Currently AWS shell plugin supports long lived credential authentication, MFA and Assume Role support with temporary credentials authentication.

Relevant log output

No response

op CLI version

No response

[fillz-noh]

I need to manage multiple environments using Terraform, so this feature is crucial. The final work will be executed through CI/CD; however, I'd like to frequently check the progress in the terminal while working.

[AndyTitu]

Thank you to all who left feedback on this issue!

I need to manage multiple environments using Terraform, so this feature is crucial. The final work will be executed through CI/CD; however, I'd like to frequently check the progress in the terminal while working.

Since #180, commands like aws s3 ls --profile dev or export AWS_PROFILE=dev; aws s3 ls are supported by the AWS shell plugin, and all the assumed profile information will be extracted from your already existing .aws/config file (or $AWS_CONFIG_FILE).

From my understanding you are requesting support for the following scenario:
If you specify source_profile = production-profile in the config file under profile dev then profile dev should use the same root credentials as production-profile.
This could be achieved today without the source_profile = production-profile directive inside the config file, by doing op plugin init aws and selecting the 1Password item that stores production-profile's root credentials. Then when you are assuming your dev profile with aws s3 ls --profile dev you will be using your production-profile root creds.

Can you help me better discover your use case, and in which scenarios the explained solution above would fall short?

[fillz-noh]

@AndyTitu Thanks for your reply.I commented because the error message was directing me to this issue.
I removed the source_profile as per your message and the profile switch was successful. Thanks for teaching me how to do this.

However, the call from terraform failed and I would like to know if there is a proper place for this issue.

[AndyTitu]

@fillz-noh Sure, glad I could help!

@AndyTitu Thanks for your reply.I commented because the error message was directing me to this issue. I removed the source_profile as per your message and the profile switch was successful. Thanks for teaching me how to do this.

However, the call from terraform failed and I would like to know if there is a proper place for this issue.

Here is the issue for supporting the terraform use case, for aws but in general for any other tool.
#140

Also, I've just opened a PR about this: #256

[jarshwah]

This could be achieved today without the source_profile = production-profile directive inside the config file

This works for the aws cli but breaks other tools like sops since it's no longer able to determine the root credentials/keys.

[AndyTitu]

@jarshwah , @fillz-noh and everyone else involved in this issue: We have just released our new beta solution for the Terraform shell plugin. This allows AWS to work with Terraform out of the box, and also comes up with a general solution for authenticating terraform providers. Would be awesome if you could give it a try and let us know how/if this simplifies your workflows by a bunch.

[jandeschuttere]

Hi

I don't know if I'm missing something else but I've been trying the beta (2.19.0-beta.01) in combination with an EKS, having set up just the op cli command and the aws plugin.

However, it's impossible for interacting with EKS using the current solution, it still seems to be requiring the source_profile. The full error output is:

Partial credentials found in assume-role, missing: source_profile or credential_source

Am I missing something or is an additional fix going to be required?

[AndyTitu]

However, it's impossible for interacting with EKS using the current solution, it still seems to be requiring the source_profile. The full error output is:

Partial credentials found in assume-role, missing: source_profile or credential_source

@jandeschuttere thanks for sharing your use case! The first thing that comes to mind when seeing that error is that the shell-plugin is not actually wrapping the tools you are using to manage EKS. Can you please share the exact command(s)/script that you are running?

[jandeschuttere]

This happens with any of the commands interacting via kubectl because it's trying to authenticate first, the authentication process is unable to complete successfully.

When I looked closer into it it seems that the introduced shell-plugin is actually setting up an alias to wrap the 1pwd command, yet the kubectl config for EKS introduces an auth provider type "exec" (see https://kubernetes.io/docs/reference/access-authn-authz/authentication/#configuration)

  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: aws
      args:
        - --region
        - ...

The alias is not able to override this behaviour. Perhaps a different config in the ~/.kube/config could be set up so that it can use the alias or introduce the wrapping logic the same way as that alias is doing but I was at least not able to easily do so.

[tr0mpa]

@jandeschuttere Hey solved it by wrapping op command to kubeconfig like this:

  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: op
      args:
      - plugin
      - run
      - --
      - aws
      - --region
      - ......