Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for credential_process from profiles #1356

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions aws/rust-runtime/aws-config/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ zeroize = "1"
bytes = "1.1.0"
http = "0.2.4"
tower = { version = "0.4.8" }
shlex = { version = "1.1.0" }
jszwedko marked this conversation as resolved.
Show resolved Hide resolved

[dev-dependencies]
futures-util = "0.3.16"
Expand Down
111 changes: 111 additions & 0 deletions aws/rust-runtime/aws-config/src/credential_process.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/

//! Credentials Provider for external process

use crate::json_credentials::{parse_json_credentials, JsonCredentials};
use aws_types::credentials::{future, CredentialsError, ProvideCredentials};
use aws_types::{credentials, Credentials};
use std::process::Command;

/// Credentials Provider
#[derive(Debug)]
pub struct CredentialProcessProvider {
credential_process: String,
}

impl ProvideCredentials for CredentialProcessProvider {
fn provide_credentials<'a>(&'a self) -> future::ProvideCredentials<'a>
where
Self: 'a,
{
future::ProvideCredentials::new(self.credentials())
}
}

impl CredentialProcessProvider {
/// Create new [`CredentialProcessProvider`]
pub fn new(credential_process: String) -> Self {
Self { credential_process }
}
jszwedko marked this conversation as resolved.
Show resolved Hide resolved

async fn credentials(&self) -> credentials::Result {
tracing::debug!(command = %self.credential_process, "loading credentials from external process");

let mut command = parse_command_str(&self.credential_process)?;

tracing::debug!(command = ?command, "parsed command");

let output = command.output().map_err(|e| {
CredentialsError::provider_error(format!(
"Error retrieving credentials from external process: {}",
e
))
})?;

tracing::debug!(command = ?command, status = ?output.status, "executed command");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The command binary name can be logged at debug, but the arguments must be logged at trace since they can have sensitive information.


if !output.status.success() {
return Err(CredentialsError::provider_error(format!(
"Error retrieving credentials from external process: exited with code: {}",
output.status
)));
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of a non-zero exit status, the contents of stderr should be included in the returned error.


let output = std::str::from_utf8(&output.stdout).map_err(|e| {
CredentialsError::provider_error(format!(
"Error retrieving credentials from external process: could not decode output as UTF-8: {}",
e
))
})?;

match parse_json_credentials(&output) {
Ok(JsonCredentials::RefreshableCredentials {
access_key_id,
secret_access_key,
session_token,
expiration,
..
}) => Ok(Credentials::new(
access_key_id,
secret_access_key,
Some(session_token.to_string()),
expiration.into(),
"IMDSv2",
)),
Ok(JsonCredentials::Error { code, message }) => {
Err(CredentialsError::provider_error(format!(
"Error retrieving credentials from external process: {} {}",
code, message
)))
}
Err(invalid) => Err(CredentialsError::provider_error(format!(
"Error retrieving credentials from external process, could not parse response: {}",
invalid
))),
}
}
}

fn parse_command_str(s: &str) -> Result<Command, CredentialsError> {
let args = shlex::split(s).ok_or_else(|| {
CredentialsError::provider_error(
"Error retrieving credentials from external process: could not parse provided value as command",
)
})?;
let mut iter = args.iter();
let mut command = Command::new(iter.next().ok_or_else(|| {
CredentialsError::provider_error(
"Error retrieving credentials from external process: provided command empty",
)
})?);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should spawn this into a task...the trouble is that we'd need to then abstract over spawning. I'll think about it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I'm open to thoughts here. I actually up refactoring this to take the Java SDK's approach of just passing the whole string through sh -c / cmd.exe /C in 14ad7a9 to avoid needing to do any parsing in the SDK itself.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should spawn this into a task...the trouble is that we'd need to then abstract over spawning. I'll think about it.

command.args(iter);
Ok(command)
}

#[cfg(test)]
mod test {
// TODO
jszwedko marked this conversation as resolved.
Show resolved Hide resolved
}
2 changes: 2 additions & 0 deletions aws/rust-runtime/aws-config/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,8 @@ pub mod sso;

pub mod connector;

pub mod credential_process;

pub(crate) mod parsing;

// Re-export types from smithy-types
Expand Down
9 changes: 9 additions & 0 deletions aws/rust-runtime/aws-config/src/profile/credentials.rs
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,14 @@ impl ProvideCredentials for ProfileFileCredentialsProvider {
///
/// Other more complex configurations are possible, consult `test-data/assume-role-tests.json`.
///
/// ### Credentials loaded from an external process
/// ```ini
/// [default]
/// credential_process = /opt/bin/awscreds-custom --username helen
/// ```
///
/// An external process can be used to provide credentials.
///
/// ### Loading Credentials from SSO
/// ```ini
/// [default]
Expand Down Expand Up @@ -493,4 +501,5 @@ mod test {
make_test!(retry_on_error);
make_test!(invalid_config);
make_test!(region_override);
make_test!(credentials_process);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will implement.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll need to figure out how to mock this—probably some sort of "fake command" interface will need to get added to our existing OS shim

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, should validate that the credentials process can be used as the source profile for assume role

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I ended up using echo for the test in which should be supported on all platforms, but happy to try to sub in command mocking if you prefer.

Ref: a787b61

}
4 changes: 4 additions & 0 deletions aws/rust-runtime/aws-config/src/profile/credentials/exec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ use aws_types::region::Region;

use super::repr::{self, BaseProvider};

use crate::credential_process::CredentialProcessProvider;
use crate::profile::credentials::ProfileFileError;
use crate::provider_config::ProviderConfig;
use crate::sso::{SsoConfig, SsoCredentialsProvider};
Expand Down Expand Up @@ -100,6 +101,9 @@ impl ProviderChain {
})?
}
BaseProvider::AccessKey(key) => Arc::new(key.clone()),
BaseProvider::CredentialProcess(credential_process) => Arc::new(
CredentialProcessProvider::new(credential_process.to_string()),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if new returned a result, we could return a profile file error here which is probably the right thing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mentioned this in #1356 (comment) but I actually ended up pulling out the command parsing logic, following suit with the Java SDK of just passing through the commands to sh -c / cmd.exe /C.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if new returned a result, we could return a profile file error here which is probably the right thing

),
BaseProvider::WebIdentityTokenRole {
role_arn,
web_identity_token_file,
Expand Down
34 changes: 33 additions & 1 deletion aws/rust-runtime/aws-config/src/profile/credentials/repr.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,13 @@ pub enum BaseProvider<'a> {
sso_role_name: &'a str,
sso_start_url: &'a str,
},

/// A profile that specifies a `credential_process`
/// ```ini
/// [profile assume-role]
/// credential_process = /opt/bin/awscreds-custom --username helen
/// ```
CredentialProcess(&'a str),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The arguments to the credentials process are allowed to contain sensitive information, so we need to be careful not to log them by default. Thus, the Debug implementation for this enum needs to do some amount of redaction on this value. I recommend creating a new-type around the &'a str, and manually implementing Debug for it such that only the part before the first space is output, followed by something like <args redacted> if there was more to the string.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for this—we should make something like:

struct<'a> Sensitive(Cow<'a, str>)
impl AsDeref<str> for Sensitive ...

we can then use it here and in the provider to avoid needing to manually do a full debug impl

}

/// A profile that specifies a role to assume
Expand Down Expand Up @@ -209,13 +216,19 @@ mod static_credentials {
pub const AWS_SECRET_ACCESS_KEY: &str = "aws_secret_access_key";
pub const AWS_SESSION_TOKEN: &str = "aws_session_token";
}

mod credential_process {
pub const CREDENTIAL_PROCESS: &str = "credential_process";
}
const PROVIDER_NAME: &str = "ProfileFile";

fn base_provider(profile: &Profile) -> Result<BaseProvider, ProfileFileError> {
dbg!("checking base provider", &profile);
// the profile must define either a `CredentialsSource` or a concrete set of access keys
match profile.get(role::CREDENTIAL_SOURCE) {
Some(source) => Ok(BaseProvider::NamedSource(source)),
None => web_identity_token_from_profile(profile)
None => credential_process_from_profile(profile)
.or_else(|| web_identity_token_from_profile(profile))
.or_else(|| sso_from_profile(profile))
.unwrap_or_else(|| Ok(BaseProvider::AccessKey(static_creds_from_profile(profile)?))),
}
jszwedko marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -363,6 +376,21 @@ fn static_creds_from_profile(profile: &Profile) -> Result<Credentials, ProfileFi
))
}

/// Load credentials from `credential_process`
///
/// Example:
/// ```ini
/// [profile B]
/// credential_process = /opt/bin/awscreds-custom --username helen
/// ```
fn credential_process_from_profile<'a>(
profile: &'a Profile,
) -> Option<Result<BaseProvider, ProfileFileError>> {
profile
.get(credential_process::CREDENTIAL_PROCESS)
.map(|credential_process| Ok(BaseProvider::CredentialProcess(credential_process)))
}

#[cfg(test)]
mod tests {
use crate::profile::credentials::repr::{resolve_chain, BaseProvider, ProfileChain};
Expand Down Expand Up @@ -427,6 +455,9 @@ mod tests {
secret_access_key: creds.secret_access_key().into(),
session_token: creds.session_token().map(|tok| tok.to_string()),
}),
BaseProvider::CredentialProcess(credential_process) => {
output.push(Provider::CredentialProcess(credential_process.into()))
}
BaseProvider::WebIdentityTokenRole {
role_arn,
web_identity_token_file,
Expand Down Expand Up @@ -477,6 +508,7 @@ mod tests {
session_token: Option<String>,
},
NamedSource(String),
CredentialProcess(String),
WebIdentityToken {
role_arn: String,
web_identity_token_file: String,
Expand Down