Skip to content

Add sts, imds, and http credential provider packages - #72

Open
arandito wants to merge 3 commits into
aws:developfrom
arandito:network-credential-providers
Open

Add sts, imds, and http credential provider packages#72
arandito wants to merge 3 commits into
aws:developfrom
arandito:network-credential-providers

Conversation

@arandito

@arandito arandito commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Description

This PR introduces three new packages that register network credential providers into the SDK's modular AWS credential chain:

  • aws-credentials-imds - EC2 Instance Metadata Service (IMDSv2) credential resolver and Ec2InstanceMetadata chain provider
  • aws-credentials-http - container HTTP credential resolver (ECS/EKS) and EcsContainer chain provider
  • aws-credentials-sts - STS AssumeRole resolvers and ProfileAssumeRole chain provider

IMDS and HTTP are direct ports of the existing smithy_aws_core.identity.imds and container resolvers, which will be deprecated. The credential resolution behavior is mostly identical. Changes are limited to:

  • Module reorganization into standalone packages (client.py/resolvers.py/providers.py split).
  • Naming cleanup (e.g. ContainerMetadataClient becomes HttpCredentialsClient, EC2Metadata becomes IMDSClient, etc).
  • ContainerCredentialsConfig is flattened into class constructor arguments to make user interface cleaner.
  • New provider/chain wiring that sources configuration from the environment and shared config profile. The original resolvers had no chain integration.
  • Config-validation errors now raise package-specific *ConfigurationError(SmithyError) types instead of bare ValueError, reserving SmithyIdentityError for resolution-time failures.

STS is new. It ships two resolvers that separate the AssumeRole call itself from the profile configuration that feeds it:

  • AssumeRoleCredentialsResolver performs the STS AssumeRole call. It takes an explicit role_arn and a source_resolver that provides the credentials used to make the call. This is the low-level resolver, with no knowledge of profiles, and can be used standalone outside the chain.
  • ProfileAssumeRoleCredentialsResolver is the profile-driven resolver used by the chain. It reads a profile from the shared config file and resolves the credential source from that profile's source_profile (chaining to another profile, including nested role chains that terminate in static credentials) or credential_source (delegating to the Environment, EcsContainer, or Ec2InstanceMetadata provider). It then hands that source to an AssumeRoleCredentialsResolver to perform the call.

Splitting the two keeps the STS call logic isolated and reusable. AssumeRoleCredentialsResolver can be constructed directly with any source resolver, while ProfileAssumeRoleCredentialsResolver owns only the profile parsing and source resolution.

Important

The underlying STS client used for Assume Role calls in imported from the aws-sdk-sts client. This means that aws-credentials-sts has a required dependency on aws-sdk-sts. This does not cause a dependency cycle as the aws-sdk-sts client will never have a required dependency on aws-credentials-sts and instead is opt in. If artifact size for aws-credentials-sts becomes a concern due to the full import of the STS client package, we can explore a slim client implementation. For now, this is the most maintainable solution.

Usage

Installing any of these packages auto-registers its provider into the SDK's credential chain via entry points. Each resolver can also be used directly:

from aws_credentials_imds import IMDSCredentialsResolver
from aws_credentials_http import ContainerCredentialsResolver
from aws_credentials_sts import AssumeRoleCredentialsResolver

# IMDS / container: HTTP client defaults to AIOHTTPClient
resolver = IMDSCredentialsResolver()
credentials = await resolver.get_identity(properties={})

# STS AssumeRole with any source resolver
resolver = AssumeRoleCredentialsResolver(
    source_resolver=source_resolver,
    role_arn="arn:aws:iam::123456789012:role/example",
)
credentials = await resolver.get_identity(properties={})

Testing

  • Ported existing test suites for IMDS and Container, with slight modification for new interfaces.
  • Added unit test coverage for new STS credential resolvers and for the three new chain providers.
  • Tested all three packages standalone and integrated in the new IdentityChain with live service calls
    • Configuration:
      • STS: Set up iam roles to assume on AWS Console
      • IMDS: Set up EC2 instance with an IMDS instance profile + role.
      • HTTP: Used amazon-ecs-local-container-endpoints to host container endpoints locally.
    • Confirmed all providers are configured automatically with ~/.aws/config profiles and environment variables.
    • Confirmed chain order is preserved when multiple sources are configured.
    • Confirmed IMDS and HTTP work as credential sources for STS providers

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@arandito
arandito requested a review from a team as a code owner July 31, 2026 15:02
retries: int = _DEFAULT_RETRIES,
):
self._http_client = http_client
self._timeout = timeout

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self._timeout seems never used. Should it be applied to the request, or should the parameter be dropped? Or do you want to to add a TODO?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why this wasn't applied to the request in the first place.

I confirmed we should apply the default timeout by looking at botocore. However, we need to apply the value to both the read timeout and connect timeout. Looks like HTTPRequestConfiguration only supports read_timeout for now which may be why it wasn't used.

I applied the value as the read_timeout in the request configuration and added a TODO to apply the value to connect_timeout once its supported.

fields.set_field(Field(name="Authorization", values=[auth_token]))
elif self.ENV_VAR_AUTH_TOKEN in os.environ:
auth_token = os.environ[self.ENV_VAR_AUTH_TOKEN]
fields.set_field(Field(name="Authorization", values=[auth_token]))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The token goes into the Authorization header unvalidated, while botocore rejects \r and \n:

def _build_headers(self):
    auth_token = None
    if self.ENV_VAR_AUTH_TOKEN_FILE in self._environ:
        auth_token_file_path = self._environ[self.ENV_VAR_AUTH_TOKEN_FILE]
        with open(auth_token_file_path) as token_file:
            auth_token = token_file.read()
    elif self.ENV_VAR_AUTH_TOKEN in self._environ:
        auth_token = self._environ[self.ENV_VAR_AUTH_TOKEN]
    if auth_token is not None:
        self._validate_auth_token(auth_token)
        return {'Authorization': auth_token}

def _validate_auth_token(self, auth_token):
    if "\r" in auth_token or "\n" in auth_token:
        raise ValueError("Auth token value is not a legal header value")

Should we add some checks as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Addressed in 524a395.

Comment thread packages/aws-credentials-http/src/aws_credentials_http/client.py
Comment thread packages/aws-credentials-http/tests/unit/test_client.py Outdated

@alexgromero alexgromero left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One callout across all three new credential packages: they add async tests, but the package local pyproject.toml files don't declare the test setup needed to run them standalone.

This worked in smithy-python because it already had that wiring, but here uv sync && uv run pytest does not install an async pytest plugin. Even with pytest-asyncio, the async tests still need either @pytest.mark.asyncio or asyncio_mode = "auto".

Could we add a shared test dependency group and pytest async configuration to each package so their test suites can run on their own?

Comment thread packages/aws-credentials-imds/src/aws_credentials_imds/client.py
Comment thread packages/aws-credentials-sts/src/aws_credentials_sts/resolvers.py
@arandito

arandito commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@alexgromero

Could we add a shared test dependency group and pytest async configuration to each package so their test suites can run on their own

I'm actually going to follow up with a PR that introduces CI + testing infrastructure using uv workspaces. This is similar to Smithy Python's setup. We will need this for our internal infrastructure to test these packages as part of our release process. For now, I'd prefer we keep the pytest configurations and dependencies centralized, instead of per-package, to avoid drift.

@Alan4506

Alan4506 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

The new commit LGTM overall.

Regarding #72 (comment), we should have a follow-up though: in smithy-http, AIOHTTPClient.send currently accepts request_config but never consumes it. We'll need follow-ups in smithy-http to actually wire read_timeout into the request and also add connect_timeout support.

Comment on lines +84 to +86
async def invalidate(self) -> None:
"""Discard cached credentials so the next resolution re-queries the endpoint."""
self._credentials = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make this a no-op for now? Clearing the cached credentials works against static stability. The refresh behavior we're moving toward expires them only when the rejected identity matches, rather than discarding them outright.

Since we can't implement the identity-match behavior yet (it needs the rejected identity passed in), a no-op seems safer than carrying over the clear. What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants