Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moves to aws sdk v2. #2111

Merged
merged 2 commits into from Jun 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
36 changes: 18 additions & 18 deletions docs/src/main/asciidoc/spring-cloud-config.adoc
Expand Up @@ -340,17 +340,17 @@ This helper is not used with the JGit library, so a JGit CredentialProvider for
AWS CodeCommit URIs follow this pattern:

```bash
https//git-codecommit.${AWS_REGION}.amazonaws.com/v1/repos/${repo}.
https://git-codecommit.${AWS_REGION}.amazonaws.com/v1/repos/${repo}
```

If you provide a username and password with an AWS CodeCommit URI, they must be the https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html[AWS accessKeyId and secretAccessKey] that provide access to the repository.
If you do not specify a username and password, the accessKeyId and secretAccessKey are retrieved by using the https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html[AWS Default Credential Provider Chain].
If you do not specify a username and password, the accessKeyId and secretAccessKey are retrieved by using the https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/credentials.html[Default Credential Provider Chain].

If your Git URI matches the CodeCommit URI pattern (shown earlier), you must provide valid AWS credentials in the username and password or in one of the locations supported by the default credential provider chain.
AWS EC2 instances may use https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html[IAM Roles for EC2 Instances].

NOTE: The `aws-java-sdk-core` jar is an optional dependency.
If the `aws-java-sdk-core` jar is not on your classpath, the AWS Code Commit credential provider is not created, regardless of the git server URI.
NOTE: The `software.amazon.awssdk:auth` jar is an optional dependency.
If the `software.amazon.awssdk:auth` jar is not on your classpath, the AWS Code Commit credential provider is not created, regardless of the git server URI.

===== Authentication with Google Cloud Source

Expand Down Expand Up @@ -940,15 +940,15 @@ NOTE: When no profile is specified `default` will be used.
==== AWS S3 Backend

Spring Cloud Config Server supports AWS S3 as a backend for configuration properties.
You can enable this feature by adding a dependency to the link:https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-s3.html[AWS Java SDK For Amazon S3].
You can enable this feature by adding a dependency to the link:https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/examples-s3.html[AWS Java SDK For Amazon S3].

[source,xml,indent=0]
.pom.xml
----
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
</dependency>
</dependencies>
----
Expand All @@ -970,22 +970,22 @@ spring:

It is also possible to specify an AWS URL to link:https://aws.amazon.com/blogs/developer/using-new-regions-and-endpoints/[override the standard endpoint] of your S3 service with `spring.cloud.config.server.awss3.endpoint`. This allows support for beta regions of S3, and other S3 compatible storage APIs.

Credentials are found using the link:https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html[Default AWS Credential Provider Chain]. Versioned and encrypted buckets are supported without further configuration.
Credentials are found using the link:https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/credentials.html[Default Credential Provider Chain]. Versioned and encrypted buckets are supported without further configuration.

Configuration files are stored in your bucket as `{application}-{profile}.properties`, `{application}-{profile}.yml` or `{application}-{profile}.json`. An optional label can be provided to specify a directory path to the file.

NOTE: When no profile is specified `default` will be used.

==== AWS Parameter Store Backend

Spring Cloud Config Server supports AWS Parameter Store as a backend for configuration properties. You can enable this feature by adding a dependency to the link:https://github.com/aws/aws-sdk-java/tree/master/aws-java-sdk-ssm[AWS Java SDK for SSM].
Spring Cloud Config Server supports AWS Parameter Store as a backend for configuration properties. You can enable this feature by adding a dependency to the link:https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/examples-ssm.html[AWS Java SDK for SSM].

[source,xml,indent=0]
.pom.xml
----
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-ssm</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>ssm</artifactId>
</dependency>
----

Expand Down Expand Up @@ -1019,7 +1019,7 @@ The following table describes the AWS Parameter Store configuration properties.
|*region*
|no
|
|The region to be used by the AWS Parameter Store client. If it's not explicitly set, the SDK tries to determine the region to use by using the link:https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-region-selection.html#default-region-provider-chain[Default Region Provider Chain].
|The region to be used by the AWS Parameter Store client. If it's not explicitly set, the SDK tries to determine the region to use by using the link:https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/region-selection.html#default-region-provider-chain[Default Region Provider Chain].

|*endpoint*
|no
Expand Down Expand Up @@ -1058,7 +1058,7 @@ The following table describes the AWS Parameter Store configuration properties.

|===

AWS Parameter Store API credentials are determined using the link:https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html#credentials-default[Default Credential Provider Chain].
AWS Parameter Store API credentials are determined using the link:https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/credentials.html#credentials-default[Default Credential Provider Chain].
Versioned parameters are already supported with the default behaviour of returning the latest version.

[NOTE]
Expand All @@ -1072,14 +1072,14 @@ Versioned parameters are already supported with the default behaviour of returni
==== AWS Secrets Manager Backend

Spring Cloud Config Server supports link:https://aws.amazon.com/secrets-manager/[AWS Secrets Manager] as a backend for configuration properties.
You can enable this feature by adding a dependency to link:https://github.com/aws/aws-sdk-java/tree/master/aws-java-sdk-secretsmanager[AWS Java SDK for Secrets Manager].
You can enable this feature by adding a dependency to link:https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/examples-secretsmanager.html[AWS Java SDK for Secrets Manager].

[source,xml,indent=0]
.pom.xml
----
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-secretsmanager</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
</dependency>
----

Expand All @@ -1102,7 +1102,7 @@ spring:

----

AWS Secrets Manager API credentials are determined using link:https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html#credentials-default[Default Credential Provider Chain].
AWS Secrets Manager API credentials are determined using link:https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/credentials.html#credentials-default[Default Credential Provider Chain].

[NOTE]
====
Expand Down Expand Up @@ -1816,7 +1816,7 @@ For instance, you might want to align the config label with your branch but make

=== Specifying Multiple URLs for the Config Server

To ensure high availability when you have multiple instances of Config Server deployed and expect one or more instances to be unavailable or unable to honor requests from time to time (such as if the Git server is down), you can either specify multiple URLs (as a comma-separated list under the `spring.cloud.config.uri` property) or have all your instances register in a Service Registry like Eureka (if using Discovery-First Bootstrap mode).
To ensure high availability when you have multiple instances of Config Server deployed and expect one or more instances to be unavailable or unable to honor requests from time to time (such as if the Git server is down), you can either specify multiple URLs (as a comma-separated list under the `spring.cloud.config.uri` property) or have all your instances register in a Service Registry like Eureka (if using Discovery-First Bootstrap mode).

The URLs listed under `spring.cloud.config.uri` are tried in the order listed. By default, the Config Client will try to fetch properties from each URL until an attempt is successful to ensure high availability.

Expand Down
23 changes: 5 additions & 18 deletions pom.xml
Expand Up @@ -28,7 +28,7 @@
<properties>
<bintray.package>config</bintray.package>
<spring-cloud-commons.version>4.0.0-SNAPSHOT</spring-cloud-commons.version>
<aws-java-sdk.version>1.11.911</aws-java-sdk.version>
<aws-java-sdk.version>2.17.195</aws-java-sdk.version>
<google-api-services-iam.version>v1-rev20201112-1.30.10</google-api-services-iam.version>
<testcontainers.version>1.16.2</testcontainers.version>
<wiremock.version>2.31.0</wiremock.version>
Expand Down Expand Up @@ -71,24 +71,11 @@
<version>${spring-cloud-commons.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>${aws-java-sdk.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>${aws-java-sdk.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-secretsmanager</artifactId>
<version>${aws-java-sdk.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-ssm</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>${aws-java-sdk.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.google.apis</groupId>
Expand Down
2 changes: 1 addition & 1 deletion spring-cloud-config-dependencies/pom.xml
Expand Up @@ -16,7 +16,7 @@
<description>Spring Cloud Config Dependencies</description>
<properties>
<jgit.version>5.12.0.202106070339-r</jgit.version>
<spring-vault.version>2.3.2</spring-vault.version>
<spring-vault.version>3.0.0-M1</spring-vault.version>
<spring-credhub.version>2.1.1.RELEASE</spring-credhub.version>
</properties>
<dependencyManagement>
Expand Down
21 changes: 13 additions & 8 deletions spring-cloud-config-server/pom.xml
Expand Up @@ -94,23 +94,28 @@
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>auth</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-secretsmanager</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>aws-core</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-ssm</artifactId>
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>ssm</artifactId>
<optional>true</optional>
</dependency>
<dependency>
Expand Down
Expand Up @@ -19,14 +19,14 @@
import java.util.List;
import java.util.Optional;

import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.secretsmanager.AWSSecretsManager;
import com.amazonaws.services.simplesystemsmanagement.AWSSimpleSystemsManagement;
import com.google.cloud.secretmanager.v1.SecretManagerServiceClient;
import jakarta.servlet.http.HttpServletRequest;
import org.apache.http.client.HttpClient;
import org.eclipse.jgit.api.TransportConfigCallback;
import org.tmatesoft.svn.core.SVNException;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient;
import software.amazon.awssdk.services.ssm.SsmClient;

import org.springframework.beans.factory.ObjectProvider;
import org.springframework.boot.actuate.health.AbstractHealthIndicator;
Expand Down Expand Up @@ -210,7 +210,7 @@ public ConfigurableHttpConnectionFactory httpClientConnectionFactory() {
}

@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(AmazonS3.class)
@ConditionalOnClass(S3Client.class)
static class AwsS3FactoryConfig {

@Bean
Expand All @@ -221,7 +221,7 @@ public AwsS3EnvironmentRepositoryFactory awsS3EnvironmentRepositoryFactory(Confi
}

@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(AWSSecretsManager.class)
@ConditionalOnClass(SecretsManagerClient.class)
static class AwsSecretsManagerFactoryConfig {

@Bean
Expand All @@ -233,7 +233,7 @@ public AwsSecretsManagerEnvironmentRepositoryFactory awsSecretsManagerEnvironmen
}

@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(AWSSimpleSystemsManagement.class)
@ConditionalOnClass(SsmClient.class)
static class AwsParameterStoreFactoryConfig {

@Bean
Expand Down
Expand Up @@ -16,8 +16,10 @@

package org.springframework.cloud.config.server.environment;

import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.client.builder.AwsSyncClientBuilder;
import java.net.URI;

import software.amazon.awssdk.awscore.client.builder.AwsClientBuilder;
import software.amazon.awssdk.regions.Region;

import org.springframework.util.StringUtils;

Expand All @@ -26,15 +28,11 @@ abstract class AwsClientBuilderConfigurer {
private AwsClientBuilderConfigurer() {
}

static void configureClientBuilder(AwsSyncClientBuilder<?, ?> clientBuilder, String region, String endpoint) {
static void configureClientBuilder(AwsClientBuilder<?, ?> clientBuilder, String region, String endpoint) {
if (StringUtils.hasText(region)) {
clientBuilder.region(Region.of(region));
if (StringUtils.hasText(endpoint)) {
AwsClientBuilder.EndpointConfiguration endpointConfiguration = new AwsClientBuilder.EndpointConfiguration(
endpoint, region);
clientBuilder.withEndpointConfiguration(endpointConfiguration);
}
else {
clientBuilder.withRegion(region);
clientBuilder.endpointOverride(URI.create(endpoint));
}
}
}
Expand Down
Expand Up @@ -26,10 +26,10 @@
import java.util.stream.Collectors;
import java.util.stream.Stream;

import com.amazonaws.services.simplesystemsmanagement.AWSSimpleSystemsManagement;
import com.amazonaws.services.simplesystemsmanagement.model.GetParametersByPathRequest;
import com.amazonaws.services.simplesystemsmanagement.model.GetParametersByPathResult;
import com.amazonaws.services.simplesystemsmanagement.model.Parameter;
import software.amazon.awssdk.services.ssm.SsmClient;
import software.amazon.awssdk.services.ssm.model.GetParametersByPathRequest;
import software.amazon.awssdk.services.ssm.model.GetParametersByPathResponse;
import software.amazon.awssdk.services.ssm.model.Parameter;

import org.springframework.cloud.config.environment.Environment;
import org.springframework.cloud.config.environment.PropertySource;
Expand All @@ -43,14 +43,13 @@
*/
public class AwsParameterStoreEnvironmentRepository implements EnvironmentRepository {

private final AWSSimpleSystemsManagement awsSsmClient;
private final SsmClient awsSsmClient;

private final ConfigServerProperties configServerProperties;

private final AwsParameterStoreEnvironmentProperties environmentProperties;

public AwsParameterStoreEnvironmentRepository(AWSSimpleSystemsManagement awsSsmClient,
ConfigServerProperties configServerProperties,
public AwsParameterStoreEnvironmentRepository(SsmClient awsSsmClient, ConfigServerProperties configServerProperties,
AwsParameterStoreEnvironmentProperties environmentProperties) {
this.awsSsmClient = awsSsmClient;
this.configServerProperties = configServerProperties;
Expand Down Expand Up @@ -137,20 +136,20 @@ private List<PropertySource> getPropertySources(Set<String> parameterPaths) {
private Map<String, String> getPropertiesByParameterPath(String path) {
Map<String, String> result = new HashMap<>();

GetParametersByPathRequest request = new GetParametersByPathRequest().withPath(path)
.withRecursive(environmentProperties.isRecursive())
.withWithDecryption(environmentProperties.isDecryptValues())
.withMaxResults(environmentProperties.getMaxResults());
GetParametersByPathRequest request = GetParametersByPathRequest.builder().path(path)
.recursive(environmentProperties.isRecursive()).withDecryption(environmentProperties.isDecryptValues())
.maxResults(environmentProperties.getMaxResults()).build();

GetParametersByPathResult response = awsSsmClient.getParametersByPath(request);
GetParametersByPathResponse response = awsSsmClient.getParametersByPath(request);

if (response != null) {
addParametersToProperties(path, response.getParameters(), result);
addParametersToProperties(path, response.parameters(), result);

while (StringUtils.hasLength(response.getNextToken())) {
response = awsSsmClient.getParametersByPath(request.withNextToken(response.getNextToken()));
while (StringUtils.hasLength(response.nextToken())) {
response = awsSsmClient
.getParametersByPath(request.toBuilder().nextToken(response.nextToken()).build());

addParametersToProperties(path, response.getParameters(), result);
addParametersToProperties(path, response.parameters(), result);
}
}

Expand All @@ -159,9 +158,9 @@ private Map<String, String> getPropertiesByParameterPath(String path) {

private void addParametersToProperties(String path, List<Parameter> parameters, Map<String, String> properties) {
for (Parameter parameter : parameters) {
String name = StringUtils.delete(parameter.getName(), path).replace(DEFAULT_PATH_SEPARATOR, ".");
String name = StringUtils.delete(parameter.name(), path).replace(DEFAULT_PATH_SEPARATOR, ".");

properties.put(name, parameter.getValue());
properties.put(name, parameter.value());
}
}

Expand Down
Expand Up @@ -16,8 +16,8 @@

package org.springframework.cloud.config.server.environment;

import com.amazonaws.services.simplesystemsmanagement.AWSSimpleSystemsManagement;
import com.amazonaws.services.simplesystemsmanagement.AWSSimpleSystemsManagementClientBuilder;
import software.amazon.awssdk.services.ssm.SsmClient;
import software.amazon.awssdk.services.ssm.SsmClientBuilder;

import org.springframework.cloud.config.server.config.ConfigServerProperties;

Expand All @@ -37,11 +37,11 @@ public AwsParameterStoreEnvironmentRepositoryFactory(ConfigServerProperties conf

@Override
public AwsParameterStoreEnvironmentRepository build(AwsParameterStoreEnvironmentProperties environmentProperties) {
AWSSimpleSystemsManagementClientBuilder clientBuilder = AWSSimpleSystemsManagementClientBuilder.standard();
SsmClientBuilder clientBuilder = SsmClient.builder();

configureClientBuilder(clientBuilder, environmentProperties.getRegion(), environmentProperties.getEndpoint());

AWSSimpleSystemsManagement client = clientBuilder.build();
SsmClient client = clientBuilder.build();

return new AwsParameterStoreEnvironmentRepository(client, configServerProperties, environmentProperties);
}
Expand Down