Setting s3region in a multi-project build

[tsuyoshizawa]

sbt fails to load sbt-s3-resolver plugin.

Stacktrace

The stacktrace says region is nothing from any provider.

[info] Loading settings from build.sbt ...
[error] com.amazonaws.SdkClientException: Unable to load region information from any provider in the chain
[error]         at com.amazonaws.regions.AwsRegionProviderChain.getRegion(AwsRegionProviderChain.java:59)
[error]         at ohnosequences.sbt.SbtS3Resolver$autoImport$.fromProviderToAWSRegion(SBTS3Resolver.scala:85)
[error]         at ohnosequences.sbt.SbtS3Resolver$.$anonfun$projectSettings$3(SBTS3Resolver.scala:137)
[error]         at sbt.internal.util.Init$Value.$anonfun$apply$3(Settings.scala:804)
[error]         at sbt.internal.util.EvaluateSettings.$anonfun$constant$1(INode.scala:197)
[error]         at sbt.internal.util.EvaluateSettings$MixedNode.evaluate0(INode.scala:214)
[error]         at sbt.internal.util.EvaluateSettings$INode.evaluate(INode.scala:159)
[error]         at sbt.internal.util.EvaluateSettings.$anonfun$submitEvaluate$1(INode.scala:82)
[error]         at sbt.internal.util.EvaluateSettings.sbt$internal$util$EvaluateSettings$$run0(INode.scala:93)
[error]         at sbt.internal.util.EvaluateSettings$$anon$3.run(INode.scala:89)
[error]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]         at java.lang.Thread.run(Thread.java:748)
[error] com.amazonaws.SdkClientException: Unable to load region information from any provider in the chain

Environment

• Scala 2.12.3
• sbt 1.0.2

project/plugins.sbt

resolvers += Resolver.jcenterRepo
addSbtPlugin("ohnosequences" % "sbt-s3-resolver" % "0.18.0")

.aws/credentials

[my-profile]
region=ap-northeast-1

build.sbt

awsProfile := "my-profile"

I also tried overwriting s3region property in my build.sbt but there was no change about the error.

[laughedelic]

Hi @tsuyoshizawa. So you say that you tried setting something like

s3region := com.amazonaws.services.s3.model.Region.US_Standard

and got the same error message? It shouldn't be the same if you set the region explicitly (not using a provider).

Is the project as simple as you describe or are there any complications like subprojects?

[laughedelic]

I also just realised that the awsProfile setting is confusing, because it's used only for the credentials provider and s3region doesn't use it in any way.

Also, regarding the region setting in the aws-cli config file, it should be ~/.aws/config not ~/.aws/credentials. The latter one is only for profile credentials. Again, default s3region provider chain will try only default profile, so if you want to use what you set in ~/.aws/config [my-profile], you can try

s3region := new com.amazonaws.regions.AwsProfileRegionProvider("my-profile")

[tsuyoshizawa]

So you say that you tried setting something like

s3region := com.amazonaws.services.s3.model.Region.US_Standard

and got the same error message?

Yes, I got the same error message.

Is the project as simple as you describe or are there any complications like subprojects?

I am trying to use subprojects.

+ root
  + core
  + libA
  + libB

I use this plugin (v0.15.0) in another project with single project, it works well.

it should be ~/.aws/config not ~/.aws/credentials.

Ah, thank you. I added the profile in ~/.aws/config.
However, the message is same...

@laughedelic Anyway, thank you for your following.
I will try another.

[laughedelic]

Now I think that it's some scoping issue. Because if you set the s3region explicitly, the error shouldn't be about region provider. Then it's an error from somewhere else, where the default value (default region provider) didn't get any value. In multiproject builds it's very possible that you set some key in one project, but it stays default somewhere else.

1. If you're using multiproject setup, I need to know in which subproject you set plugin settings and how your subprojects are connected. It would be much better if you could just provide the whole project setup (without the sources) so that I can reproduce the error.

2. About setting region in ~/.aws/config, try to do it for the [default] profile and let's see if this changes the error message, because this way default provider should pick up this setting.

P.S. Also don't hesitate to write in the gitter chat. It may be faster to resolve such issue there (if we have some common free hours between our timezones).

[tsuyoshizawa]

@laughedelic I created a repository https://github.com/tsuyoshizawa/sbt-s3-resolver-multi-project to reproduce. Could you check the repository?

s3region := new com.amazonaws.regions.AwsProfileRegionProvider("my-profile")

When I tried it on single project, it works fine.
So, I think this problem happens only multi project.

About setting region in ~/.aws/config, try to do it for the [default] profile

There is not change about error message.

P.S. Also don't hesitate to write in the gitter chat. It may be faster to resolve such issue there (if we have some common free hours between our timezones).

I see. Thank you for your cooperation.

[tsuyoshizawa]

s3region := new com.amazonaws.regions.DefaultAwsRegionProviderChain(),

BTW, could we have a chance to change the DefaultAwsRegionProviderChain?
AwsProfileRegionProvider uses default profile, so if we can pass awsProfile value to the construct, we will just set awsProfile in our project.

public class DefaultAwsRegionProviderChain extends AwsRegionProviderChain {

    public DefaultAwsRegionProviderChain() {
        super(new AwsEnvVarOverrideRegionProvider(),
              new AwsSystemPropertyRegionProvider(),
              new AwsProfileRegionProvider(),
              new InstanceMetadataRegionProvider());
    }
}

[laughedelic]

@tsuyoshizawa Thanks for putting up a demo project. So the problems are:

• default region provider doesn't find any regions defined (in env-var, sys-property, default profile or instance metadata) so it gives you an error and it's unrelated to the s3region which you set explicitly
• you set s3region only for the root project, so in other subprojects it stays unset and gives the error

You can fix it by declaring common settings that you want to apply to every subproject:

lazy val commonSettings = Seq(
  s3region := new AwsProfileRegionProvider("my-profile")
)

lazy val root = (project in file("."))
  .settings(
    commonSettings,
    // ...
  )
  .aggregate(core)

lazy val core = (project in file("core"))
  .settings(
    commonSettings,
    name := "core"
  )

Check sbt docs about it. This should solve the problem.

---

BTW, could we have a chance to change the DefaultAwsRegionProviderChain?
AwsProfileRegionProvider uses default profile, so if we can pass awsProfile value to the construct, we will just set awsProfile in our project.

Absolutely! Default s3region should take into account awsProfile setting. Could you submit a pull-request with this fix?

[tsuyoshizawa]

Thanks! I could fix this problem.

Absolutely! Default s3region should take into account awsProfile setting. Could you submit a pull-request with this fix?

Sure. I will create the PR.

[laughedelic]

Cool! I'm glad that your problem got resolved 👍

[krazavet]

Hello, I would like to know if this problem could be resolved in previous version of sbt (e.g. 0.13.x) ?

[laughedelic]

Hi @krazavet. If you use sbt-0.13 and can't upgrade to sbt-1.0, you have to use an old version (0.16.0) of this plugin:

resolvers += "Era7 maven releases" at "https://s3-eu-west-1.amazonaws.com/releases.era7.com"
addSbtPlugin("ohnosequences" % "sbt-s3-resolver" % "0.16.0")

Then if you have a similar problem with multiproject build and the region not being set, you can still try the commonSettings approach from: #56 (comment).

I also added you to the chat in case it doesn't solve your problem and you want to discuss it.