Allow easily running tests against AWS

[tiago-loureiro]

Currently we tend to test things only against fake/mocked AWS services; while this is nice from a feedback loop point of view, it would also be helpful to be able to test things against the actual AWS API and get that sort of feedback sooner rather than later.

This PR intends to streamline that workflow making some assumptions:

• Defined in the environment: AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY. This AWS_ACCESS_KEY_ID has access to the resources specified in the configuration files
• Access to a file with default settings to override settings in each service's config options. This file can either be downloaded from S3 (if you provide a path to it) or just added locally
• Whatever AWS resources used, they must be created beforehand

If the preconditions are met, one can simply run make i-aws which is equivalent to make integration except that it attempts to use real AWS resources instead.

[jschaul]

With the current config, galley tests fail with

   post cryptomessage 1:                                        [brig] E, request=N/A, GeneralError (TransportError (HttpExceptionRequest Request {
[brig]   host                 = "localhost"
[brig]   port                 = 4569
[brig]   secure               = False

(i.e. brig is trying to connect to localhost here, but I don't run the fake-AWS services locally). The configuration doesn't seem right.

[jschaul]

Did you intend to use -s $(NAME).integration-aws.yaml ? The script writes merges and writes to that file, but the Makefile doesn't use that. (same for all the Makefiles)

[tiago-loureiro]

Erm, yeah... rebase gone wrong :/ should be all green now

[jschaul]

incorrect path. Should be aws s3 cp s3://$location "${DIR}/integration-aws.yaml"

[tiago-loureiro]

👍

[jschaul]

Also, there are a few warnings in the bash script when running shellcheck on it.

[tiago-loureiro]

Also, there are a few warnings in the bash script when running shellcheck on it.

Oh nice one, thanks for the tip.

[tiago-loureiro]

Should be much cleaner now, assumes that the integration-aws.yaml file is scoped per service (example added too).

[jschaul]

Looks good, thanks!