Ignoring x-aws- extensions for local deployments

[mreferre]

I am using a fresh install of Docker Desktop 2.5.0.1 stable for Mac that comes with compose cli ver 1.0.2.

➜  docker version
Client: Docker Engine - Community
 Cloud integration: 1.0.2
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 16:58:31 2020
 OS/Arch:           darwin/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

I am using this compose file:

version: "3.8"

services:
  ecsworker-in-region:
    environment: 
        - SQS_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789/main-queue
        - EFS_SOURCE_FOLDER=/data/sourcefolder/             
        - EFS_DESTINATION_FOLDER=/data/destinationfolder/              
        - AWS_REGION=us-east-1
    image: 123456789.dkr.ecr.us-east-1.amazonaws.com/ecsworker:amd64-slim 
    volumes:
        - efs-share:/data
    x-aws-role:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Action: sqs:*
          Resource: arn:aws:sqs:us-east-1:123456789:main-queue
volumes:
  efs-share:

This deploys just fine on ECS. When I point to the local ecsLocal context though (created with docker context create ecs --local-simulation ecsLocal it throws an error - rightly so):

➜  docker context ls
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT               KUBERNETES ENDPOINT   ORCHESTRATOR
default             moby                Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                         swarm
ecsLocal *          ecs-local           ECS local endpoints                                                                           
myecscontext        ecs                                                                                                               
➜  docker compose up 
Traceback (most recent call last):
  File "docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 126, in perform_command
  File "compose/cli/main.py", line 1070, in up
  File "compose/cli/main.py", line 1066, in up
  File "compose/project.py", line 620, in up
  File "compose/project.py", line 692, in _get_convergence_plans
  File "compose/service.py", line 407, in convergence_plan
  File "compose/service.py", line 422, in _containers_have_diverged
  File "compose/service.py", line 700, in config_hash
  File "compose/utils.py", line 93, in json_hash
  File "json/__init__.py", line 238, in dumps
  File "json/encoder.py", line 199, in encode
  File "json/encoder.py", line 257, in iterencode
  File "compose/utils.py", line 93, in <lambda>
AttributeError: 'datetime.date' object has no attribute 'repr'
[21037] Failed to execute script docker-compose
exit status 255

While I initially thought (wrongly so) that the container would inherit the same IAM role assigned to the ECS task for the in-region deployment I later figured that the local application sources the AWS credentials from the local $HOME./aws./credentials file. Which makes sense.

In fact removing the following section from the compose file makes the app start fine (and work fine).

    x-aws-role:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Action: sqs:*
          Resource: arn:aws:sqs:us-east-1:123456789:main-queue

I was wondering, in the spirit of simplicity and being able to re-use the same compose file, would it make sense to add logic to compose cli so that it ignores compose file lines that do not apply to a specific context? Right now I have to have two files (one for ECS deployment and one for ECS local deployment) and maintaining two files may be error-prone.

[ndeloof]

x-aws-* should be fully ignored by docker-compose (used behind the scene by ecs local context), as long as is it valid yaml content - which is obviously the case here.
investigating

[ndeloof]

issue is caused by golang's yaml.Marshall to marshal string without quotes, and Python's loader trying to parse 2012-10-17 as a datetime type

comparable issue: helm/helm#4262

[mreferre]

Thanks @ndeloof . I see you have already created a PR for this (#918). By the way my other PR has this syntax in it so this fix seems to be a pre-req for having that to fully work.