support to import some docker_container's attributes #234

suzuki-shunsuke · 2020-01-07T01:08:41Z

import some docker_container's attributes. Please see docker/resource_docker_container_funcs.go
set Computed: true to some attributes
- hostname
- command
- entrypoint
- env
- labels
- shm_size
- ipc_mode
set DiffSuppressFunc to some attributes
- network_mode
- restart
format by gofmt

Verification codes

I share verification codes.

main.tf

provider "docker" {
}

resource "docker_image" "alpine" {
  name = "alpine:3.10.3"
}

resource "docker_container" "foo" {
  image       = docker_image.alpine.latest
  name        = "foo"
  rm          = true
  entrypoint  = ["tail"]
  start       = true
  command     = ["-f", "/dev/null"]
  read_only   = true
  shm_size    = 64 # 64MB
  privileged  = true
  user        = "root"
  working_dir = "/workspace"
  domainname  = "foo.com"
  hostname    = "foo"
  dns         = ["8.8.8.8"]
  dns_search  = ["example.com"]
  env = [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "RACK_ENV=development"
  ]
  userns_mode = "host"
  tmpfs = {
    "/run" : ""
    "/tmp" : ""
  }
  log_driver = "json-file"
  log_opts = {
    max-size : "200k"
    max-file : "10"
  }

  # these labels are set by docker-compose automatically
  labels {
    label = "com.docker.compose.config-hash"
    value = "40541480c3ba148eefff245755e35d6186d2bca9c9033340866be9a27a603fe9" # you have to fix the value
  }
  labels {
    label = "com.docker.compose.container-number"
    value = "1"
  }
  labels {
    label = "com.docker.compose.oneoff"
    value = "False"
  }
  labels {
    label = "com.docker.compose.project"
    value = "terraform-provider-docker"
  }
  labels {
    label = "com.docker.compose.service"
    value = "foo"
  }
  labels {
    label = "com.docker.compose.version"
    value = "1.24.1"
  }
  network_mode = "bridge"
  pid_mode     = "host"
  healthcheck {
    test     = ["CMD", "true"]
    interval = "1m30s"
    timeout  = "10s"
    retries  = 3
  }
  sysctls = {
    "net.core.somaxconn" : 1024
    "net.ipv4.tcp_syncookies" : 0
  }
  ipc_mode    = "shareable"
  memory      = 50
  memory_swap = 100
  capabilities {
    add  = ["ALL"]
    drop = ["SYS_ADMIN"]
  }
  devices {
    container_path = "/dev/null"
    host_path      = "/dev/null"
    permissions    = "rwm"
  }
}

resource "docker_container" "zoo" {
  image      = docker_image.alpine.latest
  name       = "zoo"
  command    = ["tail", "-f", "/dev/null"]
  cpu_shares = 2
  cpu_set    = "1"
  start      = true
  env = [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  ]
  # links = ["/foo:/zoo/foo"]
}

Using Docker Compose and `docker run` command

docker-compose.yml

---
version: "3"
services:
  foo:
    image: alpine:3.10.3
    container_name: foo
    entrypoint: ["tail"]
    command: ["-f", "/dev/null"]
    read_only: true
    shm_size: 64M
    privileged: true
    user: root
    working_dir: /workspace
    domainname: foo.com
    hostname: foo
    dns: 8.8.8.8
    dns_search: example.com
    restart: "no"
    environment:
      RACK_ENV: development
    userns_mode: "host"
    tmpfs:
    - /run
    - /tmp
    devices:
    - "/dev/null:/dev/null"
    logging:
      driver: json-file
      options:
        max-size: "200k"
        max-file: "10"
    network_mode: "bridge"
    pid: "host"
    healthcheck:
      test: ["CMD", "true"]
      interval: 1m30s
      timeout: 10s
      retries: 3
    sysctls:
      net.core.somaxconn: 1024
      net.ipv4.tcp_syncookies: 0
    ipc: host
    cap_add:
    - ALL
    cap_drop:
    - SYS_ADMIN

$ docker-compose up -d
$ docker run --cpu-shares=2 --cpuset-cpus="1" -d --name zoo alpine:3.10.3 tail -f /dev/null

terraform apply and terraform state rm

Instead of Docker Compose and docker run command, we can run terraform apply and terraform state rm too.

$ terraform apply
$ terraform state rm docker_container.foo docker_container.zoo

terraform import

$ terraform import docker_container.foo $(docker inspect foo -f "{{.ID}}")
$ terraform import docker_container.zoo $(docker inspect zoo -f "{{.ID}}")

suzuki-shunsuke · 2020-01-07T01:24:34Z

Verification precedure

main.tf

provider "docker" {
}

resource "docker_image" "alpine" {
  name = "alpine:3.10.3"
}

resource "docker_container" "foo" {
  image       = docker_image.alpine.latest
  name        = "foo"
  rm          = true
  entrypoint  = ["tail"]
  start       = true
  command     = ["-f", "/dev/null"]
  read_only   = true
  shm_size    = 64 # 64MB
  privileged  = true
  user        = "root"
  working_dir = "/workspace"
  domainname  = "foo.com"
  hostname    = "foo"
  dns         = ["8.8.8.8"]
  dns_search  = ["example.com"]
  env = [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "RACK_ENV=development"
  ]
  userns_mode = "host"
  tmpfs = {
    "/run" : ""
    "/tmp" : ""
  }
  log_driver = "json-file"
  log_opts = {
    max-size : "200k"
    max-file : "10"
  }

  # these labels are set by docker-compose automatically
  labels {
    label = "com.docker.compose.config-hash"
    value = "40541480c3ba148eefff245755e35d6186d2bca9c9033340866be9a27a603fe9" # you have to fix the value
  }
  labels {
    label = "com.docker.compose.container-number"
    value = "1"
  }
  labels {
    label = "com.docker.compose.oneoff"
    value = "False"
  }
  labels {
    label = "com.docker.compose.project"
    value = "terraform-provider-docker"
  }
  labels {
    label = "com.docker.compose.service"
    value = "foo"
  }
  labels {
    label = "com.docker.compose.version"
    value = "1.24.1"
  }
  network_mode = "bridge"
  pid_mode     = "host"
  healthcheck {
    test     = ["CMD", "true"]
    interval = "1m30s"
    timeout  = "10s"
    retries  = 3
  }
  sysctls = {
    "net.core.somaxconn" : 1024
    "net.ipv4.tcp_syncookies" : 0
  }
  ipc_mode    = "shareable"
  memory      = 50
  memory_swap = 100
  capabilities {
    add  = ["ALL"]
    drop = ["SYS_ADMIN"]
  }
  devices {
    container_path = "/dev/null"
    host_path      = "/dev/null"
    permissions    = "rwm"
  }
}

resource "docker_container" "zoo" {
  image      = docker_image.alpine.latest
  name       = "zoo"
  command    = ["tail", "-f", "/dev/null"]
  cpu_shares = 2
  cpu_set    = "1"
  start      = true
  env = [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  ]
  # links = ["/foo:/zoo/foo"]
}

commands

$ terraform version
$ terraform apply
$ terraform plan
$ terraform state rm docker_container.foo docker_container.zoo
$ terraform import docker_container.foo $(docker inspect foo -f "{{.ID}}")
$ terraform import docker_container.zoo $(docker inspect zoo -f "{{.ID}}")
$ terraform plan

results

$ terraform version
Terraform v0.12.18
+ provider.docker (unversioned)

$ terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # docker_container.foo will be created
  + resource "docker_container" "foo" {
      + attach           = false
      + bridge           = (known after apply)
      + command          = [
          + "-f",
          + "/dev/null",
        ]
      + container_logs   = (known after apply)
      + dns              = [
          + "8.8.8.8",
        ]
      + dns_search       = [
          + "example.com",
        ]
      + domainname       = "foo.com"
      + entrypoint       = [
          + "tail",
        ]
      + env              = [
          + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
          + "RACK_ENV=development",
        ]
      + exit_code        = (known after apply)
      + gateway          = (known after apply)
      + hostname         = "foo"
      + id               = (known after apply)
      + image            = (known after apply)
      + ip_address       = (known after apply)
      + ip_prefix_length = (known after apply)
      + ipc_mode         = "shareable"
      + log_driver       = "json-file"
      + log_opts         = {
          + "max-file" = "10"
          + "max-size" = "200k"
        }
      + logs             = false
      + memory           = 50
      + memory_swap      = 100
      + name             = "foo"
      + network_data     = (known after apply)
      + network_mode     = "bridge"
      + pid_mode         = "host"
      + privileged       = true
      + read_only        = true
      + restart          = "no"
      + rm               = true
      + shm_size         = 64
      + sysctls          = {
          + "net.core.somaxconn"      = "1024"
          + "net.ipv4.tcp_syncookies" = "0"
        }
      + tmpfs            = {
          + "/run" = ""
          + "/tmp" = ""
        }
      + user             = "root"
      + userns_mode      = "host"
      + working_dir      = "/workspace"

      + capabilities {
          + add  = [
              + "ALL",
            ]
          + drop = [
              + "SYS_ADMIN",
            ]
        }

      + devices {
          + container_path = "/dev/null"
          + host_path      = "/dev/null"
          + permissions    = "rwm"
        }

      + healthcheck {
          + interval     = "1m30s"
          + retries      = 3
          + start_period = "0s"
          + test         = [
              + "CMD",
              + "true",
            ]
          + timeout      = "10s"
        }

      + labels {
          + label = "com.docker.compose.config-hash"
          + value = "40541480c3ba148eefff245755e35d6186d2bca9c9033340866be9a27a603fe9"
        }
      + labels {
          + label = "com.docker.compose.container-number"
          + value = "1"
        }
      + labels {
          + label = "com.docker.compose.oneoff"
          + value = "False"
        }
      + labels {
          + label = "com.docker.compose.project"
          + value = "terraform-provider-docker"
        }
      + labels {
          + label = "com.docker.compose.service"
          + value = "foo"
        }
      + labels {
          + label = "com.docker.compose.version"
          + value = "1.24.1"
        }
    }

  # docker_container.zoo will be created
  + resource "docker_container" "zoo" {
      + attach           = false
      + bridge           = (known after apply)
      + command          = [
          + "tail",
          + "-f",
          + "/dev/null",
        ]
      + container_logs   = (known after apply)
      + cpu_set          = "1"
      + cpu_shares       = 2
      + env              = [
          + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        ]
      + exit_code        = (known after apply)
      + gateway          = (known after apply)
      + hostname         = (known after apply)
      + id               = (known after apply)
      + image            = (known after apply)
      + ip_address       = (known after apply)
      + ip_prefix_length = (known after apply)
      + ipc_mode         = (known after apply)
      + log_driver       = "json-file"
      + logs             = false
      + name             = "zoo"
      + network_data     = (known after apply)
      + read_only        = false
      + restart          = "no"
      + rm               = false
      + shm_size         = (known after apply)
    }

  # docker_image.alpine will be created
  + resource "docker_image" "alpine" {
      + id     = (known after apply)
      + latest = (known after apply)
      + name   = "alpine:3.10.3"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

docker_image.alpine: Creating...
docker_image.alpine: Creation complete after 0s [id=sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652alpine:3.10.3]
docker_container.zoo: Creating...
docker_container.foo: Creating...
docker_container.zoo: Creation complete after 0s [id=08e9f5799f00d5a6c968666cd91e683ab5a626e1aae84c761b3a05e2d9e7b9a5]
docker_container.foo: Creation complete after 0s [id=ca7f581b69edba2a71fac1146c1a3fa95a03e3e4b4fccce9dbec46cbbe9cab04]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

docker_image.alpine: Refreshing state... [id=sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652alpine:3.10.3]
docker_container.zoo: Refreshing state... [id=08e9f5799f00d5a6c968666cd91e683ab5a626e1aae84c761b3a05e2d9e7b9a5]
docker_container.foo: Refreshing state... [id=ca7f581b69edba2a71fac1146c1a3fa95a03e3e4b4fccce9dbec46cbbe9cab04]

------------------------------------------------------------------------

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

$ terraform state rm docker_container.foo docker_container.zoo
Removed docker_container.foo
Removed docker_container.zoo
Successfully removed 2 resource instance(s).

$ terraform import docker_container.foo $(docker inspect foo -f "{{.ID}}")
docker_container.foo: Importing from ID "ca7f581b69edba2a71fac1146c1a3fa95a03e3e4b4fccce9dbec46cbbe9cab04"...
docker_container.foo: Import prepared!
  Prepared docker_container for import
docker_container.foo: Refreshing state... [id=ca7f581b69edba2a71fac1146c1a3fa95a03e3e4b4fccce9dbec46cbbe9cab04]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

$ terraform import docker_container.zoo $(docker inspect zoo -f "{{.ID}}")
docker_container.zoo: Importing from ID "08e9f5799f00d5a6c968666cd91e683ab5a626e1aae84c761b3a05e2d9e7b9a5"...
docker_container.zoo: Import prepared!
  Prepared docker_container for import
docker_container.zoo: Refreshing state... [id=08e9f5799f00d5a6c968666cd91e683ab5a626e1aae84c761b3a05e2d9e7b9a5]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

docker_image.alpine: Refreshing state... [id=sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652alpine:3.10.3]
docker_container.zoo: Refreshing state... [id=08e9f5799f00d5a6c968666cd91e683ab5a626e1aae84c761b3a05e2d9e7b9a5]
docker_container.foo: Refreshing state... [id=ca7f581b69edba2a71fac1146c1a3fa95a03e3e4b4fccce9dbec46cbbe9cab04]

------------------------------------------------------------------------

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

mavogel · 2020-01-08T21:38:38Z

Hey, @suzuki-shunsuke thank you for your effort on helping to finish the import of all resources :)

Could you take a look at the failing tests? Furthermore, we'd consider adding the verify-step for the test as well as already done in the test for docker_service

{
  ResourceName:      "docker_service.foo",
  ImportState:       true,
  ImportStateVerify: true,
},

suzuki-shunsuke · 2020-01-09T07:07:24Z

@mavogel
Thank you for your comment.
I'll check in this weekend.

suzuki-shunsuke · 2020-01-09T14:48:25Z

I found that d.Get("start").(bool) is false even if "start" = true when DiffSuppressFunc of "start" is set. 🤔

https://github.com/suzuki-shunsuke/terraform-provider-docker/blob/dbf977499272d5bde494c823e8bff777e2fa0f3b/docker/resource_docker_container_funcs.go#L436
https://github.com/terraform-providers/terraform-provider-docker/pull/234/files#diff-39adaa97903875e2527ec754a4bda9a9R58-R61

When I comment out DiffSuppressFunc, d.Get("start").(bool) is false.

suzuki-shunsuke · 2020-01-09T15:11:01Z

We can avoid the above trouble by removing DiffSuppressFunc, but ideally we should set DiffSuppressFunc.

start - (Optional, bool) If true, then the Docker container will be started after creation. If false, then the container is only created.

https://www.terraform.io/docs/providers/docker/r/container.html#start

I think the attribute start is meaningful only when the container is created.
So I set DiffSuppressFunc to ignore diff.
We can't get the information about "start" from existing container.
The attribute "start" isn't the state of the container.

#234 (comment)

suzuki-shunsuke · 2020-01-10T02:01:59Z

Note that Docker image can have labels and environment variables so when we define labels or environment variables of docker_container resource we have to add labes and environment variables Docker image has to the attribute labels and env.

suzuki-shunsuke · 2020-01-10T02:12:36Z

#237

CI fails but this failure occurs at the master branch too so it has nothing to do with this pull request.

This is a test of docker_service.

https://github.com/terraform-providers/terraform-provider-docker/blob/a7f6cc93009c4052f836c279e68ecd4a96ab238d/docker/resource_docker_service_test.go#L475

=== RUN   TestAccDockerService_fullSpec
--- FAIL: TestAccDockerService_fullSpec (6.43s)
    testing.go:569: Step 0 error: Check failed: Check 71/76 error: docker_service.foo: Attribute 'endpoint_spec.0.mode' expected "vip", got ""

I don't know why this test fails.

suzuki-shunsuke · 2020-01-10T02:35:28Z

@mavogel Please review 🙏

#234 (comment)

``` panic: interface conversion: interface {} is int, not int64 ```

mavogel · 2020-01-11T21:50:30Z

@suzuki-shunsuke thank you for the awesome work. It looks like we have a flaky test here and yes it has nothing to do with your PR. I'll investigate in a separate issue and we can deactivate this test temporarily (as we did also for another flaky test only on travis)

I'll review in the next couple of days

suzuki-shunsuke · 2020-01-25T07:04:36Z

Would you review?

mavogel

LGTM :) big thanks for this PR.

suzuki-shunsuke added 5 commits January 6, 2020 18:39

feat: import docker_container

e266384

style: fix by linter

003d2fc

fix: set Computed: true

2ac26b2

fix: don't set capabilities if CapAdd and CapDrop is empty

c1365ad

fix: set DiffSuppressFunc

dbf9774

ghost added the size/M label Jan 7, 2020

suzuki-shunsuke added 4 commits January 10, 2020 00:14

Remove DiffSuppressFunc of "start" of docker_container

48c7cba

#234 (comment)

Add "Computed: true" to some attributes

ed4d7ab

Remove DiffSuppressFunc from docker_container's must_run

31fb701

Fix by gofmt

32e1867

ghost added size/L and removed size/M labels Jan 10, 2020

Fix tests of docker_container

a98fc34

suzuki-shunsuke added 2 commits January 10, 2020 13:17

Add a verification step

e1a5576

#234 (comment)

Fix to read memory_swap of docker_container

4bbdadc

suzuki-shunsuke mentioned this pull request Jan 10, 2020

Support to update docker_container #236

Merged

suzuki-shunsuke added 4 commits January 10, 2020 18:17

Support to read ulimit of docker_container

c3f3dea

Fix panic: interface conversion: interface {} is int, not int64

ac1a32f

``` panic: interface conversion: interface {} is int, not int64 ```

Support to read "mounts" of docker_container

50497c9

Fix comments

6fa530a

suzuki-shunsuke mentioned this pull request Jan 11, 2020

CI is broken (TestAccDockerService_fullSpec is failed on master branch) #237

Closed

mavogel self-requested a review January 12, 2020 16:39

mavogel added this to the v2.7.0 milestone Feb 1, 2020

mavogel mentioned this pull request Feb 1, 2020

feat: import for remaining resources #219

Open

4 tasks

mavogel approved these changes Feb 1, 2020

View reviewed changes

mavogel merged commit 33c1757 into hashicorp:master Feb 1, 2020

mavogel added the enhancement label Feb 10, 2020

pannon mentioned this pull request Apr 8, 2020

forced container replacement with 2.7.0 #242

Closed

suzuki-shunsuke mentioned this pull request Jun 3, 2020

Prevent force recreate of container about some attributes #269

Merged

mavogel mentioned this pull request Dec 25, 2020

feat: import for remaining resources kreuzwerker/terraform-provider-docker#74

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support to import some docker_container's attributes #234

support to import some docker_container's attributes #234

suzuki-shunsuke commented Jan 7, 2020 •

edited

suzuki-shunsuke commented Jan 7, 2020

mavogel commented Jan 8, 2020

suzuki-shunsuke commented Jan 9, 2020

suzuki-shunsuke commented Jan 9, 2020

suzuki-shunsuke commented Jan 9, 2020

suzuki-shunsuke commented Jan 10, 2020 •

edited

suzuki-shunsuke commented Jan 10, 2020 •

edited

suzuki-shunsuke commented Jan 10, 2020

mavogel commented Jan 11, 2020

suzuki-shunsuke commented Jan 25, 2020 •

edited

mavogel left a comment

support to import some docker_container's attributes #234

support to import some docker_container's attributes #234

Conversation

suzuki-shunsuke commented Jan 7, 2020 • edited

Verification codes

Using Docker Compose and docker run command

terraform apply and terraform state rm

terraform import

suzuki-shunsuke commented Jan 7, 2020

Verification precedure

mavogel commented Jan 8, 2020

suzuki-shunsuke commented Jan 9, 2020

suzuki-shunsuke commented Jan 9, 2020

suzuki-shunsuke commented Jan 9, 2020

suzuki-shunsuke commented Jan 10, 2020 • edited

suzuki-shunsuke commented Jan 10, 2020 • edited

suzuki-shunsuke commented Jan 10, 2020

mavogel commented Jan 11, 2020

suzuki-shunsuke commented Jan 25, 2020 • edited

mavogel left a comment

Choose a reason for hiding this comment

suzuki-shunsuke commented Jan 7, 2020 •

edited

Using Docker Compose and `docker run` command

suzuki-shunsuke commented Jan 10, 2020 •

edited

suzuki-shunsuke commented Jan 10, 2020 •

edited

suzuki-shunsuke commented Jan 25, 2020 •

edited