Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scaleway provider #7331

Merged
merged 8 commits into from
Jul 13, 2016
Merged

Add scaleway provider #7331

merged 8 commits into from
Jul 13, 2016

Conversation

nicolai86
Copy link
Contributor

@nicolai86 nicolai86 commented Jun 25, 2016

This PR adds a provider for the scaleway ARM cloud hosting provider.

One can manage the entire scaleway stack like this:

provider "scaleway" {
  api_key = "snap"
  organization = "snip"
}

resource "scaleway_ip" "base" {
  server = "${scaleway_server.base.id}"
}

resource "scaleway_server" "base" {
  name = "test"
  # ubuntu 14.04
  image = "aecaed73-51a5-4439-a127-6d8229847145"
  type = "C2S"
}

resource "scaleway_volume" "test" {
  name = "test"
  size_in_gb = 20
  type = "l_ssd"
}

resource "scaleway_volume_attachment" "test" {
  server = "${scaleway_server.base.id}"
  volume = "${scaleway_volume.test.id}"
}

resource "scaleway_security_group" "base" {
  name = "public"
  description = "public gateway"
}

resource "scaleway_security_group_rule" "http-ingress" {
  security_group = "${scaleway_security_group.base.id}"

  action = "accept"
  direction = "inbound"
  ip_range = "0.0.0.0/0"
  protocol = "TCP"
  port = 80
}


resource "scaleway_security_group_rule" "http-egress" {
  security_group = "${scaleway_security_group.base.id}"

  action = "accept"
  direction = "outbound"
  ip_range = "0.0.0.0/0"
  protocol = "TCP"
  port = 80
}

The PR includes tests & documentation.

Looking forward to your feedback!

@nicolai86
Copy link
Contributor Author

Test output (cleaned up):


TF_ACC=1 go test ./builtin/providers/scaleway -v -run=TestAccScaleway -timeout 120m
=== RUN   TestAccScalewayIP_Basic
--- PASS: TestAccScalewayIP_Basic (1.96s)
=== RUN   TestAccScalewaySecurityGroupRule_Basic
--- PASS: TestAccScalewaySecurityGroupRule_Basic (2.77s)
=== RUN   TestAccScalewaySecurityGroup_Basic
--- PASS: TestAccScalewaySecurityGroup_Basic (1.94s)
=== RUN   TestAccScalewayServer_Basic
--- PASS: TestAccScalewayServer_Basic (123.19s)
=== RUN   TestAccScalewayVolumeAttachment_Basic
--- PASS: TestAccScalewayVolumeAttachment_Basic (325.58s)
=== RUN   TestAccScalewayVolume_Basic
--- PASS: TestAccScalewayVolume_Basic (2.61s)
PASS
ok    github.com/hashicorp/terraform/builtin/providers/scaleway  458.068s

if server.State == targetState {
break
}
time.Sleep(1 * time.Second)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any issue with 1 second refresh times? I.e. will we hit their API limit or anything?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as I know there's no api limit. But maybe @moul can give some insight here.

@stack72
Copy link
Contributor

stack72 commented Jul 12, 2016

Hi @nicolai86

I left a small comments - nothing major. If we can get them fixed up then that will be fantastic.

I need to try and get in touch with Scaleway to see if we can get a testing account for terraform

Paul

@stack72 stack72 added the waiting-response An issue/pull request is waiting for a response from the community label Jul 12, 2016
@nicolai86
Copy link
Contributor Author

Hey @stack72

thank you for your feedback. Will adjust this soonish.

Best,
Raphael

@nicolai86
Copy link
Contributor Author

nicolai86 commented Jul 12, 2016

@stack72 I think I adjusted everything based on your feedback. Ready for another review I'd say.
I've also rebased on current master to make sure we're in sync.

this PR allows the entire scaleway stack to be managed with terraform

example usage looks like this:

```
provider "scaleway" {
  api_key = "snap"
  organization = "snip"
}

resource "scaleway_ip" "base" {
  server = "${scaleway_server.base.id}"
}

resource "scaleway_server" "base" {
  name = "test"
  # ubuntu 14.04
  image = "aecaed73-51a5-4439-a127-6d8229847145"
  type = "C2S"
}

resource "scaleway_volume" "test" {
  name = "test"
  size_in_gb = 20
  type = "l_ssd"
}

resource "scaleway_volume_attachment" "test" {
  server = "${scaleway_server.base.id}"
  volume = "${scaleway_volume.test.id}"
}

resource "scaleway_security_group" "base" {
  name = "public"
  description = "public gateway"
}

resource "scaleway_security_group_rule" "http-ingress" {
  security_group = "${scaleway_security_group.base.id}"

  action = "accept"
  direction = "inbound"
  ip_range = "0.0.0.0/0"
  protocol = "TCP"
  port = 80
}

resource "scaleway_security_group_rule" "http-egress" {
  security_group = "${scaleway_security_group.base.id}"

  action = "accept"
  direction = "outbound"
  ip_range = "0.0.0.0/0"
  protocol = "TCP"
  port = 80
}
```

Note that volume attachments require the server to be stopped, which can lead to
downtimes of you attach new volumes to already used servers
Sadly this is not part of the official scaleway api just yet
@stack72
Copy link
Contributor

stack72 commented Jul 12, 2016

Hi @nicolai86

This looks good to me now. I just need to get an account on scale way to test this out and we are GTG

Paul

page_title: "Provider: Scaleway"
sidebar_current: "docs-scaleway-index"
description: |-
The Docker provider is used to interact with Docker containers and images.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description should be for Scaleway :)

@stack72
Copy link
Contributor

stack72 commented Jul 13, 2016

Hi @nicolai86

left 2 more small points on the docs side of things. Just managed to test this and the tests work as expected :)

The tests are very chatty though:

=== RUN   TestAccScalewaySecurityGroup_Basic
2016/07/13 10:46:30 POST /security_groups
2016/07/13 10:46:30 GET /security_groups
2016/07/13 10:46:30 GET /security_groups/4d25daae-04f8-4dd6-b778-e3d234dafaf0
2016/07/13 10:46:31 GET /security_groups/4d25daae-04f8-4dd6-b778-e3d234dafaf0
2016/07/13 10:46:31 GET /security_groups/4d25daae-04f8-4dd6-b778-e3d234dafaf0
2016/07/13 10:46:31 GET /security_groups/4d25daae-04f8-4dd6-b778-e3d234dafaf0
2016/07/13 10:46:31 GET /security_groups/4d25daae-04f8-4dd6-b778-e3d234dafaf0
2016/07/13 10:46:31 DELETE /security_groups/4d25daae-04f8-4dd6-b778-e3d234dafaf0
--- PASS: TestAccScalewaySecurityGroup_Basic (1.56s)

Is there any way that this can be disabled? I.e can they be moved within our logging levels of INFO, DEBUG, TRACE and ERROR etc?

P.

@stack72 stack72 self-assigned this Jul 13, 2016
@nicolai86
Copy link
Contributor Author

@stack72 I've updated the docs, renamed the api_key to cleary communicate the usage, and also made sure the logs are less chatty :)

@stack72
Copy link
Contributor

stack72 commented Jul 13, 2016

Hi @nicolai86

This now looks great and the tests pass as expected. Thanks for making those logging changes!

Paul

@stack72 stack72 merged commit 9081cab into hashicorp:master Jul 13, 2016
@nicolai86 nicolai86 deleted the feature/scaleway branch July 13, 2016 20:09
@nicolai86
Copy link
Contributor Author

Hey @stack72

thanks! If there are any bugs/ problems related to this feel free to pull me into the discussion.

Raphael

@stack72
Copy link
Contributor

stack72 commented Jul 13, 2016

Will do! Thanks for all the help here :)

iceycake pushed a commit to ticketmaster/terraform that referenced this pull request Jul 22, 2016
* Add scaleway provider

this PR allows the entire scaleway stack to be managed with terraform

example usage looks like this:

```
provider "scaleway" {
  api_key = "snap"
  organization = "snip"
}

resource "scaleway_ip" "base" {
  server = "${scaleway_server.base.id}"
}

resource "scaleway_server" "base" {
  name = "test"
  # ubuntu 14.04
  image = "aecaed73-51a5-4439-a127-6d8229847145"
  type = "C2S"
}

resource "scaleway_volume" "test" {
  name = "test"
  size_in_gb = 20
  type = "l_ssd"
}

resource "scaleway_volume_attachment" "test" {
  server = "${scaleway_server.base.id}"
  volume = "${scaleway_volume.test.id}"
}

resource "scaleway_security_group" "base" {
  name = "public"
  description = "public gateway"
}

resource "scaleway_security_group_rule" "http-ingress" {
  security_group = "${scaleway_security_group.base.id}"

  action = "accept"
  direction = "inbound"
  ip_range = "0.0.0.0/0"
  protocol = "TCP"
  port = 80
}

resource "scaleway_security_group_rule" "http-egress" {
  security_group = "${scaleway_security_group.base.id}"

  action = "accept"
  direction = "outbound"
  ip_range = "0.0.0.0/0"
  protocol = "TCP"
  port = 80
}
```

Note that volume attachments require the server to be stopped, which can lead to
downtimes of you attach new volumes to already used servers

* Update IP read to handle 404 gracefully

* Read back resource on update

* Ensure IP detachment works as expected

Sadly this is not part of the official scaleway api just yet

* Adjust detachIP helper

based on feedback from @QuentinPerez in
scaleway/scaleway-cli#378

* Cleanup documentation

* Rename api_key to access_key

following @stack72 suggestion and rename the provider api_key for more clarity

* Make tests less chatty by using custom logger
@sheerun
Copy link
Contributor

sheerun commented Sep 5, 2016

@nicolai86 For some reason remote-exec provisioner is not working with scaleway resouces. It's stuck in endless loop:

scaleway_server.tunnel: Still creating... (3m20s elapsed)
scaleway_server.tunnel (remote-exec): Connecting to remote host via SSH...
scaleway_server.tunnel (remote-exec):   Host:
scaleway_server.tunnel (remote-exec):   User: root
scaleway_server.tunnel (remote-exec):   Password: false
scaleway_server.tunnel (remote-exec):   Private key: false
scaleway_server.tunnel (remote-exec):   SSH Agent: true

@nicolai86
Copy link
Contributor Author

nicolai86 commented Sep 5, 2016

@sheerun I've got a more complex example of the scaleway provider up on github which works just fine with remote provisioner.

Assuming you can SSH into your instance this should work just fine. Note that this implies either a) dynamic_ip_required = true, because otherwise your instance does not have a public accessible ip, or b) the usage of a jump host.

The above repo demos both.

@nicolai86
Copy link
Contributor Author

To summarize the use of remote-exec:

option A
use of dynamic_ip_required

resource "scaleway_server" "server" {
  image               = "${var.image}"
  type                = "${var.type}"
  dynamic_ip_required = true

  provisioner "remote-exec" {
    inline = "echo hello world"
  }
}

this works because setting dynamic_ip_required = true gives us a public IP to connect to.

option B
assuming you have a jump host with a public accessible IP (see option A to create one)
use the jump host to connect:

resource "scaleway_server" "server" {
  image = "${var.image}"
  type  = "${var.type}"

  connection {
    type         = "ssh"
    user         = "root"
    host         = "${self.private_ip}"
    bastion_host = "${var.bastion_host}"
    bastion_user = "root"
    agent        = true
  }

  provisioner "remote-exec" {
    inline = "echo hello world"
  }
}

@ghost
Copy link

ghost commented Apr 22, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 22, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement new-provider waiting-response An issue/pull request is waiting for a response from the community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants