Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure support #54

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,7 @@ vendor
*.bak
*.out
*.db
*.env
*.env
*#
*.#*
*~
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are these lines for?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emacs backup files

Copy link
Member

@bradrydzewski bradrydzewski Oct 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these should be removed and added to your global .gitignore file or in .git/info/exclude

71 changes: 67 additions & 4 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,11 @@
[[constraint]]
name = "github.com/lib/pq"
version = "1.0.0"

[[constraint]]
name = "github.com/Azure/azure-sdk-for-go"
version = "32.5.0"

[[override]]
name = "github.com/Azure/go-autorest"
version = "13.0.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the Gopkg files were removed and replaced with go.mod files. You may want to sync up your codebase with the latest version

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

K will move to go.mod

21 changes: 21 additions & 0 deletions cmd/drone-autoscaler/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"github.com/drone/autoscaler"
"github.com/drone/autoscaler/config"
"github.com/drone/autoscaler/drivers/amazon"
"github.com/drone/autoscaler/drivers/azure"
"github.com/drone/autoscaler/drivers/digitalocean"
"github.com/drone/autoscaler/drivers/google"
"github.com/drone/autoscaler/drivers/hetznercloud"
Expand Down Expand Up @@ -260,6 +261,26 @@ func setupProvider(c config.Config) (autoscaler.Provider, error) {
amazon.WithIamProfileArn(c.Amazon.IamProfileArn),
amazon.WithMarketType(c.Amazon.MarketType),
), nil
case os.Getenv("AZURE_CLIENT_ID") != "":
return azure.New(
azure.WithVMSize(c.Azure.VMSize),
azure.WithSubscriptionId(c.Azure.Subscription),
azure.WithResourceGroup(c.Azure.ResourceGroup),
azure.WithImage(c.Azure.Image),
azure.WithLocation(c.Azure.Location),
azure.WithAdminUsername(c.Azure.AdminUsername),
azure.WithAdminPassword(c.Azure.AdminPassword),
azure.WithVMName(c.Azure.VMName),
azure.WithSSHKey(c.Azure.SSHKey),
azure.WithVNet(c.Azure.VNet),
azure.WithNSG(c.Azure.NSG),
azure.WithSubnet(c.Azure.Subnet),
azure.WithVolumeSize(c.Azure.VolumeSize),
azure.WithImageOffer(c.Azure.ImageOffer),
azure.WithImagePublisher(c.Azure.ImagePub),
azure.WithImageSKU(c.Azure.ImageSKU),
azure.WithImageVersion(c.Azure.ImageVer),
), nil
case os.Getenv("OS_USERNAME") != "":
return openstack.New(
openstack.WithImage(c.OpenStack.Image),
Expand Down
22 changes: 22 additions & 0 deletions config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ type (
Environ []string
Volumes []string
Labels map[string]string `envconfig:"DRONE_AGENT_LABELS"`
DockerUsername string `envconfig:"DRONE_AGENT_DOCKER_USERNAME"`
DockerPassword string `envconfig:"DRONE_AGENT_DOCKER_PASSWORD"`
}

Runner Runner
Expand Down Expand Up @@ -93,6 +95,26 @@ type (
Datasource string `default:"database.sqlite?cache=shared&mode=rwc&_busy_timeout=9999999"`
}

Azure struct {
Subscription string `envconfig:"DRONE_AZURE_SUBSCRIPTION_ID"`
ResourceGroup string `envconfig:"DRONE_AZURE_RESOURCE_GROUP"`
Image string `envconfig:"DRONE_AZURE_IMAGE"`
Location string `envconfig:"DRONE_AZURE_LOCATION"`
AdminUsername string `envconfig:"DRONE_AZURE_ADMIN_USERNAME"`
AdminPassword string `envconfig:"DRONE_AZURE_ADMIN_PASSWORD"`
VMName string `envconfig:"DRONE_AZURE_VMNAME"`
SSHKey string `envconfig:"DRONE_AZURE_SSHKey"`
VolumeSize int32 `envconfig:"DRONE_AZURE_VOLUMESIZE"`
VMSize string `envconfig:"DRONE_AZURE_VMSIZE"`
VNet string `envconfig:"DRONE_AZURE_VNET"` // Virtual network
Subnet string `envconfig:"DRONE_AZURE_SUBNET"` // Subnetwork
NSG string `envconfig:"DRONE_AZURE_NSG"` // Network security Group
ImageOffer string `envconfig:"DRONE_AZURE_IMAGE_OFFER" default:"UbuntuServer"`
ImagePub string `envconfig:"DRONE_AZURE_IMAGE_PUBLISHER" default:"Canonical"`
ImageSKU string `envconfig:"DRONE_AZURE_IMAGE_SKU" default:"18.04-LTS"`
ImageVer string `envconfig:"DRONE_AZURE_IMAGE_VERSION" default:"latest"`
}

Amazon struct {
DeviceName string `envconfig:"DRONE_AMAZON_DEVICE_NAME"`
Image string
Expand Down
113 changes: 113 additions & 0 deletions drivers/azure/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,116 @@
// that can be found in the LICENSE file.

package azure

import (
"context"
"github.com/drone/autoscaler"
"github.com/rs/zerolog/log"
"fmt"
"time"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


"bytes"
"encoding/base64"

"github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2019-07-01/compute"
"github.com/Azure/go-autorest/autorest/to"
)

func (p *provider) computeMachineRequest(ctx context.Context, base string, opts autoscaler.InstanceCreateOpts) (compute.VirtualMachine, *string, error) {
buf := new(bytes.Buffer)
err := p.userdata.Execute(buf, &opts)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unhandled exception

cloudInit := base64.StdEncoding.EncodeToString(buf.Bytes())

nic, ip, err := p.CreateNIC(ctx, p.vnet, p.subnet, p.nsg, base+"-ip", base+"-nic")
if err != nil {
return compute.VirtualMachine{}, nil, err
}

return compute.VirtualMachine{
Location: to.StringPtr(p.location),
VirtualMachineProperties: &compute.VirtualMachineProperties{
HardwareProfile: &compute.HardwareProfile{
VMSize: p.vmSize,
},
StorageProfile: &compute.StorageProfile{
OsDisk: &compute.OSDisk{
Name: to.StringPtr(base + "-" + "osdisk"),
CreateOption: compute.DiskCreateOptionTypesFromImage,
DiskSizeGB: to.Int32Ptr(p.volumeSize),
},
ImageReference: &compute.ImageReference{
Offer: to.StringPtr(p.imageOffer),
Publisher: to.StringPtr(p.imagePublisher),
Sku: to.StringPtr(p.imageSKU),
Version: to.StringPtr(p.imageVersion),
},
},
NetworkProfile: &compute.NetworkProfile{
NetworkInterfaces: &[]compute.NetworkInterfaceReference{
{
ID: nic.ID,
NetworkInterfaceReferenceProperties: &compute.NetworkInterfaceReferenceProperties{
Primary: to.BoolPtr(true),
},
},
},
},
OsProfile: &compute.OSProfile{
ComputerName: to.StringPtr(base + "-" + "vm"),
AdminUsername: to.StringPtr(p.adminUsername),
AdminPassword: to.StringPtr(p.adminPassword),
CustomData: &cloudInit,
},
},
}, ip.PublicIPAddressPropertiesFormat.IPAddress, nil
}

func (p *provider) Create(ctx context.Context, opts autoscaler.InstanceCreateOpts) (*autoscaler.Instance, error) {
logger := log.Ctx(ctx)
logger.Debug().Msg("Azure creating an instance.")

client, err := p.getClient()
if err != nil {
return nil, err
}

base := opts.Name
computeRequest, ipaddress, err := p.computeMachineRequest(ctx, base, opts)
if err != nil {
return nil, err
}

future, err := client.CreateOrUpdate(
ctx,
p.resourceGroup,
base+"-vm",
computeRequest,
)

err = future.WaitForCompletionRef(ctx, client.Client)
if err != nil {
return nil, err
}

azureInstance, err := future.Result(client)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unhandled error

client.Start(ctx, p.resourceGroup, p.vmName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does start return any parameters? if it returns an error it needs to be handled


if err != nil {
return nil, err
}

fmt.Printf("Starting sleep: %v\n", time.Now().Unix())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use logger instead of fmt.Print or println

time.Sleep(180 * time.Second)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should probably alter this to just poll on the static IP I provide until it's available. This was a quick hack to fix an issue I had noticed around an agent getting assigned at times +1 builds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this should use polling, which is how we handle this for the other providers. You can use this example as reference https://github.com/drone/autoscaler/blob/master/drivers/amazon/create.go#L122:L175

fmt.Printf("Ending sleep: %v\n", time.Now().Unix())

instance := &autoscaler.Instance{
Provider: autoscaler.ProviderAzure,
Address: *ipaddress,
ID: base, // Set it to name because required for deallocating.
Size: string(azureInstance.HardwareProfile.VMSize),
Region: *azureInstance.Location,
Image: *azureInstance.StorageProfile.ImageReference.Sku,
}

return instance, nil
}
Loading