Salt Cloud does not recognise all Digitalocean sizes

[johje349]

Description of Issue/Question

salt-cloud -l debug -p asx-dse-cookies asx-dse-cookies1

[DEBUG   ] Sending event: tag = salt/cloud/asx-dse-cookies1/creating; data = {'profile': 'asx-dse-cookies', '_stamp': '2018-02-02T08:14:48.175113', 'name': 'asx-dse-cookies1', 'driver': 'digital_ocean', 'provider': 'do:digital_ocean', 'event': 'starting create'}
[INFO    ] Creating Cloud VM asx-dse-cookies1
[DEBUG   ] https://api.digitalocean.com/v2/sizes/
Usage: salt-cloud [options] <-m MAP | -p PROFILE> <NAME> [NAME2 ...]

salt-cloud: error: There was a profile error: The specified size, 's-12vcpu-48gb', could not be found.

It seems salt-cloud only lists 20 entries coming from the API:

https://api.digitalocean.com/v2/sizes" | jq '.' | grep slug
"slug": "512mb",
      "slug": "s-1vcpu-1gb",
      "slug": "1gb",
      "slug": "s-3vcpu-1gb",
      "slug": "s-1vcpu-2gb",
      "slug": "2gb",
      "slug": "s-2vcpu-2gb",
      "slug": "s-1vcpu-3gb",
      "slug": "c-2",
      "slug": "s-2vcpu-4gb",
      "slug": "4gb",
      "slug": "s-4vcpu-8gb",
      "slug": "8gb",
      "slug": "c-4",
      "slug": "16gb",
      "slug": "c-8",
      "slug": "s-6vcpu-16gb",
      "slug": "s-8vcpu-32gb",
      "slug": "c-16",
      "slug": "32gb",

Digitalocean recently added more droplet sizes.

The size s-12vcpu-48gb is indeed there if listing more pages or allowing more entries per page:

https://api.digitalocean.com/v2/sizes?per_page=500 | jq '.' | grep slug
"slug": "512mb",
      "slug": "s-1vcpu-1gb",
      "slug": "1gb",
      "slug": "s-3vcpu-1gb",
      "slug": "s-1vcpu-2gb",
      "slug": "2gb",
      "slug": "s-2vcpu-2gb",
      "slug": "s-1vcpu-3gb",
      "slug": "c-2",
      "slug": "s-2vcpu-4gb",
      "slug": "4gb",
      "slug": "s-4vcpu-8gb",
      "slug": "8gb",
      "slug": "c-4",
      "slug": "16gb",
      "slug": "c-8",
      "slug": "s-6vcpu-16gb",
      "slug": "s-8vcpu-32gb",
      "slug": "c-16",
      "slug": "32gb",
      "slug": "s-12vcpu-48gb",
      "slug": "48gb",
      "slug": "64gb",
      "slug": "s-16vcpu-64gb",
      "slug": "c-32",
      "slug": "c-48",
      "slug": "s-20vcpu-96gb",
      "slug": "s-24vcpu-128gb",
      "slug": "s-32vcpu-192gb",

Setup

asx-dse-cookies:
provider: do
image: ubuntu-16-04-x64
size: s-12vcpu-48gb
location: ams3
private_networking: true

Steps to Reproduce Issue

See above

Versions Report

Salt Version:
Salt: 2017.7.2

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.4.2
docker-py: Not Installed
gitdb: 0.6.4
gitpython: 1.0.1
ioflo: Not Installed
Jinja2: 2.8
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.3
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.12 (default, Dec 4 2017, 14:50:18)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.2.0
RAET: Not Installed
smmap: 0.9.0
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4

System Versions:
dist: Ubuntu 16.04 xenial
locale: UTF-8
machine: x86_64
release: 4.4.0-112-generic
system: Linux
version: Ubuntu 16.04 xenial

[johje349]

Same goes for --list-sizes, only 20 entries:

salt-cloud --list-sizes=do | grep [0-9].*:
        16gb:
        1gb:
        2gb:
        32gb:
        4gb:
        512mb:
        8gb:
        c-16:
        c-2:
        c-4:
        c-8:
        s-1vcpu-1gb:
        s-1vcpu-2gb:
        s-1vcpu-3gb:
        s-2vcpu-2gb:
        s-2vcpu-4gb:
        s-3vcpu-1gb:
        s-4vcpu-8gb:
        s-6vcpu-16gb:
        s-8vcpu-32gb:

[gtmanfred]

We should be just querying their sizes api...

https://github.com/saltstack/salt/blob/oxygen/salt/cloud/clouds/digitalocean.py#L164

If you wanted to submit a PR to fix that, we would greatly appreciate it.

Thanks!
Daniel