New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

salt-cloud no longer picking up master setting from provider file #45125

Closed
wes-novack opened this Issue Dec 21, 2017 · 9 comments

Comments

Projects
None yet
5 participants
@wes-novack

wes-novack commented Dec 21, 2017

Description of Issue/Question

With salt-master and salt-cloud 2017.7.2, salt-cloud is no longer picking up the config for minion:master:dns defined in a salt-cloud provider file. This previously worked in salt 2016.11.X

Setup

provider.conf file

myprovider:
  minion:
    master: mymaster.dns.com
  ssh_interface: private_ips
  id: 'redacted'
  key: 'redacted'
  private_key: /etc/salt/mykey.pem
  keyname: mykey
  location: us-west-2
  availability_zone: us-west-2a
  ssh_username:
    - ubuntu
  driver: ec2

profile.conf file

myprofile:
  ssh_interface: private_ips
  block_device_mappings:
    - DeviceName: /dev/sda1
      Ebs.VolumeSize: 30
      Ebs.VolumeType: gp2
  provider: myprovider
  image: ami-myamiid
  script_args: stable 2017.7.2
  size: t2.micro
  ssh_username: ubuntu
  network_interfaces:
    - DeviceIndex: 0
      SubnetId: subnet-mysubnetid
      SecurityGroupId:
        - sg-mysgid
  minion:
    grains:
      env: myenv
      roles:
        - myrole
      team: myteam
  tag:
    env: myenv
    org: myorg
    roles: myrole
    team: myteam

Steps to Reproduce Issue

salt-cloud -p myprofile myprofile-host-1
[WARNING ] There's no master defined on the 'myprofile-host-1' VM settings.

Minion does not connect to master or receive the appropriate master setting after salt-cloud spins it up and bootstraps it.

Versions Report

Salt Version:
           Salt: 2017.7.2
 
Dependency Versions:
           cffi: 1.5.2
       cherrypy: 3.2.3
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: 0.24.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.24.0
         Python: 2.7.12 (default, Nov 20 2017, 18:23:56)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-1041-aws
         system: Linux
        version: Ubuntu 16.04 xenial

Workaround

If I use the salt-cloud root config file located at /etc/salt/cloud instead of the provider file, then it works fine.

minion:
  master: mymaster.dns.com

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Dec 21, 2017

I think I have noticed this as well.

I am going to try and see if I can bisect this and find the problem.

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Dec 21, 2017

It looks like merging configurations between the provider and the profile was accidentally ripped out when we deprecated the keyword provider for driver

88936c2

I am only able to replicate this if I also have a minion dictionary in the profile as well.

Thanks for reporting.
Daniel

@gtmanfred gtmanfred added this to the Approved milestone Dec 21, 2017

@gtmanfred gtmanfred self-assigned this Dec 21, 2017

@gtmanfred gtmanfred added the P2 label Dec 21, 2017

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Dec 21, 2017

@techhat do you remember where these configs used to get merged?

Thanks,
Daniel

@techhat

This comment has been minimized.

Member

techhat commented Dec 21, 2017

I don't. I believe @rallytime did the conversion from provider to driver, she might have a better idea.

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Dec 22, 2017

Sure, but do you remember where in the code the provider and profile configs are merged?

@Reiner030

This comment has been minimized.

Reiner030 commented Mar 9, 2018

To have it also here:

Release of 2017.7.0 was "15 Aug 2017" which breaks this basic functionality... so now over ~ 7 months ago.

  • my bug report was done 2 weeks after the release 2017.7.0
  • this bug report was done 4 months after the release 2017.7.0
  • And even "Bug High Severity labels on 21 Dec 2017" was added on this issue 2,5 months later ( ~ 6,5 months after release 2017.7.0) there is still no commit/test done for this important (?) bug yet.

Also a little wondering how few people seems to use latest version/use enhanced provider file for their configuration setup...

I hope we have not to celebrate birthday for this bug - normally git history should have all changes inside ;).
And as I can check without proper Python test environment the salt-cloud states are very compact.
There is sadly no big jinja templating possible (only in map file some rare access but not in provider / profile ones).

@thetoolsmith

This comment has been minimized.

Contributor

thetoolsmith commented Mar 27, 2018

I am waiting for this as well. Installing latest salt-master on a new system, 2017.7.4.
We auto-generate our providers, profiles and cloud maps. This is a serious bug IMO.

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Mar 27, 2018

If someone wants to submit a fix for this bug. I believe it is in this function.

https://github.com/saltstack/salt/blob/2017.7/salt/cloud/__init__.py#L647

And was caused by this commit probably 88936c2

Thanks,
Daniel

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented May 15, 2018

This should be fixed by #47665

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment