Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

salt-cloud grains inheritance from provider to profile (2018 vs 2016) #49226

Closed
defanator opened this issue Aug 21, 2018 · 9 comments

Comments

Projects
None yet
5 participants
@defanator
Copy link
Contributor

commented Aug 21, 2018

Description of Issue/Question

After upgrading salt-cloud from 2016.11.6 to 2018.3.2 we've been observing a situation when provider-specific grains have not been applied to the target instance.

Setup

Excerpt from our initial configuration that was working fine with 2016.11.6:

prov1:
  driver: ec2
  minion:
    master: x.x.x.x
  grains:
    prov1_grain1: foo
    prov2_grain2: bar

profile1:
  provider: prov1
  grains:
    env: nonprod
    roles:
      - role1
      - role2
      - role2

Target instance launched with 2016.11.6 contained all the grains merged in a proper way, i.e.:

prov1_grain1: foo
prov1_grain2: bar
env: nonprod
roles:
- role1
- role2
- role3

Target instance launched with 2018.3.2 contained just the grains from profile:

env: nonprod
roles:
- role1
- role2
- role3

We finally managed to get it working by changing provider configuration to the following:

prov1:
  driver: ec2
  minion:
    master: x.x.x.x
    grains:
      prov1_grain1: foo
      prov2_grain2: bar

I.e., grains tree should be under minion now.

However, documentation examples still contain older (legacy?) syntax:

# Note: This example is for /etc/salt/cloud.providers or any file in the
# /etc/salt/cloud.providers.d/ directory.

my-ec2-southeast-public-ips:
  # Set up the location of the salt master
  #
  minion:
    master: saltmaster.example.com

  # Set up grains information, which will be common for all nodes
  # using this provider
  grains:
    node_type: broker
    release: 1.0.1

(from https://docs.saltstack.com/en/latest/topics/cloud/aws.html)

My questions so far:

  • is current behavior correct? (grains should be nested from minion in provider config, but not in profile config),
  • should a documentation be changed accordingly?

Thanks!

@gtmanfred

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2018

Can you test the 2018.3.3 branch?

#47665

This pr should fix this issue, and will be released in the 2018.3.3 and 2017.7.8 versions.

I believe this issue is a duplicate of #45125

@gtmanfred gtmanfred added the Duplicate label Aug 21, 2018

@gtmanfred gtmanfred added this to the Blocked milestone Aug 21, 2018

@defanator

This comment has been minimized.

Copy link
Contributor Author

commented Aug 21, 2018

@gtmanfred I've been running all tests on 2018.3.3 branch in fact, sorry for not mentioning it. I searched for similar issues, found #45125 and realized that packaged salt-common 2018.3.2 does not contain changes from #47655, so I switched to git repository for tests and picked 2018.3.3 branch.

# git show
commit 134f125b961c4e0307049b17da471f8f38108d64
Merge: b3a247b a2d2cd3
Author: Mike Place <mp@saltstack.com>
Date:   Sat Aug 18 08:10:31 2018 -0400

    Merge pull request #49182 from terminalmage/salt-jenkins-1078
    
    Fix hanging syndic test

# git status
On branch 2018.3.3
Your branch is up-to-date with 'origin/2018.3.3'.
nothing to commit, working directory clean

# salt --versions-report
Salt Version:
           Salt: 2018.3.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 2.0.8
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-1065-aws
         system: Linux
        version: Ubuntu 16.04 xenial
@gtmanfred

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2018

You said above

Target instance launched with 2018.3.2 contained just the grains from profile:

So, you have launched instances with 2018.3.3? and this still happens? can you make sure that all .pyc files have been cleared out?

@defanator

This comment has been minimized.

Copy link
Contributor Author

commented Aug 21, 2018

@gtmanfred I searched for similar issues, found #45125 and realized that packaged salt-common 2018.3.2 does not contain changes from #47655, so I switched to git repository for tests and picked 2018.3.3 branch.

I'll double check for stale .pyc a bit later and let you know the results.

@defanator

This comment has been minimized.

Copy link
Contributor Author

commented Aug 21, 2018

@gtmanfred I've just removed all .pyc files within salt directory, returned provider configuration to the initial format:

prov1:
  driver: ec2
  minion:
    master: x.x.x.x
  grains:
    prov1_grain1: foo
    prov2_grain2: bar

and checked with launching new instance from that profile - provider's grains are missing.

Let me know if I can provide anything else to help with investigating this one.

@gtmanfred

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2018

Ok, i can replicate this, but i need to have more time to fix it, i am marking it as a blocker for the 2018.3.4 and 2017.7.9 releases.

@gtmanfred gtmanfred self-assigned this Aug 21, 2018

@sagetherage

This comment has been minimized.

Copy link

commented Dec 11, 2018

@gtmanfred un-assigning you and looking for a replacement, today.

@sagetherage sagetherage assigned dwoz and unassigned gtmanfred Dec 11, 2018

@defanator

This comment has been minimized.

Copy link
Contributor Author

commented Dec 12, 2018

@sagetherage @dwoz any chances to get this one fixed in 2018.3.4? TIA!

@sagetherage

This comment has been minimized.

Copy link

commented Dec 12, 2018

@defanator yes, our process is to label the ticket with the releases and this is also labeled with Blocker indicating we are working on a fix for those releases and not releasing until fixed. If anything changes you will see comments added to this ticket. I hope this is helpful information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.