Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Travis CI User
committed
Jun 9, 2018
0 parents
commit 9f874c3
Showing
567 changed files
with
64,792 additions
and
0 deletions.
There are no files selected for viewing
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| <!DOCTYPE html> | ||
| <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> | ||
| <!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> | ||
| <!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> | ||
| <!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> | ||
| <head> | ||
| <meta charset="UTF-8"> | ||
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> | ||
| <title> Ansible orchestration | ||
| </title> | ||
| <meta name="description" content=""> | ||
| <meta name="viewport" content="width=device-width"> | ||
| <link rel="stylesheet" href="https://visibilityspots.github.io/blog/theme/css/normalize.css"> | ||
| <link href='https://fonts.googleapis.com/css?family=Lato' rel='stylesheet' type='text/css'> | ||
| <link href='https://fonts.googleapis.com/css?family=Oswald' rel='stylesheet' type='text/css'> | ||
| <link rel="stylesheet" href="https://visibilityspots.github.io/blog/theme/css/font-awesome.min.css"> | ||
| <link rel="stylesheet" href="https://visibilityspots.github.io/blog/theme/css/main.css"> | ||
| <link rel="stylesheet" href="https://visibilityspots.github.io/blog/theme/tipuesearch/tipuesearch.css"> | ||
|
|
||
| <link rel="stylesheet" href="https://visibilityspots.github.io/blog/theme/css/blog.css"> | ||
| <link rel="stylesheet" href="https://visibilityspots.github.io/blog/theme/css/github.css"> | ||
| <link href="https://visibilityspots.github.io/blog/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="visibilityspots Atom Feed" /> | ||
| <link href="https://visibilityspots.github.io/blog/feeds/all.rss.xml" type="application/rss+xml" rel="alternate" title="visibilityspots RSS Feed" /> | ||
| <script src="https://visibilityspots.github.io/blog/theme/js/vendor/modernizr-2.6.2.min.js"></script> | ||
| </head> | ||
| <body> | ||
| <!--[if lt IE 7]> | ||
| <p class="chromeframe">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> or <a href="http://www.google.com/chromeframe/?redirect=true">activate Google Chrome Frame</a> to improve your experience.</p> | ||
| <![endif]--> | ||
|
|
||
| <div id="wrapper"> | ||
| <header id="sidebar" class="side-shadow"> | ||
| <hgroup id="site-header"> | ||
| <a id="site-title" href="https://visibilityspots.github.io/blog"><h1>visibilityspots <i class="icon-dashboard"></i></h1></a> | ||
| <p id="site-desc"> Linux & Open-Source enthusiast | Scouting | Longboarding </p> | ||
| </hgroup> | ||
| <form id="searchform" action="https://visibilityspots.github.io/blog/search.html" onsubmit="return (this.elements['q'].value.length > 0)"> | ||
| <input id="searchbox" type="text" name="q" size="12" placeholder="search"> | ||
| </form> | ||
| <nav> | ||
| <ul id="nav-links"> | ||
| <li><a href="https://visibilityspots.github.io/blog/">Blog</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/pages/profile.html">Profile</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/pages/links.html">Links</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/pages/projects.html">Projects</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/pages/tools.html">Tools</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/pages/contact.html">Contact</a></li> | ||
| </ul> | ||
| </nav> | ||
| Categories: | ||
| <ul> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/android.html">android</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/apple.html">apple</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/automation.html">automation</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/cloud.html">cloud</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/conferences.html">conferences</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/containers.html">containers</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/home-automation.html">home-automation</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/linux.html">linux</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/networking.html">networking</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/php.html">php</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/puppet.html">puppet</a></li> | ||
| <li><a href="https://visibilityspots.github.io/blog/category/security.html">security</a></li> | ||
| </ul> | ||
| <footer id="site-info"> | ||
| <a href="http://creativecommons.org/licenses/by-nc/2.0/be/deed.nl">License</a> | 2009 - 2018 <a href="https://visibilityspots.github.io/blog">visibilityspots.github.io/blog</a> | Generated by <a href="http://getpelican.com/" target="pelican">Pelican</a> | Hosted at <a href="https://pages.github.com/" target="github pages">github pages</a> | <a href="https://visibilityspots.github.io/blog/feeds/all.atom.xml" rel="alternate">Atom</a> feed | ||
| </footer></header> | ||
| <div id="post-container"> | ||
| <ol id="post-list"> | ||
| <li> | ||
| <article class="post-entry"> | ||
| <header class="entry-header"> | ||
| <a href="https://visibilityspots.github.io/blog/ansible-orchestration.html" rel="bookmark"><h1>Ansible orchestration</h1></a> | ||
| <div><p>takes 1 minute to read</p></div> <time class="post-time" datetime="2014-10-21T23:00:00+02:00" pubdate> | ||
| last modified at Fri 21 November 2014 | ||
| </time> | ||
| </header> | ||
| <section class="post-content"> | ||
| <p>I do use <a href="https://docs.puppetlabs.com/#puppetpuppet">puppet</a> as our main configuration management tool. Together with <a href="https://docs.puppetlabs.com/#puppetdbpuppetdblatest">puppetdb</a> all our services are automatically configured from bottom to top.</p> | ||
| <p>And it rocks, getting automated as much as possible it is like easy as hell to get a server up and running. The only feature it lacked in my opinion is orchestration. I do know about <a href="http://puppetlabs.com/mcollective">collective</a> which is made for this purpose.</p> | ||
| <p>Only it's yet again using an agent which fails from time to time and eating resources which can be avoided. It's the same reason I don't use the puppet agent daemon but trigger puppet every time.</p> | ||
| <h1>orchestration</h1> | ||
| <p>We have puppet running every 15 minutes through cron, main reason is to pick up and install the latest software which has been deployed. The other reason puppet runs after installation is to make sure the configuration files were not manually manipulated and making sure necessary services are still running.</p> | ||
| <p>Using puppet for making sure services are running and configuration files are not being changed an hourly puppet run would be enough. Thing is for those deployment flows it's merely like polling. And I strongly hate polling jobs, 99% of the time they don't have to do anything. So to me it's just useless, a waste of time, energy and resources.</p> | ||
| <p>It meant that developers had to wait in worst case scenario 15 minutes before their changes where deployed on the development environment. Their changes were already processed by jenkins, packages are been made, deployed on the repository only waiting for puppet to install the latest version of them. Nobody complained, but in my opinion it was waaay too long!</p> | ||
| <p>By running puppet immediately after the package is been deployed to the repository the right order of installing, configuring and restarting the necessary services can be executed. This will gain time for deployments next to some hourly puppet cron jobs which are running just to be sure no configuration has changed manually and the services are still running.</p> | ||
| <h1>ansible</h1> | ||
| <p>So I started looking at some solution where I could trigger a puppet run on the hosts configured the software through puppet in the right environment as soon as the package is deployed to the proper repository through jenkins.</p> | ||
| <p>At first I looked into the ssh jenkins plugin, it works but has one big disadvantage. You have to configure ssh credentials for every host in jenkins and therefore you can't use abstract jenkins flows cause you need to configure in each job the specific ssh credentials.</p> | ||
| <p>I looked further and came across <a href="http://www.ansible.com">ansible</a>. You don't have to configure a client on every host, neither you have to configure a per server based jenkins configuration to get it working. It was a blessing, the only things you have to do is creating a user, his public ssh key and grant him sudo rights on every server. This can easily be done through puppet!</p> | ||
| <h1>static inventory</h1> | ||
| <p>At first I crawled through our <a href="http://www.theforeman.org">foreman</a> instance and copied over the nodes into 2 groups, development and production, the puppet environments. I also configured some stuff like ssh port and user. I refused to configure the root pw in some plain text file on the jenkins node. That's not safe at all in my opinion, instead I created an ssh key pair and distributed the public key on all servers.</p> | ||
| <p>In my fight to automate as much as possible this wasn't the most efficient way of using the inventory. Every time you removed or added a node you had to reconfigure it yourself manually in the first place. Beside the manual intervention you also have to take note how you are gonna perform that manual action? Manipulating configuration data on the production machine is not done, using a git repository which you package or adding them to puppet, which both sounds wrong. The first because it's overkill the second because it's rather data over configuration.</p> | ||
| <h1>dynamic inventory</h1> | ||
| <p>In my quest I got pointed to a <a href="https://github.com/EchoTeam/ansible-plugins">python</a> script by a colleague. Unfortunately the script isn't straight forward and the 'maintainers' hides themselves behind their footer:</p> | ||
| <div class="highlight"><pre><span></span> Notice: The puppetdb inventory plugin is not quite generic for the moment. Use more as an example. | ||
| </pre></div> | ||
|
|
||
|
|
||
| <p>Once I found out about the <a href="http://docs.ansible.com/developing_inventory.html">inventory</a> part of ansible I knew what I was looking for and saw the light by an <a href="https://blog.codecentric.de/en/2014/09/use-ansible-remote-executor-puppet-environment/">article</a> on cedecentric.de. Their was only one issue, my jenkins host which needs ansible to run isn't my puppetmaster and therefore can't list the signed certificates as used in his script.</p> | ||
| <p>But I am using <a href="https://docs.puppetlabs.com/puppetdb/latest/index.html">puppetdb</a>, and puppetdb has a great <a href="https://docs.puppetlabs.com/puppetdb/2.2/api/index.html">API</a>. So I could take advantage of it by using this great API, melting it down into an inventory script and using the json generated output through ansible.</p> | ||
| <p>So I started modifying the code example I found on codecentrec and got it working by writing a <a href="https://github.com/visibilityspots/ansible-puppet-inventory">puppetdb.sh</a> dynamic inventory script. Together with the <a href="https://github.com/visibilityspots/puppet-ansible">puppet-ansible</a> module it even got automated too!</p> | ||
| <h1>adding it to ansible-core</h1> | ||
| <p>I went to the <a href="http://events.linuxfoundation.org/events/cloudstack-collaboration-conference-europe">Cloudstack Collaboration Conference</a> in Budapest where I followed a <a href="https://github.com/runseb/runseb.github.io/blob/master/ONEPAGE.md">tutorial</a> by <a href="http://sebgoa.blogspot.hu/">Sebastien Goasguen</a>.</p> | ||
| <p>It turned out he wrote an ansible apache-libcloud inventory script and tried to pushing it in to the ansible core. This inspired my to rewrite my bash script in python so it could be added to ansible-core too.</p> | ||
| <p>After fooling around a bit in python I used the <a href="https://github.com/puppet-community/pypuppetdb">pypuppetdb</a> library so I don't have to make all the API calls natively myself through urllib request. And it turned out quite fine and I got it up and running in my setup. So those days I'm waiting on feedback from the ansible community to my <a href="https://github.com/ansible/ansible/pull/9593">pull request</a> so everyone can benefit of the joy between ansible and puppet.</p> | ||
| <h1>still need some attention</h1> | ||
| <p>I need some time to look which processes it takes to run a command through ansible so I could specify more clear the sudoers file.</p> | ||
| <p>Also the environments should be more abstract in my puppetdb.sh script without having to manually adapt the necessary puppetdb query files.</p> | ||
| <h1>drinking cocktails</h1> | ||
| <p>From now on it only takes less than 5 minutes to push your code, get it through jenkins tests into a package on an apt or yum repository got pulled into a repository and deploy it through puppet using ansible on the development servers. All without any manual action, without any cron job all automated, glued the pieces together.</p> | ||
| <p>I'll dig deeper into the whole deployment process later on, when I found time between drinking cocktails, looking at my daughter and living the dream.</p> | ||
| </section> | ||
| <hr/> | ||
| <aside class="post-meta"> | ||
| <p>Category: <a href="https://visibilityspots.github.io/blog/category/puppet.html">puppet</a></p> | ||
| <p>Tags: <a href="https://visibilityspots.github.io/blog/tag/ansible.html">ansible</a>, <a href="https://visibilityspots.github.io/blog/tag/orchestration.html">orchestration</a>, <a href="https://visibilityspots.github.io/blog/tag/tool.html">tool</a>, <a href="https://visibilityspots.github.io/blog/tag/puppet.html">puppet</a>, <a href="https://visibilityspots.github.io/blog/tag/dynamic.html">dynamic</a>, <a href="https://visibilityspots.github.io/blog/tag/inventory.html">inventory</a>, <a href="https://visibilityspots.github.io/blog/tag/puppetdb.html">puppetdb</a>, </p> | ||
| </aside> | ||
| <hr/> | ||
| <div class="comments"> | ||
| <div id="disqus_thread"></div> | ||
| <script type="text/javascript"> | ||
| var disqus_shortname = 'visibilityspots'; | ||
| (function() { | ||
| var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; | ||
| dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js'; | ||
| (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); | ||
| })(); | ||
| </script> | ||
| <noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript> | ||
| <a href="http://disqus.com" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a> | ||
| </div> | ||
| </article> | ||
| </li> | ||
| </ol> | ||
| </div> | ||
| </div> | ||
|
|
||
| <script src="https://visibilityspots.github.io/blog/theme/js/main.js"></script> | ||
| </body> | ||
| </html> |
Oops, something went wrong.