12555 - Support for Windows Platform

Add initial support for Microsoft Windows.

The main issue is that we hard coded \tmp in a few places and used
Unix signal handlers that does not exist in windows

Windows has no Syslog and the syslog module for ruby is not in the
windows installers so we do a few tweaks to the syslog logger and
tests to just nil out in windows.  We should look into adding a
logger for the windows eventlog using win32-eventlog

In a few tests we used pwd and other unix utilities, we replace
all those to calls to a few ruby invocations that is more portable

Windows does not show colors the way unix does, rather than try to
create a windows compatible color output we just default to disabling
all color when run on windos

It's hard/impossible to figure out the correct terminal dimensions on
windows so the dynamic resizing progress bar behavior on Unix is not
reproducable - on windows we just default to 80 columns which is the
default for the terminal.

^C does not interrupt IO events on windows so we took some examples from
various large projects to just start a background thread that sleeps for
a second.  The effect of this is that the ruby VM has an oppertunity
once a second to process any ^C signals etc.

The help template defaults to the directory the cofig file is in and if
it cant find the help template there it falls back to the previous
defaults to maintain consistent behavior across versions.

A first stab at some support batch files to eventually facilitate
packaging is in ext/windows, this commit specifically does not take care
of any packaging requirements yet.  There's a quick README in that
directory to assist early adopters to test this code once released.
Actual packaging will happen in a different ticket/commit.

bin/mcollectived

@@ -42,16 +42,14 @@ end
 if config.daemonize
   MCollective::Log.debug("Starting in the background (#{config.daemonize})")
 
-  MCollective::Runner.daemonize do
-    if pid
-      begin
-        File.open(pid, 'w') {|f| f.write(Process.pid) }
-      rescue Exception => e
-      end
-    end
-
-    runner = MCollective::Runner.new(configfile)
-    runner.run
+  if MCollective::Util.windows?
+    require 'mcollective/windows_daemon'
+
+    MCollective::WindowsDaemon.daemonize_runner
+  else
+    require 'mcollective/unix_daemon'
+
+    MCollective::UnixDaemon.daemonize_runner(pid)
   end
 else
   MCollective::Log.debug("Starting in the foreground")

ext/windows/README.md

@@ -0,0 +1,34 @@
+These files support installing and using mcollective on MS Windows.
+
+Here are a few instructions for people who wish to do early adopter
+testing, before 2.0 is out we hope to have this packaged into a msi
+installer but your early feedback will help.
+
+Assuming you are installing mcollective into C:\marionette-collective:
+
+ * Install Ruby from http://rubyinstaller.org/, use 1.8.7
+ * Install the following gems: stomp, win32-process, win32-service,
+   sys-admin, windows-api
+ * extract the zip file or clone the git repo into C:\marionette-collective
+ * copy the files from C:\marionette-collective\ext\windows\*.* into
+   C:\marionette-collective\bin
+ * Install any plugins and their dependencies into C:\marionette-collective\plugins
+   specifically for the package and service agents you can install Puppet via gems
+ * Edit the configuration files setting:
+   * libdir = c:\marionette-collective\plugins
+   * logfile = c:\marionette-collective\mcollective.log
+   * plugin.yaml = c:\marionette-collective\etc\facts.ysml
+   * daemonize = 1
+ * change directories to c:\marionette-collective\bin and run register_service.bat
+
+At this point you would have your service registered into the windows service
+manager but set to manual start.  If you start it there it should run ok.
+
+If it does not run:
+
+ * Look in the log files, set it to debug level
+ * If the log files are empty look at the command the service wrapper runs
+   and run it by hand.  This will show you any early exception preventing it
+   from running.  It wont succesfully start but you should see why it does
+   not get far enough to start writing logs.
+

ext/windows/environment.bat

@@ -0,0 +1,16 @@
+SET BASEDIR=%~dp0..
+SET BASEDIR=%BASEDIR:\bin\..=%
+
+SET SERVER_CONFIG=%BASEDIR%\etc\server.cfg
+SET CLIENT_CONFIG=%BASEDIR%\etc\client.cfg
+
+SET MCOLLECTIVED=%BASEDIR%\bin\mcollectived
+SET MC_STARTTYPE=manual
+REM SET MC_STARTTYPE=auto
+
+SET PATH=%BASEDIR%\bin;%PATH%
+
+SET RUBYLIB=%BASEDIR%\lib;%RUBYLIB%
+SET RUBYLIB=%RUBYLIB:\=/%
+
+SET RUBY="ruby"

ext/windows/mco.bat

@@ -0,0 +1,7 @@
+@echo off
+
+SETLOCAL
+
+call "%~dp0environment.bat" %0 %*
+
+%RUBY% -S -- mco %* --config "%CLIENT_CONFIG%"

ext/windows/register_service.bat

@@ -0,0 +1,7 @@
+@echo off
+
+SETLOCAL
+
+call "%~dp0environment.bat" %0 %*
+
+%RUBY% -S -- service_manager.rb --install

ext/windows/service_manager.rb

@@ -0,0 +1,94 @@
+require 'optparse'
+
+opt = OptionParser.new
+
+ruby_path = ENV["RUBY"].gsub('"','')
+basedir = ENV["BASEDIR"]
+libdir = ENV["RUBYLIB"]
+mcollectived = ENV["MCOLLECTIVED"]
+configfile = ENV["SERVER_CONFIG"]
+
+unless File.exist?(ruby_path)
+  ENV["PATH"].split(File::PATH_SEPARATOR).each do |path|
+    ruby = File.join(path, "ruby.exe")
+
+    if File.exist?(ruby)
+      ruby_path = ruby
+      break
+    end
+  end
+end
+
+abort("Can't find ruby.ext in the path") unless ruby_path
+
+options = {:name    => "mcollectived",
+           :display_name => "The Marionette Collective",
+           :description => "Puppet Labs server orchestration framework",
+           :command => '%s -I"%s" -- "%s" --config "%s"' % [ ruby_path, libdir, mcollectived, configfile ]}
+
+action = false
+
+opt.on("--install", "Install service") do
+  action = :install
+end
+
+opt.on("--uninstall", "Remove service") do
+  action = :uninstall
+end
+
+opt.on("--name NAME", String, "Service name (#{options[:name]})") do |n|
+  options[:name] = n
+end
+
+opt.on("--description DESCRIPTION", String, "Service description (#{options[:description]})") do |v|
+  options[:description] = v
+end
+
+opt.on("--display NAME", String, "Service display name (#{options[:display_name]})") do |n|
+  options[:display_name] = n
+end
+
+opt.on("--command COMMAND", String, "Service command (#{options[:command]})") do |c|
+  options[:command] = c
+end
+
+opt.parse!
+
+abort "Please choose an action with --install or --uninstall" unless action
+
+require 'rubygems'
+require 'win32/service'
+
+include Win32
+
+case action
+  when :install
+    if ENV["MC_STARTTYPE"] =~ /auto/i
+      start_type = Service::AUTO_START
+    else
+      start_type = Service::DEMAND_START
+    end
+
+    Service.new(
+      :service_name => options[:name],
+      :display_name => options[:display_name],
+      :description => options[:description],
+      :binary_path_name => options[:command],
+      :service_type => Service::SERVICE_WIN32_OWN_PROCESS,
+      :start_type => start_type
+    )
+
+    puts "Service %s installed" % [options[:name]]
+
+  when :uninstall
+    Service.stop(options[:name]) unless Service.status(options[:name]).current_state == 'stopped'
+
+    while Service.status(options[:name]).current_state != 'stopped'
+      puts "Waiting for service %s to stop" % [options[:name]]
+      sleep 1
+    end
+
+    Service.delete(options[:name])
+
+    puts "Service %s removed" % [options[:name]]
+end

ext/windows/unregister_service.bat

@@ -0,0 +1,7 @@
+@echo off
+
+SETLOCAL
+
+call "%~dp0environment.bat" %0 %*
+
+%RUBY% -S -- service_manager.rb --uninstall

lib/mcollective.rb

@@ -9,6 +9,8 @@
 require 'shellwords'
 require 'mcollective/monkey_patches'
 require 'tempfile'
+require 'rbconfig'
+require 'tmpdir'
 
 # == The Marionette Collective
 #

lib/mcollective/application.rb

@@ -266,6 +266,8 @@ def run
 
       validate_configuration(configuration) if respond_to?(:validate_configuration)
 
+      Util.setup_windows_sleeper if Util.windows?
+
       main
 
       disconnect

lib/mcollective/config.rb

@@ -58,7 +58,7 @@ def loadconfig(configfile)
                 when "logfacility"
                   @logfacility = val
                 when "libdir"
-                  paths = val.split(/:/)
+                  paths = val.split(File::PATH_SEPARATOR)
                   paths.each do |path|
                     @libdir << path
                     unless $LOAD_PATH.include?(path)
@@ -120,6 +120,10 @@ def loadconfig(configfile)
 
         @libdir.each {|dir| Log.warn("Cannot find libdir: #{dir}") unless File.directory?(dir)}
 
+        if @logger_type == "syslog"
+          raise "The sylog logger is not usable on the Windows platform" if Util.windows?
+        end
+
         PluginManager.loadclass("Mcollective::Facts::#{@factsource}_facts")
         PluginManager.loadclass("Mcollective::Connector::#{@connector}")
         PluginManager.loadclass("Mcollective::Security::#{@securityprovider}")
@@ -151,9 +155,8 @@ def set_config_defaults(configfile)
       @rpcauthorization = false
       @rpcauthprovider = ""
       @configdir = File.dirname(configfile)
-      @color = true
+      @color = !Util.windows?
       @configfile = configfile
-      @rpchelptemplate = "/etc/mcollective/rpc-help.erb"
       @logger_type = "file"
       @keeplogs = 5
       @max_log_size = 2097152
@@ -168,6 +171,12 @@ def set_config_defaults(configfile)
       @direct_addressing = false
       @direct_addressing_threshold = 10
       @ttl = 60
+
+      # look in the config dir for the template so users can provide their own and windows
+      # with odd paths will just work more often, but fall back to old behavior if it does
+      # not exist
+      @rpchelptemplate = File.join(File.dirname(configfile), "rpc-help.erb")
+      @rpchelptemplate = "/etc/mcollective/rpc-help.erb" unless File.exists?(@rpchelptemplate)
     end
 
     def read_plugin_config_dir(dir)

lib/mcollective/logger/syslog_logger.rb

@@ -1,9 +1,9 @@
-require 'syslog'
-
 module MCollective
   module Logger
     # Implements a syslog based logger using the standard ruby syslog class
     class Syslog_logger<Base
+      require 'syslog'
+
       include Syslog::Constants
 
       def start

lib/mcollective/monkey_patches.rb

@@ -7,6 +7,14 @@ def <=>(other)
   end unless method_defined?("<=>")
 end
 
+# This provides an alias for RbConfig to Config for versions of Ruby older then
+# # version 1.8.5. This allows us to use RbConfig in place of the older Config in
+# # our code and still be compatible with at least Ruby 1.8.1.
+# require 'rbconfig'
+unless defined? ::RbConfig
+  ::RbConfig = ::Config
+end
+
 # a method # that walks an array in groups, pass a block to
 # call the block on each sub array
 class Array

lib/mcollective/rpc/actionrunner.rb

@@ -67,7 +67,7 @@ def shell(command, infile, outfile)
         env = {"MCOLLECTIVE_REQUEST_FILE" => infile,
           "MCOLLECTIVE_REPLY_FILE"   => outfile}
 
-        Shell.new("#{command} #{infile} #{outfile}", :cwd => "/tmp", :stdout => stdout, :stderr => stderr, :environment => env)
+        Shell.new("#{command} #{infile} #{outfile}", :cwd => Dir.tmpdir, :stdout => stdout, :stderr => stderr, :environment => env)
       end
 
       def load_results(file)
@@ -123,7 +123,7 @@ def to_s
       end
 
       def tempfile(prefix)
-        Tempfile.new("mcollective_#{prefix}", "/tmp")
+        Tempfile.new("mcollective_#{prefix}", Dir.tmpdir)
       end
     end
   end

lib/mcollective/rpc/helpers.rb

@@ -49,6 +49,8 @@ def self.extract_hosts_from_array(hosts)
       def self.terminal_dimensions
         return [0, 0] unless STDOUT.tty?
 
+        return [80, 40] if Util.windows?
+
         if ENV["COLUMNS"] && ENV["LINES"]
           return [ENV["COLUMNS"].to_i, ENV["LINES"].to_i]
 

lib/mcollective/runner.rb

@@ -17,28 +17,18 @@ def initialize(configfile)
 
       @agents = Agents.new
 
-      Signal.trap("USR1") do
-        Log.info("Reloading all agents after receiving USR1 signal")
-        @agents.loadagents
-      end
-
-      Signal.trap("USR2") do
-        Log.info("Cycling logging level due to USR2 signal")
-        Log.cycle_level
-      end
-    end
+      unless Util.windows?
+        Signal.trap("USR1") do
+          Log.info("Reloading all agents after receiving USR1 signal")
+          @agents.loadagents
+        end
 
-    # Daemonize the current process
-    def self.daemonize
-      fork do
-        Process.setsid
-        exit if fork
-        Dir.chdir('/tmp')
-        STDIN.reopen('/dev/null')
-        STDOUT.reopen('/dev/null', 'a')
-        STDERR.reopen('/dev/null', 'a')
-
-        yield
+        Signal.trap("USR2") do
+          Log.info("Cycling logging level due to USR2 signal")
+          Log.cycle_level
+        end
+      else
+        Util.setup_windows_sleeper
       end
     end
 

lib/mcollective/shell.rb

@@ -28,7 +28,7 @@ def initialize(command, options={})
       @stdout = ""
       @stderr = ""
       @stdin = nil
-      @cwd = "/tmp"
+      @cwd = Dir.tmpdir
 
       options.each do |opt, val|
         case opt.to_s

lib/mcollective/unix_daemon.rb

@@ -0,0 +1,33 @@
+module MCollective
+  class UnixDaemon
+    # Daemonize the current process
+    def self.daemonize
+      fork do
+        Process.setsid
+        exit if fork
+        Dir.chdir('/tmp')
+        STDIN.reopen('/dev/null')
+        STDOUT.reopen('/dev/null', 'a')
+        STDERR.reopen('/dev/null', 'a')
+
+        yield
+      end
+    end
+
+    def self.daemonize_runner(pid=nil)
+      raise "The Unix Daemonizer can not be used on the Windows Platform" if Util.windows?
+
+      UnixDaemon.daemonize do
+        if pid
+          begin
+            File.open(pid, 'w') {|f| f.write(Process.pid) }
+          rescue Exception => e
+          end
+        end
+
+        runner = Runner.new(nil)
+        runner.run
+      end
+    end
+  end
+end