Permalink
Browse files

Removes MCollective from the Razor Micro Kernel

MCollective was running as an external service. It used a
static configuration accessible to anyone who accessed
the source, and which could connect to an external master.
This was considered a security threat in the security
evaluation.

This commit removes MCollective completely. The impact of
doing this should be zero since it wasn't currently used.
  • Loading branch information...
1 parent 4fbd56e commit 2b4fa9f3b47b48949b4ef505132eb67cf6c5c49a @thallgren thallgren committed with daniel-pittman Jan 21, 2013
View
3 README.md
@@ -18,12 +18,11 @@ This project contains the Ruby scripts/classes that are used to control the Razo
1. The **Razor Microkernel Controller**, which is actually contained in the rz_mk_control_server.rb file (which is, in turn, started up and controlled using the "Ruby Daemons" interface defined in the rz_mk_controller.rb file)
1. The **Razor Microkernel Web Server**, which can be found in the rz_mk_web_server.rb file and which is used (by the Razor Microkernel Controller) to save configuration changes from the Razor Server to the 'local filesystem' (remember, everything is in memory). During this process of saving the configuration changes, the Microkernel Web Server will actually restart the Razor Microkernel instance (in order to force it to pick up the newly saved Microkernel configuration).
1. The **Local TCE Mirror**, which is actually contained in the rz_mk_tce_mirror.rb file and which is used to install a few extensions during the post-boot configuration process.
-1. The **MCollective daemon**, which is started on the Microkernel isntances but which is not currently used. This service is being kept in place for future use by the Razor Server (or other services that need to interact with multiple instances of the Razor Microkernel simultaneously).
If the Razor Microkernel that is being built/used is a development kernel, a fifth service will also be started during the boot process (the **OpenSSH server daemon**). That service is not started in a production system in order to prevent unauthorized access to the underlying hardware through the Razor Microkernel (in fact, the openssh.tcz extension is not even installed on these production systems).
In addition, this project also includes a number of additional ruby files and configuration files that are used by these services, a list of gems that are installed dynamically each time that the Microkernel boots (under the opt/gems directory), a copy of the 'bootsync.sh' script (under the opt directory in this project), and the 'rz_mk_init.rb' script itself (which is used by that bootsync.sh script to start the appropriate Ruby-based services during the Microkernel boot process).
-Copies of the ruby scripts that appear at the top-level of this project's directory structure can be found in the /usr/local/bin directory of the Microkernel. In addition, the files that appear in the razor_microkernel subdirectory of this project are all part of the RazorMicrokernel module, and those files are placed in the /usr/local/lib/ruby/1.8 directory in the Microkernel ISO. Finally, there are two MCollective SimpleRPC agents defined in this project (in the facter-agent and configuration-agent subdirectories). The agents from these two folders are place in the /usr/local/mcollective/plugin/mcollective/agent directory in the Microkernel ISO (where they should be place if they are to be run by the MCollective daemon that runs on the Microkernel). The other two files in those project subdirectories represent the DDL for each agent and an associated test SimpleRPC client (in the form of a Ruby script). These files are meant to be placed on the Razor Server (or the MCollective Control Node if that is on a different machine).
+Copies of the ruby scripts that appear at the top-level of this project's directory structure can be found in the /usr/local/bin directory of the Microkernel. In addition, the files that appear in the razor_microkernel subdirectory of this project are all part of the RazorMicrokernel module, and those files are placed in the /usr/local/lib/ruby/1.8 directory in the Microkernel ISO.
It should be noted that this project also includes a set of scripts that are meant to be used to build a new instance of the Microkernel ISO. Instructions for building a new ISO instance using these scripts can be found in this project's Wiki. There are also a number of extensions to the standard Tiny Core Linux ISO that are bundled into the Razor Microkernel which are not included in this project. These extensions (and the other dependencies that are needed within the Razor Microkernel ISO that are not part of this project) are all downloaded dynamically when the Microkernel ISO is being built. Once again, instructions for building your own (custom) Microkernel ISO can be found in the project's Wiki.
View
BIN additional-build-files/mcollective-setup-files.tar.gz
Binary file not shown.
View
34 build-bundle-file.sh
@@ -163,8 +163,6 @@ fi
TCL_ISO_URL="$MK_BUNDLE_TCL_ISO_URL"
[ -z "$RUBY_GEMS_URL" -a -n "$MK_BUNDLE_RUBY_GEMS_URL" ] &&
RUBY_GEMS_URL="$MK_BUNDLE_RUBY_GEMS_URL"
-[ -z "$MCOLLECTIVE_URL" -a -n "$MK_BUNDLE_MCOLLECTIVE_URL" ] &&
- MCOLLECTIVE_URL="$MK_BUNDLE_MCOLLECTIVE_URL"
[ -z "$OPEN_VM_TOOLS_URL" -a -n "$MK_BUNDLE_OPEN_VM_TOOLS_URL" ] &&
OPEN_VM_TOOLS_URL="$MK_BUNDLE_OPEN_VM_TOOLS_URL"
[ -z "$GEM_SERVER_URI" -a -n "$MK_BUNDLE_GEM_SERVER_URI" ] &&
@@ -177,7 +175,6 @@ fi
[ -z "$TCL_MIRROR_URI" ] && TCL_MIRROR_URI='http://distro.ibiblio.org/tinycorelinux/4.x/x86/tcz'
[ -z "$TCL_ISO_URL" ] && TCL_ISO_URL='http://distro.ibiblio.org/tinycorelinux/4.x/x86/release/Core-current.iso'
[ -z "$RUBY_GEMS_URL" ] && RUBY_GEMS_URL='http://production.cf.rubygems.org/rubygems/rubygems-1.8.24.tgz'
-[ -z "$MCOLLECTIVE_URL" ] && MCOLLECTIVE_URL='http://puppetlabs.com/downloads/mcollective/mcollective-2.0.0.tgz'
[ -z "$OPEN_VM_TOOLS_URL" ] && OPEN_VM_TOOLS_URL='https://github.com/downloads/puppetlabs/Razor-Microkernel/mk-open-vm-tools.tar.gz'
@@ -254,15 +251,6 @@ cp -p rz_mk_*.rb tmp-build-dir/usr/local/bin
mkdir -p tmp-build-dir/usr/local/lib/ruby/1.8/razor_microkernel
cp -p razor_microkernel/*.rb tmp-build-dir/usr/local/lib/ruby/1.8/razor_microkernel
-# create copies of the MCollective agents from this project (will be placed
-# into the /usr/local/tce.installed/$mcoll_dir/plugins/mcollective/agent
-# directory in the Razor Microkernel ISO
-file=`echo $MCOLLECTIVE_URL | awk -F/ '{print $NF}'`
-mcoll_dir=`echo $file | cut -d'.' -f-3`
-mkdir -p tmp-build-dir/usr/local/tce.installed/$mcoll_dir/plugins/mcollective/agent
-cp -p configuration-agent/configuration.rb facter-agent/facteragent.rb \
- tmp-build-dir/usr/local/tce.installed/$mcoll_dir/plugins/mcollective/agent
-
# create a copy of the files from this project that will be placed into the
# /opt directory in the Razor Microkernel ISO; as part of this process will
# download the latest version of the gems in the 'gem.list' file into the
@@ -357,22 +345,6 @@ then
wget $WGET_V -P tmp-build-dir/build_dir $TCL_ISO_URL
fi
-# download the MCollective, unpack it in the appropriate location, and
-# add a couple of soft links
-file=`echo $MCOLLECTIVE_URL | awk -F/ '{print $NF}'`
-mcoll_dir=`echo $file | cut -d'.' -f-3`
-if [ $RE_USE_PREV_DL = 'no' ] || [ ! -f tmp-build-dir/$file ]
-then
- wget $WGET_V -P tmp-build-dir $MCOLLECTIVE_URL
-fi
-cd tmp-build-dir/usr/local/tce.installed
-tar zx${TAR_V}f "${TOP_DIR}/tmp-build-dir/${file}"
-cd "${TOP_DIR}/tmp-build-dir"
-rm usr/local/mcollective usr/local/bin/mcollectived 2> /dev/null
-ln -s /usr/local/tce.installed/$mcoll_dir usr/local/mcollective
-ln -s /usr/local/mcollective/bin/mcollectived usr/local/bin/mcollectived
-cd "${TOP_DIR}"
-
# add a soft-link in what will become the /usr/local/sbin directory in the
# Microkernel ISO (this fixes an issue with where Facter expects to find
# the 'dmidecode' executable)
@@ -385,11 +357,9 @@ ln -s /usr/local/sbin/dmidecode tmp-build-dir/usr/sbin 2> /dev/null
# 1. ssh-setup-files.tar.gz -> contains the setup files needed for the
# SSH/SSL (used for development access to the Microkernel); if
# the '--build-prod-image' flag is set, then this file will be skipped
-# 2. mcollective-setup-files.tar.gz -> contains the setup files needed for
-# running the mcollective daemon
-# 3. mk-open-vm-tools.tar.gz -> contains the files needed for the
+# 2. mk-open-vm-tools.tar.gz -> contains the files needed for the
# 'open_vm_tools.tcz' extension
-# 4. the etc/passwd and etc/shadow files from the Razor-Microkernel project
+# 3. the etc/passwd and etc/shadow files from the Razor-Microkernel project
# (note; if this is a production system then the etc/shadow-nologin
# file will be copied over instead of the etc/shadow file (to block
# access to the Microkernel from the console)
View
1 bundle.cfg.example
@@ -6,6 +6,5 @@ MK_BUNDLE_TYPE=prod
MK_BUNDLE_TCL_MIRROR_URI=http://localmirror.localdomain/tinycorelinux/4.x/x86/tcz
MK_BUNDLE_TCL_ISO_URL=http://localmirror.localdomain/tinycorelinux/4.x/x86/release/Core-current.iso
MK_BUNDLE_RUBY_GEMS_URL=http://localmirror.localdomain/rubygems/rubygems-1.8.24.tgz
-MK_BUNDLE_MCOLLECTIVE_URL=http://localmirror.localdomain/mcollective/mcollective-2.0.0.tgz
MK_BUNDLE_OPEN_VM_TOOLS_URL=http://localmirror.localdomain/Razor-Microkernel/mk-open-vm-tools.tar.gz
MK_BUNDLE_GEM_SERVER_URI=http://localmirror.localdomain:8808/
View
37 configuration-agent/configuration.ddl
@@ -1,37 +0,0 @@
-# DDL file for the configuration agent (defines the actions, inputs and outputs
-# for this agent for the control node)
-#
-# EMC Confidential Information, protected under EMC Bilateral Non-Disclosure Agreement.
-# Copyright © 2012 EMC Corporation, All Rights Reserved
-#
-# @author Tom McSweeney
-
-metadata :name => "Configuration Agent",
- :description => "Razor Microkernel Configuration Agent",
- :author => "Tom McSweeney",
- :license => "Apache v2",
- :version => "1.0",
- :url => "http://www.emc.com",
- :timeout => 30
-
-action "send_mk_config",
- :description => "Send a new set of configuration parameters to the Microkernel agent" do
-
- display :always # supported in 0.4.7 and newer only
-
- input :config_params,
- :prompt => "Configuration",
- :description => "The configuration parameters (as a JSON-formatted Hash Map)",
- :type => :string,
- :validation => '/^(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.([a-z]{2,5}|[0-9]{1,3})(:[0-9]{1,5})?(\/.*)?$/ix',
- :optional => false,
-
- output :response,
- :description => "The response from the Registration Servlet",
- :display_as => "Response"
-
- output :time,
- :description => "The time that the response was sent back at",
- :display_as => "Time"
-
-end
View
33 configuration-agent/configuration.rb
@@ -1,33 +0,0 @@
-# The configuration agent (hosted on the managed nodes, works with the
-# rz_web_server to save the new configuration to the filesystem on the
-# Microkernel and restart the rz_mk_contoller)
-#
-#
-
-require 'net/http'
-
-module MCollective
- module Agent
- class Configuration<RPC::Agent
- metadata :name => "Configuration Agent",
- :description => "Razor Microkernel Configuration Agent",
- :author => "Tom McSweeney",
- :license => "Apache v2",
- :version => "1.0",
- :url => "http://www.emc.com",
- :timeout => 30
-
- action "send_mk_config" do
- validate :config_params, String
- # post configuration (as a JSON string) to the local WEBrick instance
- # (which should be running at port 2156)
- uri = URI "http://localhost:2156/setMkConfig"
- json_string = request[:config_params]
- res = Net::HTTP.post_form(uri, json_string)
- reply[:response] = res.message
- reply[:time] = Time.now.strftime("%Y-%m-%d %H:%M:%S")
- end
-
- end
- end
-end
View
61 configuration-agent/test-configuration.rb
@@ -1,61 +0,0 @@
-#!/usr/bin/env ruby
-#
-# A demo SimpleRPC client that interacts with the configuration agent to set
-# the configuration of the services running in the Microkernel through the
-# MCollective (will probably construct a Razor server-side daemon that does
-# something like what's here, but that remains to be seen)
-#
-#
-
-$LOAD_PATH << "/usr/lib/ruby/1.8"
-require 'mcollective'
-require 'yaml'
-require 'json'
-
-include MCollective::RPC
-
-if !ARGV || ARGV.length != 1 then
- puts "Usage: test-configuration.rb mk_conf_filename"
- exit(-1)
-end
-
-mk_conf_filename = ARGV[0]
-if !File.exist?(mk_conf_filename) then
- puts "File '#{mk_conf_filename}' does not exist"
- exit(-1)
-end
-
-# load the Microkernel Configuration from the input (YAML) file;
-# contents of this file should look something like the following:
-# ---
-# mk:
-# razor_uri: http://192.168.5.2:8026
-# checkin_sleep: 60
-# checkin_offset: 5
-# facts:
-# exclude_pattern: /(^uptime.*$)|(^memory.*$)/
-# node:
-# register: /razor/api/node/register
-# checkin: /razor/api/node/checkin
-
-mk_conf = YAML::load(File.open(mk_conf_filename, 'r'))
-
-# then convert the resulting Hash map into a JSON string
-json_string = JSON.generate(mk_conf)
-
-# now that the setup is complete, create an rpcclient to connect to our
-# MCollective agents (and configure it so that it doesn't report progress)
-configClient = rpcclient("configuration")
-configClient.progress = false
-
-# and invoke the action on our agents that will set the Microkernel Config
-# using the json_string generated (above)
-configClient.send_mk_config(:config_params => json_string).each do |resp|
- respData = resp[:data]
- if respData then
- printf("Registration Response: '%s' [from '%s' at %s]\n",
- respData[:response], resp[:sender], respData[:time])
- else
- printf("[%s] %s\n", resp[:sender], resp[:statusmsg])
- end
-end
View
24 facter-agent/facteragent.ddl
@@ -1,24 +0,0 @@
-# DDL file for the facter agent (defines the actions, inputs and outputs
-# for this agent for the control node)
-#
-# EMC Confidential Information, protected under EMC Bilateral Non-Disclosure Agreement.
-# Copyright © 2012 EMC Corporation, All Rights Reserved
-#
-# @author Tom McSweeney
-
-metadata :name => "Facter Agent",
- :description => "Preliminary Facter Agent",
- :author => "Tom McSweeney",
- :license => "Apache v2",
- :version => "1.0",
- :url => "http://www.emc.com",
- :timeout => 60
-
-action "getall", :description => "Get facts from node using Facter" do
- display :always # supported in 0.4.7 and newer only
-
- output :facts,
- :description => "YAML representation of Facter Hash Map",
- :display_as => "Facts"
-
-end
View
32 facter-agent/facteragent.rb
@@ -1,32 +0,0 @@
-# The facter agent (hosted on the managed nodes, can be used to gather facts
-# remotely using MCollective)
-#
-#
-
-require 'rubygems'
-require 'facter'
-require 'yaml'
-
-module MCollective
- module Agent
- class Facteragent<RPC::Agent
- metadata :name => "Facter Agent",
- :description => "Preliminary Facter Agent",
- :license => "Apache v2",
- :version => "1.0",
- :url => "http://www.emc.com",
- :timeout => 60
-
- action "getall" do
- # return the facts gathered by Facter (as a YAML-formatted string)
- factMap = Hash.new
- Facter.loadfacts
- Facter.each { |fact, value|
- factMap[fact.to_sym] = value
- }
- reply[:facts] = YAML.dump(factMap)
- end
-
- end
- end
-end
View
20 facter-agent/test-facteragent.rb
@@ -1,20 +0,0 @@
-#!/usr/bin/env ruby
-#
-# A demo SimpleRPC client that interacts with the facter agent to gather
-# facts through the MCollective
-#
-#
-
-$LOAD_PATH << "/usr/share/mcollective/lib"
-require 'mcollective'
-require 'yaml'
-
-include MCollective::RPC
-
-mc = rpcclient("facteragent")
-mc.progress = false
-mc.getall().each do |resp|
- respData = resp[:data]
- facts_hash = YAML.load(respData[:facts])
- p facts_hash
-end
View
2 iso-build-files/build_initial_directories.sh
@@ -45,7 +45,7 @@ cp -rp ../extract/* .
# unpack the dependency files that were extracted earlier (these files were
# built from the current contents of the Razor-Microkernel project using the
# build-dependency-files.sh shell script, which is part of that same project)
-for file in mk-open-vm-tools.tar.gz razor-microkernel-overlay.tar.gz mcollective-setup-files.tar.gz ssh-setup-files.tar.gz; do
+for file in mk-open-vm-tools.tar.gz razor-microkernel-overlay.tar.gz ssh-setup-files.tar.gz; do
# all of these files may not exist for all Microkernels, so only try to unpack
# the files that do exist
if [ -r "../dependencies/${file}" ]; then
View
12 rz_mk_init.rb
@@ -1,7 +1,7 @@
#!/usr/bin/env ruby
#
# Used during the boot process to initialize the Microkernel (install gems
-# and start up the critical services, like MCollective)
+# and start up the critical services)
#
#
@@ -28,8 +28,7 @@
gemController.installListedGems
# Now that we've installed the facter gem, need do do a bit more work
-# first, determine where the facter gem's library is at (will need that later,
-# when we start the MCollective daemon)
+# first, determine where the facter gem's library is at
require 'rubygems'
require 'facter'
@@ -57,7 +56,7 @@
nw_is_avail = true if error_cond == RazorMicrokernel::RzNetworkUtils::SUCCESS
# if the network is available (there's an ethernet adapter that is up and
-# has a valid IP address), then start up the MCollective agent
+# has a valid IP address), then start up the controller scripts
if nw_is_avail then
# sleep 5 more seconds, just in case
@@ -84,11 +83,6 @@
%x[sudo /usr/local/bin/rz_mk_tce_mirror.rb 2>&1 > /tmp/rz_mk_tce_mirror.out]
%x[sudo /usr/local/bin/rz_mk_controller.rb start]
- # and start up the MCollective daemon
- t = %x[sudo env RUBYLIB=/usr/local/lib/ruby/1.8:/usr/local/mcollective/lib:#{facter_lib} \
- mcollectived --config /usr/local/etc/mcollective/server.cfg \
- --pidfile /var/run/mcollective.pid]
-
# finally, print out the Microkernel version number (which should be in the
# /tmp/mk_version.yaml file)
mk_version_hash = File.open("/tmp/mk-version.yaml", 'r') { |file|

0 comments on commit 2b4fa9f

Please sign in to comment.