Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Ruby Shell

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
configuration-agent
facter-agent
opt
.gitattributes
README
fact_manager.rb
mk_conf.yaml
rz_host_utils.rb
rz_mk_bundle_controller.rb
rz_mk_control_server.rb
rz_mk_controller.rb
rz_mk_init.rb
rz_mk_registration_manager.rb
rz_mk_web_server.rb
rz_network_utils.rb

README

There are five types of files contained in this project at this time. The first
file type (iso-build-files.tar.gz) contains all of the files that are needed to
build a new Microkernel ISO (i.e. a Microkernel ISO corresponding to the one
that we are using in the Razor project). Unpacking this file will result in a
directory structure that looks something like the following being created in
the current working directory:

                          microcore-current-files
                                   |
                                   |
                             tcl-4.2.1-files
                                   |
                                   |
                     -----------------------------
                     |             |             |
                     |             |             |
                   newiso       extract         tmp

In this directory structure, the "extract" subdirectory contains all of the
files that were extracted from the core.gz file that is contained in the
original Core-current.iso file from the Tiny Core Linux (v4.2.1) distribution.
The "tmp" directory, on the other hand, contains all of the files that were
used to create the customized version of this same core.gz file (the
customized version that appears in the Microkernel ISO that we are using in
Razor). The new core.gz file was then used to replace the file that appears
in the boot subdirectory of "newiso" subdirectory (which originally contained
a copy of the contents of the Core-current.iso file).  As such, the "newiso"
subdirectory contains all of the contents of the ISO file that we are using
in the Razor project.

If the iso-build-files.tar.gz tarfile is extracted, the microcore-current-files
subdirectory contains an another file (in addition to the three directories
mentioned above).  That file is a shell script (rebuild_iso.sh) that can be
used to build a new ISO after changes are made to the contents of the "tmp"
or "newiso" directories.  The shell script will run all of the appropriate
commands (there are about 8 of them) to build the new core.gz file,
place it in the appropriate location in the newiso directory, and then build
the new ISO from the contents of that directory. When the script finishes,
a new ISO will be built that can be used to boot machines.

The second type of file are the gems that appear under the "gems"
subdirectory in the repository.   Currently, there is only one gem
included in this folder (v1.0.21 of the Bundler gem), and this will
likely continue to be the case.  A copy of this folder appears as the
/opt/gems directory in our ISO image, and the gems that appear in this
folder in the ISO are meant to be installed when the ISO used to boot
a node.

The third type of file are the bundles that appear in the "bundles"
subdirectory.  These files are copies of the bundles that appear in the
current ISO (in the /opt/bundles subdirectory) and are meant to be used
(with the bundler gem, see above) to install additional functionality into
the OS image when it is booted.

The fourth type of file are the Ruby scripts/classes that are used to
control the Microkernel image boot process.  These files all appear at
the root of this project, and copies of these meant to be installed in
scripts/classes appear in the /usr/local/bin directory of the Microkernel
image.

The list file type is the Microkernel image itself (which can be found
under the Image-Files subdirectory of this project).  There are two versions
of Tiny Core Linux that have been used to build these image files, and the
image files in the repository are broken out into two separate subdirectories
accordingly (based on the Tiny Core Linux distribution that they were built
from).  Each image in this subdirectory includes the following extensions to
the "stock" Tiny Core Linux distro that it was built from:

        1. Ruby (v1.8.7)
        2. Bash
        3. OpenSSH (v5.8p1)
        4. MCollective (v1.2.1)
        5. RubyGems (v1.8.15)
        6. dmidecode (v2.11)  <- (Added in v0.0.2.0)

When these ISOs are used as the boot image for a node, scripts will
automatically be run after the node boots that will start an SSH daemon
and the MCollective daemon post-boot.  Once the MCollective daemon is started,
the node will be controllable from the MCollective Control Node.

At this time, the MCollective daemon's configuration (which appears in the
/usr/local/etc/mcollective directory of the Microkernel's filesystem)
is "hard-coded" into the ISO, as is the public/private host key pair that
is used by the SSH daemon.  Eventually, the MCollective daemon configuration
will be obtained from a Node.js instance, but that instance doesn't exist yet
(so for testing purposes we'll continue to use the "fixed" MCollective
configuration, at least for now).  The "fixed" public/private host key pair
(which is the same for all Microkernel instances in the current ISO) is less
of an issue, because this service will be pulled out of any ISO that is rolled
out into a production environment (to prevent command-line access to the nodes
when they are being managed by the Microkernel).

In addition to the extensions that were mentioned previously, we have also made
the following changes to the "stock" Tiny Core Linux distro when building out
the current version of the Razor Microkernel (v0.0.2.0):

        1. We have reduced the boot prompt timeout (defined in the
           /boot/isolinux/isolinux.cfg file in the ISO itself) to
           approximately 10 seconds (the original value was 30 seconds)

        2. We have modified the /opt/bootlocal.sh script that is include
           in the microcore.gz file used by the Microkernel (which can be
           found in the ISO's /boot directory) so that when the Microkernel
           is booted it will automatically:

                a. Starts the SSH daemon
                b. Installs the Bundler RubyGem
                c. Uses the Bundler RubyGem to install the stomp, facter,
                   daemons, json_pure, and bluepill bundles

           Once the stomp bundle is installed, that same script changes the
           hostname (in the /etc/hosts and /etc/hostname files using the 'sed'
           command and also logically using the 'hostname' command) and then
           starts the MCollective agent (but it only performs these two tasks
           after the network is available; this ensures that MAC address for
           the 'eth0' adapter is available through Facter and the that
           the MCollective agent will be able to establish a connection to
           the MCollective Control Node).

         3. We have added a pair of MCollective agents to the
            /usr/local/mcollective/plugins/mcollective/agent subdirectory.
            One agent is a test agent that simply echos back the message
            sent to it, along with the time that message was receive. The
            second agent (facteragent.rb) provides access to "Facter" on
            each instance of the Microkernel through the MCollective.

         4. We have constructed a wrapper (rz_mk_controller_control.rb)
            around the rz_mk_controller.rb script that daemonizes the
            latter script and allows us to easily control it from the
            command line using the former.  The usage for the "control"
            script is as follows:

            Usage: rz_mk_controller_control.rb <command> <options> -- <application options>

            * where <command> is one of:
              start         start an instance of the application
              stop          stop all instances of the application
              restart       stop all instances and restart them afterwards
              reload        send a SIGHUP to all instances of the application
              run           start the application and stay on top
              zap           set the application to a stopped state
              status        show status (PID) of application instances

            * and where <options> may contain several of the following:

                -t, --ontop                      Stay on top (does not daemonize)
                -f, --force                      Force operation
                -n, --no_wait                    Do not wait for processes to stop

            Common options:
                -h, --help                       Show this message
                    --version                    Show version

            As you can see, options are provided to start, stop, restart, reload,
            and check the status of the underlying rz_mk_controller process.

         5. We added an MCollective agent (the Configuration) agent to the
            Microkernel.  Currently, this agent has one action defined
            (set_registration_uri) but additional actions can be added in the
            future.  The "set_registration_uri" action takes one argument
            (which MUST be a URI) and passes that string to the Microkernel
            Controller using a POST operation.

         6. We added an action to the Microkernel Controller (a WEBrick instance)
            that is invoked by the Configuration Agent in order to pass the
            Registration URI that should be used by the Microkernel instance
            to the Microkernel Controller.

         7. We modified the Microkernel Controller so that a pattern can be
            defined within the Controller for Facter "facts" that should be
            excluded from the registration details that are passed to the
            Razor server.  Currently, only "facts" with names that start
            with the string "uptime" or "memory" are excluded, but this
            could be changed later.

         8. We added code to the Microkernel Controller (code that is invoked
            by the servlet that handles the action defined in #6, above) that
            actually POSTs a JSON-formatted string representation of the facts
            discovered using Facter to the registration URI received from the
            Configuration agent.  Eventually this functionality will be
            triggered by a service in the Razor server that periodically sends
            out the registration URI (perhaps even a different URI to different
            types of nodes), but for now, this functionality can be triggered
            directly from the MCollective Control Node using the
            'test-configuration.rb' script (which can be found in the
            'configuration-agent' subdirectory).  An example of using this
            script from the command line is something like the following:

            # test-configuration.rb http://192.168.5.2:8026/razor/slice/node/register

            This command tells all of the configuration-agents to use the
            URI "http://192.168.5.2:8026/razor/slice/node/register" for
            registration of any new facts that they might detect.  The
            Microkernel Controller will add the node's UUID and a state to
            the end of this URI to complete the URI for the POST operation that
            is used to pass the details gathered from Facter for that node
            to the Razor server.

All of these changes (and the procedure that was followed to make them) can be
found in the following pages:

        1. Remastering the Microkernel ISO:

        http://dcade.lss.emc.com/confluence/display/ASI/Remastering+the+Microkernel+ISO

        2. Reconfiguring the Microkernel to include the "extras" that are
           described above:

        http://dcade.lss.emc.com/confluence/display/ASI/Microkernel+Configuration

        3. Configuring an MCollective Agent

        http://dcade.lss.emc.com/confluence/display/ASI/Configuring+an+MCollective+Agent

TO DO Items:

        1. Add action for "keep-alive" (will have to deal with responses from the
           keep-alive server that tell the Microkernel what to do next; "nil" is a
           valid command option)

        2. Add ability to deal with multiple registration servers

        3. Add additional "Configuration" options to the Configuration Agent
           (and similar actions in the Microkernel Controller as new Servlets)
Something went wrong with that request. Please try again.