Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Commits on Sep 13, 2011
  1. (#8045) Remove ext/debian/changelog

    The ext/debian/changelog file is quite outdated and gives the appearance
    of a very old (outdated) package.  The rake task for the debian package
    builds generates the changelog on the fly, so the file can be safely
    Signed-off-by: Michael Stahnke <>
  2. (#8045) Debian package build should produce source files

    Previously the rake taks to build a Debian package would not contian the
    source files.  Commit changes behvaior to build source files as well.
    Patch from: Max Bowsher
    Signed-off-by: Michael Stahnke <>
Commits on Sep 12, 2011
  1. @ripienaar

    8466 - When identities are supplied , do not do discovery

    ripienaar authored
    Allow the Message object to be of type :direct_request which would force
    direct requests
    Change identity filters to be and and not or since that makes more sense
    for identities.  But also if we're sending messages with many identities
    attached we'd only want to match if ours is anywhere in the list, an
    and based selection would always fail.
    If identities are provided on the CLI then we use those and avoid discovery.
    The RPC::Client#discover function can now be passed a list of hosts in
    either string, array or JSON formats.  It will extract from this data
    discovery data and use that instead of the old discovery methods.  In
    this use case it will force direct_addressing mode.
    The rpc application can now take JSON on STDIN and will use the above
    capabilities to fill in discovery data
Commits on Aug 29, 2011
  1. @ripienaar

    8325 - A default TTL should be supported

    ripienaar authored
    Make tests more robust by sending in a hash where one is expected
    rather than an empty string which causes failures.
  2. @ripienaar

    8325 - A default TTL should be supported

    ripienaar authored
    Use UTC time for all calculations wrt TTL
    Also logs some more details about messages that fail TTL validation
  3. @ripienaar

    8325 - A default TTL should be supported

    ripienaar authored
    As messages are now possibly sent to queues and can hang around the
    middleware for a very long time we need to support a TTL.  Nodes that
    go off-line between discovery and request being sent may come back
    a month later and find these old messages sitting there and then take
    the actions which would not be desirable.
    Requests will now get a default TTL of 60 seconds and during validation
    of a message any messages older than this will not validate.
    The various security plugins will accept a TTL when encoding the messages
    and default to 60 if not given, similarly the Message object will default
    to 60 TTL.
    As a side effect security plugins now need a TTL when encoding requests
    which means we now need to upgrade all security plugins in use when going
    to a version running this version.
    Future tickets will allow the TTL to be set per message.
  4. @ripienaar

    8775 - Give version precision for registration_collective feature

    ripienaar authored
    Improve the documentation for the registration_collective configuration
    by adding hints about what versions support this and how older ones behave
  5. @ripienaar

    9133 - Update classesfile default location

    ripienaar authored
    Update changelog
  6. @ripienaar

    9133 - Update classesfile default location

    Matt Robinson authored ripienaar committed
    The default classesfile location was set to a location that Puppet never
    defaulted to.  It was missing the state part of the path, and the more
    modern classesfile location in puppet also has lib in the path.  This is
    now the default.
Commits on Aug 6, 2011
  1. @ripienaar

    7730 - Expose reload actions to EL init script

    ripienaar authored
    Update changelog
  2. @dcarley @ripienaar

    7730 - Expose reload actions to EL init script

    dcarley authored ripienaar committed
    reload sends USR1 to reload agents.
    reload-loglevel sends USR2 to cycle logging level.
  3. @ripienaar

    8703 - RPC::Agent#authorized_by loads auth plugins on each request

    ripienaar authored
    The authorized_by helper creates a method called authorization_hook and
    loads up a class from disk that provides the implimentation as a plugin.
    We used to only create 1 instance of a agent so this only happened once
    but recently we started creating new instances of the agent on each
    request which means this steps gets done many times.
    This change avoids creating the hook if it's already defined thus avoiding
    the undesired disk loads
  4. @ripienaar

    8799 - RPC Validators should include boolean

    ripienaar authored
    DDL files can declare a input data item can be boolean which means
    we would like to validate that on the agent side.
    The validators had no boolean validation so this adds one.
    Also do a small bit of test maintenance to avoid spurious failures
  5. @ploubser @ripienaar

    8550 - Documentation out of date/misleading

    ploubser authored ripienaar committed
    Updated out of date documentation
  6. @ripienaar

    8807 - printrpc should justify long text results better

    ripienaar authored
    Tweak the justification of multi line text results so it all
    lines up nicely
Commits on Aug 2, 2011
  1. @ploubser @ripienaar

    7822 - MCollective should return a --version

    ploubser authored ripienaar committed
    Added Version to mco script and catching the SystemExit exception cleanly.
    This works with the builtin --version handling in OptionParser to
    support --version on the command line
  2. @ploubser @ripienaar

    8497 - Discovery agent needs normal meta data set

    ploubser authored ripienaar committed
    Updated discovery agent's meta data
Commits on Jul 25, 2011
  1. @ripienaar

    8419 - Filters with invalid syntax do not fail, possible unintentiona…

    ripienaar authored
    …l execution in environment.
    We now raise exceptions when fact filter parsing fails, the -W helper
    relied on it returning false.
    This adds support for the new behavior to -W
  2. @ripienaar

    8461 - Improve support for compound_filters

    ripienaar authored
    Fix logic used in processing compound filters that were introduced
    during a previous refactor.
    Arrays of compound filters were expected but the option parser did
    not send such an array.
Commits on Jul 18, 2011
  1. @ripienaar

    8419 - Filters with invalid syntax do not fail, possible unintentiona…

    ripienaar authored
    …l execution in environment.
    If users supplied fact filters on the CLI or in ruby code that did not
    match our expected valid fact formats the effect would be as if the
    filter was never specified.
    This would result in requests being run on the wrong machines.  We
    now raise an exception rather than just returning false
  2. @ripienaar

    8461 - Improve support for compound_filters

    ripienaar authored
    During the work in #8181 a new filter type was introduced but
    a few integration points were overlooked.  The result of these
    were that for some kinds of requests no filter were applied.
    This improves the Util#empty_filter method to return an empty
    compound filter and also improves various aspects of the RPC::Client
    class to be aware of the compound filter.
Commits on Jul 14, 2011
  1. @ripienaar
  2. @ploubser @ripienaar

    8181 - Add a rich discovery language to the CLI and elsewhere

    ploubser authored ripienaar committed
    Add a small filter language that allows rich expressions to be used
    when doing discovery.
    A simple parser/scanner was included in MCollective::Matcher and
    it's available on all SimpleRPC clients with the -S option and
    on any MCollective::RPC::Client using the #compound_filter method.
Commits on Jul 8, 2011
  1. @ripienaar

    8063 - Remove the RUBYLIB setting from the rc script

    ripienaar authored
    The RC scripts were setting the RUBYLIB environment variable, this
    should not be needed as the operatinsystem should do the right thing.
Commits on Jul 7, 2011
  1. @ripienaar

    8280 - RPC::Helpers#rpcresults should support JSON

    ripienaar authored
    Add the ability for the printrpc helper to return JSON data.
    Any application that is using the printrpc helper to display
    data will now be able to just add -j to the command line and
    return JSON instead.
    If STDOUT is a tty the JSON will be pretty formatted else it
    will just be a big blob of machine friendly JSON.
    The basic use case is to eventually allow piping RPC results
    from one query into the following query - perhaps with some
    grep like filtering - thus avoiding discovery and allowing
    totally arbitrary actions on arbitrary sets of hosts.
    This also adjusts the progress bar and the stats to only display
    if STDOUT is a tty.
  2. @ripienaar

    7988 - MCollective::Message should support point to point comms

    ripienaar authored
    This adds a new direct request method to the Message object
    that will communicate with nodes directly.
    With this feature enabled requests that goes out to a number
    of nodes below a certain threshold (10 by default) will use
    the new point to point mode while the rest will use broadcast.
    This should be transparent to the end user.  But is disabled
    by default
    The idea with this is to enable communication with off-line
    machines but also to optimize the communication where you need
    to send a message to only 10 or so nodes and creating 10 messages
    would be more efficient than sending a broadcast message to all of
    1000s of nodes,
    The other area where this is important is in the web space where
    users can easily create arbitrary hosts lists by clicking on nodes.
    In this scenario the grouping based on facts, classes etc does not
    apply well.  Having a direct mode means you could supply arbitrary
    hosts lists.
    Ultimately this is the ground work for making discovery pluggable
    and optional.  In the case of the web scenario above when the web
    application provides a list of hostnames to communicate with the
    mcollective infrastructure will not do a discovery but simply go
    and communicate with those nodes.
    This specific commit begins making the various internals aware of
    this new mode, it adds 2 configuration options but leaves this mode
    disabled by default.
    The stomp connector will create queues per node to enable this,
    doing it this way is pretty inefficient and wont work at scale
    a new ActiveMQ connector will be created that use a much better
    but ActiveMQ specific approach to solving the queues per server
    Followup work needs to be done around request TTLs to avoid down
    machines from doing lots of unintended work.
Commits on Jun 30, 2011
  1. @ripienaar

    8117 - Release 1.2.1

    ripienaar authored
    Update changelog and release notes for rel 1.2.1
Commits on Jun 25, 2011
  1. @ripienaar

    8097 - MC::Client#discovered_req should support the Message object

    ripienaar authored
    During the work for #7619 the Client#receive were changed to return
    Message objects but the #discovered_req method that is mostly unused
    were not made aware of this change.
    This commit rectifies that.
Commits on Jun 21, 2011
  1. @ripienaar

    7960 - Add support for stomp gem 1.1.9 logging options

    ripienaar authored
    When using the pooled configuration syntax you now automatically
    get detailed logging at info level for connection events on the
    Stomp connetion.
    This is based on a feature in Stomp 1.1.9 where you can have a
    class that responds to some predictable method names to create
    call back based logging.  The stomp plugin constains a class
    M::C::Stomp::EventLogger that has these methods and does the
    logging using the normal logging framework.
  2. @maguec @ripienaar

    7946 - fix example configurations for activemq in mcollective

    maguec authored ripienaar committed
    add in jetty and the environment variables so the web console is active
  3. @ripienaar

    7961 - on the server, missing DDLs should be a debug log not warning

    ripienaar authored
    Decrease log level on missing DDL file messages in the agent from
    warning to debug as at present there's no real reason for them to
    exist on the servers anyway.
Commits on Jun 20, 2011
  1. @ripienaar

    7225 - mcollectived should be able to subscribe to a per identity queue

    ripienaar authored
    Add a new config option direct_addressing that will cause the main runner
    to request :directed subscriptions from the middleware plugins.
    This setting is off by default since the default stomp plugin really
    isn't going to be able to do really intelligent stuff with this and
    on big networks it will have big impacts on middleware.  The proposed
    ActiveMQ specific plugin will be able to do this in a way that does
    not have such a huge impact on the middleware.
    For Stomp we've added a new config option queueprefix that compliments
    topicprefix and the plugin will simply subscribe to a queue that has a
    MD5 of the configured identity.  We're using the md5 as thats the safest
    way to ensure there's no illegal characters in the resulting queue names.
    The intention with this ability is so that we can address individual nodes
    directly without broadcasting.  The use cases for this are - among others:
    - Web apps where people will be clicking arbitrary hosts, the library will
      fall back to direct mode in that case
    - For calls to M::RPC::Client#custom_request where the list of expected
      replies are some arbitrary low number like 20.  In a big network talking
      to those hosts directly will have a lower impact than the broadcast
    - Add the potential to do batched requests.  With the broadcast mode
      it's a bit all or nothing unless restrictive filters are generated.
      By combining the broadcast for discovery with directed requests in
      groups of n hosts we can slowly go through a network rolling out
      change in managable batches.
    - This will enable truely pluggable discovery.  External resources
      like BPM systems can supply a list of hosts they wish to talk to
      and we can do that easily.
    - Queued requests for off-line machines
    None of these abilities are in the libraries now, this is just the
    enabling architecture.
Commits on Jun 13, 2011
  1. @ripienaar

    7619 - Creating of destinations and messages should be left to the co…

    ripienaar authored
    Previously the structure of the middleware was baked into the mcollective core
    we had assumptions about how the middleware looked and this made it hard for
    connector plugins to be effective.  Connector plugins had to abide by the
    internal assumptions rather than have the freedom to impliment middleware however
    they wanted.  We deliberately threw away message headers as received from the
    middleware as they were considered too middleware specific.  This was a mistake
    as effective use of the middleware relies on these.
    We now have a MC::Message object that encapsulates all the properties of a mcollective
    request or reply but in a way that does not assume a structure of the middleware.
    The connector plugin now gets to make all decisions about the middleware structure.
    All creation of topics, targets, subscriptions etc gets delegated to the connector
    Additionally the Message object now takes care of validation, encoding, decoding
    and base64 processing of messages as these functions only apply to message objects.
    This simplified a lot of code especially in the main runner loop and improve out
    ability to extend mcollective for the work related to #7226
    The entire server and client logic has been reworked around this new object and
    lots of dead code in the Util class was removed.
    In the case where a Message object is actually a reply to an earlier Message the
    earlier message is stored and the connector plugin will have access to the full
    request that initiated a reply.  The connector will thus have access to all the
    headers received and can make routing decision based on those.  The core has no
    use for message headers, this is purely to support the connectors. This will make
    doing #7246 and #6938 much eaier
    This change also breaks backward compatability with early 1.1.x and earlier, this is
    acceptable as the current production release is 1.2.x and we maintain compatability
    with that.  The message protocol now has proper headers for the agent and collective
    a message was sent to so the parsing of :msgtarget was retired.  So this header was
    removed and old servers just wont understand these messages and we wont understand
    old clients.
Commits on Jun 8, 2011
  1. @ripienaar

    7796 - Release 1.3.0

    ripienaar authored
    Update Rakefile and release notes for 1.3.0 release
Commits on Jun 7, 2011
  1. @ripienaar

    7822 - MCollective should return a --version

    ripienaar authored
    Back out this change as it prevented further option parsing from
Something went wrong with that request. Please try again.