Merge tag '2.4.0' into stable

2.4.0

CHANGELOG.md

@@ -1,5 +1,12 @@
 # Jerakia Release Notes
 
+## 2.4.0
+
+* Feature: API now supports optional data encoding type of messagepack.  The default behaviour is still JSON as with previous versions, but now if you specify a content type of `application/x-msgpack` (and encode your payload with msgpack) then Jerakia will return responses in msgpack instead of JSON.  This has some small performance gains but also solves an issue with JSON encoding integer hash keys as strings (https://github.com/crayfishx/jerakia/issues/95).  This is currently an opt-in change, however we aim to make msgpack the default serialization format for Jerakia 3. Thanks to @hhenkel for the contribution (https://github.com/crayfishx/jerakia/pull/96)
+
+* Enhancement: Better error reporting and softer handling of situations where there is no logfile.  Jerakia will no longer cause an exception to be thrown if the logfile cannot be opened, instead it will direct logs to `STDOUT` with a warning to the user, and continue executing.  The previous behaviour was frustrating, especially when using the rubygem installation method that doesn't set up all the scaffolding for you.  Other errors such as failure to open the database files are also better reported and more understandable.  Any `Jerakia::Error` thrown on the CLI is now captured and reported back in an easier to understand way
+
+
 ## 2.3.0
 
 * Enhancement:  Added `--bind`, `--port` and `--token_ttl` options to `jerakia server` to override these values from the CLI

Gemfile

@@ -14,13 +14,13 @@ gem "sinatra"
 gem "dm-sqlite-adapter"
 gem "rake"
 gem "rack", "~> 1.6.5"
+gem "msgpack"
 
 group(:development, :test) do
   gem 'rspec-core'
   gem 'rspec'
   gem 'mocha'
   gem "puppet", ENV['PUPPET_GEM_VERSION'] || '~> 4.8.0'
-  gem "msgpack" # required for integration tests with jerakia-puppet
   gem "webmock-rspec-helper"
   if RUBY_VERSION < '2.1'
     gem "nokogiri", '1.6.8'

README.md

@@ -11,14 +11,15 @@ jerakia
 
 [![Join the chat at https://gitter.im/crayfishx/jerakia](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/crayfishx/jerakia?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
-A pluggable and extendable data lookup system
+A pluggable and extendable hierarchical data lookup system
 
 ## Introduction ##
 
-Jerakia is a pluggable hierarchical data lookup engine.  It is not a database, Jerakia itself does not store any data but rather gives a single point of access to your data via a variety of back end data sources.   Jerakia is inspired by Hiera, and can be used a drop in replacement. Hiera itself is a good tool, however it suffers from some degree of limitation in its architecture that makes solving complex edge cases a challenge. Jerakia is an attempt at a different way of approaching data lookup management.  Jerakia started out as a prototype experiment to replace hiera in order to solve a number of complicated requirements for a particular project, over time it matured a bit and we decided to open source it and move it towards a standalone data lookup system.
+Jerakia is a pluggable hierarchical data lookup engine.  It is not a database, Jerakia gives a single point of access to your data via a variety of back end data sources and provides a hierarchical way to search key value data.   Jerakia is inspired by Hiera, and can be used along side or instead of Hiera in Puppet implementations.  Jerakia can also integrate with other tools, such as Ansible and more.
 
-The main goals of Jerakia are:
+The main goals and features of Jerakia are:
 
+* Hierarchical key value data lookups
 * Extendable framework to solve even the most complex edge cases
 * Decoupled from any particular configuration management system
 * Pluggable framework to encourage community plugin development
@@ -35,7 +36,7 @@ Features include:
 Documentation is kept on the [Official Website](http://jerakia.io)
 
 ## Other documentation ##
-
+* [Blog post: Understanding Hierarchichal Data Lookups](https://www.craigdunn.org/2017/08/understanding-hierarchical-data-lookups/)
 * [Blog post part 1: Solving real world problems with Jerakia](http://www.craigdunn.org/2015/09/solving-real-world-problems-with-jerakia/)
 * [Blog post part 2: Extending Jerakia with lookup plugins](http://www.craigdunn.org/2015/09/extending-jerakia-with-lookup-plugins/)
 * [Blog post: Managing Puppet Secrets with Jerakia and Vault](http://www.craigdunn.org/2017/04/managing-puppet-secrets-with-jerakia-and-vault/)
@@ -53,9 +54,10 @@ There are various integration options for making requests to Jerakia.
 * Command line tool
 * Ruby API
 * REST API
-* Hiera 5 Backend
+* [Hiera 5 Backend](http://forge.puppet.com/crayfishx/jerakia)
+* [Ansible Lookup Plugin](https://github.com/crayfishx/ansible-jerakia)
 
-Legacy (see jerakia-puppet):
+Legacy (see [jerakia-puppet](https://github.com/crayfishx/jerakia-puppet) ):
 
 * Puppet data binding terminus
 * Hiera 3.x Backend

bin/jerakia

@@ -4,5 +4,9 @@ lib = File.expand_path('../../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 require 'jerakia/cli'
-
-Jerakia::CLI.start(ARGV)
+begin
+  Jerakia::CLI.start(ARGV)
+rescue Jerakia::Error => e
+  STDERR.puts "Error(#{e.class}): #{e.message}"
+  exit 1
+end

jerakia.gemspec

@@ -20,5 +20,6 @@ Gem::Specification.new do |s|
   s.add_dependency 'data_mapper', '~> 1.2'
   s.add_dependency 'dm-sqlite-adapter', '~> 1.2'
   s.add_dependency 'thin', '~> 1.6'
+  s.add_dependency 'msgpack', '~> 1.1'
 
 end

lib/jerakia/cli.rb

@@ -16,7 +16,6 @@ class CLI < Thor
     include Jerakia::CLI::Secret
     include Jerakia::CLI::Config
 
-
     desc 'version', 'Version information'
     def version
       puts Jerakia::VERSION

lib/jerakia/log.rb

@@ -3,8 +3,9 @@ class Jerakia::Log < Jerakia
   def initialize(level = :info, file = '/var/log/jerakia.log')
     begin
       @@logger = Logger.new(file)
-    rescue Errno::EACCES => e
-      raise Jerakia::Error, "Error opening log file, #{e.message}"
+    rescue Errno::EACCES, Errno::ENOENT => e
+      @@logger = Logger.new(STDOUT)
+      info("Failed to open logfile: #{e.message}, logs will be directed to STDOUT")
     end
 
     @@level = level

lib/jerakia/server/auth/token.rb

@@ -26,8 +26,12 @@ def self.default_repository_name
         property :last_seen, DateTime, :default => DateTime.now
       end
 
-      DataMapper.finalize
-      DataMapper.auto_upgrade!
+     begin
+        DataMapper.finalize
+        DataMapper.auto_upgrade!
+      rescue DataObjects::ConnectionError => e
+        raise Jerakia::Error, "Unable to open database file in #{Jerakia.config[:databasedir]}: #{e.message}"
+      end
     end
   end
 end

lib/jerakia/server/rest.rb

@@ -1,8 +1,9 @@
 require 'sinatra'
 require 'jerakia'
 require 'jerakia/server/auth'
-require 'json'
 require 'jerakia/scope/server'
+require 'json'
+require 'msgpack'
 
 class Jerakia
   class Server
@@ -44,20 +45,44 @@ def authenticate!
         @authorized_tokens[token] = Time.now
       end
 
+      def encode_result(data)
+        case @content_type
+        when :json
+          return data.to_json
+        when :msgpack
+          return data.to_msgpack
+        end
+      end
+
+      def determine_content_type!
+        if not env.key?('CONTENT_TYPE') or env['CONTENT_TYPE'] == "application/json"
+          content_type 'application/json'
+          @content_type = :json
+        elsif env['CONTENT_TYPE'] == "application/x-msgpack"
+          content_type 'application/x-msgpack'
+          @content_type = :msgpack
+        else
+          wrong_media_type("Content type #{env['CONTENT_TYPE']} not supported", 415)
+        end
+      end
+
       before do
         authenticate!
-        content_type 'application/json'
+        determine_content_type!
       end
 
       get '/' do
         auth_denied
       end
 
+      def wrong_media_type(message, status_code=415)
+        halt(status_code, {'Content-Type' => 'text/plain'}, message)
+      end
+
       def request_failed(message, status_code=501)
-        halt(status_code, {
-          :status => 'failed',
-          :message => message,
-        }.to_json)
+        halt(status_code, 
+             encode_result({ :status => 'failed', 
+                             :message => message }))
       end
 
       def mandatory_params(mandatory, params)
@@ -98,42 +123,36 @@ def mandatory_params(mandatory, params)
         rescue Jerakia::Error => e
           request_failed(e.message, 501)
         end
-        {
-          :status => 'ok',
-          :payload => answer.payload
-        }.to_json
+        encode_result({ :status => 'ok', 
+                        :payload => answer.payload})
       end
 
       get '/v1/scope/:realm/:identifier' do
         resource = Jerakia::Scope::Server.find(params['realm'], params['identifier'])
         if resource.nil?
-          halt(404, { :status => 'failed', :message => "No scope data found" }.to_json)
+          halt(404, 
+               encode_result({:status => 'failed', 
+                              :message => "No scope data found"}))
         else
-          {
-            :status => 'ok',
-            :payload => resource.scope
-          }.to_json
+          encode_result({:status => 'ok',
+                         :payload => resource.scope})
         end
       end
 
       put '/v1/scope/:realm/:identifier' do
         scope = JSON.parse(request.body.read)
         uuid = Jerakia::Scope::Server.store(params['realm'], params['identifier'], scope)
-        {
-          :status => 'ok',
-          :uuid => uuid
-        }.to_json
+        encode_result({:status => 'ok',
+                       :uuid => uuid})
       end
 
       get '/v1/scope/:realm/:identifier/uuid' do
         resource = Jerakia::Scope::Server.find(params['realm'], params['identifier'])
         if resource.nil?
           request_failed('No scope data found', 404)
         else
-          {
-            :status => 'ok',
-            :uuid => resource.uuid
-          }.to_json
+          encode_result({:status => 'ok',
+                         :uuid => resource.uuid})
         end
       end
     end

lib/jerakia/version.rb

@@ -3,5 +3,5 @@ class Jerakia
   #
   # This should be updated when a new gem is released and it is read from the gemspec file
   #
-  VERSION = '2.3.0'.freeze
+  VERSION = '2.4.0'.freeze
 end

test/fixtures/var/lib/jerakia/data/common/test.yaml

@@ -61,4 +61,7 @@ databases:
     boo:
       bear: baz
 
-
+msgpack:
+  values:
+    1: "Test 1"
+    '2': "Test 2"