feature: Add filter to lift keys from a map or nest keys in a map

[michiel]

This adds a new filter, called 'nest', that allows keys to be,

• Nested under another key using a wildcard
• Lifted up from a nest

Examples,

Nest 1

[INPUT]
    Name mem
    Tag  mem.local

[OUTPUT]
    Name  stdout
    Match *

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard Mem.*
    Nest_under LAYER1

Output,

[0] mem.local: [1524795567.010244325, {"Swap.total"=>1046524, "Swap.used"=>0, "Swap.free"=>1046524, "LAYER1"=>{"Mem.total"=>4050908, "Mem.used"=>1101372, "Mem.free"=>2949536}}]

{
  "Swap.total"=>1046524,
  "Swap.used"=>0,
  "Swap.free"=>1046524,
  "LAYER1"=>{
   "Mem.total"=>4050908,
   "Mem.used"=>1101372,
   "Mem.free"=>2949536
  }
}

Nest 1 - Lift 1

[INPUT]
    Name mem
    Tag  mem.local

[OUTPUT]
    Name  stdout
    Match *

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard Mem.*
    Nest_under LAYER1

[FILTER]
    Name nest
    Match *
    Operation lift
    Nested_under LAYER1

Output,

[0] mem.local: [1524795757.002111309, {"Swap.total"=>1046524, "Swap.used"=>0, "Swap.free"=>1046524, "Mem.total"=>4050908, "Mem.used"=>1112044, "Mem.free"=>2938864}]

{
  "Swap.total"=>1046524, 
  "Swap.used"=>0,
  "Swap.free"=>1046524, 
  "Mem.total"=>4050908, 
  "Mem.used"=>1112044, 
  "Mem.free"=>2938864
}

Nest 1/2/3

[INPUT]
    Name mem
    Tag  mem.local

[OUTPUT]
    Name  stdout
    Match *

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard Mem.*
    Nest_under LAYER1

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard LAYER1*
    Nest_under LAYER2

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard LAYER2*
    Nest_under LAYER3

Output,

 [0] mem.local: [1524795923.009867831, {"Swap.total"=>1046524, "Swap.used"=>0, "Swap.free"=>1046524, "LAYER3"=>{"LAYER2"=>{"LAYER1"=>{"Mem.total"=>4050908, "Mem.used"=>1112036, "Mem.free"=>2938872}}}}]

 {
   "Swap.total"=>1046524,
   "Swap.used"=>0,
   "Swap.free"=>1046524,
   "LAYER3"=>{
     "LAYER2"=>{
       "LAYER1"=>{
         "Mem.total"=>4050908,
         "Mem.used"=>1112036,
         "Mem.free"=>2938872
       }
     }
   }
 }

Nest 1/2/3 - Lift 3/2

[INPUT]
    Name mem
    Tag  mem.local

[OUTPUT]
    Name  stdout
    Match *

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard Mem.*
    Nest_under LAYER1

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard LAYER1*
    Nest_under LAYER2

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard LAYER2*
    Nest_under LAYER3

[FILTER]
    Name nest
    Match *
    Operation lift
    Nested_under LAYER3

[FILTER]
    Name nest
    Match *
    Operation lift
    Nested_under LAYER2

Output,

[0] mem.local: [1524795834.019236872, {"Swap.total"=>1046524, "Swap.used"=>0, "Swap.free"=>1046524, "LAYER1"=>{"Mem.total"=>4050908, "Mem.used"=>1112152, "Mem.free"=>2938756}}]

{
  "Swap.total"=>1046524,
  "Swap.used"=>0,
  "Swap.free"=>1046524, 
  "LAYER1"=>{
    "Mem.total"=>4050908, 
    "Mem.used"=>1112152, 
    "Mem.free"=>2938756
  }
}

[edsiper]

Please add missing header here:

/*  Fluent Bit
 *  ==========
 *  Copyright (C) 2015-2017 Treasure Data Inc.

[michiel]

Added in 77ac171

[michiel]

Documentation PR added at fluent/fluent-bit-docs#62

[michiel]

Documentation PR updated to reflect addition of 'lift' operation

See https://github.com/michiel/fluent-bit-docs/blob/feature/add_filter_nest/filter/nest.md

[edsiper]

thanks!