Skip to content

Commit

Permalink
Disallow version 1 of audit event configuration
Browse files Browse the repository at this point in the history 
All modules use version 2 (which added the filtering_permitted
attribute). Drop support for version 1 and update the documentation.

Allow 'sync' (which we don't support "yet") and "enabled" to be
optional and default to sync = false and enabled = true to remove
the amount of "boilerplate" info in the per-event specification.

Change-Id: I68a336eaf470a6e319983a30435c27813d1332f8
Reviewed-on: https://review.couchbase.org/c/kv_engine/+/182776
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@couchbase.com>
  • Loading branch information
@trondn
trondn committed Nov 21, 2022
1 parent 6628d89 commit 48beb95
Show file tree
Hide file tree
Showing 11 changed files with 169 additions and 251 deletions.
188 changes: 61 additions & 127 deletions auditd/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,102 +73,67 @@ module are found in `kv_engine/auditd/etc/auditd_descriptor.json`.
## The Per Module Events Descriptor File

An events descriptor JSON structure contains a definition of the
events for the module. The JSON structure contains a version field;
stating the auditd format version used. Version 1 and with the
introduction of Vulcan version 2 is supported.
events for the module, and the following attributes must be present:

The second field contains the module name. This needs
to match the name used in the module descriptor file. The third field
is a list of all the events that are defined for the module. An
example event descriptor file for the auditd module is given below.
* version (number) - the version of the auditd format to be used.
Only version 2 is supported
* module (string) - the name of the module. Must match the name used
in the module descriptor file.
* events (array) - an array of objects where each object defines a single
audit event.

Example:

{
"version" : 1,
"module" : "auditd",
"events" : [
{ "id" : 4096,
"name" : "configured audit daemon",
"description" : "loaded configuration file for audit daemon",
"sync" : false,
"enabled" : true,
"mandatory_fields" : {
"timestamp" : "",
"real_userid" : {"domain" : "", "user" : ""},
"hostname" : "",
"version" : 1,
"auditd_enabled" : true,
"rotate_interval" : 1,
"log_path" : ""
},
"optional_fields" : {}
},

{ "id" : 4097,
"name" : "enabled audit daemon",
"description" : "The audit daemon is now enabled",
"sync" : false,
"enabled" : true,
"mandatory_fields" : {
"timestamp" : "",
"real_userid" : {"domain" : "", "user" : ""}
},
"optional_fields" : {}
},

{ "id" : 4098,
"name" : "disabled audit daemon",
"description" : "The audit daemon is now disabled",
"sync" : false,
"enabled" : true,
"mandatory_fields" : {
"timestamp" : "",
"real_userid" : {"domain" : "", "user" : ""}
},
"optional_fields" : {}
},

{ "id" : 4099,
"name" : "shutting down audit daemon",
"description" : "The audit daemon is being shutdown",
"sync" : false,
"enabled" : true,
"mandatory_fields" : {
"timestamp" : "",
"real_userid" : {"domain" : "", "user" : ""}
},
"optional_fields" : {}
}
]
}


The module defines 4 events; for each event 7 fields must be specified:

* id (number) - the event id; it must be >= startid and <= startid + 0xFFF
* name (string) - short textual name of the event
* description (string) - longer name / description of the event
* sync (bool) - whether the event is synchronous. Currently only async events
are supported
* enabled (bool) - whether the event should be outputted in the audit log.
This feature can be used to depreciate an event, if required.
* mandatory_fields (object) - field(s) required for a valid instance of the
event. In version 1 of the audit format *timestamp* and *real_userid*
fields are required (see below). Other bespoke fields can be added.
* optional_fields - optional field(s) valid in an instance of the event.
Three standard optional_fields are defined in version 1; *sessionID*,
*remote* and *effective_userid*. However additional bespoke fields can
be added, if required. Note: it is valid to have an empty optional_fields,
i.e. {}.
* filtering_permitted (bool) - whether the event can be filtered or not.
"version": 2,
"module": "auditd",
"events": [
{
"id": 4096,
"name": "configured audit daemon",
"description": "loaded configuration file for audit daemon",
"sync": false,
"enabled": true,
"filtering_permitted": false,
"mandatory_fields": {
"timestamp": "",
"real_userid": {
"domain": "",
"user": ""
},
"hostname": "",
"version": 1,
"auditd_enabled": true,
"rotate_interval": 1,
"log_path": "",
"descriptors_path": ""
},
"optional_fields": {}
}
]
}

The module defines 1 event; for each event the following fields may be specified:

* id (number, mandatory) - the event id; it must be >= startid and <= startid + 0xFFF
* name (string, mandatory) - short textual name of the event
* description (string, mandatory) - longer name / description of the event
* sync (bool, optional) - whether the event is synchronous.
Currently only async events are supported (default if not specified)
* enabled (bool, optional) - whether the event should be enabled by default.
* filtering_permitted (bool, optional) - whether the event can be filtered or not.
If the attribute is not defined then it is defaulted that the event
cannot be filtered. Note: we don't want to permit any ns_server or audit
events from being filtered.
cannot be filtered.
* mandatory_fields (object, mandatory) - field(s) required for a valid instance of the
event.
* optional_fields (object, optional) - optional field(s) valid in an instance of the event.
Note: it is valid to have an empty optional_fields, i.e. {}.

### Defining the format for mandatory and optional fields

The format of mandatory and optional fields are defined to allow
validation of the events submitted to auditd. Note: validation is not currently
implemented.
validation of the events submitted to auditd. Note: Only simple validation
is implemented (that all the mandatory fields are present).

Field types can be any valid JSON type. The type is specified by
providing the following default values:
Expand All @@ -183,14 +148,12 @@ The example JSON structure below shows the definition for the 4
pre-defined mandatory fields; *timestamp*, *real_userid*, *remote*, *local*,
and the 2 pre-defined optional fields; *sessionID* and *effective_userid*.

{"version" : 1,
{"version" : 2,
"module" : "example",
"events" : [
{ "id" : 8192,
"name" : "example event",
"description" : "this is the full description of the example event",
"sync" : false,
"enabled" : true,
"mandatory_fields" : {
"timestamp" : "",
"real_userid" : {"domain" : "", "user" : ""},
Expand All @@ -205,42 +168,16 @@ and the 2 pre-defined optional fields; *sessionID* and *effective_userid*.
]
}

A similar example for Version 2 where filtering_permitted is defined is provided
below:

{"version" : 2,
"module" : "example",
"events" : [
{ "id" : 8192,
"name" : "example event",
"description" : "this is the full description of the example event",
"sync" : false,
"enabled" : true,
"filtering_permitted" : true,
"mandatory_fields" : {
"timestamp" : "",
"real_userid" : {"domain" : "", "user" : ""}
},
"optional_fields" : {
"sessionid" : ""
"remote" : {"ip" : "", "port" : 1}
"effective_userid" : {"domain" : "", "user" : ""}
}
}
]
}

#### Pre-defined Mandatory Fields
#### Pre-defined Fields

* timestamp - Contains the date and time of the event, in ISO 8601 format.
* `timestamp` - Contains the date and time of the event, in ISO 8601 format.
Uses local time with timezone offset (from UTC) in hours and minutes.
Records microsecond granularity using 3 digits after decimal point.
* real_userid - comprises a "domain", which states where the user is
defined: "local" (in Couchbase) or "external" (outside Couchbase; LDAP etc).
It then contains the user string.

Note: In version 2 the real_user_id has been changed from
`{"source" : "", "user" : ""}` to `{"domain" : "", "user" : ""}`.
* `real_userid` and `effective_userid` - comprises a "domain", which states
where the user is defined: "local" (in Couchbase) or "external" (outside
Couchbase; LDAP etc). It then contains the user string. Example:
` {"domain":"local","user":"@ns_server"}`
* `remote` and `local` - An object containing two fields `ip` and `port`

#### Pre-defined Optional Fields

Expand All @@ -257,9 +194,6 @@ Note: In version 2 the real_user_id has been changed from
"_admin" (query auth's as the internal admin); effective user ID is what
ever the client's ID is.

Note: Similar to real_user_id the notation has changed from version 1 to
version 2, to be `{"domain" : "", "user" : ""}`.

#### Duration

* The query team include durations in some audit events. We want to
Expand Down
3 changes: 2 additions & 1 deletion auditd/generator/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ add_library(audit_generator_utilities OBJECT
generator_module.h
generator_utilities.cc
generator_utilities.h)
target_link_libraries(audit_generator_utilities PRIVATE nlohmann_json::nlohmann_json)
target_link_libraries(audit_generator_utilities
PRIVATE nlohmann_json::nlohmann_json fmt::fmt)
kv_enable_pch(audit_generator_utilities)

add_executable(auditevent_generator
Expand Down
47 changes: 22 additions & 25 deletions auditd/generator/generator_event.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,34 +9,31 @@
* the file licenses/APL2.txt.
*/
#include "generator_event.h"
#include "generator_utilities.h"

#include <gsl/gsl-lite.hpp>
#include <nlohmann/json.hpp>
#include <sstream>

Event::Event(const nlohmann::json& json) {
id = json.at("id");
name = json.at("name").get<std::string>();
description = json.at("description").get<std::string>();
sync = json.at("sync");
enabled = json.at("enabled");
auto cFilteringPermitted = json.value("filtering_permitted", -1);
mandatory_fields = json.at("mandatory_fields").dump();
optional_fields = json.at("optional_fields").dump();
void to_json(nlohmann::json& json, const Event& event) {
json = nlohmann::json{
{"id", event.id},
{"name", event.name},
{"description", event.description},
{"sync", event.sync},
{"enabled", event.enabled},
{"filtering_permitted", event.filtering_permitted},
{"mandatory_fields", nlohmann::json::parse(event.mandatory_fields)},
{"optional_fields", nlohmann::json::parse(event.optional_fields)}};
}

if (cFilteringPermitted != -1) {
filtering_permitted = gsl::narrow_cast<bool>(cFilteringPermitted);
void from_json(const nlohmann::json& j, Event& event) {
j.at("id").get_to(event.id);
j.at("name").get_to(event.name);
j.at("description").get_to(event.description);
event.sync = j.value("sync", false);
event.enabled = j.value("enabled", true);
event.filtering_permitted = j.value("filtering_permitted", false);
event.mandatory_fields = j.at("mandatory_fields").dump();
if (j.contains("optional_fields")) {
event.optional_fields = j.at("optional_fields").dump();
} else {
filtering_permitted = false;
}

auto num_elem = json.size();
if ((cFilteringPermitted == -1 && num_elem != 7) ||
(cFilteringPermitted != -1 && num_elem != 8)) {
std::stringstream ss;
ss << "Unknown elements for " << name << ": " << std::endl
<< json << std::endl;
throw std::runtime_error(ss.str());
event.optional_fields = "{}";
}
}
23 changes: 5 additions & 18 deletions auditd/generator/generator_event.h
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
/* -*- Mode: C++; tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- */
/*
* Copyright 2018-Present Couchbase, Inc.
*
Expand All @@ -11,7 +10,6 @@

#pragma once

#include <gsl/gsl-lite.hpp>
#include <nlohmann/json_fwd.hpp>
#include <cstdint>
#include <string>
Expand All @@ -20,21 +18,7 @@
* The Event class represents the information needed for a single
* audit event entry.
*/
class Event {
public:
Event() = delete;

/**
* Construct and initialize a new Event structure based off the
* provided JSON. See ../README.md for information about the
* layout of the JSON element.
*
* @param entry
* @throws std::runtime_error for errors accessing the expected
* elements
*/
explicit Event(const nlohmann::json& json);

struct Event {
/// The identifier for this entry
uint32_t id;
/// The name of the entry
Expand All @@ -45,7 +29,7 @@ class Event {
bool sync;
/// Set to true if this entry is enabled (or should be dropped)
bool enabled;
/// Set to true if the user may enable filtering for the enry
/// Set to true if the user may enable filtering for the entry
bool filtering_permitted;
/// The textual representation of the JSON describing mandatory
/// fields in the event (NOTE: this is currently not enforced
Expand All @@ -56,3 +40,6 @@ class Event {
/// by the audit daemon)
std::string optional_fields;
};

void to_json(nlohmann::json& json, const Event& event);
void from_json(const nlohmann::json& j, Event& event);
36 changes: 25 additions & 11 deletions auditd/generator/generator_event_test.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,13 +76,8 @@ TEST_F(EventParseTest, TestCorrectInput) {
* Verify that we detect that a mandatory field is missing
*/
TEST_F(EventParseTest, MandatoryFields) {
for (const auto& tag : std::vector<std::string>{{"id",
"name",
"description",
"sync",
"enabled",
"mandatory_fields",
"optional_fields"}}) {
for (const auto& tag : std::vector<std::string>{
{"id", "name", "description", "mandatory_fields"}}) {
auto removed = json.at(tag);
json.erase(tag);
try {
Expand All @@ -101,8 +96,27 @@ TEST_F(EventParseTest, MandatoryFields) {
TEST_F(EventParseTest, OptionalFields) {
// "filtering_permitted" is optional, and should be set to false if it
// is missing
auto removed = json.at("filtering_permitted");
json.erase("filtering_permitted");
Event event(json);
ASSERT_FALSE(event.filtering_permitted);
{
json.erase("filtering_permitted");
Event event(json);
EXPECT_FALSE(event.filtering_permitted);
}

{
json.erase("sync");
Event event(json);
EXPECT_FALSE(event.sync);
}

{
json.erase("enabled");
Event event(json);
EXPECT_TRUE(event.enabled);
}

{
json.erase("optional_fields");
Event event(json);
EXPECT_EQ("{}", event.optional_fields);
}
}
Loading

0 comments on commit 48beb95

Please sign in to comment.