audit-logging: Filter uninteresting conversations out of audit log #8242

Merged
merged 4 commits into from Dec 22, 2017

Conversation

Projects
None yet
5 participants
Member

babbageclunk commented Dec 19, 2017

Description of change

Read-only API requests (like status) aren't interesting from the perspective of audit logging. To avoid burying important information in the log we buffer the conversation and any uninteresting requests/responses until we see an interesting request, at which point we flush the buffer and write out any later messages immediately.

This means that any logged conversation will have the full set of requests even if some of them are only read-only.

QA steps

Bootstrap a controller with auditing enabled. Run various read-only commands, check that none of them generate audit log entries. Run commands that change state (setting config, deploying apps, adding and removing models, units and machines). Check that all of those show up in the audit log.

babbageclunk added some commits Dec 18, 2017

Filter uninteresting conversations out of audit log
Read-only API requests (like status) aren't interesting from the
perspective of audit logging. To avoid burying important information in
the log we buffer the conversation and any uninteresting
requests/responses until we see an interesting request, at which point
we flush the buffer and write out any later messages immediately.

This means that any logged conversation will have the full set of
requests even if some of them are only read-only.

Looks pretty neat!
(I love the fact that you went with observer pattern :D my favourite!)
My only concern at this stage is that Login call itself is not audited.

apiserver/admin.go
@@ -160,8 +160,11 @@ func (a *admin) getAuditRecorder(req params.LoginRequest, authResult *authResult
if !authResult.userLogin || a.srv.auditLogger == nil {
return nil, nil
}
+ // Wrap the audit logger in a filter that prevents us from logging
+ // lots of readonly conversations (like juju status requests).
@anastasiamac

anastasiamac Dec 19, 2017

Member

nit - I'd prefer to surround 'juju status' in quotes :)

@@ -967,9 +969,23 @@ func (s *loginSuite) TestLoginAddsAuditConversation(c *gc.C) {
err := conn.APICall("Admin", 3, "", "Login", request, &result)
c.Assert(err, jc.ErrorIsNil)
c.Assert(result.UserInfo, gc.NotNil)
+ // Nothing's logged at this point because there haven't been any
@anastasiamac

anastasiamac Dec 19, 2017

Member

I am a bit concerned that "Login" request itself is not considered "interesting"... Most of auditing is actually VERY interested in "login" and "login" attempts :D

@babbageclunk

babbageclunk Dec 19, 2017

Member

Hmm. Since every client API interaction requires a login, including all logins would kind of defeat the purpose of filtering here - my watch juju status will still fill up the log with noise. I think you're right about logging failed login attempts though. I'll add a card for it and do it as a separate PR, if that's ok - it's pretty separate from this change.

apiserver/admin_test.go
@@ -1003,10 +1032,23 @@ func (s *loginSuite) TestAuditLoggingFailurePreventsLogin(c *gc.C) {
request := &params.LoginRequest{
AuthTag: user.Tag().String(),
Credentials: password,
- CLIArgs: "hey you guys",
+
@anastasiamac

anastasiamac Dec 19, 2017

Member

Probably, don't need a newline?

@babbageclunk

babbageclunk Dec 19, 2017

Member

Oops, due to some inexpert keyboard mashing.

apiserver/observer/auditfilter.go
+ "Client.GetModelConstraints",
+ "Client.StatusHistory",
+ "Controller.AllModels",
+ "Controller.ControllerConfig",
@anastasiamac

anastasiamac Dec 19, 2017

Member

Is this a read-only call? I seem to recall that it was also a mutator based on whether the data was passed-in or not...

@babbageclunk

babbageclunk Dec 19, 2017

Member

No, this one doesn't accept any params.

apiserver/observer/auditfilter.go
+ "Controller.AllModels",
+ "Controller.ControllerConfig",
+ "Controller.GetControllerAccess",
+ "Controller.ModelConfig",
@anastasiamac

anastasiamac Dec 19, 2017

Member

a mutator also, perhaps?

@babbageclunk

babbageclunk Dec 19, 2017

Member

No, it also doesn't accept args.

apiserver/observer/auditfilter.go
+ "Action.ApplicationsCharmsActions",
+ "Action.FindActionsByNames",
+ "Action.FindActionTagsByPrefix",
+ "Application.Get",
@wgrant

wgrant Dec 19, 2017

Application.Get returns the config, which will usually have secrets.

@anastasiamac

anastasiamac Dec 19, 2017

Member

We will not be recording it :D
This is a black list of Facade.Method - meaning "do not audit these calls". This will also ignore all arguments, parameters and return values to/from these methods.

@wgrant

wgrant Dec 19, 2017

Right, but if you're going to audit things, surely you want to audit things that will be the easiest way to exfiltrate secrets.

@babbageclunk

babbageclunk Dec 19, 2017

Member

Yeah, that's a good point - and I don't think users will do it so often in the normal course of events that it'll obscure other information in the logs.

@babbageclunk

babbageclunk Dec 19, 2017

Member

Oops, I meant to say, so I'll remove it from the filter list.

I think it's a nice initial cut - thank you :)
+1 from me but there seems to be some test failures.
Also since William raised some concerns, maybe worthwhile getting a 2nd review from @jameinel or @wupeka ?

Owner

jameinel commented Dec 19, 2017

I would be much happier if this filtering was being done on read rather than on write. Because you can't ever recover the information if it was never written in the first place.
Do we know that audit log is growing out of control without pruning on write? (We may need this because audit log is growing too fast without it.)
But deciding "what is interesting" feels very much where different operators can differ in their desire. So at a minimum I'd want to see a way to enable/disable this. We could default it to being on, with a way to disable it.

Member

babbageclunk commented Dec 19, 2017

Thanks John - yeah, looking at it now I think I went overboard on the filtering. Maybe a better way would be minimal filtering by default (just Client.FullStatus) with the list of methods to filter being a configuration setting (I guess just a comma-separated list). That way operators could remove anything they decided didn't need to be logged.

Cut back filter list to just remove "juju status"
It would be better not to assume what operators will want to include in
the audit log. Instead, just filter out the noisiest API method, and in
a follow-up change we'll allow users to configure this list to remove
other methods they don't care about from the log.
Member

babbageclunk commented Dec 22, 2017

$$merge$$

Contributor

jujubot commented Dec 22, 2017

Status: merge request accepted. Url: http://ci.jujucharms.com/job/github-merge-juju

@jujubot jujubot merged commit ef55053 into juju:2.3 Dec 22, 2017

1 check passed

continuous-integration/jenkins/pr-merge This commit looks good
Details

@babbageclunk babbageclunk deleted the babbageclunk:audit-filter branch Dec 22, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment