Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x-pack/filebeat/input/entityanalytics/provider/okta: allow fine-grain control of API requests #36492

Merged
merged 2 commits into from
Sep 5, 2023
Merged

x-pack/filebeat/input/entityanalytics/provider/okta: allow fine-grain control of API requests #36492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8fcccc0
x-pack/filebeat/input/entityanalytics/provider/okta: allow fine-grain…
efd6 Sep 3, 2023
42489fb
address pr comment
efd6 Sep 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Reduce HTTPJSON metrics allocations. {pull}36282[36282]
- Add support for a simplified input configuraton when running under Elastic-Agent {pull}36390[36390]
- Make HTTPJSON response body decoding errors more informative. {pull}36481[36481]
- Allow fine-grained control of entity analytics API requests for Okta provider. {issue}36440[36440] {pull}36492[36492]

*Auditbeat*

Expand Down
9 changes: 9 additions & 0 deletions x-pack/filebeat/docs/inputs/input-entity-analytics.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -545,6 +545,7 @@ Example configuration:
enabled: true
id: okta-1
provider: okta
dataset: "all"
sync_interval: "12h"
update_interval: "30m"
okta_domain: "OKTA_DOMAIN"
Expand All @@ -570,6 +571,14 @@ Whether the input should collect device and device-associated user details
from the Okta API. Device details must be activated on the Okta account for
this option.

[float]
===== `dataset`

The datasets to collect from the API. This can be one of "all", "users" or "devices",
or may be left empty for the default behavior which is to collect all entities.
When the `dataset` is set to "devices", some user entity data is collected in order
to populate the registered users and registered owner fields for each device.

[float]
===== `sync_interval`

Expand Down
15 changes: 10 additions & 5 deletions x-pack/filebeat/input/entityanalytics/provider/okta/conf.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ package okta

import (
"errors"
"strings"
"time"

"github.com/elastic/elastic-agent-libs/transport/httpcommon"
Expand Down Expand Up @@ -41,10 +42,10 @@ type conf struct {
OktaDomain string `config:"okta_domain" validate:"required"`
OktaToken string `config:"okta_token" validate:"required"`

// WantDevices indicates that device details
// should be collected. This is optional as
// the devices API is not necessarily activated.
WantDevices bool `config:"collect_device_details"`
// Dataset specifies the datasets to collect from
// the API. It can be ""/"all", "users", or
// "devices".
Dataset string `config:"dataset"`

// SyncInterval is the time between full
// synchronisation operations.
Expand Down Expand Up @@ -159,7 +160,11 @@ func (c *conf) Validate() error {
return errInvalidUpdateInterval
case c.SyncInterval <= c.UpdateInterval:
return errSyncBeforeUpdate
default:
}
switch strings.ToLower(c.Dataset) {
case "", "all", "users", "devices":
return nil
default:
return errors.New("dataset must be 'all', 'users', 'devices' or empty")
}
}
17 changes: 15 additions & 2 deletions x-pack/filebeat/input/entityanalytics/provider/okta/okta.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
"io"
"net/http"
"net/url"
"strings"
"time"

"github.com/hashicorp/go-retryablehttp"
Expand Down Expand Up @@ -338,6 +339,13 @@
// any existing deltaLink will be ignored, forcing a full synchronization from Okta.
// Returns a set of modified users by ID.
func (p *oktaInput) doFetchUsers(ctx context.Context, state *stateStore, fullSync bool) ([]*User, error) {
switch strings.ToLower(p.cfg.Dataset) {
case "", "all", "users":
default:
p.logger.Debugf("Skipping user collection from API: dataset=%s", p.cfg.Dataset)
return nil, nil
}

var (
query url.Values
err error
Expand Down Expand Up @@ -391,7 +399,7 @@

next, err := okta.Next(h)
if err != nil {
if err == io.EOF {

Check failure on line 402 in x-pack/filebeat/input/entityanalytics/provider/okta/okta.go

View workflow job for this annotation

GitHub Actions / lint (windows) 

comparing with == will fail on wrapped errors. Use errors.Is to check for a specific error (errorlint)
break
}
p.logger.Debugf("received %d users from API", len(users))
Expand All @@ -418,7 +426,10 @@
// synchronization from Okta.
// Returns a set of modified devices by ID.
func (p *oktaInput) doFetchDevices(ctx context.Context, state *stateStore, fullSync bool) ([]*Device, error) {
if !p.cfg.WantDevices {
switch strings.ToLower(p.cfg.Dataset) {
case "", "all", "devices":
default:
p.logger.Debugf("Skipping device collection from API: dataset=%s", p.cfg.Dataset)
return nil, nil
}

Expand Down Expand Up @@ -482,12 +493,14 @@

// Users are not stored in the state as they are in doFetchUsers. We expect
// them to already have been discovered/stored from that call and are stored
// associated with the device undecorated with discovery state.
// associated with the device undecorated with discovery state. Or, if the
// the dataset is set to "devices", then we have been asked not to care about
// this detail.
batch[i].Users = append(batch[i].Users, users...)

next, err := okta.Next(h)
if err != nil {
if err == io.EOF {

Check failure on line 503 in x-pack/filebeat/input/entityanalytics/provider/okta/okta.go

View workflow job for this annotation

GitHub Actions / lint (windows) 

comparing with == will fail on wrapped errors. Use errors.Is to check for a specific error (errorlint)
break
}
p.logger.Debugf("received %d devices from API", len(devices))
Expand Down Expand Up @@ -516,7 +529,7 @@

next, err := okta.Next(h)
if err != nil {
if err == io.EOF {

Check failure on line 532 in x-pack/filebeat/input/entityanalytics/provider/okta/okta.go

View workflow job for this annotation

GitHub Actions / lint (windows) 

comparing with == will fail on wrapped errors. Use errors.Is to check for a specific error (errorlint)
break
}
p.logger.Debugf("received %d devices from API", len(devices))
Expand Down