[Okta] Add event categories and types

[moxarth-rathod]

Proposed commit message

okta: add missing event categories

[git-generate]
MAPPINGS=https://gist.githubusercontent.com/moxarth-rathod/a0f59f128a207deb162ab31b64eb19d4/raw/91c0935490b409fb50f2e3065617cc01cf8f25cc/okta_event_categories_with_ecs.json
SCRIPT=https://gist.githubusercontent.com/moxarth-rathod/a0f59f128a207deb162ab31b64eb19d4/raw/a70f0cfce18d05d12c0d9d579c87e456f81fa00d/main.go

cd packages/okta
mkdir gen
(
  cd gen
  curl -O $MAPPINGS
  curl -O $SCRIPT
  curl -O https://developer.okta.com/docs/okta-event-types.csv
  go mod init gen
  go get github.com/efd6/dispear@v0.0.0-20251110014810-dfeb192cf303
  go mod tidy
  go run . > ../data_stream/system/elasticsearch/ingest_pipeline/ecs_category_type.yml
)
sed -i '/if: '"'"'\["policy.evaluate_sign_on"].contains(ctx.okta?.event_type)/ {
  n
  a\
  - pipeline:\
      tag: pipeline_ecs_category_type\
      name: '\''{{ IngestPipeline "ecs_category_type" }}'\''
}' data_stream/system/elasticsearch/ingest_pipeline/default.yml
rm -rf gen
elastic-package test pipeline -g
elastic-package changelog add --description "Sets the ECS event categorization fields for system data stream." --type enhancement --next minor --link elastic#15860

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

• Clone integrations repo.
• Install elastic package locally.
• Start elastic stack using elastic-package.
• Move to integrations/packages/okta directory.
• Run the following command to run tests.

elastic-package test -v

Related issues

• Closes [Okta] Add Missing Event Categories #15235

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[ShourieG]

@moxarth-rathod, looks good, majority of the pipeline was autogenerated so looks fine, please fix the PR no. and i'll approve.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[moxarth-rathod]

@terrancedejesus @ShourieG @efd6 fyi, I had to adjust some of the suggested mappings because certain event categories contained invalid event types. For example, the access.request.cancel event category was set to iam with event types access and deletion, but iam only allows the following event types: admin, change, creation, deletion, group, info, and user. Therefore, I removed access to align with the allowed values.

[efd6]

Can you confirm that you have used the generator shown in #15235 (comment) with the definitions in #15235 (comment)?

[moxarth-rathod]

Can you confirm that you have used the generator shown in #15235 (comment) with the definitions in #15235 (comment)?

Yes, but I made a few changes during testing, including adding null handling. Also, some event types were invalid for their respective categories, so I've updated those definitions accordingly.

[andrewkroh]

Can you please share the exact code used to produce this in a gist for me to replicate and validate .

[efd6]

My preferred approach would be to have something like (until we can commit this kind of data) (exact text depends on how the definition data is ingested by the generator and will require some additions to handle dep modules):

informative commit message title

[git-generate]
MAPPINGS=<link to git or other stable location for the JSON definitions>
curl -o okta_event_categories_with_ecs.json $MAPPINGS
cat >main.go <<END
package main
…
END
go run main.go > packages/okta/…
rm -i main.go okta_event_categories_with_ecs.json

[moxarth-rathod]

@andrewkroh i've prepared a gist for the script to generate a pipeline. I'll also update the commit message with gist and proper steps to run it.

[andrewkroh]

Thanks. And how is the Okta JSON file created? It looks like it is a conversion of the CSV given at the link below to JSON, and then some categories were assigned?

It would be good to be able to easily identify when our pipeline is no longer in sync with the Okta event types (i.e. there are new event types that need to be added and manually categorized). So it might be better if the inputs to the Go script were the raw CSV from okta.com plus a table of event types to ECS categories. And the script fails when there are uncategorized event types.

For an example, we already need to update that Okta JSON content because there are new events types in the 2025.10 okta release.

https://developer.okta.com/docs/reference/api/event-types/
https://developer.okta.com/docs/okta-event-types.csv

[moxarth-rathod]

Thanks @andrewkroh for the suggestion! I've made the changes in pipeline. I've also updated the script in gist for your reference (please refer PR description for steps to run script).

[efd6]

The CSV is not currently used. That will need to be fixed.

I've revised the commit message to confirm that it works; this required some work.

okta: add missing event categories

[git-generate]
MAPPINGS=https://gist.githubusercontent.com/moxarth-rathod/a0f59f128a207deb162ab31b64eb19d4/raw/d709c826a19b5f857ad29e0bc414815bfc90e709/okta_event_categories_with_ecs.json
SCRIPT=https://gist.githubusercontent.com/moxarth-rathod/a0f59f128a207deb162ab31b64eb19d4/raw/d709c826a19b5f857ad29e0bc414815bfc90e709/main.go

cd packages/okta
mkdir gen
(
  cd gen
  curl -O $MAPPINGS
  curl -O $SCRIPT
  curl -O https://developer.okta.com/docs/okta-event-types.csv
  go mod init gen
  go get github.com/efd6/dispear@v0.0.0-20251110014810-dfeb192cf303
  go mod tidy
  go run . > ../data_stream/system/elasticsearch/ingest_pipeline/ecs_category_type.yml
)
rm -rf gen

When using git-generate it's best to keep a linear history (no merging of main back into the change) and if you can, keeping all the changes in control of the script. This is harder when squashes are used, but it can be done if you add

elastic-package test pipeline -g
elastic-package test system -g
elastic-package build
elastic-package changelog add --description "Sets the ECS event categorization fields for system data stream." --type enhancement --next minor --link 15860

to the end of the script (along with a sed edit to inject the pipeline processor which is left as an exercise for the reader).

This all should be in the commit message of the commit that made the changes, not for a collection of commits that get squashed.

I'm a little uncomfortable with reaching out to a gist for the code, but I could not get local Go code from the commit message.

[moxarth-rathod]

@efd6 I've made the changes as you suggested.

[terrancedejesus]

@terrancedejesus @ShourieG @efd6 fyi, I had to adjust some of the suggested mappings because certain event categories contained invalid event types. For example, the access.request.cancel event category was set to iam with event types access and deletion, but iam only allows the following event types: admin, change, creation, deletion, group, info, and user. Therefore, I removed access to align with the allowed values.

Thank you for doing so and sharing why. I reviewed the docs briefly this morning. I am surprised iam does not allow at least allowed and denied event types? For example:

• OAuth request for token (IAM) - access was allowed to this resource
• SAML request for access (IAM) - access was denied to this resource

Especially when we consider IAM covering behavior such as token issuance. Are these boundaries set within the elasticsearch codebase or at an integration level? Understandably it is out-of-scope for this PR but I can carry the conversation forward to the appropriate team.

[efd6]

It looks like there are duplicate tags in the definitions. For example,

  {
    "event": "directory.user_profile.update",
    "description": "Update universal directory user profile directory.user_profile.update. Triggered when an event hook has been activated. Used to notify admin
s that an event hook has been activated. When triggered, this events contains information about the activated event hook. Triggered when an event hook has been 
created. Used to notify admins that an event hook has been created. When triggered, this events contains information about the created event hook. Triggered whe
n an event hook has been deactivated. Used to notify admins that an event hook has been deactivated. When triggered, this events contains information about the 
deactivated event hook. Triggered when an event hook has been deleted. Used to notify admins that an event hook has been deleted. When triggered, this events co
ntains information about the deleted event hook. Triggered when an event hook delivery fails. Used to identify when an event hook from Okta is not successfully 
delivered to the configured endpoint. Note that the event is triggered only when the delivery is unsuccessful. Triggered when an event hook has been updated. Us
ed to notify admins that an event hook has been updated. When triggered, this events contains information about the updated event hook. Triggered when attemptin
g to verify an event hook. Used to notify admins about the outcome of event hook endpoint URL verification. Note that the event is fired even when the verificat
ion is unsuccessful.",
    "tags": [
      "event-hook",
      "event-hook",
      "event-hook",
      "event-hook",
      "event-hook",
      "event-hook",
      "event-hook"
    ],
    "since": "2019.03.4",
    "ecs_category": "iam",
    "ecs_event_type": [
      "change",
      "info",
      "user"
    ]
  },

We can make the generator remove these, or we can ask that the input be sanitised. I'm thinking a more robust generator is a safer options.

diff --git a/main.go b/main.go
index 2cc3b44..c3a1e7b 100644
--- a/main.go
+++ b/main.go
@@ -7,6 +7,7 @@ import (
        "log"
        "os"
        "path/filepath"
+       "sort"
        "strings"
 
        // For okta_event_categories_with_ecs.json.
@@ -79,13 +80,13 @@ func main() {
        for _, m := range mapping {
                d := make(map[string]any)
                if len(m.Type) != 0 {
-                       d["type"] = m.Type
+                       d["type"] = unique(m.Type)
                }
                if len(m.Category) != 0 {
                        d["category"] = []string{m.Category}
                }
                if len(m.Tags) != 0 {
-                       d["tags"] = m.Tags
+                       d["tags"] = unique(m.Tags)
                }
                if len(d) == 0 {
                        continue
@@ -127,3 +128,21 @@ Okta event type mapping to ECS event type and category.`, filepath.Base(os.Args[
 }
 
 const errorFormat = "Processor '{{{ _ingest.on_failure_processor_type }}}' {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}'"
+
+func unique(ss []string) []string {
+       if len(ss) < 2 {
+               return ss
+       }
+       sort.Strings(ss)
+       curr := 0
+       for i, s := range ss {
+               if s == ss[curr] {
+                       continue
+               }
+               curr++
+               if curr < i {
+                       ss[curr], ss[i] = ss[i], ""
+               }
+       }
+       return ss[:curr+1]
+}

[efd6]

There is also this

  {
    "event": "iam.role.update",
    "description": "Custom admin role update. Use this event to track and audit when a custom admin role was updated. This event contains information about the 
updated name and description of the role. Triggered when an inline hook in activated. Used to identify when an inline hook lifecycle status was changed to activ
ated. When triggered, this events contains information about the activated inline hook. Triggered when an inline hook has been created. Used to notify admins th
at an inline hook has been created. When triggered, this events contains information about the created inline hook. Triggered when an inline hook is deactivated
. Used to identify when an inline hook lifecycle status was changed to deactivated. When triggered, this events contains information about the deactivated inlin
e hook. Triggered when an inline hook has been deleted. Used to notify admins that an inline hook has been deleted. When triggered, this events contains informa
tion about the deleted inline hook. Triggered when an inline hook has been executed. Used to notify admins about the outcome of execution of an inline hook. Note that the event is fired when the execution is unsuccessful. Triggered after Okta has finished processing response from an inline hook. Used to notify admins about the outcome of processing response from an inline hook. Note that the event is fired even when the processing is unsuccessful. Triggered when an inline hook has been modified. Used to notify admins that an inline hook has been updated. When triggered, this events contains information about the updated inline hook. Triggered when attempting to verify an inline hook. Used to notify admins about the outcome of inline hook endpoint URL verification. Note that the event is fired even when the verification is unsuccessful.",
    "tags": [
      "admin-role",
      "event-hook-eligible",
      "inline-hook",
      "inline-hook",
      "inline-hook",
      "inline-hook",
      "event-hook-eligibleinline-hook",
      "inline-hook",
      "inline-hook",
      "inline-hook"
    ],
    "since": "2019.01.2",
    "ecs_category": "iam",
    "ecs_event_type": [
      "change",
      "info"
    ]
  },

"event-hook-eligibleinline-hook" looks suspicious.

@terrancedejesus Can you go over the JSON definitions and check that they are not corrupted?

[terrancedejesus]

@terrancedejesus Can you go over the JSON definitions and check that they are not corrupted?

Yes my initial parsing of the tags was incorrect. I've fixed this and attached the updated file. The original CSV file for these from Okta is here --> https://developer.okta.com/docs/reference/api/event-types/

okta_event_categories_with_ecs_fixed.json

[efd6]

@moxarth-rathod Please update the generator and input JSON.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: fa53789

History

• 💚 Build #34212 succeeded 793acf2
• 💚 Build #34091 succeeded 7af08ac
• 💚 Build #33804 succeeded c95a593
• 💔 Build #33803 failed 0e6bbac
• 💚 Build #33649 succeeded 799e6ea

cc @moxarth-rathod

[moxarth-rathod]

@moxarth-rathod Please update the generator and input JSON.

@efd6 done, i've made the changes.

[efd6]

This LGTM, but I'll wait for @terrancedejesus to go over the behaviour before approving.

[terrancedejesus]

This LGTM, but I'll wait for @terrancedejesus to go over the behaviour before approving.

Is there a way to test the changes? Is there a data example we can generate to review?

[efd6]

@terrancedejesus I would suggest just taking a look at the behaviour that is shown in the pipeline tests.

[terrancedejesus]

@terrancedejesus I would suggest just taking a look at the behaviour that is shown in the pipeline tests.

@efd6 Apologies for the late response as I've been OOO. I reviewed the buildkite jobs / tasks. LGTM!

[efd6]

Thanks

[elastic-vault-github-plugin-prod]

Package okta - 3.12.0 containing this change is available at https://epr.elastic.co/package/okta/3.12.0/