[filebeat][httpjson] - Added support for Okta Oauth2 provider

[ShourieG]

Type of change

• Enhancement
• Docs

Proposed commit message

Added support for Okta as a separate OAuth2 provider. This was required because specific use cases
require access tokens with okta.* scopes, which involves a specific 2 step auth mechanism using a private
RSA JWK to generate a JWT which is in turn used to fetch an access token. Standard client_id & client_secrets don't
work in this scenario.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
  - [] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

1. Acquire an okta dev or user account with privilages to mint tokens with the okta.* scopes.
2. Log into your okta account, navigate to Applications on the left hand side, and click on the "Create App Integration" button and create an API Services application.
3. Click on the created app and note down the Client Id and select the option for Public key / Private key.
4. Generate your own Private/Public key pair in the JWK format (PEM is not supported atm) and save it in a credentials json file or copy it to use directly in the config.
5. Tokens with okta.* scopes are generally minted from Okta Org Auth server and not the default/custom authorization server.
6. The standard Okta Org Auth server endpoint to mint tokens is https://<your_okta_org>.okta.com/oauth2/v1/token
7. Use the httpjson config sample to configure the input to read okta system logs.
   Other details could also be read according to your requirements, the scope and endpoint url need to be changed accordingly.

Sample configuration for reading system logs via Okta System Log API:

filebeat.inputs:
- type: httpjson
  enabled: true
  id: my-httpjson-id
  request.url: https://<your_okta_org>/api/v1/logs
  interval: 1h
  auth.oauth2.provider: okta
  auth.oauth2.client.id: 0oaajljieokFZLyKXcU7
  auth.oauth2.token_url: https://<your_okta_org>/oauth2/v1/token
  auth.oauth2.scopes:
  - okta.logs.read
  auth.oauth2.okta.jwk_file: /path/to/your/jwk.json
  request.rate_limit:
  limit: '[[.last_response.header.Get "X-Rate-Limit-Limit"]]'
  remaining: '[[.last_response.header.Get "X-Rate-Limit-Remaining"]]'
  reset: '[[.last_response.header.Get "X-Rate-Limit-Reset"]]'
  request.transforms:
    - set:
        target: url.params.limit
        value: '1000'
    - set:
        target: url.params.since
        value: "[[.cursor.published]]"
        default: '[[formatDate (now (parseDuration "-24h")) "RFC3339"]]'
  response.pagination:
    - set:
        target: url.value
        value: '[[ getRFC5988Link "next" .last_response.header.Link ]]'
        fail_on_template_error: true

  cursor:
    published:
      value: "[[.last_event.published]]"

Related issues

• Relates Okta Oauth Support

Use cases

Screenshots

Logs

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @ShourieG? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-08-12T04:19:53.281+0000

• Duration: 124 min 12 sec

Test stats 🧪

Test	Results
Failed	0
Passed	28112
Skipped	2015
Total	30127

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b okta/oauth2 upstream/okta/oauth2
git merge upstream/main
git push upstream okta/oauth2

[kcreddy]

The change LGTM.
But it would be nice if someone with better knowledge of httpjson to have a review the change and module upgrades as well.

[ShourieG]

@kcreddy @bhapas have resolved all your comments, If everything looks good can anyone approve. I'll still need a beats tech lead approval nonetheless.

[ShourieG]

@andrewkroh will require an approval from your end as a beats tech lead review is mandatory here.

[andrewkroh]

I don't see any tests except for the configuration. What manual testing has been performed? Ideally there would be some test to exercise the TokenSource implementation.

[ShourieG]

I don't see any tests except for the configuration. What manual testing has been performed? Ideally there would be some test to exercise the TokenSource implementation.

@andrewkroh Have tested it extensively with an okta dev account and used a httpjson config similar to the okta integration to poll system logs and everything thing is working as expected.

[andrewkroh]

Have tested it extensively with an okta dev account

Could you please add those setup steps to the PR description in case someone else needs to replicate this testing in the future.

[ShourieG]

@andrewkroh I have resolved all the PR suggestions and updated the PR description with steps to test this feature locally.