[1password] Add user fields for actor_details, object_details, and aux_details. #6851

Jin-Dhaliwal · 2023-07-06T19:24:55Z

What does this PR do?

We've added new details fields to the data model that we want to consumed through this integration. An example of the original message would be:

                 {
			"uuid": "2CA5O6WU4VOPE225X64BMDJ6M3",
			"timestamp": "2023-06-30T18:02:23.273264Z",
			"actor_uuid": "PPQYFXGORNDIRHFJ43QNOEKZ6A",
			"actor_details": {
				"uuid": "PPQYFXGORNDIRHFJ43QNOEKZ6A",
				"name": "Insights Test",
				"email": "jin.dhaliwal+insightstest@agilebits.com"
			},
			"action": "trvlaway",
			"object_type": "user",
			"object_uuid": "X3TLF2JNIVDKJAXU67SNSHCFBU",
			"object_details": {
				"uuid": "X3TLF2JNIVDKJAXU67SNSHCFBU",
				"name": "Member 7",
				"email": "jin.dhaliwal+m7@agilebits.com"
			},
			"session": {
				"uuid": "RD4MLUQFIZDW5G6MUFFBIYUUBM",
				"login_time": "2023-06-30T18:01:42.300478Z",
				"device_uuid": "gdbhmdzsecieklx5wmgr7qy6oe",
				"ip": "172.19.0.1"
			}
		}

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

Integration tests pass
Integration includes the new details fields in processed event data

How to test this PR locally

Follow the testing steps outlined here: https://developer.1password.com/docs/events-api/reference/#post-apiv1auditevents

Get the elastic stack up and running.
Follow the integration steps to set up the 1Password integration. (You will need a 1password business account)

You can create a test account on start.b5dev.com and use Stripe test card 4242424242424242
You'll need to change the URL in the integration setup to events.b5dev.com

On your 1Password account add some members/groups/vaults/items. This will generate events that the integration will capture.
Confirm that the audit events dashboard contains those events and the new details fields are present.

Screenshots

elasticmachine · 2023-07-06T19:29:14Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-08-03T05:02:59.463+0000
Duration: 16 min 44 sec

Test stats 🧪

Test	Results
Failed	0
Passed	22
Skipped	0
Total	22

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-07-07T11:57:49Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

efd6 · 2023-07-10T00:38:05Z

/test

elasticmachine · 2023-07-10T00:54:49Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`3/3`)	💚
Files	100.0% (`3/3`)	💚 2.842
Classes	100.0% (`3/3`)	💚 2.842
Methods	90.909% (`30/33`)	👎 -1.208
Lines	88.182% (`291/330`)	👎 -2.503
Conditionals	100.0% (`0/0`)	💚

kcreddy

Thanks for the contribution!

Could you also update ingest pipeline to add User ECS mapping? For example, you could add user.name field from actor_details.name field.

More examples on User ECS field that you could map: https://www.elastic.co/guide/en/ecs/current/ecs-user-usage.html

packages/1password/data_stream/audit_events/fields/fields.yml

Co-authored-by: Krishna Chaitanya Reddy Burri <krish.reddy91@gmail.com>

Jin-Dhaliwal · 2023-07-17T13:50:45Z

Anything left on this one? Would like to get this out asap because we've released the new version of the API and customers will want to start ingesting the new fields.

Jin-Dhaliwal · 2023-07-26T18:57:44Z

@kcreddy is there any other changes needed here or is this ready for merge now?

kcreddy · 2023-07-27T03:07:16Z

/test

kcreddy · 2023-07-27T05:39:23Z

/test

kcreddy · 2023-07-27T14:00:56Z

Hey @Jin-Dhaliwal
Sincere apologies for the delay 🙏🏼 .
Looks like the CI is failing due to README being not updated. Could you please run below command and commit ?

elastic-package build && elastic-package format && elastic-package lint && elastic-package check && elastic-package build

Jin-Dhaliwal · 2023-07-27T15:21:25Z

@kcreddy done!

kcreddy · 2023-07-27T16:48:54Z

/test

kcreddy · 2023-07-28T06:12:51Z

@Jin-Dhaliwal
Thanks for fixing the README. Now the CI is failing on pipeline tests. Looks like after the user ECS fields are added, the pipeline tests are not updated.

Can you run pipeline tests after bringing the stack up?
eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate and then commit.

Jin-Dhaliwal · 2023-07-28T14:04:28Z

@kcreddy thanks for the help. done

kcreddy · 2023-07-29T13:04:26Z

/test

kcreddy · 2023-07-31T06:51:25Z

/test

kcreddy · 2023-07-31T06:58:37Z

@Jin-Dhaliwal can you please fix the merge conflicts? The CI wouldn't run otherwise. Thanks!

Jin-Dhaliwal · 2023-08-02T15:51:53Z

/test

Jin-Dhaliwal · 2023-08-02T15:57:07Z

@kcreddy OK conflicts resolved

kcreddy · 2023-08-02T15:57:25Z

/test

Jin-Dhaliwal · 2023-08-02T19:04:36Z

How can I fix those failures in CI? I'm not sure what happened. Reading at face value seems like service that the test step needed failed to start?

Is there a way to restart those?

elastic-package test passes locally

kcreddy · 2023-08-03T05:02:47Z

/test

kcreddy

LGTM 👍🏼

kcreddy · 2023-08-03T11:19:13Z

How can I fix those failures in CI? I'm not sure what happened. Reading at face value seems like service that the test step needed failed to start?
Is there a way to restart those?

Hey, @Jin-Dhaliwal the issue was with the CI system, your changes are good.
Once again thanks so much for the contribution 😄

elasticmachine · 2023-08-03T11:41:05Z

Package 1password - 1.18.0 containing this change is available at https://epr.elastic.co/search?package=1password

Add user detail fields for actor, object, and aux_info.

7f1c2f6

Jin-Dhaliwal requested a review from a team as a code owner July 6, 2023 19:24

Add changelog entry.

93c6e16

Jin-Dhaliwal changed the title ~~Draft: Add user detail fields for actor, object, and aux_info.~~ Add user detail fields for actor, object, and aux_info. Jul 6, 2023

andrewkroh added Team:Security-External Integrations Integration:1password 1Password labels Jul 7, 2023

andrewkroh changed the title ~~Add user detail fields for actor, object, and aux_info.~~ [1password] Add user detail fields for actor, object, and aux_info. Jul 7, 2023

Jin-Dhaliwal added 2 commits July 7, 2023 15:21

Merge branch 'main' into 1password/add-user-detail-fields

1959a59

update manifest version.

f4d5218

kcreddy requested changes Jul 12, 2023

View reviewed changes

packages/1password/data_stream/audit_events/fields/fields.yml Outdated Show resolved Hide resolved

packages/1password/data_stream/audit_events/fields/fields.yml Outdated Show resolved Hide resolved

packages/1password/data_stream/audit_events/fields/fields.yml Outdated Show resolved Hide resolved

kcreddy changed the title ~~[1password] Add user detail fields for actor, object, and aux_info.~~ [1password] Add user fields for actor_details, object_details, and aux_details. Jul 12, 2023

kcreddy added the enhancement New feature or request label Jul 12, 2023

Jin-Dhaliwal and others added 4 commits July 12, 2023 11:27

Update packages/1password/data_stream/audit_events/fields/fields.yml

9c87921

Co-authored-by: Krishna Chaitanya Reddy Burri <krish.reddy91@gmail.com>

Update packages/1password/data_stream/audit_events/fields/fields.yml

4acd675

Co-authored-by: Krishna Chaitanya Reddy Burri <krish.reddy91@gmail.com>

Update packages/1password/data_stream/audit_events/fields/fields.yml

2034339

Co-authored-by: Krishna Chaitanya Reddy Burri <krish.reddy91@gmail.com>

Add ecs user mappings for user name/email.

7123b22

Update README.

c1bcedb

Update pipeline tests.

60b65cc

Merge branch 'main' into 1password/add-user-detail-fields

e2dddfe

kcreddy approved these changes Aug 3, 2023

View reviewed changes

kcreddy merged commit f3200ea into elastic:main Aug 3, 2023
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1password] Add user fields for actor_details, object_details, and aux_details. #6851

[1password] Add user fields for actor_details, object_details, and aux_details. #6851

Jin-Dhaliwal commented Jul 6, 2023 •

edited

Loading

elasticmachine commented Jul 6, 2023 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Jul 7, 2023

efd6 commented Jul 10, 2023

elasticmachine commented Jul 10, 2023 •

edited

Loading

kcreddy left a comment

Jin-Dhaliwal commented Jul 17, 2023

Jin-Dhaliwal commented Jul 26, 2023

kcreddy commented Jul 27, 2023

kcreddy commented Jul 27, 2023

kcreddy commented Jul 27, 2023 •

edited

Loading

Jin-Dhaliwal commented Jul 27, 2023

kcreddy commented Jul 27, 2023

kcreddy commented Jul 28, 2023 •

edited

Loading

Jin-Dhaliwal commented Jul 28, 2023

kcreddy commented Jul 29, 2023

kcreddy commented Jul 31, 2023

kcreddy commented Jul 31, 2023

Jin-Dhaliwal commented Aug 2, 2023

Jin-Dhaliwal commented Aug 2, 2023

kcreddy commented Aug 2, 2023

Jin-Dhaliwal commented Aug 2, 2023 •

edited

Loading

kcreddy commented Aug 3, 2023

kcreddy left a comment

kcreddy commented Aug 3, 2023

elasticmachine commented Aug 3, 2023

[1password] Add user fields for actor_details, object_details, and aux_details. #6851

[1password] Add user fields for actor_details, object_details, and aux_details. #6851

Conversation

Jin-Dhaliwal commented Jul 6, 2023 • edited Loading

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Screenshots

elasticmachine commented Jul 6, 2023 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

elasticmachine commented Jul 7, 2023

efd6 commented Jul 10, 2023

elasticmachine commented Jul 10, 2023 • edited Loading

🌐 Coverage report

kcreddy left a comment

Choose a reason for hiding this comment

Jin-Dhaliwal commented Jul 17, 2023

Jin-Dhaliwal commented Jul 26, 2023

kcreddy commented Jul 27, 2023

kcreddy commented Jul 27, 2023

kcreddy commented Jul 27, 2023 • edited Loading

Jin-Dhaliwal commented Jul 27, 2023

kcreddy commented Jul 27, 2023

kcreddy commented Jul 28, 2023 • edited Loading

Jin-Dhaliwal commented Jul 28, 2023

kcreddy commented Jul 29, 2023

kcreddy commented Jul 31, 2023

kcreddy commented Jul 31, 2023

Jin-Dhaliwal commented Aug 2, 2023

Jin-Dhaliwal commented Aug 2, 2023

kcreddy commented Aug 2, 2023

Jin-Dhaliwal commented Aug 2, 2023 • edited Loading

kcreddy commented Aug 3, 2023

kcreddy left a comment

Choose a reason for hiding this comment

kcreddy commented Aug 3, 2023

elasticmachine commented Aug 3, 2023

Jin-Dhaliwal commented Jul 6, 2023 •

edited

Loading

elasticmachine commented Jul 6, 2023 •

edited

Loading

elasticmachine commented Jul 10, 2023 •

edited

Loading

kcreddy commented Jul 27, 2023 •

edited

Loading

kcreddy commented Jul 28, 2023 •

edited

Loading

Jin-Dhaliwal commented Aug 2, 2023 •

edited

Loading