Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitwarden] Add support for the member data stream #8352

Merged
merged 3 commits into from Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 14 additions & 2 deletions packages/bitwarden/_dev/build/docs/README.md
Expand Up @@ -2,20 +2,22 @@

## Overview

The [Bitwarden](https://bitwarden.com) integration allows users to monitor collections, groups, events and policies. Bitwarden is a free and open-source password management service that stores sensitive information such as website credentials in an encrypted vault. The Bitwarden platform offers a variety of client applications including a web interface, desktop applications, browser extensions, mobile apps and a command-line interface. Bitwarden offers a cloud-hosted service as well as the ability to deploy the solution on-premises.
The [Bitwarden](https://bitwarden.com) integration allows users to monitor collections, events, groups, members and policies. Bitwarden is a free and open-source password management service that stores sensitive information such as website credentials in an encrypted vault. The Bitwarden platform offers a variety of client applications including a web interface, desktop applications, browser extensions, mobile apps and a command-line interface. Bitwarden offers a cloud-hosted service as well as the ability to deploy the solution on-premises.

Use the Bitwarden integration to collect and parse data from the REST APIs. Then visualize that data in Kibana.

## Data streams

The Bitwarden integration collects four types of data: collections, events, groups and policies.
The Bitwarden integration collects five types of data: Collections, Events, Groups, Members and Policies.

**Collections** returns a list of an organization's collections.

**Events** returns a list of an organization's event logs.

**Groups** returns a list of an organization's groups.

**Members** returns the details of an organization's members.

**Policies** returns a list of an organization's policies.

Reference for [Rest APIs](https://bitwarden.com/help/api/) of Bitwarden.
Expand Down Expand Up @@ -70,6 +72,16 @@ This is the `Group` dataset.

{{fields "group"}}

### Member

This is the `Member` dataset.

#### Example

{{event "member"}}

{{fields "member"}}

### Policy

This is the `Policy` dataset.
Expand Down
10 changes: 10 additions & 0 deletions packages/bitwarden/_dev/deploy/docker/files/config.yml
Expand Up @@ -48,3 +48,13 @@ rules:
body: >
{"object":"list","data":[{"enabled":true,"data":{"defaultType":"password","minLength":5,"useUpper":true,"useLower":true,"useNumbers":true,"useSpecial":true,"minNumbers":1,"minSpecial":1,"minNumberWords":3,"capitalize":true,"includeNumber":true},"object":"policy","id":"539a36c5-e0d2-4cf9-979e-51ecf5cf6593","type":0}]}

- path: /public/members
methods: ['GET']
request_headers:
Authorization:
- 'Bearer xxxx'
responses:
- status_code: 200
body: >
{"object":"list","data":[{"type":0,"accessAll":true,"externalId":"external_id_123456","resetPasswordEnrolled":true,"object":"member","id":"1234","userId":"48b47ee1-493e-4c67-aef7-014996c40eca","name":"John Smith","email":"jsmith@example.com","twoFactorEnabled":true,"status":0,"collections":null}]}

5 changes: 5 additions & 0 deletions packages/bitwarden/changelog.yml
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.8.0"
changes:
- description: Add support for the member data stream.
type: enhancement
link: https://github.com/elastic/integrations/pull/8352
- version: "1.7.0"
changes:
- description: Improve 'event.original' check to avoid errors if set.
Expand Down
2 changes: 1 addition & 1 deletion packages/bitwarden/data_stream/collection/manifest.yml
Expand Up @@ -9,7 +9,7 @@ streams:
- name: interval
type: text
title: Interval
description: "Duration between requests to the Bitwarden. NOTE: Supported units for this parameter are h/m/s."
description: Duration between requests to the Bitwarden. Supported units for this parameter are h/m/s.
default: 1h
multi: false
required: true
Expand Down
12 changes: 6 additions & 6 deletions packages/bitwarden/data_stream/collection/sample_event.json
@@ -1,8 +1,8 @@
{
"@timestamp": "2023-04-18T11:44:01.141Z",
"@timestamp": "2023-10-31T07:31:24.050Z",
"agent": {
"ephemeral_id": "0601b1ca-3a76-4d9a-9ed7-3da5b4333d2d",
"id": "ff2a1bfe-20b0-4bab-ad84-8609f33b69f8",
"ephemeral_id": "bf237146-2d4b-427b-b731-6dadb1dfdd90",
"id": "fa60f5ca-bf95-4706-9195-907dd5f9b537",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.4.1"
Expand All @@ -25,15 +25,15 @@
"version": "8.10.0"
},
"elastic_agent": {
"id": "ff2a1bfe-20b0-4bab-ad84-8609f33b69f8",
"id": "fa60f5ca-bf95-4706-9195-907dd5f9b537",
"snapshot": false,
"version": "8.4.1"
},
"event": {
"agent_id_status": "verified",
"created": "2023-04-18T11:44:01.141Z",
"created": "2023-10-31T07:31:24.050Z",
"dataset": "bitwarden.collection",
"ingested": "2023-04-18T11:44:04Z",
"ingested": "2023-10-31T07:31:27Z",
"kind": "event",
"original": "{\"externalId\":\"external_id_123456\",\"groups\":null,\"id\":\"539a36c5-e0d2-4cf9-979e-51ecf5cf6593\",\"object\":\"collection\"}",
"type": [
Expand Down
4 changes: 2 additions & 2 deletions packages/bitwarden/data_stream/event/manifest.yml
Expand Up @@ -9,15 +9,15 @@ streams:
- name: interval
type: text
title: Interval
description: "Duration between requests to the Bitwarden. NOTE: Supported units for this parameter are h/m/s."
description: Duration between requests to the Bitwarden. Supported units for this parameter are h/m/s.
default: 5m
multi: false
required: true
show_user: true
- name: initial_interval
type: text
title: Initial Interval
description: "How far back to pull the events from Bitwarden. NOTE: Supported units for this parameter are h/m/s."
description: How far back to pull the events from Bitwarden. Supported units for this parameter are h/m/s.
multi: false
required: true
show_user: true
Expand Down
10 changes: 5 additions & 5 deletions packages/bitwarden/data_stream/event/sample_event.json
@@ -1,8 +1,8 @@
{
"@timestamp": "2023-02-22T09:00:21.728Z",
"agent": {
"ephemeral_id": "03059a2a-a7ad-4677-a95d-00b24272a9af",
"id": "ff2a1bfe-20b0-4bab-ad84-8609f33b69f8",
"ephemeral_id": "23334f92-55ed-4a8f-b7c3-9e36ff9d73a2",
"id": "fa60f5ca-bf95-4706-9195-907dd5f9b537",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.4.1"
Expand Down Expand Up @@ -49,7 +49,7 @@
"version": "8.10.0"
},
"elastic_agent": {
"id": "ff2a1bfe-20b0-4bab-ad84-8609f33b69f8",
"id": "fa60f5ca-bf95-4706-9195-907dd5f9b537",
"snapshot": false,
"version": "8.4.1"
},
Expand All @@ -59,9 +59,9 @@
"iam",
"authentication"
],
"created": "2023-04-18T11:45:04.623Z",
"created": "2023-10-31T07:32:17.783Z",
"dataset": "bitwarden.event",
"ingested": "2023-04-18T11:45:08Z",
"ingested": "2023-10-31T07:32:21Z",
"kind": "event",
"original": "{\"actingUserId\":\"a2549f79-a71f-4eb9-9234-eb7247333f94\",\"collectionId\":\"bce212a4-25f3-4888-8a0a-4c5736d851e0\",\"date\":\"2023-02-22T09:00:21.728Z\",\"device\":0,\"groupId\":\"f29a2515-91d2-4452-b49b-5e8040e6b0f4\",\"ipAddress\":\"172.16.254.1\",\"itemId\":\"3767a302-8208-4dc6-b842-030428a1cfad\",\"memberId\":\"e68b8629-85eb-4929-92c0-b84464976ba4\",\"object\":\"event\",\"policyId\":\"f29a2515-91d2-4452-b49b-5e8040e6b0f4\",\"type\":1000}",
"outcome": "success",
Expand Down
2 changes: 1 addition & 1 deletion packages/bitwarden/data_stream/group/fields/fields.yml
Expand Up @@ -6,7 +6,7 @@
fields:
- name: access_all
type: boolean
description: Determines if this group can access all collections within the organization, or only the associated collections. If set to {true}, this option overrides any collection assignments.
description: Determines if this group can access all collections within the organization, or only the associated collections. If set to true, this option overrides any collection assignments.
- name: collection
type: group
fields:
Expand Down
2 changes: 1 addition & 1 deletion packages/bitwarden/data_stream/group/manifest.yml
Expand Up @@ -9,7 +9,7 @@ streams:
- name: interval
type: text
title: Interval
description: "Duration between requests to the Bitwarden. NOTE: Supported units for this parameter are h/m/s."
description: Duration between requests to the Bitwarden. Supported units for this parameter are h/m/s.
default: 1h
multi: false
required: true
Expand Down
12 changes: 6 additions & 6 deletions packages/bitwarden/data_stream/group/sample_event.json
@@ -1,8 +1,8 @@
{
"@timestamp": "2023-04-18T11:46:13.418Z",
"@timestamp": "2023-10-31T07:33:12.430Z",
"agent": {
"ephemeral_id": "88e47b12-e16a-4b3e-8170-e610d78e0566",
"id": "ff2a1bfe-20b0-4bab-ad84-8609f33b69f8",
"ephemeral_id": "2531708a-f7fa-48b6-913e-7d5d7d08b29b",
"id": "fa60f5ca-bf95-4706-9195-907dd5f9b537",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.4.1"
Expand Down Expand Up @@ -33,7 +33,7 @@
"version": "8.10.0"
},
"elastic_agent": {
"id": "ff2a1bfe-20b0-4bab-ad84-8609f33b69f8",
"id": "fa60f5ca-bf95-4706-9195-907dd5f9b537",
"snapshot": false,
"version": "8.4.1"
},
Expand All @@ -42,9 +42,9 @@
"category": [
"iam"
],
"created": "2023-04-18T11:46:13.418Z",
"created": "2023-10-31T07:33:12.430Z",
"dataset": "bitwarden.group",
"ingested": "2023-04-18T11:46:16Z",
"ingested": "2023-10-31T07:33:15Z",
"kind": "event",
"original": "{\"accessAll\":true,\"collections\":[{\"id\":\"bfbc8338-e329-4dc0-b0c9-317c2ebf1a09\",\"readOnly\":true}],\"externalId\":\"external_id_123456\",\"id\":\"539a36c5-e0d2-4cf9-979e-51ecf5cf6593\",\"name\":\"Development Team\",\"object\":\"group\"}",
"type": [
Expand Down
@@ -0,0 +1,4 @@
fields:
tags:
- preserve_original_event
- preserve_duplicate_custom_fields
@@ -0,0 +1,2 @@
{"type":0,"accessAll":true,"externalId":"external_id_123456","resetPasswordEnrolled":true,"object":"member","id":"539a36c5-e0d2-4cf9-979e-51ecf5cf6593","userId":"48b47ee1-493e-4c67-aef7-014996c40eca","name":"John Smith","email":"jsmith@example.com","twoFactorEnabled":true,"status":0,"collections":[{"id":"bfbc8338-e329-4dc0-b0c9-317c2ebf1a09","readOnly":true}]}
{"object":"list","data":[],"continuationToken":null}
@@ -0,0 +1,68 @@
{
"expected": [
{
"bitwarden": {
"member": {
"access_all": true,
"collection": [
{
"id": "bfbc8338-e329-4dc0-b0c9-317c2ebf1a09",
"read_only": true
}
],
"email": "jsmith@example.com",
"external": {
"id": "external_id_123456"
},
"id": "539a36c5-e0d2-4cf9-979e-51ecf5cf6593",
"name": "John Smith",
"reset_password_enrolled": true,
"status": {
"name": "Invited",
"value": "0"
},
"two_factor_enabled": true,
"type": {
"name": "Owner",
"value": "0"
},
"user": {
"id": "48b47ee1-493e-4c67-aef7-014996c40eca"
}
},
"object": "member"
},
"ecs": {
"version": "8.10.0"
},
"event": {
"category": [
"iam"
],
"kind": "event",
"original": "{\"type\":0,\"accessAll\":true,\"externalId\":\"external_id_123456\",\"resetPasswordEnrolled\":true,\"object\":\"member\",\"id\":\"539a36c5-e0d2-4cf9-979e-51ecf5cf6593\",\"userId\":\"48b47ee1-493e-4c67-aef7-014996c40eca\",\"name\":\"John Smith\",\"email\":\"jsmith@example.com\",\"twoFactorEnabled\":true,\"status\":0,\"collections\":[{\"id\":\"bfbc8338-e329-4dc0-b0c9-317c2ebf1a09\",\"readOnly\":true}]}",
"type": [
"user"
]
},
"related": {
"user": [
"539a36c5-e0d2-4cf9-979e-51ecf5cf6593",
"48b47ee1-493e-4c67-aef7-014996c40eca",
"John Smith",
"jsmith@example.com"
]
},
"tags": [
"preserve_original_event",
"preserve_duplicate_custom_fields"
],
"user": {
"email": "jsmith@example.com",
"id": "539a36c5-e0d2-4cf9-979e-51ecf5cf6593",
"name": "John Smith"
}
},
null
]
}
@@ -0,0 +1,13 @@
input: httpjson
service: bitwarden
vars:
url: http://{{Hostname}}:{{Port}}
client_id: xxxx
client_secret: xxxx
token_url: http://{{Hostname}}:{{Port}}/connect/token
data_stream:
vars:
preserve_original_event: true
preserve_duplicate_custom_fields: true
assert:
hit_count: 1
@@ -0,0 +1,50 @@
config_version: 2
interval: {{interval}}
{{#if enable_request_tracer}}
request.tracer.filename: "../../logs/httpjson/http-request-trace-*.ndjson"
{{/if}}
request.method: GET
request.url: {{url}}/public/members
{{#if proxy_url}}
request.proxy_url: {{proxy_url}}
{{/if}}
{{#if ssl}}
request.ssl: {{ssl}}
{{/if}}
{{#if http_client_timeout}}
request.timeout: {{http_client_timeout}}
{{/if}}
auth.oauth2:
client.id: {{client_id}}
client.secret: {{client_secret}}
token_url: {{token_url}}
scopes: api.organization
endpoint_params:
grant_type: client_credentials
request.rate_limit:
reset: '[[ add (toInt (.last_response.header.Get "Retry-After")) ((now).Unix) ]]'
remaining: '0' # hardcoded to 0 since bitwarden doesn't return remaining header
response.pagination:
- set:
target: url.params.continuationToken
value: '[[if index .last_response.body "continuationToken"]][[.last_response.body.continuationToken]][[end]]'
fail_on_template_error: true
response.split:
target: body.data
tags:
{{#if preserve_original_event}}
- preserve_original_event
{{/if}}
{{#if preserve_duplicate_custom_fields}}
- preserve_duplicate_custom_fields
{{/if}}
{{#each tags as |tag|}}
- {{tag}}
{{/each}}
{{#contains "forwarded" tags}}
publisher_pipeline.disable_host: true
{{/contains}}
{{#if processors}}
processors:
{{processors}}
{{/if}}