diff --git a/packages/axonius/_dev/build/docs/README.md b/packages/axonius/_dev/build/docs/README.md index e05eb9cad76..f69c328e86a 100644 --- a/packages/axonius/_dev/build/docs/README.md +++ b/packages/axonius/_dev/build/docs/README.md @@ -50,6 +50,26 @@ This integration collects log messages of the following type: - nat_rules (endpoint: `/api/v2/nat_rules`) - network_routes (endpoint: `/api/v2/network_routes`) +- `Identity`: Collect details of all identity assets including: + - users (endpoint: `/api/v2/users`) + - groups (endpoint: `/api/v2/groups`) + - security_roles (endpoint: `/api/v2/security_roles`) + - organizational_units (endpoint: `/api/v2/organizational_units`) + - accounts (endpoint: `/api/v2/accounts`) + - certificates (endpoint: `/api/v2/certificates`) + - permissions (endpoint: `/api/v2/permissions`) + - latest_rules (endpoint: `/api/v2/latest_rules`) + - profiles (endpoint: `/api/v2/profiles`) + - job_titles (endpoint: `/api/v2/job_titles`) + - access_review_campaign_instances (endpoint: `/api/v2/access_review_campaign_instances`) + - access_review_approval_items (endpoint: `/api/v2/access_review_approval_items`) + +### Supported use cases + +Integrating the Axonius Identity Datastream with Elastic SIEM provides a unified view of users, groups, roles, organizational units, accounts, permissions, certificates, profiles, and access review activity. Metrics and breakdowns help teams quickly assess identity posture by highlighting active, inactive, suspended, and external users, as well as patterns across user types and departments. + +Tables showing top email addresses and cloud providers add context into frequently used identities and their sources. These insights help security and IAM teams detect identity anomalies, validate account hygiene, and maintain strong visibility into access across the organization. + ### Supported use cases Integrating the Axonius Adapter, User, Gateway, Exposure, Alert, Incident, Storage, Ticket, and Network data streams with Elastic SIEM provides centralized, end-to-end visibility across data ingestion, identity posture, network configuration, vulnerability exposure, security events, storage assets, ticketing, and network activity. Together, these data streams help analysts understand how data flows into the platform, how it maps to users and access, how gateways and network assets operate, where risks and exposures exist, and how alerts evolve into incidents and tracked issues. @@ -136,12 +156,14 @@ Destinations indices are aliased to `logs-axonius_latest.`. | `logs-axonius.gateway-*` | `logs-axonius_latest.dest_gateway-*` | `logs-axonius_latest.gateway` | | `logs-axonius.incident-*` | `logs-axonius_latest.dest_incident-*` | `logs-axonius_latest.incident` | | `logs-axonius.user-*` | `logs-axonius_latest.dest_user-*` | `logs-axonius_latest.user` | -| `logs-axonius.storage-*` | `logs-axonius_latest.dest_storage-*` | `logs-axonius_latest.storage` | -| `logs-axonius.ticket-*` | `logs-axonius_latest.dest_ticket-*` | `logs-axonius_latest.ticket` -| `logs-axonius.network-*` | `logs-axonius_latest.dest_network-*` | `logs-axonius_latest.network` +| `logs-axonius.storage-*` | `logs-axonius_latest.dest_storage-*` | `logs-axonius_latest.storage` | +| `logs-axonius.ticket-*` | `logs-axonius_latest.dest_ticket-*` | `logs-axonius_latest.ticket` | +| `logs-axonius.network-*` | `logs-axonius_latest.dest_network-*` | `logs-axonius_latest.network` | +| `logs-axonius.identity-*` | `logs-axonius_latest.dest_identity-*` | `logs-axonius_latest.identity` | + **Note:** Assets deleted from Axonius may reappear in a future discovery cycle if they are still present in connected data sources and get re-detected. Because the exact duration for which a deleted asset may remain dormant before being rediscovered is unknown, the transform retention period is set to **90 days** to reduce the risk of data loss for such assets. This means deleted assets will continue to appear in dashboards for up to 90 days after deletion. -The network destination index is a content-based deduplicated view, not an entity-level latest-state view like the other data streams (for example `user` and `gateway`), which rely on a unique entity identifier and reflect the latest state of each entity. +The network and identity destination indices are a content-based deduplicated view, not an entity-level latest-state view like the other data streams (for example `user` and `gateway`), which rely on a unique entity identifier and reflect the latest state of each entity. ## Troubleshooting @@ -243,6 +265,16 @@ The `network` data stream provides network events from axonius. {{ event "network" }} +### Identity + +The `identity` data stream provides identity asset logs from axonius. + +#### identity fields + +{{ fields "identity" }} + +{{event "identity"}} + ### Inputs used {{/* All inputs used by this package will be automatically listed here. */}} {{ inputDocs }} @@ -277,7 +309,20 @@ These APIs are used with this integration: * firewalls (endpoint: `/api/v2/firewalls`) * nat_rules (endpoint: `/api/v2/nat_rules`) * network_routes (endpoint: `/api/v2/network_routes`) +* Identity: + * users (endpoint: `/api/v2/users`) + * groups (endpoint: `/api/v2/groups`) + * security_roles (endpoint: `/api/v2/security_roles`) + * organizational_units (endpoint: `/api/v2/organizational_units`) + * accounts (endpoint: `/api/v2/accounts`) + * certificates (endpoint: `/api/v2/certificates`) + * permissions (endpoint: `/api/v2/permissions`) + * latest_rules (endpoint: `/api/v2/latest_rules`) + * profiles (endpoint: `/api/v2/profiles`) + * job_titles (endpoint: `/api/v2/job_titles`) + * access_review_campaign_instances (endpoint: `/api/v2/access_review_campaign_instances`) + * access_review_approval_items (endpoint: `/api/v2/access_review_approval_items`) ### ILM Policy -To facilitate adapter, user, gateway and assets data including exposures, alert findings, incidents, storage and ticket source data stream-backed indices `.ds-logs-axonius.adapter-*`, `.ds-logs-axonius.user-*`, `.ds-logs-axonius.gateway-*`, `.ds-logs-axonius.exposure-*`, `.ds-logs-axonius.alert_finding-*`, `.ds-logs-axonius.incident-*`, `.ds-logs-axonius.storage-*` and `.ds-logs-axonius.ticket-*` respectively are allowed to contain duplicates from each polling interval. ILM policies `logs-axonius.adapter-default_policy`, `logs-axonius.user-default_policy`, `logs-axonius.gateway-default_policy`, `logs-axonius.exposure-default_policy`, `logs-axonius.alert_finding-default_policy`, `logs-axonius.incident-default_policy`, `logs-axonius.storage-default_policy` and `logs-axonius.ticket-default_policy` are added to these source indices, so it doesn't lead to unbounded growth. This means that in these source indices data will be deleted after `30 days` from ingested date. +To facilitate adapter, user, gateway and assets data including exposures, alert findings, incidents, storage and ticket, network and identity source data stream-backed indices `.ds-logs-axonius.adapter-*`, `.ds-logs-axonius.user-*`, `.ds-logs-axonius.gateway-*`, `.ds-logs-axonius.exposure-*`, `.ds-logs-axonius.alert_finding-*`, `.ds-logs-axonius.incident-*`, `.ds-logs-axonius.storage-*`, `.ds-logs-axonius.ticket-*`, `.ds-logs-axonius.network-*` and `.ds-logs-axonius.identity-*` respectively are allowed to contain duplicates from each polling interval. ILM policies `logs-axonius.adapter-default_policy`, `logs-axonius.user-default_policy`, `logs-axonius.gateway-default_policy`, `logs-axonius.exposure-default_policy`, `logs-axonius.alert_finding-default_policy`, `logs-axonius.incident-default_policy`, `logs-axonius.storage-default_policy`, `logs-axonius.ticket-default_policy`, `logs-axonius.network-default_policy` and `logs-axonius.identity-default_policy` are added to these source indices, so it doesn't lead to unbounded growth. This means that in these source indices data will be deleted after `30 days` from ingested date. diff --git a/packages/axonius/_dev/deploy/docker/files/config.yml b/packages/axonius/_dev/deploy/docker/files/config.yml index 7b536c14692..025509030dc 100644 --- a/packages/axonius/_dev/deploy/docker/files/config.yml +++ b/packages/axonius/_dev/deploy/docker/files/config.yml @@ -3133,7 +3133,7 @@ rules: "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", "is_data_from_cache": true, "page": { - "number": 2, + "number": 1, "size": 2, "totalPages": 2, "totalResources": 4 @@ -4107,6 +4107,1292 @@ rules: `}} - path: /api/v2/assets/network_routes + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 0, + "totalPages": 0, + "totalResources": 0 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + - path: /api/v2/assets/users + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"next_page":"xyz".*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [ + { + "internal_axon_id": "5e0d2725484dde8e602ea9f97aa67d71", + "adapters": [ + "aws_adapter" + ], + "asset_type": "users", + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:20 GMT", + "adapter_categories": [ + "ERP", + "SaaS Management", + "HR" + ], + "client_used": "67fd0998fe1c8e812a176b9f", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:20 GMT", + "application_and_account_name": "workday/workday-demo", + "display_name": "William Mcallister", + "employee_id": "880290", + "employee_type": "Worker", + "fetch_time": "Tue, 09 Dec 2025 12:02:19 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:27:14 GMT", + "first_name": "William", + "from_last_fetch": true, + "hr_employment_status": "Employed", + "id": "b5089852-7138-47ee-aac3-38359caf8754", + "is_active": true, + "is_fetched_from_adapter": true, + "is_managed_by_application": true, + "is_permission_adapter": true, + "is_saas_user": true, + "is_user_active": true, + "is_user_inactive": false, + "last_fetch_connection_id": "67fd0998fe1c8e812a176b9f", + "last_fetch_connection_label": "workday-demo", + "last_name": "Mcallister", + "mail": "william.mcallister@demo.local", + "nested_applications": [ + { + "active_from_direct_adapter": true, + "app_accounts": [ + { + "name": "workday-demo" + } + ], + "app_display_name": "workday-demo", + "assignment_type": "Direct", + "is_from_direct_adapter": true, + "is_managed": true, + "name": "Workday", + "parents": [ + { + "name": "", + "value": "" + } + ], + "relation_direct_name": "Workday", + "source_application": "Workday", + "value": "Workday_67fd0998fe1c8e812a176b9f", + "vendor_category": "HR" + } + ], + "nested_associated_devices": [], + "nested_grants_last_updated": "Tue, 09 Dec 2025 12:10:06 GMT", + "nested_grants_managers_last_updated": "Tue, 09 Dec 2025 12:10:10 GMT", + "nested_groups": [], + "nested_managers": [ + { + "assignment_type": "Indirect", + "parents": [ + { + "name": "Irene James", + "parent_type": "User", + "value": "irene.james@demo.local" + } + ], + "value": "" + } + ], + "nested_permissions": [], + "nested_resources": [], + "nested_roles": [], + "not_fetched_count": 0, + "pretty_id": "AX-2400172294", + "sm_entity_type": "saas_user", + "source_application": "Workday", + "tenant_number": [ + "4" + ], + "user_apps": [ + { + "active_from_direct_adapter": true, + "app_accounts": [ + { + "name": "workday-demo" + } + ], + "app_display_name": "workday-demo", + "app_id": "Workday_67fd0998fe1c8e812a176b9f", + "app_name": "Workday", + "is_from_direct_adapter": true, + "is_managed": true, + "is_saas_application": true, + "relation_direct_name": "Workday", + "source_application": "Workday", + "vendor_category": "HR" + } + ], + "user_department": "R&D", + "user_full_name": "William Mcallister", + "user_manager": "yvonne.gordon@demo.local", + "user_manager_mail": "yvonne.gordon@demo.local", + "user_remote_id": "c36808f9-305b-4e92-acfb-dfabfc2f0cb3", + "user_title": "R&D Engineer", + "username": "william.mcallister@demo.local" + }, + "initial_plugin_unique_name": "workday_adapter_0", + "plugin_name": "workday_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "workday_adapter_0", + "quick_id": "workday_adapter_0!b5089852-7138-47ee-aac3-38359caf8754", + "type": "entitydata" + } + ] + } + ], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 2, + "size": 2, + "totalPages": 2, + "totalResources": 3 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/users + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [ + { + "internal_axon_id": "bc11b2989fc0f69708b6865d172a49fe", + "adapters": [ + "aws_adapter", + "zoom_adapter" + ], + "adapter_list_length": 12, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:24 GMT", + "adapter_categories": [ + "Cloud Infra" + ], + "client_used": "67fd09ab731ccb57309230fc", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:24 GMT", + "application_and_account_name": "aws/aws-demo", + "aws_iam_identity_type": "IAM User", + "fetch_time": "Tue, 09 Dec 2025 12:02:22 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:27:03 GMT", + "from_last_fetch": true, + "id": "06C9040CA7D1D8B6F01F6", + "internal_is_admin": false, + "is_admin": false, + "is_fetched_from_adapter": true, + "is_managed_by_application": true, + "is_permission_adapter": true, + "is_saas_user": true, + "last_fetch_connection_id": "67fd09ab731ccb57309230fc", + "last_fetch_connection_label": "aws-demo", + "nested_associated_devices": [], + "nested_grants_last_updated": "Tue, 09 Dec 2025 12:10:06 GMT", + "nested_grants_managers_last_updated": "Tue, 09 Dec 2025 12:10:10 GMT", + "nested_groups": [], + "nested_managers": [], + "nested_permissions": [], + "nested_resources": [], + "nested_roles": [], + "not_fetched_count": 0, + "sm_entity_type": "saas_user", + "source_application": "AWS", + "tenant_number": [ + "2" + ], + "user_apps": [ + { + "active_from_direct_adapter": true, + "app_accounts": [ + { + "name": "aws-demo" + } + ], + "app_display_name": "aws-demo", + "app_id": "AWS_67fd09ab731ccb57309230fc", + "app_name": "AWS", + "is_from_direct_adapter": true, + "is_managed": true, + "is_saas_application": true, + "relation_direct_name": "AWS", + "source_application": "AWS", + "vendor_category": "Other" + } + ], + "user_is_password_enabled": false, + "user_pass_last_used": "Sat, 12 Apr 2025 21:58:16 GMT", + "user_path": "/", + "user_permissions": [], + "user_remote_id": "63d52bb0-7ce0-4467-9004-2b19c06b86ae", + "user_sid": "helen.jordan@demo.local@demo.local", + "username": "helen.jordan@demo.local" + }, + "initial_plugin_unique_name": "aws_adapter_0", + "plugin_name": "aws_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "aws_adapter_0", + "quick_id": "aws_adapter_0!06C9040CA7D1D8B6F01F6", + "type": "entitydata" + }, + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:11 GMT", + "adapter_categories": [ + "Directory", + "IAM", + "SaaS Management" + ], + "client_used": "67fd09bbfe1c8e812a176bb5", + "data": { + "account_disabled": true, + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:11 GMT", + "application_and_account_name": "microsoft/azure_ad-demo", + "associated_groups": [ + { + "display_name": "developers-group", + "remote_id": "a3e70162" + } + ], + "azure_account_id": "c8103abe-eda9-472b-894a-6260bb2ba8cc", + "cloud_provider": "Azure", + "email_activity": { + "is_deleted": false, + "product_license": "MICROSOFT FABRIC (FREE)+MICROSOFT TEAMS PHONE STANDARD+MICROSOFT DEFENDER FOR OFFICE365 (PLAN 2)+MICROSOFT 365 AUDIO CONFERENCING+ENTERPRISE MOBILITY + SECURITY E3+OFFICE365 E3+MICROSOFT 365 E3 EXTRA FEATURES", + "read_count": 2321, + "receive_count": 6965, + "report_date": "Fri, 10 Jan 2025 20:34:43 GMT", + "report_period": 90, + "send_count": 3030 + }, + "fetch_time": "Tue, 09 Dec 2025 12:02:03 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:27:00 GMT", + "from_last_fetch": true, + "has_administrative_permissions": true, + "id": "c8103abe-eda9-472b-894a-6260bb2ba8cc", + "internal_is_admin": false, + "is_admin": false, + "is_fetched_from_adapter": true, + "is_latest_last_seen": true, + "is_managed_by_application": true, + "is_permission_adapter": true, + "is_saas_user": true, + "is_user_external": false, + "last_fetch_connection_id": "67fd09bbfe1c8e812a176bb5", + "last_fetch_connection_label": "azure_ad-demo", + "last_logon": "Sun, 30 Nov 2025 18:50:39 GMT", + "last_seen": "Mon, 10 Nov 2025 22:18:25 GMT", + "mail": "helen.jordan@demo.local", + "nested_applications": [ + { + "app_display_name": "Calendly", + "assignment_type": "Direct", + "extension_type": "User Consent", + "is_managed": false, + "is_unmanaged_extension": true, + "name": "Calendly", + "parents": [ + { + "name": "", + "value": "" + } + ], + "permissions": [ + { + "name": "openid" + } + ], + "relation_extension_name": "Calendly", + "source_application": "Microsoft", + "value": "2E2a2e7c9f758BDcC0E2", + "vendor_category": "Productivity" + } + ], + "nested_associated_devices": [], + "nested_grants_last_updated": "Tue, 09 Dec 2025 12:10:06 GMT", + "nested_grants_managers_last_updated": "Tue, 09 Dec 2025 12:10:10 GMT", + "nested_groups": [ + { + "assignment_type": "Direct", + "name": "Office365 Users", + "parents": [ + { + "name": "", + "value": "" + } + ], + "value": "d8e66837" + } + ], + "nested_managers": [], + "nested_resources": [], + "nested_roles": [], + "not_fetched_count": 0, + "sm_entity_type": "saas_user", + "source_application": "Microsoft", + "tenant_number": [ + "2" + ], + "user_apps": [], + "user_created": "Fri, 28 Jun 2024 08:49:28 GMT", + "user_permissions": [ + { + "is_admin": false, + "name": "OnlineMeetings.ReadWrite" + } + ], + "user_remote_id": "63d52bb0-7ce0-4467-9004-2b19c06b86ae", + "user_type": "Member", + "username": "helen.jordan@demo.local" + }, + "initial_plugin_unique_name": "azure_ad_adapter_0", + "plugin_name": "azure_ad_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "azure_ad_adapter_0", + "quick_id": "azure_ad_adapter_0!c8103abe-eda9-472b-894a-6260bb2ba8cc", + "type": "entitydata" + } + ] + }, + { + "internal_axon_id": "ddca721dfbf352b3fca31546ec23cb0c", + "adapters": [ + "workday_adapter" + ], + "adapter_list_length": 12, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:11 GMT", + "adapter_categories": [ + "Directory", + "IAM", + "SaaS Management" + ], + "client_used": "67fd09bbfe1c8e812a176bb5", + "data": { + "account_disabled": false, + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:11 GMT", + "admin_roles": [ + { + "display_name": "Application Administrator", + "id": "ce4625e3-77dd-4ca5-955e-fab8bcb69d18" + } + ], + "application_and_account_name": "microsoft/azure_ad-demo", + "associated_groups": [ + { + "display_name": "Network", + "remote_id": "570d01c7" + } + ], + "azure_account_id": "87dee520-a55d-43f5-aaa9-775691047e43", + "cloud_provider": "Azure", + "fetch_time": "Tue, 09 Dec 2025 12:02:01 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:26:57 GMT", + "from_last_fetch": true, + "has_administrative_permissions": true, + "id": "87dee520-a55d-43f5-aaa9-775691047e43", + "internal_is_admin": false, + "is_admin": false, + "is_fetched_from_adapter": true, + "is_latest_last_seen": true, + "is_managed_by_application": true, + "is_permission_adapter": true, + "is_saas_user": true, + "is_user_external": false, + "last_fetch_connection_id": "67fd09bbfe1c8e812a176bb5", + "last_fetch_connection_label": "azure_ad-demo", + "last_logon": "Sun, 07 Dec 2025 03:33:50 GMT", + "last_seen": "Wed, 26 Nov 2025 06:54:58 GMT", + "mail": "chris.swarey@demo.local", + "nested_applications": [], + "nested_associated_devices": [], + "nested_grants_last_updated": "Tue, 09 Dec 2025 12:10:03 GMT", + "nested_grants_managers_last_updated": "Tue, 09 Dec 2025 12:10:09 GMT", + "nested_groups": [], + "nested_managers": [], + "nested_permissions": [], + "nested_resources": [], + "nested_roles": [], + "not_fetched_count": 0, + "sm_entity_type": "saas_user", + "source_application": "Microsoft", + "tenant_number": [ + "1" + ], + "user_apps": [], + "user_created": "Sun, 27 Oct 2024 11:19:15 GMT", + "user_permissions": [ + { + "is_admin": true, + "name": "openid" + } + ], + "user_remote_id": "27722fde-2498-4302-b125-71b5166694b1", + "user_type": "Member", + "username": "chris.swarey@demo.local" + }, + "initial_plugin_unique_name": "azure_ad_adapter_0", + "plugin_name": "azure_ad_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "azure_ad_adapter_0", + "quick_id": "azure_ad_adapter_0!87dee520-a55d-43f5-aaa9-775691047e43", + "type": "entitydata" + } + ] + } + ], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 2, + "totalPages": 2, + "totalResources": 3 + }, + "next_page": "xyz", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/groups + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 0, + "totalPages": 0, + "totalResources": 0 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/security_roles + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [ + { + "internal_axon_id": "bbe87f5753a6cf1500ed8c8249d3a3dd", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MAINTAINER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "1af78d57c74550c0a7bb", + "id_raw": "4e8a1f5a1b3143e68b9c26d894f9a41f", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MAINTAINER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-3152576343", + "remote_id": "3e82J945I67S92T50g3471", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "4" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!1af78d57c74550c0a7bb", + "type": "entitydata" + } + ] + }, + { + "internal_axon_id": "4e54b5a92713c1b42ea35b1bb71af0f7", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MEMBER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "92674aa848660cff18c2", + "id_raw": "1a93c0f49df342bd8c6e3a6d9af3e014", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MEMBER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-1314174377", + "remote_id": "8U24X75fN307L845Y63219", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "3" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!92674aa848660cff18c2", + "type": "entitydata" + } + ] + } + ], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 2, + "totalPages": 1, + "totalResources": 2 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/organizational_units + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 0, + "totalPages": 0, + "totalResources": 0 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/accounts + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 0, + "totalPages": 0, + "totalResources": 0 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/certificates + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [ + { + "internal_axon_id": "bbe87f5753a6cf1500ed8c8249d3a3dd", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MAINTAINER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "1af78d57c74550c0a7bb", + "id_raw": "4e8a1f5a1b3143e68b9c26d894f9a41f", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MAINTAINER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-3152576343", + "remote_id": "3e82J945I67S92T50g3471", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "4" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!1af78d57c74550c0a7bb", + "type": "entitydata" + } + ] + }, + { + "internal_axon_id": "4e54b5a92713c1b42ea35b1bb71af0f7", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MEMBER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "92674aa848660cff18c2", + "id_raw": "1a93c0f49df342bd8c6e3a6d9af3e014", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MEMBER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-1314174377", + "remote_id": "8U24X75fN307L845Y63219", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "3" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!92674aa848660cff18c2", + "type": "entitydata" + } + ] + } + ], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 2, + "totalPages": 1, + "totalResources": 2 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/permissions + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [ + { + "internal_axon_id": "bbe87f5753a6cf1500ed8c8249d3a3dd", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MAINTAINER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "1af78d57c74550c0a7bb", + "id_raw": "4e8a1f5a1b3143e68b9c26d894f9a41f", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MAINTAINER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-3152576343", + "remote_id": "3e82J945I67S92T50g3471", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "4" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!1af78d57c74550c0a7bb", + "type": "entitydata" + } + ] + } + ], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 1, + "totalPages": 1, + "totalResources": 1 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/latest_rules + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 0, + "totalPages": 0, + "totalResources": 0 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/profiles + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [ + { + "internal_axon_id": "bbe87f5753a6cf1500ed8c8249d3a3dd", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MAINTAINER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "1af78d57c74550c0a7bb", + "id_raw": "4e8a1f5a1b3143e68b9c26d894f9a41f", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MAINTAINER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-3152576343", + "remote_id": "3e82J945I67S92T50g3471", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "4" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!1af78d57c74550c0a7bb", + "type": "entitydata" + } + ] + }, + { + "internal_axon_id": "4e54b5a92713c1b42ea35b1bb71af0f7", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MEMBER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "92674aa848660cff18c2", + "id_raw": "1a93c0f49df342bd8c6e3a6d9af3e014", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MEMBER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-1314174377", + "remote_id": "8U24X75fN307L845Y63219", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "3" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!92674aa848660cff18c2", + "type": "entitydata" + } + ] + } + ], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 2, + "totalPages": 1, + "totalResources": 2 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/job_titles + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 0, + "totalPages": 0, + "totalResources": 0 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/access_review_campaign_instances + methods: ['POST'] + request_headers: + Content-Type: application/json + api-key: xxxx + api-secret: xxxx + request_body: /.*"page":{"limit":2}.*/ + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: |- + {{ minify_json ` + { + "assets": [ + { + "internal_axon_id": "bbe87f5753a6cf1500ed8c8249d3a3dd", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MAINTAINER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "1af78d57c74550c0a7bb", + "id_raw": "4e8a1f5a1b3143e68b9c26d894f9a41f", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MAINTAINER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-3152576343", + "remote_id": "3e82J945I67S92T50g3471", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "4" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!1af78d57c74550c0a7bb", + "type": "entitydata" + } + ] + }, + { + "internal_axon_id": "4e54b5a92713c1b42ea35b1bb71af0f7", + "adapters": [ + "github_adapter" + ], + "adapter_list_length": 1, + "specific_data": [ + { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176ba9", + "data": { + "accurate_for_datetime": "Tue, 09 Dec 2025 12:02:32 GMT", + "application_and_account_name": "github/github-prod", + "display_name": "REPOSITORY_MEMBER", + "fetch_time": "Tue, 09 Dec 2025 12:02:31 GMT", + "first_fetch_time": "Mon, 14 Apr 2025 13:25:24 GMT", + "from_last_fetch": true, + "id": "92674aa848660cff18c2", + "id_raw": "1a93c0f49df342bd8c6e3a6d9af3e014", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09a9fe1c8e812a176ba9", + "last_fetch_connection_label": "github-prod", + "name": "REPOSITORY_MEMBER", + "not_fetched_count": 0, + "permissions": [], + "pretty_id": "AX-1314174377", + "remote_id": "8U24X75fN307L845Y63219", + "sm_entity_type": "security_role", + "software_cves": [], + "source_application": "GitHub", + "tenant_number": [ + "3" + ], + "type": "SecurityRoles" + }, + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!92674aa848660cff18c2", + "type": "entitydata" + } + ] + } + ], + "meta": { + "cache_last_updated": "Mon, 01 Dec 2025 13:37:38 GMT", + "is_data_from_cache": true, + "page": { + "number": 1, + "size": 2, + "totalPages": 1, + "totalResources": 2 + }, + "next_page": "abcd", + "expand_row": false, + "optimized_view": false, + "relation_fields_data": false + } + } + `}} + + - path: /api/v2/assets/access_review_approval_items methods: ['POST'] request_headers: Content-Type: application/json diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-account.log b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-account.log new file mode 100644 index 00000000000..815fe2d68f5 --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-account.log @@ -0,0 +1,3 @@ +{"asset_type":"accounts","event":{"data":{"active_users":1500,"active_users_saved_query_id":"665db21284bdd7fee6703009","admin_non_operational_users":0,"admin_non_operational_users_saved_query_id":"665db21a84bdd7fee67032aa","admin_operational_active_users":0,"admin_operational_active_users_saved_query_id":"665db21584bdd7fee6703148","admin_operational_inactive_users":0,"admin_operational_inactive_users_saved_query_id":"665db22a84bdd7fee6703692","admin_operational_users":0,"admin_operational_users_saved_query_id":"665db21684bdd7fee6703174","admins":0,"admins_saved_query_id":"665db22584bdd7fee67035a4","application_id":"198ef9c063EC253aEcEe","application_name":"VMware","asset_entity_info":"ARP","connected_assets":["storage_account_id::ac4d42d9-5b05-4f67-ba8d-c770fcaa17d6"],"connection_label":"okta-demo","created_date":"Sun, 23 Feb 2025 00:25:17 GMT","deleted_users":249,"deleted_users_saved_query_id":"665db22684bdd7fee67035bb","direct_not_sso_users":34,"direct_not_sso_users_saved_query_id":"665db21384bdd7fee67030ab","domains":[{"name":"salesforce.demo.local"}],"email":"example@domain.com","external_users":0,"external_users_saved_query_id":"665db21084bdd7fee6702f98","gce_account_id":"gce_account_id_1","inactive_users":96,"inactive_users_saved_query_id":"665db20b84bdd7fee6702d68","is_managed_by_direct_app":true,"last_enrichment_run":"Wed, 12 Nov 2025 00:02:18 GMT","managed_non_operational_users":371,"managed_non_operational_users_saved_query_id":"665db20e84bdd7fee6702e8b","managed_operational_users":1596,"managed_operational_users_saved_query_id":"665db20e84bdd7fee6702e83","managed_users":1602,"managed_users_by_app":1602,"managed_users_by_app_saved_query_id":"665db22784bdd7fee6703613","managed_users_by_sso":1568,"managed_users_by_sso_saved_query_id":"665db20e84bdd7fee6702ea0","managed_users_saved_query_id":"665db22284bdd7fee67034e4","orphaned_users":28,"orphaned_users_saved_query_id":"665db21784bdd7fee67031fb","paid_users":247,"paid_users_saved_query_id":"665db21984bdd7fee6703296","project_ids":["id1","id2"],"project_tags":[{"inherited":"dunno","key":"key1","namespaced_tag_key":"tkey","namespaced_tag_value":"tvalue","value":"value1"}],"projects_roles":[{"project_id":"id123","role_name":"role12"}],"relatable_ids":["image_id::87dec736-1ce1-4476-8a68-c37dcdffe9c2"],"remote_account_id":"racid","status":"closed","suspended_users":241,"suspended_users_saved_query_id":"665db20d84bdd7fee6702e2b","unlinked_users":0,"unlinked_users_saved_query_id":"665db22384bdd7fee67034fc"}}} +{"adapter_list_length":1,"adapters":["qualys_scans_adapter"],"asset_type":"accounts","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:04:58 GMT","adapter_categories":["VA Tool"],"client_used":"67fd09dffe1c8e812a176bcb","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:04:58 GMT","application_and_account_name":"qualys/qualys_scans-demo","application_name":"Qualys","asset_type":"Connected Adapter","domains":[{}],"fetch_time":"Tue, 09 Dec 2025 00:04:58 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:29:14 GMT","from_last_fetch":true,"id":"38561584016c3bd50997","id_raw":"qualys_scans-demo_67fd09dffe1c8e812a176bcb","is_fetched_from_adapter":true,"is_managed_by_direct_app":true,"name":"Qualys/qualys_scans-demo","not_fetched_count":0,"roles":["roles/clou."], "source_application":"Qualys","type":"Accounts"},"initial_plugin_unique_name":"qualys_scans_adapter_0","plugin_name":"qualys_scans_adapter","plugin_type":"Adapter","plugin_unique_name":"qualys_scans_adapter_0","quick_id":"qualys_scans_adapter_0!38561584016c3bd50997","type":"entitydata"},"internal_axon_id":"e2781b0b477c77303261aec45a40cb44"} +{"adapter_list_length":2,"adapters":["github_adapter","okta_adapter"],"asset_type":"accounts","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:02:32 GMT","adapter_categories":["Software Development Version Control","SaaS Management"],"client_used":"67fd09a9fe1c8e812a176baa","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:02:32 GMT","active_users":0,"active_users_saved_query_id":"67fd1261709940e80b240c24","admin_non_operational_users":0,"admin_non_operational_users_saved_query_id":"67fd1261709940e80b240c68","admin_operational_active_users":0,"admin_operational_active_users_saved_query_id":"67fd1261709940e80b240c7e","admin_operational_inactive_users":0,"admin_operational_inactive_users_saved_query_id":"67fd1261709940e80b240c3f","admin_operational_users":0,"admin_operational_users_saved_query_id":"67fd1261709940e80b240c5d","admins":0,"admins_saved_query_id":"67fd1261709940e80b240c38","application_and_account_name":"github/github-dev","application_name":"GitHub","asset_type":"Connected Adapter","deleted_users":0,"deleted_users_saved_query_id":"67fd1261709940e80b240cb7","direct_not_sso_users":27,"direct_not_sso_users_saved_query_id":"67fd1261709940e80b240c86","domains":[{"name":"github.demo.local"}],"external_users":0,"external_users_saved_query_id":"67fd1261709940e80b240cc4","fetch_time":"Tue, 09 Dec 2025 00:02:32 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:27:53 GMT","from_last_fetch":true,"id":"697947761f2f779c8df5","id_raw":"github-dev_67fd09a9fe1c8e812a176baa","inactive_users":339,"inactive_users_saved_query_id":"67fd1261709940e80b240c4a","is_fetched_from_adapter":true,"is_managed_by_direct_app":true,"last_enrichment_run":"Tue, 09 Dec 2025 00:10:07 GMT","managed_non_operational_users":0,"managed_non_operational_users_saved_query_id":"67fd1261709940e80b240c56","managed_operational_users":339,"managed_operational_users_saved_query_id":"67fd1261709940e80b240c71","managed_users":339,"managed_users_by_app":180,"managed_users_by_app_saved_query_id":"67fd1261709940e80b240c9a","managed_users_by_sso":312,"managed_users_by_sso_saved_query_id":"67fd1261709940e80b240ca6","managed_users_saved_query_id":"67fd1261709940e80b240c17","name":"GitHub/github-dev","not_fetched_count":0,"orphaned_users":27,"orphaned_users_saved_query_id":"67fd1261709940e80b240c8f","paid_users":0,"paid_users_saved_query_id":"67fd1261709940e80b240ccc","source_application":"GitHub","suspended_users":0,"suspended_users_saved_query_id":"67fd1261709940e80b240cae","type":"Accounts","unlinked_users":159,"unlinked_users_saved_query_id":"67fd1261709940e80b240c2b"},"initial_plugin_unique_name":"github_adapter_0","plugin_name":"github_adapter","plugin_type":"Adapter","plugin_unique_name":"github_adapter_0","quick_id":"github_adapter_0!697947761f2f779c8df5","type":"entitydata"},"internal_axon_id":"5592c1d0fc4c0fdeebb6c98530a7c777"} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-account.log-expected.json b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-account.log-expected.json new file mode 100644 index 00000000000..6ddcf050ef1 --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-account.log-expected.json @@ -0,0 +1,292 @@ +{ + "expected": [ + { + "axonius": { + "identity": { + "active_users": 1500, + "active_users_saved_query_id": "665db21284bdd7fee6703009", + "admin_non_operational_users": 0, + "admin_non_operational_users_saved_query_id": "665db21a84bdd7fee67032aa", + "admin_operational_active_users": 0, + "admin_operational_active_users_saved_query_id": "665db21584bdd7fee6703148", + "admin_operational_inactive_users": 0, + "admin_operational_inactive_users_saved_query_id": "665db22a84bdd7fee6703692", + "admin_operational_users": 0, + "admin_operational_users_saved_query_id": "665db21684bdd7fee6703174", + "admins": 0, + "admins_saved_query_id": "665db22584bdd7fee67035a4", + "application_id": "198ef9c063EC253aEcEe", + "application_name": "VMware", + "asset_entity_info": "ARP", + "asset_type": "accounts", + "connected_assets": [ + "storage_account_id::ac4d42d9-5b05-4f67-ba8d-c770fcaa17d6" + ], + "connection_label": "okta-demo", + "created_date": "2025-02-23T00:25:17.000Z", + "deleted_users": 249, + "deleted_users_saved_query_id": "665db22684bdd7fee67035bb", + "direct_not_sso_users": 34, + "direct_not_sso_users_saved_query_id": "665db21384bdd7fee67030ab", + "domains": [ + { + "name": "salesforce.demo.local" + } + ], + "email": "example@domain.com", + "external_users": 0, + "external_users_saved_query_id": "665db21084bdd7fee6702f98", + "gce_account_id": "gce_account_id_1", + "inactive_users": 96, + "inactive_users_saved_query_id": "665db20b84bdd7fee6702d68", + "is_managed_by_direct_app": true, + "last_enrichment_run": "2025-11-12T00:02:18.000Z", + "managed_non_operational_users": 371, + "managed_non_operational_users_saved_query_id": "665db20e84bdd7fee6702e8b", + "managed_operational_users": 1596, + "managed_operational_users_saved_query_id": "665db20e84bdd7fee6702e83", + "managed_users": 1602, + "managed_users_by_app": 1602, + "managed_users_by_app_saved_query_id": "665db22784bdd7fee6703613", + "managed_users_by_sso": 1568, + "managed_users_by_sso_saved_query_id": "665db20e84bdd7fee6702ea0", + "managed_users_saved_query_id": "665db22284bdd7fee67034e4", + "orphaned_users": 28, + "orphaned_users_saved_query_id": "665db21784bdd7fee67031fb", + "paid_users": 247, + "paid_users_saved_query_id": "665db21984bdd7fee6703296", + "project_ids": [ + "id1", + "id2" + ], + "project_tags": [ + { + "inherited": "dunno", + "key": "key1", + "namespaced_tag_key": "tkey", + "namespaced_tag_value": "tvalue", + "value": "value1" + } + ], + "projects_roles": [ + { + "project_id": "id123", + "role_name": "role12" + } + ], + "relatable_ids": [ + "image_id::87dec736-1ce1-4476-8a68-c37dcdffe9c2" + ], + "remote_account_id": "racid", + "status": "closed", + "suspended_users": 241, + "suspended_users_saved_query_id": "665db20d84bdd7fee6702e2b", + "transform_unique_id": "06QLxqxkDWILMiPUnx82BIz87Lc=", + "unlinked_users": 0, + "unlinked_users_saved_query_id": "665db22384bdd7fee67034fc" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "created": "2025-02-23T00:25:17.000Z", + "kind": "event", + "original": "{\"asset_type\":\"accounts\",\"event\":{\"data\":{\"active_users\":1500,\"active_users_saved_query_id\":\"665db21284bdd7fee6703009\",\"admin_non_operational_users\":0,\"admin_non_operational_users_saved_query_id\":\"665db21a84bdd7fee67032aa\",\"admin_operational_active_users\":0,\"admin_operational_active_users_saved_query_id\":\"665db21584bdd7fee6703148\",\"admin_operational_inactive_users\":0,\"admin_operational_inactive_users_saved_query_id\":\"665db22a84bdd7fee6703692\",\"admin_operational_users\":0,\"admin_operational_users_saved_query_id\":\"665db21684bdd7fee6703174\",\"admins\":0,\"admins_saved_query_id\":\"665db22584bdd7fee67035a4\",\"application_id\":\"198ef9c063EC253aEcEe\",\"application_name\":\"VMware\",\"asset_entity_info\":\"ARP\",\"connected_assets\":[\"storage_account_id::ac4d42d9-5b05-4f67-ba8d-c770fcaa17d6\"],\"connection_label\":\"okta-demo\",\"created_date\":\"Sun, 23 Feb 2025 00:25:17 GMT\",\"deleted_users\":249,\"deleted_users_saved_query_id\":\"665db22684bdd7fee67035bb\",\"direct_not_sso_users\":34,\"direct_not_sso_users_saved_query_id\":\"665db21384bdd7fee67030ab\",\"domains\":[{\"name\":\"salesforce.demo.local\"}],\"email\":\"example@domain.com\",\"external_users\":0,\"external_users_saved_query_id\":\"665db21084bdd7fee6702f98\",\"gce_account_id\":\"gce_account_id_1\",\"inactive_users\":96,\"inactive_users_saved_query_id\":\"665db20b84bdd7fee6702d68\",\"is_managed_by_direct_app\":true,\"last_enrichment_run\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"managed_non_operational_users\":371,\"managed_non_operational_users_saved_query_id\":\"665db20e84bdd7fee6702e8b\",\"managed_operational_users\":1596,\"managed_operational_users_saved_query_id\":\"665db20e84bdd7fee6702e83\",\"managed_users\":1602,\"managed_users_by_app\":1602,\"managed_users_by_app_saved_query_id\":\"665db22784bdd7fee6703613\",\"managed_users_by_sso\":1568,\"managed_users_by_sso_saved_query_id\":\"665db20e84bdd7fee6702ea0\",\"managed_users_saved_query_id\":\"665db22284bdd7fee67034e4\",\"orphaned_users\":28,\"orphaned_users_saved_query_id\":\"665db21784bdd7fee67031fb\",\"paid_users\":247,\"paid_users_saved_query_id\":\"665db21984bdd7fee6703296\",\"project_ids\":[\"id1\",\"id2\"],\"project_tags\":[{\"inherited\":\"dunno\",\"key\":\"key1\",\"namespaced_tag_key\":\"tkey\",\"namespaced_tag_value\":\"tvalue\",\"value\":\"value1\"}],\"projects_roles\":[{\"project_id\":\"id123\",\"role_name\":\"role12\"}],\"relatable_ids\":[\"image_id::87dec736-1ce1-4476-8a68-c37dcdffe9c2\"],\"remote_account_id\":\"racid\",\"status\":\"closed\",\"suspended_users\":241,\"suspended_users_saved_query_id\":\"665db20d84bdd7fee6702e2b\",\"unlinked_users\":0,\"unlinked_users_saved_query_id\":\"665db22384bdd7fee67034fc\"}}}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "example@domain.com" + ] + }, + "service": { + "id": "198ef9c063EC253aEcEe", + "name": "VMware" + }, + "tags": [ + "preserve_duplicate_custom_fields" + ], + "user": { + "domain": "domain.com", + "email": "example@domain.com" + } + }, + { + "@timestamp": "2025-12-09T00:04:58.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:04:58.000Z", + "adapter_list_length": 1, + "adapters": [ + "qualys_scans_adapter" + ], + "application_and_account_name": "qualys/qualys_scans-demo", + "application_name": "Qualys", + "asset_type": "accounts", + "data_asset_type": "Connected Adapter", + "event": { + "accurate_for_datetime": "2025-12-09T00:04:58.000Z", + "adapter_categories": [ + "VA Tool" + ], + "client_used": "67fd09dffe1c8e812a176bcb", + "initial_plugin_unique_name": "qualys_scans_adapter_0", + "plugin_name": "qualys_scans_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "qualys_scans_adapter_0", + "quick_id": "qualys_scans_adapter_0!38561584016c3bd50997", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T00:04:58.000Z", + "first_fetch_time": "2025-04-14T13:29:14.000Z", + "from_last_fetch": true, + "id": "38561584016c3bd50997", + "id_raw": "qualys_scans-demo_67fd09dffe1c8e812a176bcb", + "internal_axon_id": "e2781b0b477c77303261aec45a40cb44", + "is_fetched_from_adapter": true, + "is_managed_by_direct_app": true, + "name": "Qualys/qualys_scans-demo", + "not_fetched_count": 0, + "roles_accounts": [ + "roles/clou." + ], + "source_application": "Qualys", + "transform_unique_id": "o1/wQWHAgVHLeNNsa0aSzPLTx74=", + "type": "Accounts" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"qualys_scans_adapter\"],\"asset_type\":\"accounts\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:04:58 GMT\",\"adapter_categories\":[\"VA Tool\"],\"client_used\":\"67fd09dffe1c8e812a176bcb\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:04:58 GMT\",\"application_and_account_name\":\"qualys/qualys_scans-demo\",\"application_name\":\"Qualys\",\"asset_type\":\"Connected Adapter\",\"domains\":[{}],\"fetch_time\":\"Tue, 09 Dec 2025 00:04:58 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:29:14 GMT\",\"from_last_fetch\":true,\"id\":\"38561584016c3bd50997\",\"id_raw\":\"qualys_scans-demo_67fd09dffe1c8e812a176bcb\",\"is_fetched_from_adapter\":true,\"is_managed_by_direct_app\":true,\"name\":\"Qualys/qualys_scans-demo\",\"not_fetched_count\":0,\"roles\":[\"roles/clou.\"], \"source_application\":\"Qualys\",\"type\":\"Accounts\"},\"initial_plugin_unique_name\":\"qualys_scans_adapter_0\",\"plugin_name\":\"qualys_scans_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"qualys_scans_adapter_0\",\"quick_id\":\"qualys_scans_adapter_0!38561584016c3bd50997\",\"type\":\"entitydata\"},\"internal_axon_id\":\"e2781b0b477c77303261aec45a40cb44\"}", + "type": [ + "info" + ] + }, + "service": { + "name": "Qualys" + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:02:32.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:02:32.000Z", + "active_users": 0, + "active_users_saved_query_id": "67fd1261709940e80b240c24", + "adapter_list_length": 2, + "adapters": [ + "github_adapter", + "okta_adapter" + ], + "admin_non_operational_users": 0, + "admin_non_operational_users_saved_query_id": "67fd1261709940e80b240c68", + "admin_operational_active_users": 0, + "admin_operational_active_users_saved_query_id": "67fd1261709940e80b240c7e", + "admin_operational_inactive_users": 0, + "admin_operational_inactive_users_saved_query_id": "67fd1261709940e80b240c3f", + "admin_operational_users": 0, + "admin_operational_users_saved_query_id": "67fd1261709940e80b240c5d", + "admins": 0, + "admins_saved_query_id": "67fd1261709940e80b240c38", + "application_and_account_name": "github/github-dev", + "application_name": "GitHub", + "asset_type": "accounts", + "data_asset_type": "Connected Adapter", + "deleted_users": 0, + "deleted_users_saved_query_id": "67fd1261709940e80b240cb7", + "direct_not_sso_users": 27, + "direct_not_sso_users_saved_query_id": "67fd1261709940e80b240c86", + "domains": [ + { + "name": "github.demo.local" + } + ], + "event": { + "accurate_for_datetime": "2025-12-09T00:02:32.000Z", + "adapter_categories": [ + "Software Development Version Control", + "SaaS Management" + ], + "client_used": "67fd09a9fe1c8e812a176baa", + "initial_plugin_unique_name": "github_adapter_0", + "plugin_name": "github_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "github_adapter_0", + "quick_id": "github_adapter_0!697947761f2f779c8df5", + "type": "entitydata" + }, + "external_users": 0, + "external_users_saved_query_id": "67fd1261709940e80b240cc4", + "fetch_time": "2025-12-09T00:02:32.000Z", + "first_fetch_time": "2025-04-14T13:27:53.000Z", + "from_last_fetch": true, + "id": "697947761f2f779c8df5", + "id_raw": "github-dev_67fd09a9fe1c8e812a176baa", + "inactive_users": 339, + "inactive_users_saved_query_id": "67fd1261709940e80b240c4a", + "internal_axon_id": "5592c1d0fc4c0fdeebb6c98530a7c777", + "is_fetched_from_adapter": true, + "is_managed_by_direct_app": true, + "last_enrichment_run": "2025-12-09T00:10:07.000Z", + "managed_non_operational_users": 0, + "managed_non_operational_users_saved_query_id": "67fd1261709940e80b240c56", + "managed_operational_users": 339, + "managed_operational_users_saved_query_id": "67fd1261709940e80b240c71", + "managed_users": 339, + "managed_users_by_app": 180, + "managed_users_by_app_saved_query_id": "67fd1261709940e80b240c9a", + "managed_users_by_sso": 312, + "managed_users_by_sso_saved_query_id": "67fd1261709940e80b240ca6", + "managed_users_saved_query_id": "67fd1261709940e80b240c17", + "name": "GitHub/github-dev", + "not_fetched_count": 0, + "orphaned_users": 27, + "orphaned_users_saved_query_id": "67fd1261709940e80b240c8f", + "paid_users": 0, + "paid_users_saved_query_id": "67fd1261709940e80b240ccc", + "source_application": "GitHub", + "suspended_users": 0, + "suspended_users_saved_query_id": "67fd1261709940e80b240cae", + "transform_unique_id": "NGzJreKl2cvKT/Mp9NTeXRSOHZg=", + "type": "Accounts", + "unlinked_users": 159, + "unlinked_users_saved_query_id": "67fd1261709940e80b240c2b" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":2,\"adapters\":[\"github_adapter\",\"okta_adapter\"],\"asset_type\":\"accounts\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:02:32 GMT\",\"adapter_categories\":[\"Software Development Version Control\",\"SaaS Management\"],\"client_used\":\"67fd09a9fe1c8e812a176baa\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:02:32 GMT\",\"active_users\":0,\"active_users_saved_query_id\":\"67fd1261709940e80b240c24\",\"admin_non_operational_users\":0,\"admin_non_operational_users_saved_query_id\":\"67fd1261709940e80b240c68\",\"admin_operational_active_users\":0,\"admin_operational_active_users_saved_query_id\":\"67fd1261709940e80b240c7e\",\"admin_operational_inactive_users\":0,\"admin_operational_inactive_users_saved_query_id\":\"67fd1261709940e80b240c3f\",\"admin_operational_users\":0,\"admin_operational_users_saved_query_id\":\"67fd1261709940e80b240c5d\",\"admins\":0,\"admins_saved_query_id\":\"67fd1261709940e80b240c38\",\"application_and_account_name\":\"github/github-dev\",\"application_name\":\"GitHub\",\"asset_type\":\"Connected Adapter\",\"deleted_users\":0,\"deleted_users_saved_query_id\":\"67fd1261709940e80b240cb7\",\"direct_not_sso_users\":27,\"direct_not_sso_users_saved_query_id\":\"67fd1261709940e80b240c86\",\"domains\":[{\"name\":\"github.demo.local\"}],\"external_users\":0,\"external_users_saved_query_id\":\"67fd1261709940e80b240cc4\",\"fetch_time\":\"Tue, 09 Dec 2025 00:02:32 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:27:53 GMT\",\"from_last_fetch\":true,\"id\":\"697947761f2f779c8df5\",\"id_raw\":\"github-dev_67fd09a9fe1c8e812a176baa\",\"inactive_users\":339,\"inactive_users_saved_query_id\":\"67fd1261709940e80b240c4a\",\"is_fetched_from_adapter\":true,\"is_managed_by_direct_app\":true,\"last_enrichment_run\":\"Tue, 09 Dec 2025 00:10:07 GMT\",\"managed_non_operational_users\":0,\"managed_non_operational_users_saved_query_id\":\"67fd1261709940e80b240c56\",\"managed_operational_users\":339,\"managed_operational_users_saved_query_id\":\"67fd1261709940e80b240c71\",\"managed_users\":339,\"managed_users_by_app\":180,\"managed_users_by_app_saved_query_id\":\"67fd1261709940e80b240c9a\",\"managed_users_by_sso\":312,\"managed_users_by_sso_saved_query_id\":\"67fd1261709940e80b240ca6\",\"managed_users_saved_query_id\":\"67fd1261709940e80b240c17\",\"name\":\"GitHub/github-dev\",\"not_fetched_count\":0,\"orphaned_users\":27,\"orphaned_users_saved_query_id\":\"67fd1261709940e80b240c8f\",\"paid_users\":0,\"paid_users_saved_query_id\":\"67fd1261709940e80b240ccc\",\"source_application\":\"GitHub\",\"suspended_users\":0,\"suspended_users_saved_query_id\":\"67fd1261709940e80b240cae\",\"type\":\"Accounts\",\"unlinked_users\":159,\"unlinked_users_saved_query_id\":\"67fd1261709940e80b240c2b\"},\"initial_plugin_unique_name\":\"github_adapter_0\",\"plugin_name\":\"github_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"github_adapter_0\",\"quick_id\":\"github_adapter_0!697947761f2f779c8df5\",\"type\":\"entitydata\"},\"internal_axon_id\":\"5592c1d0fc4c0fdeebb6c98530a7c777\"}", + "type": [ + "info" + ] + }, + "service": { + "name": "GitHub" + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + } + ] +} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-certificate.log b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-certificate.log new file mode 100644 index 00000000000..61c1299fb28 --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-certificate.log @@ -0,0 +1,3 @@ +{"internal_axon_id":"d2e5e38e803587f6beec0ebf20841e35","adapters":["webscan_adapter"],"asset_type":"certificates","adapter_list_length":1,"event":{"accurate_for_datetime":"Mon, 08 Dec 2025 12:01:42 GMT","adapter_categories":["ITAM\/ITSM"],"client_used":"67fd09ab3c68ed1b541bb4af","data":{"accurate_for_datetime":"Mon, 08 Dec 2025 12:01:42 GMT","alt_names":[{"name":"test.demo.com","name_type":"DNS"}],"application_and_account_name":"web server information\/webscan-demo","asset_type":"Webscan Certificate","begins_on":"Thu, 15 Feb 2024 18:24:34 GMT","bit_size":256,"expires_on":"Thu, 27 Nov 2025 17:35:21 GMT","fetch_time":"Mon, 08 Dec 2025 12:01:41 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:26:31 GMT","from_last_fetch":true,"id":"4c5eb5e23e36805ac57d","id_raw":"bbb6246e-7056-4120-b96f-87f3d38a080e","is_fetched_from_adapter":true,"issuer":{"common_name":"GlobalSign Root CA","country_name":"US","organization":"GlobalSign nv-sa"},"last_fetch_connection_id":"67fd09ab3c68ed1b541bb4af","last_fetch_connection_label":"webscan-demo","name":"GlobalSign Root CA","not_fetched_count":0,"pretty_id":"AX-3538281358","serial_number":"22053078225666923629770456953932","source_application":"Web Server Information","subject":{"common_name":"GlobalSign Root CA","country_name":"US","locality":"San Francisco","organization":"GlobalSign nv-sa","state":"California"},"tenant_number":["4"],"type":"Certificate","version":"2"},"initial_plugin_unique_name":"webscan_adapter_0","plugin_name":"webscan_adapter","plugin_type":"Adapter","plugin_unique_name":"webscan_adapter_0","quick_id":"webscan_adapter_0!4c5eb5e23e36805ac57d","type":"entitydata"}} +{"adapter_list_length":1,"adapters":["webscan_adapter"],"asset_type":"certificates","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:01:30 GMT","adapter_categories":["ITAM/ITSM"],"client_used":"67fd09ab3c68ed1b541bb4af","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:01:30 GMT","alt_names":[{"name":"subdomain.demo.com","name_type":"DNS"},{"name":"login.demo.com","name_type":"DNS"}],"application_and_account_name":"web server information/webscan-demo","asset_type":"Webscan Certificate","begins_on":"Wed, 05 Jun 2024 20:39:03 GMT","bit_size":4096,"expires_on":"Mon, 22 Dec 2025 18:05:33 GMT","fetch_time":"Tue, 09 Dec 2025 00:01:30 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:26:31 GMT","from_last_fetch":true,"id":"c59079ddd4349aa5c745","id_raw":"3778ea79-f4c5-4a65-b0a6-e1304b698c45","is_fetched_from_adapter":true,"issuer":{"common_name":"Baltimore CyberTrust Root","country_name":"US","organization":"Baltimore"},"last_fetch_connection_id":"67fd09ab3c68ed1b541bb4af","last_fetch_connection_label":"webscan-demo","name":"Baltimore CyberTrust Root","not_fetched_count":0,"pretty_id":"AX-3061421597","serial_number":"00145161702969659350434254970802","source_application":"Web Server Information","subject":{"common_name":"Baltimore CyberTrust Root","country_name":"US","locality":"San Francisco","organization":"Baltimore","state":"California"},"tenant_number":["4"],"type":"Certificate","version":"2"},"initial_plugin_unique_name":"webscan_adapter_0","plugin_name":"webscan_adapter","plugin_type":"Adapter","plugin_unique_name":"webscan_adapter_0","quick_id":"webscan_adapter_0!c59079ddd4349aa5c745","type":"entitydata"},"internal_axon_id":"b679961d638ed4affd8b2027fe73af6d"} +{"adapter_list_length":1,"adapters":["webscan_adapter"],"asset_type":"certificates","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:01:30 GMT","adapter_categories":["ITAM/ITSM"],"client_used":"67fd09ab3c68ed1b541bb4af","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:01:30 GMT","alt_names":[{"name":"subdomain.demo.com","name_type":"DNS"},{"name":"test.demo.com","name_type":"DNS"},{"name":"login.demo.com","name_type":"DNS"}],"application_and_account_name":"web server information/webscan-demo","asset_type":"Webscan Certificate","begins_on":"Wed, 06 Nov 2024 21:18:38 GMT","bit_size":256,"expires_on":"Sun, 11 Jan 2026 15:12:32 GMT","fetch_time":"Tue, 09 Dec 2025 00:01:30 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:26:31 GMT","from_last_fetch":true,"id":"358c45934b0b5dde6187","id_raw":"8b25e8ce-c084-455b-a2eb-60dd0790dbcf","is_fetched_from_adapter":true,"issuer":{"common_name":"DST Root CA X3","country_name":"US","organization":"Digital Signature Trust Co."},"last_fetch_connection_id":"67fd09ab3c68ed1b541bb4af","last_fetch_connection_label":"webscan-demo","name":"DST Root CA X3","not_fetched_count":0,"pretty_id":"AX-1987793400","serial_number":"50251828753248654275669930302432","source_application":"Web Server Information","subject":{"common_name":"DST Root CA X3","country_name":"US","locality":"San Francisco","organization":"Digital Signature Trust Co.","state":"California"},"tenant_number":["3"],"type":"Certificate","version":"2"},"initial_plugin_unique_name":"webscan_adapter_0","plugin_name":"webscan_adapter","plugin_type":"Adapter","plugin_unique_name":"webscan_adapter_0","quick_id":"webscan_adapter_0!358c45934b0b5dde6187","type":"entitydata"},"internal_axon_id":"767b51f89d768697b64836746401a5cf"} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-certificate.log-expected.json b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-certificate.log-expected.json new file mode 100644 index 00000000000..a768e1c8e26 --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-certificate.log-expected.json @@ -0,0 +1,376 @@ +{ + "expected": [ + { + "@timestamp": "2025-12-08T12:01:42.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-08T12:01:42.000Z", + "adapter_list_length": 1, + "adapters": [ + "webscan_adapter" + ], + "alt_names": [ + { + "name": "test.demo.com", + "name_type": "DNS" + } + ], + "application_and_account_name": "web server information/webscan-demo", + "asset_type": "certificates", + "begins_on": "2024-02-15T18:24:34.000Z", + "bit_size": 256, + "data_asset_type": "Webscan Certificate", + "event": { + "accurate_for_datetime": "2025-12-08T12:01:42.000Z", + "adapter_categories": [ + "ITAM/ITSM" + ], + "client_used": "67fd09ab3c68ed1b541bb4af", + "initial_plugin_unique_name": "webscan_adapter_0", + "plugin_name": "webscan_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "webscan_adapter_0", + "quick_id": "webscan_adapter_0!4c5eb5e23e36805ac57d", + "type": "entitydata" + }, + "expires_on": "2025-11-27T17:35:21.000Z", + "fetch_time": "2025-12-08T12:01:41.000Z", + "first_fetch_time": "2025-04-14T13:26:31.000Z", + "from_last_fetch": true, + "id": "4c5eb5e23e36805ac57d", + "id_raw": "bbb6246e-7056-4120-b96f-87f3d38a080e", + "internal_axon_id": "d2e5e38e803587f6beec0ebf20841e35", + "is_fetched_from_adapter": true, + "issuer": { + "common_name": "GlobalSign Root CA", + "country_name": "US", + "organization": "GlobalSign nv-sa" + }, + "last_fetch_connection_id": "67fd09ab3c68ed1b541bb4af", + "last_fetch_connection_label": "webscan-demo", + "name": "GlobalSign Root CA", + "not_fetched_count": 0, + "pretty_id": "AX-3538281358", + "serial_number": "22053078225666923629770456953932", + "source_application": "Web Server Information", + "subject": { + "common_name": "GlobalSign Root CA", + "country_name": "US", + "locality": "San Francisco", + "organization": "GlobalSign nv-sa", + "state": "California" + }, + "tenant_number": [ + 4 + ], + "transform_unique_id": "Odx9fE8sV+1lCFDeADL7zugotXk=", + "type": "Certificate", + "version": "2" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "end": "2025-11-27T17:35:21.000Z", + "kind": "event", + "original": "{\"internal_axon_id\":\"d2e5e38e803587f6beec0ebf20841e35\",\"adapters\":[\"webscan_adapter\"],\"asset_type\":\"certificates\",\"adapter_list_length\":1,\"event\":{\"accurate_for_datetime\":\"Mon, 08 Dec 2025 12:01:42 GMT\",\"adapter_categories\":[\"ITAM\\/ITSM\"],\"client_used\":\"67fd09ab3c68ed1b541bb4af\",\"data\":{\"accurate_for_datetime\":\"Mon, 08 Dec 2025 12:01:42 GMT\",\"alt_names\":[{\"name\":\"test.demo.com\",\"name_type\":\"DNS\"}],\"application_and_account_name\":\"web server information\\/webscan-demo\",\"asset_type\":\"Webscan Certificate\",\"begins_on\":\"Thu, 15 Feb 2024 18:24:34 GMT\",\"bit_size\":256,\"expires_on\":\"Thu, 27 Nov 2025 17:35:21 GMT\",\"fetch_time\":\"Mon, 08 Dec 2025 12:01:41 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:26:31 GMT\",\"from_last_fetch\":true,\"id\":\"4c5eb5e23e36805ac57d\",\"id_raw\":\"bbb6246e-7056-4120-b96f-87f3d38a080e\",\"is_fetched_from_adapter\":true,\"issuer\":{\"common_name\":\"GlobalSign Root CA\",\"country_name\":\"US\",\"organization\":\"GlobalSign nv-sa\"},\"last_fetch_connection_id\":\"67fd09ab3c68ed1b541bb4af\",\"last_fetch_connection_label\":\"webscan-demo\",\"name\":\"GlobalSign Root CA\",\"not_fetched_count\":0,\"pretty_id\":\"AX-3538281358\",\"serial_number\":\"22053078225666923629770456953932\",\"source_application\":\"Web Server Information\",\"subject\":{\"common_name\":\"GlobalSign Root CA\",\"country_name\":\"US\",\"locality\":\"San Francisco\",\"organization\":\"GlobalSign nv-sa\",\"state\":\"California\"},\"tenant_number\":[\"4\"],\"type\":\"Certificate\",\"version\":\"2\"},\"initial_plugin_unique_name\":\"webscan_adapter_0\",\"plugin_name\":\"webscan_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"webscan_adapter_0\",\"quick_id\":\"webscan_adapter_0!4c5eb5e23e36805ac57d\",\"type\":\"entitydata\"}}", + "start": "2024-02-15T18:24:34.000Z", + "type": [ + "info" + ] + }, + "file": { + "x509": { + "issuer": { + "common_name": [ + "GlobalSign Root CA" + ], + "country": [ + "US" + ], + "organization": [ + "GlobalSign nv-sa" + ] + }, + "serial_number": "22053078225666923629770456953932", + "subject": { + "common_name": [ + "GlobalSign Root CA" + ], + "country": [ + "US" + ], + "locality": [ + "San Francisco" + ], + "organization": [ + "GlobalSign nv-sa" + ], + "state_or_province": [ + "California" + ] + } + } + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:01:30.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:01:30.000Z", + "adapter_list_length": 1, + "adapters": [ + "webscan_adapter" + ], + "alt_names": [ + { + "name": "subdomain.demo.com", + "name_type": "DNS" + }, + { + "name": "login.demo.com", + "name_type": "DNS" + } + ], + "application_and_account_name": "web server information/webscan-demo", + "asset_type": "certificates", + "begins_on": "2024-06-05T20:39:03.000Z", + "bit_size": 4096, + "data_asset_type": "Webscan Certificate", + "event": { + "accurate_for_datetime": "2025-12-09T00:01:30.000Z", + "adapter_categories": [ + "ITAM/ITSM" + ], + "client_used": "67fd09ab3c68ed1b541bb4af", + "initial_plugin_unique_name": "webscan_adapter_0", + "plugin_name": "webscan_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "webscan_adapter_0", + "quick_id": "webscan_adapter_0!c59079ddd4349aa5c745", + "type": "entitydata" + }, + "expires_on": "2025-12-22T18:05:33.000Z", + "fetch_time": "2025-12-09T00:01:30.000Z", + "first_fetch_time": "2025-04-14T13:26:31.000Z", + "from_last_fetch": true, + "id": "c59079ddd4349aa5c745", + "id_raw": "3778ea79-f4c5-4a65-b0a6-e1304b698c45", + "internal_axon_id": "b679961d638ed4affd8b2027fe73af6d", + "is_fetched_from_adapter": true, + "issuer": { + "common_name": "Baltimore CyberTrust Root", + "country_name": "US", + "organization": "Baltimore" + }, + "last_fetch_connection_id": "67fd09ab3c68ed1b541bb4af", + "last_fetch_connection_label": "webscan-demo", + "name": "Baltimore CyberTrust Root", + "not_fetched_count": 0, + "pretty_id": "AX-3061421597", + "serial_number": "00145161702969659350434254970802", + "source_application": "Web Server Information", + "subject": { + "common_name": "Baltimore CyberTrust Root", + "country_name": "US", + "locality": "San Francisco", + "organization": "Baltimore", + "state": "California" + }, + "tenant_number": [ + 4 + ], + "transform_unique_id": "fYuXqtfJLKJ24laKfs2pcNkuELA=", + "type": "Certificate", + "version": "2" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "end": "2025-12-22T18:05:33.000Z", + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"webscan_adapter\"],\"asset_type\":\"certificates\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:01:30 GMT\",\"adapter_categories\":[\"ITAM/ITSM\"],\"client_used\":\"67fd09ab3c68ed1b541bb4af\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:01:30 GMT\",\"alt_names\":[{\"name\":\"subdomain.demo.com\",\"name_type\":\"DNS\"},{\"name\":\"login.demo.com\",\"name_type\":\"DNS\"}],\"application_and_account_name\":\"web server information/webscan-demo\",\"asset_type\":\"Webscan Certificate\",\"begins_on\":\"Wed, 05 Jun 2024 20:39:03 GMT\",\"bit_size\":4096,\"expires_on\":\"Mon, 22 Dec 2025 18:05:33 GMT\",\"fetch_time\":\"Tue, 09 Dec 2025 00:01:30 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:26:31 GMT\",\"from_last_fetch\":true,\"id\":\"c59079ddd4349aa5c745\",\"id_raw\":\"3778ea79-f4c5-4a65-b0a6-e1304b698c45\",\"is_fetched_from_adapter\":true,\"issuer\":{\"common_name\":\"Baltimore CyberTrust Root\",\"country_name\":\"US\",\"organization\":\"Baltimore\"},\"last_fetch_connection_id\":\"67fd09ab3c68ed1b541bb4af\",\"last_fetch_connection_label\":\"webscan-demo\",\"name\":\"Baltimore CyberTrust Root\",\"not_fetched_count\":0,\"pretty_id\":\"AX-3061421597\",\"serial_number\":\"00145161702969659350434254970802\",\"source_application\":\"Web Server Information\",\"subject\":{\"common_name\":\"Baltimore CyberTrust Root\",\"country_name\":\"US\",\"locality\":\"San Francisco\",\"organization\":\"Baltimore\",\"state\":\"California\"},\"tenant_number\":[\"4\"],\"type\":\"Certificate\",\"version\":\"2\"},\"initial_plugin_unique_name\":\"webscan_adapter_0\",\"plugin_name\":\"webscan_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"webscan_adapter_0\",\"quick_id\":\"webscan_adapter_0!c59079ddd4349aa5c745\",\"type\":\"entitydata\"},\"internal_axon_id\":\"b679961d638ed4affd8b2027fe73af6d\"}", + "start": "2024-06-05T20:39:03.000Z", + "type": [ + "info" + ] + }, + "file": { + "x509": { + "issuer": { + "common_name": [ + "Baltimore CyberTrust Root" + ], + "country": [ + "US" + ], + "organization": [ + "Baltimore" + ] + }, + "serial_number": "00145161702969659350434254970802", + "subject": { + "common_name": [ + "Baltimore CyberTrust Root" + ], + "country": [ + "US" + ], + "locality": [ + "San Francisco" + ], + "organization": [ + "Baltimore" + ], + "state_or_province": [ + "California" + ] + } + } + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:01:30.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:01:30.000Z", + "adapter_list_length": 1, + "adapters": [ + "webscan_adapter" + ], + "alt_names": [ + { + "name": "subdomain.demo.com", + "name_type": "DNS" + }, + { + "name": "test.demo.com", + "name_type": "DNS" + }, + { + "name": "login.demo.com", + "name_type": "DNS" + } + ], + "application_and_account_name": "web server information/webscan-demo", + "asset_type": "certificates", + "begins_on": "2024-11-06T21:18:38.000Z", + "bit_size": 256, + "data_asset_type": "Webscan Certificate", + "event": { + "accurate_for_datetime": "2025-12-09T00:01:30.000Z", + "adapter_categories": [ + "ITAM/ITSM" + ], + "client_used": "67fd09ab3c68ed1b541bb4af", + "initial_plugin_unique_name": "webscan_adapter_0", + "plugin_name": "webscan_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "webscan_adapter_0", + "quick_id": "webscan_adapter_0!358c45934b0b5dde6187", + "type": "entitydata" + }, + "expires_on": "2026-01-11T15:12:32.000Z", + "fetch_time": "2025-12-09T00:01:30.000Z", + "first_fetch_time": "2025-04-14T13:26:31.000Z", + "from_last_fetch": true, + "id": "358c45934b0b5dde6187", + "id_raw": "8b25e8ce-c084-455b-a2eb-60dd0790dbcf", + "internal_axon_id": "767b51f89d768697b64836746401a5cf", + "is_fetched_from_adapter": true, + "issuer": { + "common_name": "DST Root CA X3", + "country_name": "US", + "organization": "Digital Signature Trust Co." + }, + "last_fetch_connection_id": "67fd09ab3c68ed1b541bb4af", + "last_fetch_connection_label": "webscan-demo", + "name": "DST Root CA X3", + "not_fetched_count": 0, + "pretty_id": "AX-1987793400", + "serial_number": "50251828753248654275669930302432", + "source_application": "Web Server Information", + "subject": { + "common_name": "DST Root CA X3", + "country_name": "US", + "locality": "San Francisco", + "organization": "Digital Signature Trust Co.", + "state": "California" + }, + "tenant_number": [ + 3 + ], + "transform_unique_id": "a8UeB3YaIU4lCPAgIx2o5XbQ29k=", + "type": "Certificate", + "version": "2" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "end": "2026-01-11T15:12:32.000Z", + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"webscan_adapter\"],\"asset_type\":\"certificates\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:01:30 GMT\",\"adapter_categories\":[\"ITAM/ITSM\"],\"client_used\":\"67fd09ab3c68ed1b541bb4af\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:01:30 GMT\",\"alt_names\":[{\"name\":\"subdomain.demo.com\",\"name_type\":\"DNS\"},{\"name\":\"test.demo.com\",\"name_type\":\"DNS\"},{\"name\":\"login.demo.com\",\"name_type\":\"DNS\"}],\"application_and_account_name\":\"web server information/webscan-demo\",\"asset_type\":\"Webscan Certificate\",\"begins_on\":\"Wed, 06 Nov 2024 21:18:38 GMT\",\"bit_size\":256,\"expires_on\":\"Sun, 11 Jan 2026 15:12:32 GMT\",\"fetch_time\":\"Tue, 09 Dec 2025 00:01:30 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:26:31 GMT\",\"from_last_fetch\":true,\"id\":\"358c45934b0b5dde6187\",\"id_raw\":\"8b25e8ce-c084-455b-a2eb-60dd0790dbcf\",\"is_fetched_from_adapter\":true,\"issuer\":{\"common_name\":\"DST Root CA X3\",\"country_name\":\"US\",\"organization\":\"Digital Signature Trust Co.\"},\"last_fetch_connection_id\":\"67fd09ab3c68ed1b541bb4af\",\"last_fetch_connection_label\":\"webscan-demo\",\"name\":\"DST Root CA X3\",\"not_fetched_count\":0,\"pretty_id\":\"AX-1987793400\",\"serial_number\":\"50251828753248654275669930302432\",\"source_application\":\"Web Server Information\",\"subject\":{\"common_name\":\"DST Root CA X3\",\"country_name\":\"US\",\"locality\":\"San Francisco\",\"organization\":\"Digital Signature Trust Co.\",\"state\":\"California\"},\"tenant_number\":[\"3\"],\"type\":\"Certificate\",\"version\":\"2\"},\"initial_plugin_unique_name\":\"webscan_adapter_0\",\"plugin_name\":\"webscan_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"webscan_adapter_0\",\"quick_id\":\"webscan_adapter_0!358c45934b0b5dde6187\",\"type\":\"entitydata\"},\"internal_axon_id\":\"767b51f89d768697b64836746401a5cf\"}", + "start": "2024-11-06T21:18:38.000Z", + "type": [ + "info" + ] + }, + "file": { + "x509": { + "issuer": { + "common_name": [ + "DST Root CA X3" + ], + "country": [ + "US" + ], + "organization": [ + "Digital Signature Trust Co." + ] + }, + "serial_number": "50251828753248654275669930302432", + "subject": { + "common_name": [ + "DST Root CA X3" + ], + "country": [ + "US" + ], + "locality": [ + "San Francisco" + ], + "organization": [ + "Digital Signature Trust Co." + ], + "state_or_province": [ + "California" + ] + } + } + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + } + ] +} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-common-config.yml b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..37e8fa225fd --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,3 @@ +fields: + tags: + - preserve_duplicate_custom_fields diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-group.log b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-group.log new file mode 100644 index 00000000000..2a5d8c88f2b --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-group.log @@ -0,0 +1,3 @@ +{"asset_type":"groups","event":{"data":{"first_seen":"Wed, 12 Nov 2025 00:02:18 GMT","roles":[{"display_name":"SharePoint Administrator","remote_id":"c9d7cac3e29346d3a4c259303bab613f"}],"user_count_link":[{"bracketWeight":0,"compOp":"IN","field":"event.data.mail","leftBracket":0,"logicOp":"and","not":false,"rightBracket":0,"value":["anna.pullman@demo.local","ryan.smith@demo.local"]}]}}} +{"adapter_list_length":1,"adapters":["okta_adapter"],"asset_type":"groups","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:09:21 GMT","action_if_exists":"update","associated_adapters":[],"association_type":"Tag","data":{"user_count":633,"user_count_link":[{"bracketWeight":0,"compOp":"equals","field":"specific_data.data.nested_groups.name","leftBracket":0,"logicOp":"","not":false,"rightBracket":0,"value":"Server-Operators"}]},"entity":"groups","hidden_for_gui":true,"name":"static_analysis_0_UserCountSchema","plugin_name":"static_analysis","plugin_unique_name":"static_analysis_0_UserCountSchema","type":"adapterdata"},"internal_axon_id":"f2230d26ee294785fa7cedb347ee946b"} +{"adapter_list_length":1,"adapters":["service_now_adapter"],"asset_type":"groups","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:02:40 GMT","adapter_categories":["CMDB","ITAM/ITSM","Ticketing","SaaS Management"],"client_used":"67fd0999fe1c8e812a176ba2","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:02:40 GMT","application_and_account_name":"servicenow/servicenow-dev","display_name":"Server","fetch_time":"Tue, 09 Dec 2025 00:02:34 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:27:11 GMT","first_seen":"Wed, 14 Jun 2023 10:54:10 GMT","from_last_fetch":true,"groups":[{"display_name":"sales-group","name":"sales-group","remote_id":"c0d5e3a8"}],"has_administrative_permissions":false,"id":"ee198fc2ad0d8e96d043","id_raw":"ba25c305-3dbd-4389-a778-a2b24ac9c021","is_fetched_from_adapter":true,"last_fetch_connection_id":"67fd0999fe1c8e812a176ba2","last_fetch_connection_label":"servicenow-dev","name":"Server","nested_applications":[],"nested_associated_devices":[],"nested_grants_last_updated":"Tue, 09 Dec 2025 00:09:42 GMT","nested_groups":[{"assignment_type":"Direct","group_name":"sales-group","name":"sales-group","parents":[{"name":"","value":""}],"value":"c0d5e3a8"},{"assignment_type":"Indirect","group_name":"test-group","name":"test-group","parents":[{"name":"sales-group","parent_type":"Group","value":"c0d5e3a8"}],"value":"36ef470c"},{"assignment_type":"Indirect","group_name":"guests","name":"guests","parents":[{"name":"sales-group","parent_type":"Group","value":"c0d5e3a8"}],"value":"aed2627e"},{"assignment_type":"Indirect","group_name":"Server Operators","name":"Server Operators","parents":[{"name":"sales-group","parent_type":"Group","value":"c0d5e3a8"}],"value":"9ba02066"},{"assignment_type":"Indirect","group_name":"r\\u0026d testing","name":"r\\u0026d testing","parents":[{"name":"sales-group","parent_type":"Group","value":"c0d5e3a8"}],"value":"714c4251"},{"assignment_type":"Indirect","group_name":"lab production","name":"lab production","parents":[{"name":"sales-group","parent_type":"Group","value":"c0d5e3a8"}],"value":"26205d34"}],"nested_permissions":[],"nested_resources":[],"nested_roles":[{"assignment_type":"Direct","name":"sn_dependentclient.dependentclient_user","parents":[{"name":"","value":""}],"value":"38ceb28b936146f7acbb7438b54283fc"},{"assignment_type":"Direct","name":"log_points","parents":[{"name":"","value":""}],"value":"8d0a58c77a064bfd94e8ca3ee1cab36f"},{"assignment_type":"Direct","name":"decision_table_admin","parents":[{"name":"","value":""}],"value":"5ad93ef2d50c4fa997a72d4903271a87"},{"assignment_type":"Indirect","name":"catalog_template_editor","parents":[{"name":"sales-group","parent_type":"Group","value":"c0d5e3a8"}],"value":"340f6b1c033f49a9b3e2b0895f382329"},{"assignment_type":"Indirect","name":"export_rest_api","parents":[{"name":"sales-group","parent_type":"Group","value":"c0d5e3a8"}],"value":"c8e6c40ab18149aea54c3b5da4b1d849"}],"not_fetched_count":0,"permissions":[],"remote_id":"95c933af","roles":[{"display_name":"sn_dependentclient.dependentclient_user","remote_id":"38ceb28b936146f7acbb7438b54283fc"},{"display_name":"log_points","remote_id":"8d0a58c77a064bfd94e8ca3ee1cab36f"},{"display_name":"decision_table_admin","remote_id":"5ad93ef2d50c4fa997a72d4903271a87"}],"sm_entity_type":"group","software_cves":[],"source_application":"ServiceNow","tenant_number":["2"],"type":"Groups"},"initial_plugin_unique_name":"service_now_adapter_0","plugin_name":"service_now_adapter","plugin_type":"Adapter","plugin_unique_name":"service_now_adapter_0","quick_id":"service_now_adapter_0!ee198fc2ad0d8e96d043","type":"entitydata"},"internal_axon_id":"2285fd71b4ec307bb4054a9dc1c5aefb"} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-group.log-expected.json b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-group.log-expected.json new file mode 100644 index 00000000000..4e2a311fb13 --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-group.log-expected.json @@ -0,0 +1,326 @@ +{ + "expected": [ + { + "axonius": { + "identity": { + "asset_type": "groups", + "first_seen": "2025-11-12T00:02:18.000Z", + "roles": [ + { + "display_name": "SharePoint Administrator", + "remote_id": "c9d7cac3e29346d3a4c259303bab613f" + } + ], + "transform_unique_id": "ln52qz4dwMaH33vfZ343CfOfWZc=", + "user_count_link": [ + { + "bracketWeight": 0.0, + "compOp": "IN", + "field": "event.data.mail", + "leftBracket": 0.0, + "logicOp": "and", + "not": false, + "rightBracket": 0.0, + "value": [ + "anna.pullman@demo.local", + "ryan.smith@demo.local" + ] + } + ] + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"asset_type\":\"groups\",\"event\":{\"data\":{\"first_seen\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"roles\":[{\"display_name\":\"SharePoint Administrator\",\"remote_id\":\"c9d7cac3e29346d3a4c259303bab613f\"}],\"user_count_link\":[{\"bracketWeight\":0,\"compOp\":\"IN\",\"field\":\"event.data.mail\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":[\"anna.pullman@demo.local\",\"ryan.smith@demo.local\"]}]}}}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "{0=anna.pullman@demo.local, 1=ryan.smith@demo.local}" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:09:21.000Z", + "axonius": { + "identity": { + "adapter_list_length": 1, + "adapters": [ + "okta_adapter" + ], + "asset_type": "groups", + "event": { + "accurate_for_datetime": "2025-12-09T00:09:21.000Z", + "action_if_exists": "update", + "association_type": "Tag", + "entity": "groups", + "hidden_for_gui": true, + "name": "static_analysis_0_UserCountSchema", + "plugin_name": "static_analysis", + "plugin_unique_name": "static_analysis_0_UserCountSchema", + "type": "adapterdata" + }, + "internal_axon_id": "f2230d26ee294785fa7cedb347ee946b", + "transform_unique_id": "Yi7EQyFEUxTvkDiRCrt/u91p7u8=", + "user_count": 633, + "user_count_link": [ + { + "bracketWeight": 0.0, + "compOp": "equals", + "field": "specific_data.data.nested_groups.name", + "leftBracket": 0.0, + "not": false, + "rightBracket": 0.0, + "value": "Server-Operators" + } + ] + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "action": "update", + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"okta_adapter\"],\"asset_type\":\"groups\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:09:21 GMT\",\"action_if_exists\":\"update\",\"associated_adapters\":[],\"association_type\":\"Tag\",\"data\":{\"user_count\":633,\"user_count_link\":[{\"bracketWeight\":0,\"compOp\":\"equals\",\"field\":\"specific_data.data.nested_groups.name\",\"leftBracket\":0,\"logicOp\":\"\",\"not\":false,\"rightBracket\":0,\"value\":\"Server-Operators\"}]},\"entity\":\"groups\",\"hidden_for_gui\":true,\"name\":\"static_analysis_0_UserCountSchema\",\"plugin_name\":\"static_analysis\",\"plugin_unique_name\":\"static_analysis_0_UserCountSchema\",\"type\":\"adapterdata\"},\"internal_axon_id\":\"f2230d26ee294785fa7cedb347ee946b\"}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "Server-Operators" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:02:40.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:02:40.000Z", + "adapter_list_length": 1, + "adapters": [ + "service_now_adapter" + ], + "application_and_account_name": "servicenow/servicenow-dev", + "asset_type": "groups", + "display_name": "Server", + "event": { + "accurate_for_datetime": "2025-12-09T00:02:40.000Z", + "adapter_categories": [ + "CMDB", + "ITAM/ITSM", + "Ticketing", + "SaaS Management" + ], + "client_used": "67fd0999fe1c8e812a176ba2", + "initial_plugin_unique_name": "service_now_adapter_0", + "plugin_name": "service_now_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "service_now_adapter_0", + "quick_id": "service_now_adapter_0!ee198fc2ad0d8e96d043", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T00:02:34.000Z", + "first_fetch_time": "2025-04-14T13:27:11.000Z", + "first_seen": "2023-06-14T10:54:10.000Z", + "from_last_fetch": true, + "groups": [ + { + "display_name": "sales-group", + "name": "sales-group", + "remote_id": "c0d5e3a8" + } + ], + "has_administrative_permissions": false, + "id": "ee198fc2ad0d8e96d043", + "id_raw": "ba25c305-3dbd-4389-a778-a2b24ac9c021", + "internal_axon_id": "2285fd71b4ec307bb4054a9dc1c5aefb", + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd0999fe1c8e812a176ba2", + "last_fetch_connection_label": "servicenow-dev", + "name": "Server", + "nested_grants_last_updated": "2025-12-09T00:09:42.000Z", + "nested_groups": [ + { + "assignment_type": "Direct", + "group_name": "sales-group", + "name": "sales-group", + "value": "c0d5e3a8" + }, + { + "assignment_type": "Indirect", + "group_name": "test-group", + "name": "test-group", + "parents": [ + { + "name": "sales-group", + "parent_type": "Group", + "value": "c0d5e3a8" + } + ], + "value": "36ef470c" + }, + { + "assignment_type": "Indirect", + "group_name": "guests", + "name": "guests", + "parents": [ + { + "name": "sales-group", + "parent_type": "Group", + "value": "c0d5e3a8" + } + ], + "value": "aed2627e" + }, + { + "assignment_type": "Indirect", + "group_name": "Server Operators", + "name": "Server Operators", + "parents": [ + { + "name": "sales-group", + "parent_type": "Group", + "value": "c0d5e3a8" + } + ], + "value": "9ba02066" + }, + { + "assignment_type": "Indirect", + "group_name": "r\\u0026d testing", + "name": "r\\u0026d testing", + "parents": [ + { + "name": "sales-group", + "parent_type": "Group", + "value": "c0d5e3a8" + } + ], + "value": "714c4251" + }, + { + "assignment_type": "Indirect", + "group_name": "lab production", + "name": "lab production", + "parents": [ + { + "name": "sales-group", + "parent_type": "Group", + "value": "c0d5e3a8" + } + ], + "value": "26205d34" + } + ], + "nested_roles": [ + { + "assignment_type": "Direct", + "name": "sn_dependentclient.dependentclient_user", + "value": "38ceb28b936146f7acbb7438b54283fc" + }, + { + "assignment_type": "Direct", + "name": "log_points", + "value": "8d0a58c77a064bfd94e8ca3ee1cab36f" + }, + { + "assignment_type": "Direct", + "name": "decision_table_admin", + "value": "5ad93ef2d50c4fa997a72d4903271a87" + }, + { + "assignment_type": "Indirect", + "name": "catalog_template_editor", + "parents": [ + { + "name": "sales-group", + "parent_type": "Group", + "value": "c0d5e3a8" + } + ], + "value": "340f6b1c033f49a9b3e2b0895f382329" + }, + { + "assignment_type": "Indirect", + "name": "export_rest_api", + "parents": [ + { + "name": "sales-group", + "parent_type": "Group", + "value": "c0d5e3a8" + } + ], + "value": "c8e6c40ab18149aea54c3b5da4b1d849" + } + ], + "not_fetched_count": 0, + "remote_id": "95c933af", + "roles": [ + { + "display_name": "sn_dependentclient.dependentclient_user", + "remote_id": "38ceb28b936146f7acbb7438b54283fc" + }, + { + "display_name": "log_points", + "remote_id": "8d0a58c77a064bfd94e8ca3ee1cab36f" + }, + { + "display_name": "decision_table_admin", + "remote_id": "5ad93ef2d50c4fa997a72d4903271a87" + } + ], + "sm_entity_type": "group", + "source_application": "ServiceNow", + "tenant_number": [ + 2 + ], + "transform_unique_id": "+cCrMPCIJIKUJ2cPA1kZ8wH3Le4=", + "type": "Groups" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"service_now_adapter\"],\"asset_type\":\"groups\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:02:40 GMT\",\"adapter_categories\":[\"CMDB\",\"ITAM/ITSM\",\"Ticketing\",\"SaaS Management\"],\"client_used\":\"67fd0999fe1c8e812a176ba2\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:02:40 GMT\",\"application_and_account_name\":\"servicenow/servicenow-dev\",\"display_name\":\"Server\",\"fetch_time\":\"Tue, 09 Dec 2025 00:02:34 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:27:11 GMT\",\"first_seen\":\"Wed, 14 Jun 2023 10:54:10 GMT\",\"from_last_fetch\":true,\"groups\":[{\"display_name\":\"sales-group\",\"name\":\"sales-group\",\"remote_id\":\"c0d5e3a8\"}],\"has_administrative_permissions\":false,\"id\":\"ee198fc2ad0d8e96d043\",\"id_raw\":\"ba25c305-3dbd-4389-a778-a2b24ac9c021\",\"is_fetched_from_adapter\":true,\"last_fetch_connection_id\":\"67fd0999fe1c8e812a176ba2\",\"last_fetch_connection_label\":\"servicenow-dev\",\"name\":\"Server\",\"nested_applications\":[],\"nested_associated_devices\":[],\"nested_grants_last_updated\":\"Tue, 09 Dec 2025 00:09:42 GMT\",\"nested_groups\":[{\"assignment_type\":\"Direct\",\"group_name\":\"sales-group\",\"name\":\"sales-group\",\"parents\":[{\"name\":\"\",\"value\":\"\"}],\"value\":\"c0d5e3a8\"},{\"assignment_type\":\"Indirect\",\"group_name\":\"test-group\",\"name\":\"test-group\",\"parents\":[{\"name\":\"sales-group\",\"parent_type\":\"Group\",\"value\":\"c0d5e3a8\"}],\"value\":\"36ef470c\"},{\"assignment_type\":\"Indirect\",\"group_name\":\"guests\",\"name\":\"guests\",\"parents\":[{\"name\":\"sales-group\",\"parent_type\":\"Group\",\"value\":\"c0d5e3a8\"}],\"value\":\"aed2627e\"},{\"assignment_type\":\"Indirect\",\"group_name\":\"Server Operators\",\"name\":\"Server Operators\",\"parents\":[{\"name\":\"sales-group\",\"parent_type\":\"Group\",\"value\":\"c0d5e3a8\"}],\"value\":\"9ba02066\"},{\"assignment_type\":\"Indirect\",\"group_name\":\"r\\\\u0026d testing\",\"name\":\"r\\\\u0026d testing\",\"parents\":[{\"name\":\"sales-group\",\"parent_type\":\"Group\",\"value\":\"c0d5e3a8\"}],\"value\":\"714c4251\"},{\"assignment_type\":\"Indirect\",\"group_name\":\"lab production\",\"name\":\"lab production\",\"parents\":[{\"name\":\"sales-group\",\"parent_type\":\"Group\",\"value\":\"c0d5e3a8\"}],\"value\":\"26205d34\"}],\"nested_permissions\":[],\"nested_resources\":[],\"nested_roles\":[{\"assignment_type\":\"Direct\",\"name\":\"sn_dependentclient.dependentclient_user\",\"parents\":[{\"name\":\"\",\"value\":\"\"}],\"value\":\"38ceb28b936146f7acbb7438b54283fc\"},{\"assignment_type\":\"Direct\",\"name\":\"log_points\",\"parents\":[{\"name\":\"\",\"value\":\"\"}],\"value\":\"8d0a58c77a064bfd94e8ca3ee1cab36f\"},{\"assignment_type\":\"Direct\",\"name\":\"decision_table_admin\",\"parents\":[{\"name\":\"\",\"value\":\"\"}],\"value\":\"5ad93ef2d50c4fa997a72d4903271a87\"},{\"assignment_type\":\"Indirect\",\"name\":\"catalog_template_editor\",\"parents\":[{\"name\":\"sales-group\",\"parent_type\":\"Group\",\"value\":\"c0d5e3a8\"}],\"value\":\"340f6b1c033f49a9b3e2b0895f382329\"},{\"assignment_type\":\"Indirect\",\"name\":\"export_rest_api\",\"parents\":[{\"name\":\"sales-group\",\"parent_type\":\"Group\",\"value\":\"c0d5e3a8\"}],\"value\":\"c8e6c40ab18149aea54c3b5da4b1d849\"}],\"not_fetched_count\":0,\"permissions\":[],\"remote_id\":\"95c933af\",\"roles\":[{\"display_name\":\"sn_dependentclient.dependentclient_user\",\"remote_id\":\"38ceb28b936146f7acbb7438b54283fc\"},{\"display_name\":\"log_points\",\"remote_id\":\"8d0a58c77a064bfd94e8ca3ee1cab36f\"},{\"display_name\":\"decision_table_admin\",\"remote_id\":\"5ad93ef2d50c4fa997a72d4903271a87\"}],\"sm_entity_type\":\"group\",\"software_cves\":[],\"source_application\":\"ServiceNow\",\"tenant_number\":[\"2\"],\"type\":\"Groups\"},\"initial_plugin_unique_name\":\"service_now_adapter_0\",\"plugin_name\":\"service_now_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"service_now_adapter_0\",\"quick_id\":\"service_now_adapter_0!ee198fc2ad0d8e96d043\",\"type\":\"entitydata\"},\"internal_axon_id\":\"2285fd71b4ec307bb4054a9dc1c5aefb\"}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "Server" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ], + "user": { + "name": "Server" + } + } + ] +} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-identity-common.log b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-identity-common.log new file mode 100644 index 00000000000..363812a472c --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-identity-common.log @@ -0,0 +1,5 @@ +{"asset_type":"permissions","adapter_list_length":1,"adapters":["axonius_findings_adapter"],"internal_axon_id":"386843f6309d91e89a8e50ffc2e003cf","event":{"accurate_for_datetime":"Wed, 12 Nov 2025 00:02:19 GMT","action_if_exists":"update","adapter_categories":["Cloud Infra","Containers","Virtualization"],"association_type":"Tag","client_used":"67fd09bdfe1c8e812a176bbd","data":{"accurate_for_datetime":"Wed, 12 Nov 2025 00:02:19 GMT","application_and_account_name":"chef\/chef-demo","display_name":"William Mcallister","fetch_time":"Wed, 12 Nov 2025 00:02:18 GMT","first_fetch_time":"Wed, 12 Nov 2025 00:02:18 GMT","from_last_fetch":true,"groups":[{"display_name":"developers-group","name":"developers-group","remote_id":"a3e70162"}],"has_administrative_permissions":true,"id":"esx-monitor1871068-stg.healthcare-subsidiary.com","is_admin":false,"is_fetched_from_adapter":true,"is_managed_by_sso":false,"last_fetch_connection_id":"67fd09bdfe1c8e812a176bbd","last_fetch_connection_label":"chef-demo","nested_applications":[{"active_from_direct_adapter":true,"app_accounts":[{"name":"aws-demo"}],"app_display_name":"aws-demo","app_links":["https:\/\/impl.workday.com\/mytest"],"assignment_type":"Direct","extension_type":"User Consent","has_administrative_permissions":true,"is_deleted":true,"is_from_direct_adapter":true,"is_managed":true,"is_suspended":true,"is_unmanaged_extension":true,"is_user_external":true,"is_user_paid":false,"last_access":"Sat, 12 Apr 2025 22:59:20 GMT","last_access_count":7,"last_access_count_60_days":37,"last_access_count_90_days":67,"name":"AWS","parents":[{"name":"name1","value":"value1"}],"permissions":[{"name":"User.Read"}],"relation_direct_name":"AWS","relation_discovery_name":"relation_discovery_name","relation_extension_name":"Me\u0107kano","relation_sso_name":"Office365","source_application":"AWS","value":"AWS_67fd09ab731ccb57309230fc","vendor_category":"Other"}],"nested_associated_devices":[],"nested_grants_last_updated":"Sat, 12 Apr 2025 22:59:20 GMT","nested_groups":[{"assignment_type":"Direct","group_name":"developers-group","name":"Users","parents":[{"name":"sample_name","parent_type":"Group","value":"parents_value"}],"value":"aa8ceb7e"}],"nested_permissions":[{"assignment_type":"Direct","parents":[{"name":"parent_name","parent_type":"Group","value":"sample_value"}],"value":"User.Read"}],"nested_resources":[{"assignment_type":"Direct","name":"demo\/ml","parents":[{"name":"name2","value":"value2"}],"value":"https://axonius2-dev-ed.develop.my.salesforce.com/138/"}],"nested_roles":[{"assignment_type":"Indirect","name":"Application Administrator","parents":[{"name":"developers-group","parent_type":"Group","value":"a3e70162"}],"value":"a90eb3906d8a426d958ef805a006cdf7"}],"not_fetched_count":0,"permissions":[{"name":"User.Read"}],"pretty_id":"AX-2032710998","sm_entity_type":"saas_application","source_application":"Chef","tenant_number":["3"],"id_raw":"axonius_catalog_adapter!Zoom","name":"monitor1871068-stg","remote_id":"02A10b5B981B277a2FAd","type":"SaasApplications","operational_users_count":1,"total_users_count":1,"asset_type":"Endpoint","aws_arn":"arn:aws:iam::601111735606:role\/service-role\/aws-opsworks-cm-ec2-role","is_built_in":"true","is_privileged":"false"},"entity":"devices","hidden_for_gui":true,"initial_plugin_unique_name":"chef_adapter_0","name":"cisa_enrichment_0","plugin_name":"chef_adapter","plugin_type":"Adapter","plugin_unique_name":"chef_adapter_0","quick_id":"chef_adapter_0!esx-monitor1871068-stg.healthcare-subsidiary.com","type":"entitydata"}} +{"adapter_list_length":3,"adapters":["okta_adapter","oracle_cloud_adapter","workday_adapter"],"asset_type":"job_titles","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:02:17 GMT","adapter_categories":["Cloud Infra"],"client_used":"67fd09f2731ccb5730923116","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:02:17 GMT","application_and_account_name":"oracle cloud/oracle_cloud-demo","fetch_time":"Tue, 09 Dec 2025 00:02:13 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:27:16 GMT","from_last_fetch":true,"id":"db11e404b96303faedfe","id_raw":"67fd09f2731ccb5730923116_Sales Senior Director","is_fetched_from_adapter":true,"last_fetch_connection_id":"67fd09f2731ccb5730923116","last_fetch_connection_label":"oracle_cloud-demo","name":"Sales Senior Director","not_fetched_count":0,"operational_users_count":1,"source_application":"Oracle Cloud","total_users_count":1,"type":"JobTitles"},"initial_plugin_unique_name":"oracle_cloud_adapter_0","plugin_name":"oracle_cloud_adapter","plugin_type":"Adapter","plugin_unique_name":"oracle_cloud_adapter_0","quick_id":"oracle_cloud_adapter_0!db11e404b96303faedfe","type":"entitydata"},"internal_axon_id":"c6827a92611637e89be05a32e682c95f"} +{"adapter_list_length":1,"adapters":["service_now_adapter"],"asset_type":"organizational_units","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:03:29 GMT","adapter_categories":["CMDB","ITAM/ITSM","Ticketing","SaaS Management"],"client_used":"67fd09aa731ccb57309230f8","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:03:29 GMT","application_and_account_name":"servicenow/servicenow-prod","fetch_time":"Tue, 09 Dec 2025 00:03:24 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:28:25 GMT","from_last_fetch":true,"id":"20e6ecda6ecdf2a0cf90","id_raw":"67fd09aa731ccb57309230f8_Finance","is_fetched_from_adapter":true,"last_fetch_connection_id":"67fd09aa731ccb57309230f8","last_fetch_connection_label":"servicenow-prod","name":"Finance","not_fetched_count":0,"operational_users_count":86,"source_application":"ServiceNow","total_users_count":86,"type":"OrganizationalUnits"},"initial_plugin_unique_name":"service_now_adapter_0","plugin_name":"service_now_adapter","plugin_type":"Adapter","plugin_unique_name":"service_now_adapter_0","quick_id":"service_now_adapter_0!20e6ecda6ecdf2a0cf90","type":"entitydata"},"internal_axon_id":"365c146be1ec21a38d84252aef88ebef"} +{"adapter_list_length":1,"adapters":["azure_ad_adapter"],"asset_type":"permissions","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:01:51 GMT","adapter_categories":["Directory","IAM","SaaS Management"],"client_used":"67fd09bbfe1c8e812a176bb5","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:01:51 GMT","application_and_account_name":"microsoft/azure_ad-demo","fetch_time":"Tue, 09 Dec 2025 00:01:39 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:26:45 GMT","from_last_fetch":true,"id":"13b8aecb3f7bc5e14876","id_raw":"Calendars.Read.Shared","is_admin":true,"is_fetched_from_adapter":true,"last_fetch_connection_id":"67fd09bbfe1c8e812a176bb5","last_fetch_connection_label":"azure_ad-demo","name":"Calendars.Read.Shared","not_fetched_count":0,"source_application":"Microsoft","type":"Permissions"},"initial_plugin_unique_name":"azure_ad_adapter_0","plugin_name":"azure_ad_adapter","plugin_type":"Adapter","plugin_unique_name":"azure_ad_adapter_0","quick_id":"azure_ad_adapter_0!13b8aecb3f7bc5e14876","type":"entitydata"},"internal_axon_id":"d5c35a74659f99ac755bd4c7a43e809d"} +{"adapter_list_length":1,"adapters":["service_now_adapter"],"asset_type":"security_roles","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:03:27 GMT","adapter_categories":["CMDB","ITAM/ITSM","Ticketing","SaaS Management"],"client_used":"67fd09aa731ccb57309230f8","data":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:03:27 GMT","application_and_account_name":"servicenow/servicenow-prod","description":"Allows access to the REST endpoints for Performance Dashboards data.","display_name":"pdb_user","fetch_time":"Tue, 09 Dec 2025 00:03:21 GMT","first_fetch_time":"Mon, 14 Apr 2025 13:28:22 GMT","from_last_fetch":true,"id":"5fd90eca19122eb8384b","id_raw":"900ea76192fd44f7b8ac76334115c835","is_admin":false,"is_fetched_from_adapter":true,"last_fetch_connection_id":"67fd09aa731ccb57309230f8","last_fetch_connection_label":"servicenow-prod","name":"pdb_user","not_fetched_count":0,"permissions":[],"sm_entity_type":"security_role","software_cves":[],"source_application":"ServiceNow","tenant_number":["3"],"type":"SecurityRoles"},"initial_plugin_unique_name":"service_now_adapter_0","plugin_name":"service_now_adapter","plugin_type":"Adapter","plugin_unique_name":"service_now_adapter_0","quick_id":"service_now_adapter_0!5fd90eca19122eb8384b","type":"entitydata"},"internal_axon_id":"30c4b7337f11c79ba953655eab05591e"} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-identity-common.log-expected.json b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-identity-common.log-expected.json new file mode 100644 index 00000000000..8b9b6f4e7ae --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-identity-common.log-expected.json @@ -0,0 +1,466 @@ +{ + "expected": [ + { + "@timestamp": "2025-11-12T00:02:19.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-11-12T00:02:19.000Z", + "adapter_list_length": 1, + "adapters": [ + "axonius_findings_adapter" + ], + "application_and_account_name": "chef/chef-demo", + "asset_type": "permissions", + "aws_arn": "arn:aws:iam::601111735606:role/service-role/aws-opsworks-cm-ec2-role", + "data_asset_type": "Endpoint", + "display_name": "William Mcallister", + "event": { + "accurate_for_datetime": "2025-11-12T00:02:19.000Z", + "action_if_exists": "update", + "adapter_categories": [ + "Cloud Infra", + "Containers", + "Virtualization" + ], + "association_type": "Tag", + "client_used": "67fd09bdfe1c8e812a176bbd", + "entity": "devices", + "hidden_for_gui": true, + "initial_plugin_unique_name": "chef_adapter_0", + "name": "cisa_enrichment_0", + "plugin_name": "chef_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "chef_adapter_0", + "quick_id": "chef_adapter_0!esx-monitor1871068-stg.healthcare-subsidiary.com", + "type": "entitydata" + }, + "fetch_time": "2025-11-12T00:02:18.000Z", + "first_fetch_time": "2025-11-12T00:02:18.000Z", + "from_last_fetch": true, + "groups": [ + { + "display_name": "developers-group", + "name": "developers-group", + "remote_id": "a3e70162" + } + ], + "has_administrative_permissions": true, + "id": "esx-monitor1871068-stg.healthcare-subsidiary.com", + "id_raw": "axonius_catalog_adapter!Zoom", + "internal_axon_id": "386843f6309d91e89a8e50ffc2e003cf", + "is_admin": false, + "is_built_in": true, + "is_fetched_from_adapter": true, + "is_managed_by_sso": false, + "is_privileged": false, + "last_fetch_connection_id": "67fd09bdfe1c8e812a176bbd", + "last_fetch_connection_label": "chef-demo", + "name": "monitor1871068-stg", + "nested_applications": [ + { + "active_from_direct_adapter": true, + "app_accounts": [ + { + "name": "aws-demo" + } + ], + "app_display_name": "aws-demo", + "app_links": [ + "https://impl.workday.com/mytest" + ], + "assignment_type": "Direct", + "extension_type": "User Consent", + "has_administrative_permissions": true, + "is_deleted": true, + "is_from_direct_adapter": true, + "is_managed": true, + "is_suspended": true, + "is_unmanaged_extension": true, + "is_user_external": true, + "is_user_paid": false, + "last_access": "2025-04-12T22:59:20.000Z", + "last_access_count": 7, + "last_access_count_60_days": 37, + "last_access_count_90_days": 67, + "name": "AWS", + "parents": [ + { + "name": "name1", + "value": "value1" + } + ], + "permissions": [ + { + "name": "User.Read" + } + ], + "relation_direct_name": "AWS", + "relation_discovery_name": "relation_discovery_name", + "relation_extension_name": "Mećkano", + "relation_sso_name": "Office365", + "source_application": "AWS", + "value": "AWS_67fd09ab731ccb57309230fc", + "vendor_category": "Other" + } + ], + "nested_grants_last_updated": "2025-04-12T22:59:20.000Z", + "nested_groups": [ + { + "assignment_type": "Direct", + "group_name": "developers-group", + "name": "Users", + "parents": [ + { + "name": "sample_name", + "parent_type": "Group", + "value": "parents_value" + } + ], + "value": "aa8ceb7e" + } + ], + "nested_permissions": [ + { + "assignment_type": "Direct", + "parents": [ + { + "name": "parent_name", + "parent_type": "Group", + "value": "sample_value" + } + ], + "value": "User.Read" + } + ], + "nested_resources": [ + { + "assignment_type": "Direct", + "name": "demo/ml", + "parents": [ + { + "name": "name2", + "value": "value2" + } + ], + "value": "https://axonius2-dev-ed.develop.my.salesforce.com/138/" + } + ], + "nested_roles": [ + { + "assignment_type": "Indirect", + "name": "Application Administrator", + "parents": [ + { + "name": "developers-group", + "parent_type": "Group", + "value": "a3e70162" + } + ], + "value": "a90eb3906d8a426d958ef805a006cdf7" + } + ], + "not_fetched_count": 0, + "operational_users_count": 1, + "permissions_list": [ + { + "name": "User.Read" + } + ], + "pretty_id": "AX-2032710998", + "remote_id": "02A10b5B981B277a2FAd", + "sm_entity_type": "saas_application", + "source_application": "Chef", + "tenant_number": [ + 3 + ], + "total_users_count": 1, + "transform_unique_id": "RY0PNolwfrCUO8b8wPAhSRzV0tg=", + "type": "SaasApplications" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "action": "update", + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"asset_type\":\"permissions\",\"adapter_list_length\":1,\"adapters\":[\"axonius_findings_adapter\"],\"internal_axon_id\":\"386843f6309d91e89a8e50ffc2e003cf\",\"event\":{\"accurate_for_datetime\":\"Wed, 12 Nov 2025 00:02:19 GMT\",\"action_if_exists\":\"update\",\"adapter_categories\":[\"Cloud Infra\",\"Containers\",\"Virtualization\"],\"association_type\":\"Tag\",\"client_used\":\"67fd09bdfe1c8e812a176bbd\",\"data\":{\"accurate_for_datetime\":\"Wed, 12 Nov 2025 00:02:19 GMT\",\"application_and_account_name\":\"chef\\/chef-demo\",\"display_name\":\"William Mcallister\",\"fetch_time\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"first_fetch_time\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"from_last_fetch\":true,\"groups\":[{\"display_name\":\"developers-group\",\"name\":\"developers-group\",\"remote_id\":\"a3e70162\"}],\"has_administrative_permissions\":true,\"id\":\"esx-monitor1871068-stg.healthcare-subsidiary.com\",\"is_admin\":false,\"is_fetched_from_adapter\":true,\"is_managed_by_sso\":false,\"last_fetch_connection_id\":\"67fd09bdfe1c8e812a176bbd\",\"last_fetch_connection_label\":\"chef-demo\",\"nested_applications\":[{\"active_from_direct_adapter\":true,\"app_accounts\":[{\"name\":\"aws-demo\"}],\"app_display_name\":\"aws-demo\",\"app_links\":[\"https:\\/\\/impl.workday.com\\/mytest\"],\"assignment_type\":\"Direct\",\"extension_type\":\"User Consent\",\"has_administrative_permissions\":true,\"is_deleted\":true,\"is_from_direct_adapter\":true,\"is_managed\":true,\"is_suspended\":true,\"is_unmanaged_extension\":true,\"is_user_external\":true,\"is_user_paid\":false,\"last_access\":\"Sat, 12 Apr 2025 22:59:20 GMT\",\"last_access_count\":7,\"last_access_count_60_days\":37,\"last_access_count_90_days\":67,\"name\":\"AWS\",\"parents\":[{\"name\":\"name1\",\"value\":\"value1\"}],\"permissions\":[{\"name\":\"User.Read\"}],\"relation_direct_name\":\"AWS\",\"relation_discovery_name\":\"relation_discovery_name\",\"relation_extension_name\":\"Me\\u0107kano\",\"relation_sso_name\":\"Office365\",\"source_application\":\"AWS\",\"value\":\"AWS_67fd09ab731ccb57309230fc\",\"vendor_category\":\"Other\"}],\"nested_associated_devices\":[],\"nested_grants_last_updated\":\"Sat, 12 Apr 2025 22:59:20 GMT\",\"nested_groups\":[{\"assignment_type\":\"Direct\",\"group_name\":\"developers-group\",\"name\":\"Users\",\"parents\":[{\"name\":\"sample_name\",\"parent_type\":\"Group\",\"value\":\"parents_value\"}],\"value\":\"aa8ceb7e\"}],\"nested_permissions\":[{\"assignment_type\":\"Direct\",\"parents\":[{\"name\":\"parent_name\",\"parent_type\":\"Group\",\"value\":\"sample_value\"}],\"value\":\"User.Read\"}],\"nested_resources\":[{\"assignment_type\":\"Direct\",\"name\":\"demo\\/ml\",\"parents\":[{\"name\":\"name2\",\"value\":\"value2\"}],\"value\":\"https://axonius2-dev-ed.develop.my.salesforce.com/138/\"}],\"nested_roles\":[{\"assignment_type\":\"Indirect\",\"name\":\"Application Administrator\",\"parents\":[{\"name\":\"developers-group\",\"parent_type\":\"Group\",\"value\":\"a3e70162\"}],\"value\":\"a90eb3906d8a426d958ef805a006cdf7\"}],\"not_fetched_count\":0,\"permissions\":[{\"name\":\"User.Read\"}],\"pretty_id\":\"AX-2032710998\",\"sm_entity_type\":\"saas_application\",\"source_application\":\"Chef\",\"tenant_number\":[\"3\"],\"id_raw\":\"axonius_catalog_adapter!Zoom\",\"name\":\"monitor1871068-stg\",\"remote_id\":\"02A10b5B981B277a2FAd\",\"type\":\"SaasApplications\",\"operational_users_count\":1,\"total_users_count\":1,\"asset_type\":\"Endpoint\",\"aws_arn\":\"arn:aws:iam::601111735606:role\\/service-role\\/aws-opsworks-cm-ec2-role\",\"is_built_in\":\"true\",\"is_privileged\":\"false\"},\"entity\":\"devices\",\"hidden_for_gui\":true,\"initial_plugin_unique_name\":\"chef_adapter_0\",\"name\":\"cisa_enrichment_0\",\"plugin_name\":\"chef_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"chef_adapter_0\",\"quick_id\":\"chef_adapter_0!esx-monitor1871068-stg.healthcare-subsidiary.com\",\"type\":\"entitydata\"}}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "William Mcallister" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ], + "user": { + "name": "William Mcallister" + } + }, + { + "@timestamp": "2025-12-09T00:02:17.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:02:17.000Z", + "adapter_list_length": 3, + "adapters": [ + "okta_adapter", + "oracle_cloud_adapter", + "workday_adapter" + ], + "application_and_account_name": "oracle cloud/oracle_cloud-demo", + "asset_type": "job_titles", + "event": { + "accurate_for_datetime": "2025-12-09T00:02:17.000Z", + "adapter_categories": [ + "Cloud Infra" + ], + "client_used": "67fd09f2731ccb5730923116", + "initial_plugin_unique_name": "oracle_cloud_adapter_0", + "plugin_name": "oracle_cloud_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "oracle_cloud_adapter_0", + "quick_id": "oracle_cloud_adapter_0!db11e404b96303faedfe", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T00:02:13.000Z", + "first_fetch_time": "2025-04-14T13:27:16.000Z", + "from_last_fetch": true, + "id": "db11e404b96303faedfe", + "id_raw": "67fd09f2731ccb5730923116_Sales Senior Director", + "internal_axon_id": "c6827a92611637e89be05a32e682c95f", + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09f2731ccb5730923116", + "last_fetch_connection_label": "oracle_cloud-demo", + "name": "Sales Senior Director", + "not_fetched_count": 0, + "operational_users_count": 1, + "source_application": "Oracle Cloud", + "total_users_count": 1, + "transform_unique_id": "Djt3PwVdApFWmEQnn5Cro1ulZtQ=", + "type": "JobTitles" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":3,\"adapters\":[\"okta_adapter\",\"oracle_cloud_adapter\",\"workday_adapter\"],\"asset_type\":\"job_titles\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:02:17 GMT\",\"adapter_categories\":[\"Cloud Infra\"],\"client_used\":\"67fd09f2731ccb5730923116\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:02:17 GMT\",\"application_and_account_name\":\"oracle cloud/oracle_cloud-demo\",\"fetch_time\":\"Tue, 09 Dec 2025 00:02:13 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:27:16 GMT\",\"from_last_fetch\":true,\"id\":\"db11e404b96303faedfe\",\"id_raw\":\"67fd09f2731ccb5730923116_Sales Senior Director\",\"is_fetched_from_adapter\":true,\"last_fetch_connection_id\":\"67fd09f2731ccb5730923116\",\"last_fetch_connection_label\":\"oracle_cloud-demo\",\"name\":\"Sales Senior Director\",\"not_fetched_count\":0,\"operational_users_count\":1,\"source_application\":\"Oracle Cloud\",\"total_users_count\":1,\"type\":\"JobTitles\"},\"initial_plugin_unique_name\":\"oracle_cloud_adapter_0\",\"plugin_name\":\"oracle_cloud_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"oracle_cloud_adapter_0\",\"quick_id\":\"oracle_cloud_adapter_0!db11e404b96303faedfe\",\"type\":\"entitydata\"},\"internal_axon_id\":\"c6827a92611637e89be05a32e682c95f\"}", + "type": [ + "info" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:03:29.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:03:29.000Z", + "adapter_list_length": 1, + "adapters": [ + "service_now_adapter" + ], + "application_and_account_name": "servicenow/servicenow-prod", + "asset_type": "organizational_units", + "event": { + "accurate_for_datetime": "2025-12-09T00:03:29.000Z", + "adapter_categories": [ + "CMDB", + "ITAM/ITSM", + "Ticketing", + "SaaS Management" + ], + "client_used": "67fd09aa731ccb57309230f8", + "initial_plugin_unique_name": "service_now_adapter_0", + "plugin_name": "service_now_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "service_now_adapter_0", + "quick_id": "service_now_adapter_0!20e6ecda6ecdf2a0cf90", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T00:03:24.000Z", + "first_fetch_time": "2025-04-14T13:28:25.000Z", + "from_last_fetch": true, + "id": "20e6ecda6ecdf2a0cf90", + "id_raw": "67fd09aa731ccb57309230f8_Finance", + "internal_axon_id": "365c146be1ec21a38d84252aef88ebef", + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09aa731ccb57309230f8", + "last_fetch_connection_label": "servicenow-prod", + "name": "Finance", + "not_fetched_count": 0, + "operational_users_count": 86, + "source_application": "ServiceNow", + "total_users_count": 86, + "transform_unique_id": "8jB2KnVqyD+UVJcwYda7VulakrE=", + "type": "OrganizationalUnits" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"service_now_adapter\"],\"asset_type\":\"organizational_units\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:03:29 GMT\",\"adapter_categories\":[\"CMDB\",\"ITAM/ITSM\",\"Ticketing\",\"SaaS Management\"],\"client_used\":\"67fd09aa731ccb57309230f8\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:03:29 GMT\",\"application_and_account_name\":\"servicenow/servicenow-prod\",\"fetch_time\":\"Tue, 09 Dec 2025 00:03:24 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:28:25 GMT\",\"from_last_fetch\":true,\"id\":\"20e6ecda6ecdf2a0cf90\",\"id_raw\":\"67fd09aa731ccb57309230f8_Finance\",\"is_fetched_from_adapter\":true,\"last_fetch_connection_id\":\"67fd09aa731ccb57309230f8\",\"last_fetch_connection_label\":\"servicenow-prod\",\"name\":\"Finance\",\"not_fetched_count\":0,\"operational_users_count\":86,\"source_application\":\"ServiceNow\",\"total_users_count\":86,\"type\":\"OrganizationalUnits\"},\"initial_plugin_unique_name\":\"service_now_adapter_0\",\"plugin_name\":\"service_now_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"service_now_adapter_0\",\"quick_id\":\"service_now_adapter_0!20e6ecda6ecdf2a0cf90\",\"type\":\"entitydata\"},\"internal_axon_id\":\"365c146be1ec21a38d84252aef88ebef\"}", + "type": [ + "info" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:01:51.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:01:51.000Z", + "adapter_list_length": 1, + "adapters": [ + "azure_ad_adapter" + ], + "application_and_account_name": "microsoft/azure_ad-demo", + "asset_type": "permissions", + "event": { + "accurate_for_datetime": "2025-12-09T00:01:51.000Z", + "adapter_categories": [ + "Directory", + "IAM", + "SaaS Management" + ], + "client_used": "67fd09bbfe1c8e812a176bb5", + "initial_plugin_unique_name": "azure_ad_adapter_0", + "plugin_name": "azure_ad_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "azure_ad_adapter_0", + "quick_id": "azure_ad_adapter_0!13b8aecb3f7bc5e14876", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T00:01:39.000Z", + "first_fetch_time": "2025-04-14T13:26:45.000Z", + "from_last_fetch": true, + "id": "13b8aecb3f7bc5e14876", + "id_raw": "Calendars.Read.Shared", + "internal_axon_id": "d5c35a74659f99ac755bd4c7a43e809d", + "is_admin": true, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09bbfe1c8e812a176bb5", + "last_fetch_connection_label": "azure_ad-demo", + "name": "Calendars.Read.Shared", + "not_fetched_count": 0, + "source_application": "Microsoft", + "transform_unique_id": "Dem0CCOo74mVb3qAVjwK4Oi1h1E=", + "type": "Permissions" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"azure_ad_adapter\"],\"asset_type\":\"permissions\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:01:51 GMT\",\"adapter_categories\":[\"Directory\",\"IAM\",\"SaaS Management\"],\"client_used\":\"67fd09bbfe1c8e812a176bb5\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:01:51 GMT\",\"application_and_account_name\":\"microsoft/azure_ad-demo\",\"fetch_time\":\"Tue, 09 Dec 2025 00:01:39 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:26:45 GMT\",\"from_last_fetch\":true,\"id\":\"13b8aecb3f7bc5e14876\",\"id_raw\":\"Calendars.Read.Shared\",\"is_admin\":true,\"is_fetched_from_adapter\":true,\"last_fetch_connection_id\":\"67fd09bbfe1c8e812a176bb5\",\"last_fetch_connection_label\":\"azure_ad-demo\",\"name\":\"Calendars.Read.Shared\",\"not_fetched_count\":0,\"source_application\":\"Microsoft\",\"type\":\"Permissions\"},\"initial_plugin_unique_name\":\"azure_ad_adapter_0\",\"plugin_name\":\"azure_ad_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"azure_ad_adapter_0\",\"quick_id\":\"azure_ad_adapter_0!13b8aecb3f7bc5e14876\",\"type\":\"entitydata\"},\"internal_axon_id\":\"d5c35a74659f99ac755bd4c7a43e809d\"}", + "type": [ + "info" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:03:27.000Z", + "axonius": { + "identity": { + "accurate_for_datetime": "2025-12-09T00:03:27.000Z", + "adapter_list_length": 1, + "adapters": [ + "service_now_adapter" + ], + "application_and_account_name": "servicenow/servicenow-prod", + "asset_type": "security_roles", + "description": "Allows access to the REST endpoints for Performance Dashboards data.", + "display_name": "pdb_user", + "event": { + "accurate_for_datetime": "2025-12-09T00:03:27.000Z", + "adapter_categories": [ + "CMDB", + "ITAM/ITSM", + "Ticketing", + "SaaS Management" + ], + "client_used": "67fd09aa731ccb57309230f8", + "initial_plugin_unique_name": "service_now_adapter_0", + "plugin_name": "service_now_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "service_now_adapter_0", + "quick_id": "service_now_adapter_0!5fd90eca19122eb8384b", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T00:03:21.000Z", + "first_fetch_time": "2025-04-14T13:28:22.000Z", + "from_last_fetch": true, + "id": "5fd90eca19122eb8384b", + "id_raw": "900ea76192fd44f7b8ac76334115c835", + "internal_axon_id": "30c4b7337f11c79ba953655eab05591e", + "is_admin": false, + "is_fetched_from_adapter": true, + "last_fetch_connection_id": "67fd09aa731ccb57309230f8", + "last_fetch_connection_label": "servicenow-prod", + "name": "pdb_user", + "not_fetched_count": 0, + "sm_entity_type": "security_role", + "source_application": "ServiceNow", + "tenant_number": [ + 3 + ], + "transform_unique_id": "bi+UrgpCSzpYOMwUWoVDY0OfcMk=", + "type": "SecurityRoles" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":1,\"adapters\":[\"service_now_adapter\"],\"asset_type\":\"security_roles\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:03:27 GMT\",\"adapter_categories\":[\"CMDB\",\"ITAM/ITSM\",\"Ticketing\",\"SaaS Management\"],\"client_used\":\"67fd09aa731ccb57309230f8\",\"data\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:03:27 GMT\",\"application_and_account_name\":\"servicenow/servicenow-prod\",\"description\":\"Allows access to the REST endpoints for Performance Dashboards data.\",\"display_name\":\"pdb_user\",\"fetch_time\":\"Tue, 09 Dec 2025 00:03:21 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:28:22 GMT\",\"from_last_fetch\":true,\"id\":\"5fd90eca19122eb8384b\",\"id_raw\":\"900ea76192fd44f7b8ac76334115c835\",\"is_admin\":false,\"is_fetched_from_adapter\":true,\"last_fetch_connection_id\":\"67fd09aa731ccb57309230f8\",\"last_fetch_connection_label\":\"servicenow-prod\",\"name\":\"pdb_user\",\"not_fetched_count\":0,\"permissions\":[],\"sm_entity_type\":\"security_role\",\"software_cves\":[],\"source_application\":\"ServiceNow\",\"tenant_number\":[\"3\"],\"type\":\"SecurityRoles\"},\"initial_plugin_unique_name\":\"service_now_adapter_0\",\"plugin_name\":\"service_now_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"service_now_adapter_0\",\"quick_id\":\"service_now_adapter_0!5fd90eca19122eb8384b\",\"type\":\"entitydata\"},\"internal_axon_id\":\"30c4b7337f11c79ba953655eab05591e\"}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "pdb_user" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ], + "user": { + "name": "pdb_user" + } + } + ] +} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-user.log b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-user.log new file mode 100644 index 00000000000..bb88f4c4eaa --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-user.log @@ -0,0 +1,3 @@ +{"asset_type":"users","event":{"associated_adapter_plugin_name":"chef_adapter","data":{"account_disabled":false,"active":["sample-active-value"],"admin_roles":[{"display_name":"_SEED_ADMIN_ROLE","id":"89617862826397410"}],"associated_devices":[{"device_associated_saas_apps_names":["sample-app"],"device_caption":["sample-caption"],"device_id":["device-12345"],"device_labels":["label1"],"device_model":["sample-model"],"device_os_distribution":["Ubuntu"],"device_os_edition":["sample-edition"],"device_os_end_of_life":["2030-01-01"],"device_os_type":["Linux"],"device_os_version":["20.04"],"device_preferred_mac_address":["00:0C:29:12:52:47"],"device_serial":["SN12345678"],"internal_axon_id":["internal-axon-1"]}],"associated_employees":[{"internal_axon_id":["ax-employee-1"],"username":["employee.username@demo.local"]}],"associated_groups":[{"display_name":"Users","remote_id":"aa8ceb7e"}],"associated_licenses":[{"adapter_connection_label":["chef-demo"],"internal_axon_id":["lic-123"],"license_name":["Standard License"],"pricing_unit":["per-user"],"related_vendor_name":["VendorName"],"unit_price":["100"]}],"aws_iam_identity_type":"IAM User","azure_account_id":"c85155ee-451d-4839-ac97-b1ecad769374","breaches_data":[{"added_date":"Thu, 06 Dec 2018 19:11:27 GMT","breach_date":"Fri, 06 Dec 2018 19:11:27 GMT","data_classes":["sample-data-class"],"domain":"adapt.io","is_fabricated":false,"is_retired":false,"is_sensitive":false,"is_spam_list":false,"is_verified":true,"logo_path":"https:\/\/haveibeenpwned.com\/Content\/Images\/PwnedLogos\/List.png","modified_date":"Thu, 06 Dec 2018 19:11:27 GMT","name":"YouveBeenScraped","pwn_count":66147869,"title":"You've Been Scraped"}],"class_name":"cmdb_ci_vm","cloud_provider":"Azure","distinct_associated_devices_count":1,"email_activity":{"is_deleted":false,"product_license":"MICROSOFT FABRIC (FREE)+MICROSOFT TEAMS PHONE STANDARD+MICROSOFT DEFENDER FOR OFFICE365 (PLAN 2)+MICROSOFT 365 AUDIO CONFERENCING+ENTERPRISE MOBILITY + SECURITY E3+OFFICE365 E3+MICROSOFT 365 E3 EXTRA FEATURES","read_count":740,"receive_count":5181,"report_date":"Mon, 03 Jun 2024 00:00:00 GMT","report_period":90,"send_count":4677},"email_notification":{"alternative_host_reminder":false,"cancel_meeting_reminder":false,"jbh_reminder":true},"employee_id":"880290","employee_number":"880290","employee_type":"local","feature":{"cn_meeting":false,"in_meeting":false,"large_meeting":false,"meeting_capacity":70,"webinar":true,"zoom_phone":false},"first_name":"William","hire_date":"Wed, 12 Nov 2025 00:02:18 GMT","hr_employment_status":"Employed","in_meeting":{"allow_live_streaming":false,"annotation":false,"attendee_on_hold":true,"auto_saving_chat":false,"breakout_room":false,"chat":true,"closed_caption":true,"co_host":true,"data_center_regions":["AU","CA","CN","DE","HK","IE","IN","LA","NL","SG","TY","US"],"e2e_encryption":true,"entry_exit_chime":true,"far_end_camera_control":true,"feedback":true,"group_hd":true,"non_verbal_feedback":false,"polling":true,"private_chat":false,"record_play_voice":false,"remote_control":false,"remote_support":false,"share_dual_camera":true,"show_meeting_control_toolbar":false,"virtual_background":true,"waiting_room":false,"workplace_by_facebook":true},"internal_is_admin":false,"is_active":true,"is_delegated_admin":false,"is_from_sso_provider":true,"is_latest_last_seen":true,"is_managed_by_application":true,"is_mfa_enforced":false,"is_mfa_enrolled":true,"is_non_editable":false,"is_paid":false,"is_permission_adapter":true,"is_saas_user":true,"is_user_active":true,"is_user_deleted":true,"is_user_external":true,"is_user_inactive":false,"is_user_suspended":true,"last_client_version":"5.10.7.7748(mac)","last_login_attempt":"Wed, 12 Nov 2025 00:02:18 GMT","last_logon":"Wed, 12 Nov 2025 00:02:18 GMT","last_name":"Mcallister","last_password_change":"Wed, 12 Nov 2025 00:02:18 GMT","last_seen":"Wed, 12 Nov 2025 00:02:18 GMT","mail":"william.mcallister@demo.local","manager_id":"yvonne.gordon@demo.local","max_added_date":"Sat, 12 Apr 2025 22:59:20 GMT","max_breach_date":"Sat, 12 Apr 2025 22:59:20 GMT","max_modified_date":"Sat, 12 Apr 2025 22:59:20 GMT","nested_grants_managers_last_updated":"Sat, 12 Apr 2025 22:59:20 GMT","nested_managers":[{"assignment_type":"Direct","parents":[{"name":"pname1","parent_type":"User","value":"pvalue1"}],"value":"yvonne.gordon@demo.local"}],"nested_permissions":[{"has_administrative_permissions":true,"is_admin":true}],"oracle_cloud_cis_incompliant":[{"rule_cis_version":1,"rule_section":"1.11"}],"password_never_expires":false,"password_not_required":false,"pmi":4279269702,"provider_name":"OKTA","provider_type":"OKTA","recovery_question_set":false,"schedule_meeting":{"audio_type":"both","force_pmi_jbh_password":true,"host_video":false,"join_before_host":true,"participants_video":false,"pstn_password_protected":false,"require_password_for_instant_meetings":false,"require_password_for_pmi_meetings":true,"require_password_for_scheduled_meetings":true,"require_password_for_scheduling_new_meetings":false,"use_pmi_for_instant_meetings":true,"use_pmi_for_scheduled_meetings":true},"shirt_size":"M","snow_full_name":"William Mcallister","snow_location":"Seattle","status_changed":"Sun, 13 Apr 2025 00:29:22 GMT","telephony":{"show_international_numbers_link":true,"third_party_audio":true},"timezone":"Asia\/Shanghai","tsp":{"call_out":true,"show_international_numbers_link":true},"u_department":"R&D","u_vip":false,"updated_on":"Sun, 13 Apr 2025 00:29:22 GMT","user_apps":[{"active_from_direct_adapter":true,"app_accounts":[{"name":"aws-demo"}],"app_display_name":"aws-demo","app_id":"AWS_67fd09ab731ccb57309230fc","app_links":["https:\/\/demo.my.salesforce.com?so=00C4G0000005h79"],"app_name":"AWS","extension_type":"User Consent","is_from_direct_adapter":true,"is_managed":true,"is_saas_application":true,"is_unmanaged_extension":true,"is_user_deleted":true,"is_user_external":true,"is_user_paid":false,"is_user_suspended":true,"last_access":"Sun, 13 Apr 2025 00:29:22 GMT","permissions":[{"name":"User.Read"}],"relation_direct_name":"AWS","relation_discovery_name":"relation_discovery_name1","relation_extension_name":"Me\u0107kano","relation_sso_name":"Office365","source_application":"AWS","vendor_category":"Other"}],"user_country":"United States","user_created":"Sun, 13 Apr 2025 00:29:22 GMT","user_department":"R&D","user_factors":[{"created":"Sun, 13 Apr 2025 00:29:22 GMT","factor_status":"ACTIVE","factor_type":"Push Notifications","is_enabled":true,"last_updated":"Sun, 13 Apr 2025 00:29:22 GMT","name":"user_factor_name1","provider":"OKTA","strength":"Strong","vendor_name":"OKTA"}],"user_full_name":"William Mcallister","user_is_password_enabled":false,"user_manager":"yvonne.gordon@demo.local","user_manager_mail":"yvonne.gordon@demo.local","user_pass_last_used":"Sun, 13 Apr 2025 00:29:22 GMT","user_path":"\/","user_permissions":[{"is_admin":true,"name":"User.Read"}],"user_related_resources":[{"id":185980621,"name":"demo\/ml","type":"repository"}],"user_remote_id":"c36808f9-305b-4e92-acfb-dfabfc2f0cb3","user_sid":"william.mcallister@demo.local@demo.local","user_status":"SUSPENDED","user_telephone_number":"-6733","user_title":"R&D Engineer","user_type":"Guest","username":"william.mcallister@demo.local","verified":true}}} +{"adapter_list_length":2,"adapters":["active_directory_adapter","azure_ad_adapter"],"asset_type":"users","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:09:44 GMT","action_if_exists":"update","associated_adapters":[],"association_type":"Tag","data":{"associated_employees":[{"internal_axon_id":["762427561140fc5497b0d9007d922c57"],"username":["jessica.stowell@demo.local"]},{"internal_axon_id":["aeb3a136070d4e24452148cf1ef65d0c"],"username":["omar.caudill@demo.local"]},{"internal_axon_id":["400aaffbddcf2e13e366090dd03b6825"],"username":["donald.bouyer@demo.local"]},{"internal_axon_id":["d3cadc6e64a03bcc4f4be877d5fc8477"],"username":["david.plummer@demo.local"]},{"internal_axon_id":["4c884d940f167756a36c65e7814f5207"],"username":["greg.trujillo@demo.local"]},{"internal_axon_id":["428b88be91c19776323838324dbf94fb"],"username":["guillermo.krause@demo.local"]}]},"entity":"users","hidden_for_gui":true,"name":"static_analysis_0_associated_employees","plugin_name":"static_analysis","plugin_unique_name":"static_analysis_0_associated_employees","type":"adapterdata"},"internal_axon_id":"d6c6f476f36f29a1bd29e99f30a293ac"} +{"adapter_list_length":7,"adapters":["google_mdm_adapter","okta_adapter","service_now_adapter","slack_adapter","tenable_io_adapter","workday_adapter","zoom_adapter"],"asset_type":"users","event":{"accurate_for_datetime":"Tue, 09 Dec 2025 00:09:44 GMT","action_if_exists":"update","associated_adapters":[],"association_type":"Tag","data":{"associated_employees":[{"internal_axon_id":["c0c9e1b4bc17295912be5cc2ce43ff1f"],"username":["clyde.bejaran@demo.local"]}]},"entity":"users","hidden_for_gui":true,"name":"static_analysis_0_associated_employees","plugin_name":"static_analysis","plugin_unique_name":"static_analysis_0_associated_employees","type":"adapterdata"},"internal_axon_id":"e2078572d687b8d1461d7c5878ebddf9"} diff --git a/packages/axonius/data_stream/identity/_dev/test/pipeline/test-user.log-expected.json b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-user.log-expected.json new file mode 100644 index 00000000000..ec2d683f5c7 --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/pipeline/test-user.log-expected.json @@ -0,0 +1,600 @@ +{ + "expected": [ + { + "axonius": { + "identity": { + "account_disabled": false, + "active": [ + "sample-active-value" + ], + "admin_roles": [ + { + "display_name": "_SEED_ADMIN_ROLE", + "id": "89617862826397410" + } + ], + "asset_type": "users", + "associated_devices": [ + { + "device_associated_saas_apps_names": [ + "sample-app" + ], + "device_caption": [ + "sample-caption" + ], + "device_id": [ + "device-12345" + ], + "device_labels": [ + "label1" + ], + "device_model": [ + "sample-model" + ], + "device_os_distribution": [ + "Ubuntu" + ], + "device_os_edition": [ + "sample-edition" + ], + "device_os_end_of_life": [ + "2030-01-01" + ], + "device_os_type": [ + "Linux" + ], + "device_os_version": [ + "20.04" + ], + "device_preferred_mac_address": [ + "00-0C-29-12-52-47" + ], + "device_serial": [ + "SN12345678" + ], + "internal_axon_id": [ + "internal-axon-1" + ] + } + ], + "associated_employees": [ + { + "internal_axon_id": [ + "ax-employee-1" + ], + "username": [ + "employee.username@demo.local" + ] + } + ], + "associated_groups": [ + { + "display_name": "Users", + "remote_id": "aa8ceb7e" + } + ], + "associated_licenses": [ + { + "adapter_connection_label": [ + "chef-demo" + ], + "internal_axon_id": [ + "lic-123" + ], + "license_name": [ + "Standard License" + ], + "pricing_unit": [ + "per-user" + ], + "related_vendor_name": [ + "VendorName" + ], + "unit_price": [ + "100" + ] + } + ], + "aws_iam_identity_type": "IAM User", + "azure_account_id": "c85155ee-451d-4839-ac97-b1ecad769374", + "breaches_data": [ + { + "added_date": "2018-12-06T19:11:27.000Z", + "breach_date": "2018-12-06T19:11:27.000Z", + "data_classes": [ + "sample-data-class" + ], + "domain": "adapt.io", + "is_fabricated": false, + "is_retired": false, + "is_sensitive": false, + "is_spam_list": false, + "is_verified": true, + "logo_path": "https://haveibeenpwned.com/Content/Images/PwnedLogos/List.png", + "modified_date": "2018-12-06T19:11:27.000Z", + "name": "YouveBeenScraped", + "pwn_count": 66147869, + "title": "You've Been Scraped" + } + ], + "class_name": "cmdb_ci_vm", + "cloud_provider": "Azure", + "distinct_associated_devices_count": 1, + "email_activity": { + "is_deleted": false, + "product_license": "MICROSOFT FABRIC (FREE)+MICROSOFT TEAMS PHONE STANDARD+MICROSOFT DEFENDER FOR OFFICE365 (PLAN 2)+MICROSOFT 365 AUDIO CONFERENCING+ENTERPRISE MOBILITY + SECURITY E3+OFFICE365 E3+MICROSOFT 365 E3 EXTRA FEATURES", + "read_count": 740, + "receive_count": 5181, + "report_date": "2024-06-03T00:00:00.000Z", + "report_period": 90, + "send_count": 4677 + }, + "email_notification": { + "alternative_host_reminder": false, + "cancel_meeting_reminder": false, + "jbh_reminder": true + }, + "employee_id": "880290", + "employee_number": "880290", + "employee_type": "local", + "event": { + "associated_adapter_plugin_name": "chef_adapter" + }, + "feature": { + "cn_meeting": false, + "in_meeting": false, + "large_meeting": false, + "meeting_capacity": 70, + "webinar": true, + "zoom_phone": false + }, + "first_name": "William", + "hire_date": "2025-11-12T00:02:18.000Z", + "hr_employment_status": "Employed", + "in_meeting": { + "allow_live_streaming": false, + "annotation": false, + "attendee_on_hold": true, + "auto_saving_chat": false, + "breakout_room": false, + "chat": true, + "closed_caption": true, + "co_host": true, + "data_center_regions": [ + "AU", + "CA", + "CN", + "DE", + "HK", + "IE", + "IN", + "LA", + "NL", + "SG", + "TY", + "US" + ], + "e2e_encryption": true, + "entry_exit_chime": true, + "far_end_camera_control": true, + "feedback": true, + "group_hd": true, + "non_verbal_feedback": false, + "polling": true, + "private_chat": false, + "record_play_voice": false, + "remote_control": false, + "remote_support": false, + "share_dual_camera": true, + "show_meeting_control_toolbar": false, + "virtual_background": true, + "waiting_room": false, + "workplace_by_facebook": true + }, + "internal_is_admin": false, + "is_active": true, + "is_delegated_admin": false, + "is_from_sso_provider": true, + "is_latest_last_seen": true, + "is_managed_by_application": true, + "is_mfa_enforced": false, + "is_mfa_enrolled": true, + "is_non_editable": false, + "is_paid": false, + "is_permission_adapter": true, + "is_saas_user": true, + "is_user_active": true, + "is_user_deleted": true, + "is_user_external": true, + "is_user_inactive": false, + "is_user_suspended": true, + "last_client_version": "5.10.7.7748(mac)", + "last_login_attempt": "2025-11-12T00:02:18.000Z", + "last_logon": "2025-11-12T00:02:18.000Z", + "last_name": "Mcallister", + "last_password_change": "2025-11-12T00:02:18.000Z", + "last_seen": "2025-11-12T00:02:18.000Z", + "mail": "william.mcallister@demo.local", + "manager_id": "yvonne.gordon@demo.local", + "max_added_date": "2025-04-12T22:59:20.000Z", + "max_breach_date": "2025-04-12T22:59:20.000Z", + "max_modified_date": "2025-04-12T22:59:20.000Z", + "nested_grants_managers_last_updated": "2025-04-12T22:59:20.000Z", + "nested_managers": [ + { + "assignment_type": "Direct", + "parents": [ + { + "name": "pname1", + "parent_type": "User", + "value": "pvalue1" + } + ], + "value": "yvonne.gordon@demo.local" + } + ], + "nested_permissions": [ + { + "has_administrative_permissions": true, + "is_admin": true + } + ], + "oracle_cloud_cis_incompliant": [ + { + "rule_cis_version": 1.0, + "rule_section": "1.11" + } + ], + "password_never_expires": false, + "password_not_required": false, + "pmi": "4279269702", + "provider_name": "OKTA", + "provider_type": "OKTA", + "recovery_question_set": false, + "schedule_meeting": { + "audio_type": "both", + "force_pmi_jbh_password": true, + "host_video": false, + "join_before_host": true, + "participants_video": false, + "pstn_password_protected": false, + "require_password_for_instant_meetings": false, + "require_password_for_pmi_meetings": true, + "require_password_for_scheduled_meetings": true, + "require_password_for_scheduling_new_meetings": false, + "use_pmi_for_instant_meetings": true, + "use_pmi_for_scheduled_meetings": true + }, + "shirt_size": "M", + "snow_full_name": "William Mcallister", + "snow_location": "Seattle", + "status_changed": "2025-04-13T00:29:22.000Z", + "telephony": { + "show_international_numbers_link": true, + "third_party_audio": true + }, + "timezone": "Asia/Shanghai", + "transform_unique_id": "pZdP191Co8AJ452qSsO22/77T/c=", + "tsp": { + "call_out": true, + "show_international_numbers_link": true + }, + "u_department": "R&D", + "u_vip": false, + "updated_on": "2025-04-13T00:29:22.000Z", + "user_apps": [ + { + "active_from_direct_adapter": true, + "app_accounts": [ + { + "name": "aws-demo" + } + ], + "app_display_name": "aws-demo", + "app_id": "AWS_67fd09ab731ccb57309230fc", + "app_links": [ + "https://demo.my.salesforce.com?so=00C4G0000005h79" + ], + "app_name": "AWS", + "extension_type": "User Consent", + "is_from_direct_adapter": true, + "is_managed": true, + "is_saas_application": true, + "is_unmanaged_extension": true, + "is_user_deleted": true, + "is_user_external": true, + "is_user_paid": false, + "is_user_suspended": true, + "last_access": "2025-04-13T00:29:22.000Z", + "permissions": [ + { + "name": "User.Read" + } + ], + "relation_direct_name": "AWS", + "relation_discovery_name": "relation_discovery_name1", + "relation_extension_name": "Mećkano", + "relation_sso_name": "Office365", + "source_application": "AWS", + "vendor_category": "Other" + } + ], + "user_country": "United States", + "user_created": "2025-04-13T00:29:22.000Z", + "user_department": "R&D", + "user_factors": [ + { + "created": "2025-04-13T00:29:22.000Z", + "factor_status": "ACTIVE", + "factor_type": "Push Notifications", + "is_enabled": true, + "last_updated": "2025-04-13T00:29:22.000Z", + "name": "user_factor_name1", + "provider": "OKTA", + "strength": "Strong", + "vendor_name": "OKTA" + } + ], + "user_full_name": "William Mcallister", + "user_is_password_enabled": false, + "user_manager": "yvonne.gordon@demo.local", + "user_manager_mail": "yvonne.gordon@demo.local", + "user_pass_last_used": "2025-04-13T00:29:22.000Z", + "user_path": "/", + "user_permissions": [ + { + "is_admin": true, + "name": "User.Read" + } + ], + "user_related_resources": [ + { + "id": "185980621", + "name": "demo/ml", + "type": "repository" + } + ], + "user_remote_id": "c36808f9-305b-4e92-acfb-dfabfc2f0cb3", + "user_sid": "william.mcallister@demo.local@demo.local", + "user_status": "SUSPENDED", + "user_telephone_number": "-6733", + "user_title": "R&D Engineer", + "user_type": "Guest", + "username": "william.mcallister@demo.local", + "verified": true + } + }, + "cloud": { + "account": { + "id": "c85155ee-451d-4839-ac97-b1ecad769374" + }, + "provider": "Azure", + "service": { + "name": "IAM User" + } + }, + "device": { + "id": [ + "device-12345" + ], + "model": { + "name": [ + "sample-model" + ] + }, + "serial_number": [ + "SN12345678" + ] + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "category": [ + "iam" + ], + "created": "2025-04-13T00:29:22.000Z", + "kind": "event", + "original": "{\"asset_type\":\"users\",\"event\":{\"associated_adapter_plugin_name\":\"chef_adapter\",\"data\":{\"account_disabled\":false,\"active\":[\"sample-active-value\"],\"admin_roles\":[{\"display_name\":\"_SEED_ADMIN_ROLE\",\"id\":\"89617862826397410\"}],\"associated_devices\":[{\"device_associated_saas_apps_names\":[\"sample-app\"],\"device_caption\":[\"sample-caption\"],\"device_id\":[\"device-12345\"],\"device_labels\":[\"label1\"],\"device_model\":[\"sample-model\"],\"device_os_distribution\":[\"Ubuntu\"],\"device_os_edition\":[\"sample-edition\"],\"device_os_end_of_life\":[\"2030-01-01\"],\"device_os_type\":[\"Linux\"],\"device_os_version\":[\"20.04\"],\"device_preferred_mac_address\":[\"00:0C:29:12:52:47\"],\"device_serial\":[\"SN12345678\"],\"internal_axon_id\":[\"internal-axon-1\"]}],\"associated_employees\":[{\"internal_axon_id\":[\"ax-employee-1\"],\"username\":[\"employee.username@demo.local\"]}],\"associated_groups\":[{\"display_name\":\"Users\",\"remote_id\":\"aa8ceb7e\"}],\"associated_licenses\":[{\"adapter_connection_label\":[\"chef-demo\"],\"internal_axon_id\":[\"lic-123\"],\"license_name\":[\"Standard License\"],\"pricing_unit\":[\"per-user\"],\"related_vendor_name\":[\"VendorName\"],\"unit_price\":[\"100\"]}],\"aws_iam_identity_type\":\"IAM User\",\"azure_account_id\":\"c85155ee-451d-4839-ac97-b1ecad769374\",\"breaches_data\":[{\"added_date\":\"Thu, 06 Dec 2018 19:11:27 GMT\",\"breach_date\":\"Fri, 06 Dec 2018 19:11:27 GMT\",\"data_classes\":[\"sample-data-class\"],\"domain\":\"adapt.io\",\"is_fabricated\":false,\"is_retired\":false,\"is_sensitive\":false,\"is_spam_list\":false,\"is_verified\":true,\"logo_path\":\"https:\\/\\/haveibeenpwned.com\\/Content\\/Images\\/PwnedLogos\\/List.png\",\"modified_date\":\"Thu, 06 Dec 2018 19:11:27 GMT\",\"name\":\"YouveBeenScraped\",\"pwn_count\":66147869,\"title\":\"You've Been Scraped\"}],\"class_name\":\"cmdb_ci_vm\",\"cloud_provider\":\"Azure\",\"distinct_associated_devices_count\":1,\"email_activity\":{\"is_deleted\":false,\"product_license\":\"MICROSOFT FABRIC (FREE)+MICROSOFT TEAMS PHONE STANDARD+MICROSOFT DEFENDER FOR OFFICE365 (PLAN 2)+MICROSOFT 365 AUDIO CONFERENCING+ENTERPRISE MOBILITY + SECURITY E3+OFFICE365 E3+MICROSOFT 365 E3 EXTRA FEATURES\",\"read_count\":740,\"receive_count\":5181,\"report_date\":\"Mon, 03 Jun 2024 00:00:00 GMT\",\"report_period\":90,\"send_count\":4677},\"email_notification\":{\"alternative_host_reminder\":false,\"cancel_meeting_reminder\":false,\"jbh_reminder\":true},\"employee_id\":\"880290\",\"employee_number\":\"880290\",\"employee_type\":\"local\",\"feature\":{\"cn_meeting\":false,\"in_meeting\":false,\"large_meeting\":false,\"meeting_capacity\":70,\"webinar\":true,\"zoom_phone\":false},\"first_name\":\"William\",\"hire_date\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"hr_employment_status\":\"Employed\",\"in_meeting\":{\"allow_live_streaming\":false,\"annotation\":false,\"attendee_on_hold\":true,\"auto_saving_chat\":false,\"breakout_room\":false,\"chat\":true,\"closed_caption\":true,\"co_host\":true,\"data_center_regions\":[\"AU\",\"CA\",\"CN\",\"DE\",\"HK\",\"IE\",\"IN\",\"LA\",\"NL\",\"SG\",\"TY\",\"US\"],\"e2e_encryption\":true,\"entry_exit_chime\":true,\"far_end_camera_control\":true,\"feedback\":true,\"group_hd\":true,\"non_verbal_feedback\":false,\"polling\":true,\"private_chat\":false,\"record_play_voice\":false,\"remote_control\":false,\"remote_support\":false,\"share_dual_camera\":true,\"show_meeting_control_toolbar\":false,\"virtual_background\":true,\"waiting_room\":false,\"workplace_by_facebook\":true},\"internal_is_admin\":false,\"is_active\":true,\"is_delegated_admin\":false,\"is_from_sso_provider\":true,\"is_latest_last_seen\":true,\"is_managed_by_application\":true,\"is_mfa_enforced\":false,\"is_mfa_enrolled\":true,\"is_non_editable\":false,\"is_paid\":false,\"is_permission_adapter\":true,\"is_saas_user\":true,\"is_user_active\":true,\"is_user_deleted\":true,\"is_user_external\":true,\"is_user_inactive\":false,\"is_user_suspended\":true,\"last_client_version\":\"5.10.7.7748(mac)\",\"last_login_attempt\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"last_logon\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"last_name\":\"Mcallister\",\"last_password_change\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"last_seen\":\"Wed, 12 Nov 2025 00:02:18 GMT\",\"mail\":\"william.mcallister@demo.local\",\"manager_id\":\"yvonne.gordon@demo.local\",\"max_added_date\":\"Sat, 12 Apr 2025 22:59:20 GMT\",\"max_breach_date\":\"Sat, 12 Apr 2025 22:59:20 GMT\",\"max_modified_date\":\"Sat, 12 Apr 2025 22:59:20 GMT\",\"nested_grants_managers_last_updated\":\"Sat, 12 Apr 2025 22:59:20 GMT\",\"nested_managers\":[{\"assignment_type\":\"Direct\",\"parents\":[{\"name\":\"pname1\",\"parent_type\":\"User\",\"value\":\"pvalue1\"}],\"value\":\"yvonne.gordon@demo.local\"}],\"nested_permissions\":[{\"has_administrative_permissions\":true,\"is_admin\":true}],\"oracle_cloud_cis_incompliant\":[{\"rule_cis_version\":1,\"rule_section\":\"1.11\"}],\"password_never_expires\":false,\"password_not_required\":false,\"pmi\":4279269702,\"provider_name\":\"OKTA\",\"provider_type\":\"OKTA\",\"recovery_question_set\":false,\"schedule_meeting\":{\"audio_type\":\"both\",\"force_pmi_jbh_password\":true,\"host_video\":false,\"join_before_host\":true,\"participants_video\":false,\"pstn_password_protected\":false,\"require_password_for_instant_meetings\":false,\"require_password_for_pmi_meetings\":true,\"require_password_for_scheduled_meetings\":true,\"require_password_for_scheduling_new_meetings\":false,\"use_pmi_for_instant_meetings\":true,\"use_pmi_for_scheduled_meetings\":true},\"shirt_size\":\"M\",\"snow_full_name\":\"William Mcallister\",\"snow_location\":\"Seattle\",\"status_changed\":\"Sun, 13 Apr 2025 00:29:22 GMT\",\"telephony\":{\"show_international_numbers_link\":true,\"third_party_audio\":true},\"timezone\":\"Asia\\/Shanghai\",\"tsp\":{\"call_out\":true,\"show_international_numbers_link\":true},\"u_department\":\"R&D\",\"u_vip\":false,\"updated_on\":\"Sun, 13 Apr 2025 00:29:22 GMT\",\"user_apps\":[{\"active_from_direct_adapter\":true,\"app_accounts\":[{\"name\":\"aws-demo\"}],\"app_display_name\":\"aws-demo\",\"app_id\":\"AWS_67fd09ab731ccb57309230fc\",\"app_links\":[\"https:\\/\\/demo.my.salesforce.com?so=00C4G0000005h79\"],\"app_name\":\"AWS\",\"extension_type\":\"User Consent\",\"is_from_direct_adapter\":true,\"is_managed\":true,\"is_saas_application\":true,\"is_unmanaged_extension\":true,\"is_user_deleted\":true,\"is_user_external\":true,\"is_user_paid\":false,\"is_user_suspended\":true,\"last_access\":\"Sun, 13 Apr 2025 00:29:22 GMT\",\"permissions\":[{\"name\":\"User.Read\"}],\"relation_direct_name\":\"AWS\",\"relation_discovery_name\":\"relation_discovery_name1\",\"relation_extension_name\":\"Me\\u0107kano\",\"relation_sso_name\":\"Office365\",\"source_application\":\"AWS\",\"vendor_category\":\"Other\"}],\"user_country\":\"United States\",\"user_created\":\"Sun, 13 Apr 2025 00:29:22 GMT\",\"user_department\":\"R&D\",\"user_factors\":[{\"created\":\"Sun, 13 Apr 2025 00:29:22 GMT\",\"factor_status\":\"ACTIVE\",\"factor_type\":\"Push Notifications\",\"is_enabled\":true,\"last_updated\":\"Sun, 13 Apr 2025 00:29:22 GMT\",\"name\":\"user_factor_name1\",\"provider\":\"OKTA\",\"strength\":\"Strong\",\"vendor_name\":\"OKTA\"}],\"user_full_name\":\"William Mcallister\",\"user_is_password_enabled\":false,\"user_manager\":\"yvonne.gordon@demo.local\",\"user_manager_mail\":\"yvonne.gordon@demo.local\",\"user_pass_last_used\":\"Sun, 13 Apr 2025 00:29:22 GMT\",\"user_path\":\"\\/\",\"user_permissions\":[{\"is_admin\":true,\"name\":\"User.Read\"}],\"user_related_resources\":[{\"id\":185980621,\"name\":\"demo\\/ml\",\"type\":\"repository\"}],\"user_remote_id\":\"c36808f9-305b-4e92-acfb-dfabfc2f0cb3\",\"user_sid\":\"william.mcallister@demo.local@demo.local\",\"user_status\":\"SUSPENDED\",\"user_telephone_number\":\"-6733\",\"user_title\":\"R&D Engineer\",\"user_type\":\"Guest\",\"username\":\"william.mcallister@demo.local\",\"verified\":true}}}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "employee.username@demo.local", + "Users", + "William", + "Mcallister", + "william.mcallister@demo.local", + "yvonne.gordon@demo.local", + "William Mcallister" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ], + "threat": { + "enrichments": { + "indicator": { + "first_seen": [ + "2018-12-06T19:11:27.000Z" + ] + } + } + }, + "user": { + "domain": "demo.local", + "email": "william.mcallister@demo.local", + "full_name": "William Mcallister", + "name": "william.mcallister@demo.local" + } + }, + { + "@timestamp": "2025-12-09T00:09:44.000Z", + "axonius": { + "identity": { + "adapter_list_length": 2, + "adapters": [ + "active_directory_adapter", + "azure_ad_adapter" + ], + "asset_type": "users", + "associated_employees": [ + { + "internal_axon_id": [ + "762427561140fc5497b0d9007d922c57" + ], + "username": [ + "jessica.stowell@demo.local" + ] + }, + { + "internal_axon_id": [ + "aeb3a136070d4e24452148cf1ef65d0c" + ], + "username": [ + "omar.caudill@demo.local" + ] + }, + { + "internal_axon_id": [ + "400aaffbddcf2e13e366090dd03b6825" + ], + "username": [ + "donald.bouyer@demo.local" + ] + }, + { + "internal_axon_id": [ + "d3cadc6e64a03bcc4f4be877d5fc8477" + ], + "username": [ + "david.plummer@demo.local" + ] + }, + { + "internal_axon_id": [ + "4c884d940f167756a36c65e7814f5207" + ], + "username": [ + "greg.trujillo@demo.local" + ] + }, + { + "internal_axon_id": [ + "428b88be91c19776323838324dbf94fb" + ], + "username": [ + "guillermo.krause@demo.local" + ] + } + ], + "event": { + "accurate_for_datetime": "2025-12-09T00:09:44.000Z", + "action_if_exists": "update", + "association_type": "Tag", + "entity": "users", + "hidden_for_gui": true, + "name": "static_analysis_0_associated_employees", + "plugin_name": "static_analysis", + "plugin_unique_name": "static_analysis_0_associated_employees", + "type": "adapterdata" + }, + "internal_axon_id": "d6c6f476f36f29a1bd29e99f30a293ac", + "transform_unique_id": "1G9GgsyNMtN5iGQk4+HZeJxjHWo=" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "action": "update", + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":2,\"adapters\":[\"active_directory_adapter\",\"azure_ad_adapter\"],\"asset_type\":\"users\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:09:44 GMT\",\"action_if_exists\":\"update\",\"associated_adapters\":[],\"association_type\":\"Tag\",\"data\":{\"associated_employees\":[{\"internal_axon_id\":[\"762427561140fc5497b0d9007d922c57\"],\"username\":[\"jessica.stowell@demo.local\"]},{\"internal_axon_id\":[\"aeb3a136070d4e24452148cf1ef65d0c\"],\"username\":[\"omar.caudill@demo.local\"]},{\"internal_axon_id\":[\"400aaffbddcf2e13e366090dd03b6825\"],\"username\":[\"donald.bouyer@demo.local\"]},{\"internal_axon_id\":[\"d3cadc6e64a03bcc4f4be877d5fc8477\"],\"username\":[\"david.plummer@demo.local\"]},{\"internal_axon_id\":[\"4c884d940f167756a36c65e7814f5207\"],\"username\":[\"greg.trujillo@demo.local\"]},{\"internal_axon_id\":[\"428b88be91c19776323838324dbf94fb\"],\"username\":[\"guillermo.krause@demo.local\"]}]},\"entity\":\"users\",\"hidden_for_gui\":true,\"name\":\"static_analysis_0_associated_employees\",\"plugin_name\":\"static_analysis\",\"plugin_unique_name\":\"static_analysis_0_associated_employees\",\"type\":\"adapterdata\"},\"internal_axon_id\":\"d6c6f476f36f29a1bd29e99f30a293ac\"}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "jessica.stowell@demo.local", + "omar.caudill@demo.local", + "donald.bouyer@demo.local", + "david.plummer@demo.local", + "greg.trujillo@demo.local", + "guillermo.krause@demo.local" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2025-12-09T00:09:44.000Z", + "axonius": { + "identity": { + "adapter_list_length": 7, + "adapters": [ + "google_mdm_adapter", + "okta_adapter", + "service_now_adapter", + "slack_adapter", + "tenable_io_adapter", + "workday_adapter", + "zoom_adapter" + ], + "asset_type": "users", + "associated_employees": [ + { + "internal_axon_id": [ + "c0c9e1b4bc17295912be5cc2ce43ff1f" + ], + "username": [ + "clyde.bejaran@demo.local" + ] + } + ], + "event": { + "accurate_for_datetime": "2025-12-09T00:09:44.000Z", + "action_if_exists": "update", + "association_type": "Tag", + "entity": "users", + "hidden_for_gui": true, + "name": "static_analysis_0_associated_employees", + "plugin_name": "static_analysis", + "plugin_unique_name": "static_analysis_0_associated_employees", + "type": "adapterdata" + }, + "internal_axon_id": "e2078572d687b8d1461d7c5878ebddf9", + "transform_unique_id": "Lx92Br6FupBVJnGMm25nSs5SUHw=" + } + }, + "ecs": { + "version": "9.2.0" + }, + "event": { + "action": "update", + "category": [ + "iam" + ], + "kind": "event", + "original": "{\"adapter_list_length\":7,\"adapters\":[\"google_mdm_adapter\",\"okta_adapter\",\"service_now_adapter\",\"slack_adapter\",\"tenable_io_adapter\",\"workday_adapter\",\"zoom_adapter\"],\"asset_type\":\"users\",\"event\":{\"accurate_for_datetime\":\"Tue, 09 Dec 2025 00:09:44 GMT\",\"action_if_exists\":\"update\",\"associated_adapters\":[],\"association_type\":\"Tag\",\"data\":{\"associated_employees\":[{\"internal_axon_id\":[\"c0c9e1b4bc17295912be5cc2ce43ff1f\"],\"username\":[\"clyde.bejaran@demo.local\"]}]},\"entity\":\"users\",\"hidden_for_gui\":true,\"name\":\"static_analysis_0_associated_employees\",\"plugin_name\":\"static_analysis\",\"plugin_unique_name\":\"static_analysis_0_associated_employees\",\"type\":\"adapterdata\"},\"internal_axon_id\":\"e2078572d687b8d1461d7c5878ebddf9\"}", + "type": [ + "info" + ] + }, + "related": { + "user": [ + "clyde.bejaran@demo.local" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + } + ] +} diff --git a/packages/axonius/data_stream/identity/_dev/test/system/test-default-config.yml b/packages/axonius/data_stream/identity/_dev/test/system/test-default-config.yml new file mode 100644 index 00000000000..1119b114726 --- /dev/null +++ b/packages/axonius/data_stream/identity/_dev/test/system/test-default-config.yml @@ -0,0 +1,13 @@ +input: cel +service: axonius +vars: + url: http://{{Hostname}}:{{Port}} + api_key: xxxx + secret_key: xxxx +data_stream: + vars: + preserve_original_event: true + preserve_duplicate_custom_fields: true + batch_size: 2 +assert: + hit_count: 4 diff --git a/packages/axonius/data_stream/identity/agent/stream/cel.yml.hbs b/packages/axonius/data_stream/identity/agent/stream/cel.yml.hbs new file mode 100644 index 00000000000..beca4579dd9 --- /dev/null +++ b/packages/axonius/data_stream/identity/agent/stream/cel.yml.hbs @@ -0,0 +1,140 @@ +config_version: 2 +interval: {{interval}} +resource.tracer: + enabled: {{enable_request_tracer}} + filename: "../../logs/cel/http-request-trace-*.ndjson" + maxbackups: 5 +{{#if proxy_url}} +resource.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +resource.ssl: {{ssl}} +{{/if}} +{{#if http_client_timeout}} +resource.timeout: {{http_client_timeout}} +{{/if}} +resource.url: {{url}} + +state: + api_key: {{api_key}} + secret_key: {{secret_key}} + batch_size: {{batch_size}} + asset_type_list: + {{#each asset_type_list as |asset|}} + - {{asset}} + {{/each}} + +redact: + fields: + - api_key + - secret_key +program: | + ( + state.?worklist.asset_type_list[0].hasValue() ? + state + : + state.with( + { + "worklist": { + "asset_type_list": state.asset_type_list, + }, + } + ) + ).as(state, + state.with( + request( + "POST", + state.url.trim_right("/") + "/api/v2/assets/" + state.worklist.asset_type_list[0] + ).with( + { + "Header": { + "Content-Type": ["application/json"], + "api-key": [state.api_key], + "api-secret": [state.secret_key], + }, + "Body": { + "include_metadata": true, + "page": { + "limit": state.batch_size, + }, + ?"next_page": state.?worklist.?next_page, + "fields": ["specific_data"], + "use_cache_entry": false, + "include_details": false, + }.encode_json(), + } + ).do_request().as(resp, resp.StatusCode == 200 ? + resp.Body.decode_json().as(body, + { + "events": (has(body.assets) && size(body.assets) > 0) ? + body.assets.map(assets, + assets.specific_data.orValue([]).map(d, + { + "message": { + ?"internal_axon_id": assets.?internal_axon_id, + ?"adapters": assets.?adapters, + ?"adapter_list_length": assets.?adapter_list_length, + ?"labels": assets.?labels, + "asset_type": state.worklist.asset_type_list[0], + "event": d, + }.encode_json(), + } + ) + ).flatten() + : + [], + "worklist": { + "asset_type_list": (int(body.?meta.page.number.orValue(0)) < int(body.?meta.page.totalPages.orValue(0))) ? + state.worklist.asset_type_list + : + tail(state.worklist.asset_type_list), + ?"next_page": (int(body.?meta.page.number.orValue(0)) < int(body.?meta.page.totalPages.orValue(0))) ? + body.?meta.next_page + : + optional.none(), + }, + "want_more": int(body.?meta.page.number.orValue(0)) < int(body.?meta.page.totalPages.orValue(0)) || + size(state.worklist.asset_type_list) > 1, + } + ) + : + { + "events": [ + { + "message": { + "error": { + "code": string(resp.StatusCode), + "id": string(resp.Status), + "message": "POST:" + state.url.trim_right("/") + "/api/v2/assets/" + state.worklist.asset_type_list[0] + ( + size(resp.Body) != 0 ? + string(resp.Body) + : + string(resp.Status) + " (" + string(resp.StatusCode) + ")" + ), + "asset_type": state.worklist.asset_type_list[0], + }, + }.encode_json(), + }, + ], + "worklist": { + "asset_type_list": tail(state.worklist.asset_type_list), + }, + "want_more": size(state.worklist.asset_type_list) > 1, + } + ) + ) + ) +tags: +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/axonius/data_stream/identity/elasticsearch/ilm/default_policy.json b/packages/axonius/data_stream/identity/elasticsearch/ilm/default_policy.json new file mode 100644 index 00000000000..a2258ec38f8 --- /dev/null +++ b/packages/axonius/data_stream/identity/elasticsearch/ilm/default_policy.json @@ -0,0 +1,23 @@ +{ + "policy": { + "phases": { + "hot": { + "actions": { + "rollover": { + "max_age": "2d", + "max_size": "50gb" + }, + "set_priority": { + "priority": 100 + } + } + }, + "delete": { + "min_age": "30d", + "actions": { + "delete": {} + } + } + } + } +} diff --git a/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/default.yml b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..2e6a0cc54af --- /dev/null +++ b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,704 @@ +--- +description: Pipeline for processing identity logs. +processors: + - set: + field: ecs.version + tag: set_ecs_version + value: 9.2.0 + - terminate: + description: error message set and no data to process. + tag: terminate_data_collection_error + if: ctx.error?.message != null && ctx.message == null && ctx.event?.original == null + + # remove agentless metadata + - remove: + description: Removes the fields added by Agentless as metadata, as they can collide with ECS fields. + tag: remove_agentless_tags + if: ctx.organization instanceof String && ctx.division instanceof String && ctx.team instanceof String + field: + - organization + - division + - team + ignore_missing: true + - rename: + field: message + tag: rename_message_to_event_original + target_field: event.original + ignore_missing: true + description: Renames the original `message` field to `event.original` to store a copy of the original message. The `event.original` field is not touched if the document already has one; it may happen when Logstash sends the document. + if: ctx.event?.original == null + - remove: + field: message + tag: remove_message + ignore_missing: true + description: The `message` field is no longer required if the document has an `event.original` field. + if: ctx.event?.original != null + - json: + field: event.original + tag: json_event_original + target_field: axonius.identity + # Fingerprint the full raw payload because the Axonius API does not expose a stable entity ID for identity assets; the transform keys on this value for deduplication. + - fingerprint: + fields: + - event.original + tag: fingerprint_event_original + target_field: axonius.identity.transform_unique_id + ignore_missing: true + - set: + tag: set_event_kind + field: event.kind + value: event + - append: + field: event.category + value: iam + tag: category_iam + - append: + field: event.type + value: info + tag: type_info + + - convert: + field: axonius.identity.adapter_list_length + tag: convert_adapter_list_length_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.adapter_list_length + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - script: + tag: script_rename_fields + lang: painless + description: Renames all axonius.identity.event.data.* fields to root level (removes event.data prefix). Prefixes conflicting keys with 'data_' to avoid overwriting custom data-stream-root-level fields. + source: |- + if (ctx.axonius?.identity?.event?.data instanceof Map) { + Map eventData = ctx.axonius.identity.event.data; + for (String key : new ArrayList(eventData.keySet())) { + if (ctx.axonius.identity.containsKey(key)) { + ctx.axonius.identity['data_' + key] = eventData[key]; + } else { + ctx.axonius.identity[key] = eventData[key]; + } + } + ctx.axonius.identity.event.remove('data'); + } + - date: + field: axonius.identity.event.accurate_for_datetime + tag: date_event_accurate_for_datetime + target_field: axonius.identity.event.accurate_for_datetime + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.event?.accurate_for_datetime != null && ctx.axonius.identity.event.accurate_for_datetime != '' + on_failure: + - remove: + field: axonius.identity.event.accurate_for_datetime + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: '@timestamp' + tag: set_@timestamp_from_identity_event_accurate_for_datetime + copy_from: axonius.identity.event.accurate_for_datetime + ignore_empty_value: true + - set: + field: event.action + tag: set_event_action_from_identity_event_action_if_exists + copy_from: axonius.identity.event.action_if_exists + ignore_empty_value: true + - lowercase: + field: event.action + tag: lowercase_event_action + ignore_missing: true + - split: + field: event.action + tag: split_event_action + separator: \s+ + ignore_missing: true + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - join: + field: event.action + tag: join_event_action + separator: '-' + if: ctx.event?.action != null && ctx.event.action != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.accurate_for_datetime + tag: date_accurate_for_datetime + target_field: axonius.identity.accurate_for_datetime + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.accurate_for_datetime != null && ctx.axonius.identity.accurate_for_datetime != '' + on_failure: + - remove: + field: axonius.identity.accurate_for_datetime + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: user.name + tag: set_user_email_from_display_name + copy_from: axonius.identity.display_name + ignore_empty_value: true + - append: + field: related.user + tag: append_display_name_into_related_user + value: '{{{axonius.identity.display_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.display_name != null + - foreach: + field: axonius.identity.tenant_number + tag: foreach_identity_tenant_number + if: ctx.axonius?.identity?.tenant_number instanceof List + processor: + convert: + field: _ingest._value + tag: convert_identity_tenant_number_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.fetch_time + tag: date_fetch_time + target_field: axonius.identity.fetch_time + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.fetch_time != null && ctx.axonius.identity.fetch_time != '' + on_failure: + - remove: + field: axonius.identity.fetch_time + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.first_fetch_time + tag: date_first_fetch_time + target_field: axonius.identity.first_fetch_time + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.first_fetch_time != null && ctx.axonius.identity.first_fetch_time != '' + on_failure: + - remove: + field: axonius.identity.first_fetch_time + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.from_last_fetch + tag: convert_from_last_fetch_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.from_last_fetch + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.has_administrative_permissions + tag: convert_has_administrative_permissions_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.has_administrative_permissions + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.is_admin + tag: convert_is_admin_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.is_admin + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.is_built_in + tag: convert_is_built_in_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.is_built_in + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.is_fetched_from_adapter + tag: convert_is_fetched_from_adapter_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.is_fetched_from_adapter + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.is_managed_by_sso + tag: convert_is_managed_by_sso_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.is_managed_by_sso + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.is_privileged + tag: convert_is_privileged_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.is_privileged + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_active_from_direct_adapter + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.active_from_direct_adapter + tag: convert_nested_applications_active_from_direct_adapter_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.active_from_direct_adapter + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_has_administrative_permissions + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.has_administrative_permissions + tag: convert_nested_applications_has_administrative_permissions_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.has_administrative_permissions + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_is_deleted + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.is_deleted + tag: convert_nested_applications_is_deleted_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_deleted + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_is_from_direct_adapter + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.is_from_direct_adapter + tag: convert_nested_applications_is_from_direct_adapter_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_from_direct_adapter + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_is_managed + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.is_managed + tag: convert_nested_applications_is_managed_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_managed + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_is_suspended + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.is_suspended + tag: convert_nested_applications_is_suspended_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_suspended + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_is_unmanaged_extension + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.is_unmanaged_extension + tag: convert_nested_applications_is_unmanaged_extension_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_unmanaged_extension + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_is_user_external + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.is_user_external + tag: convert_nested_applications_is_user_external_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_user_external + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_is_user_paid + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.is_user_paid + tag: convert_nested_applications_is_user_paid_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_user_paid + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_last_access + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + date: + field: _ingest._value.last_access + tag: date_nested_applications_last_access + target_field: _ingest._value.last_access + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + on_failure: + - remove: + field: _ingest._value.last_access + ignore_missing: true + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_last_access_count + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.last_access_count + tag: convert_nested_applications_last_access_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.last_access_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_last_access_count_60_days + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.last_access_count_60_days + tag: convert_nested_applications_last_access_count_60_days_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.last_access_count_60_days + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_applications + tag: foreach_nested_applications_last_access_count_90_days + if: ctx.axonius?.identity?.nested_applications instanceof List + processor: + convert: + field: _ingest._value.last_access_count_90_days + tag: convert_nested_applications_last_access_count_90_days_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.last_access_count_90_days + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.nested_grants_last_updated + tag: date_nested_grants_last_updated + target_field: axonius.identity.nested_grants_last_updated + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.nested_grants_last_updated != null && ctx.axonius.identity.nested_grants_last_updated != '' + on_failure: + - remove: + field: axonius.identity.nested_grants_last_updated + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_resources + tag: foreach_nested_resources_value + if: ctx.axonius?.identity?.nested_resources instanceof List + processor: + convert: + field: _ingest._value.value + tag: convert_nested_resources_value_to_keyword + type: string + ignore_missing: true + - convert: + field: axonius.identity.not_fetched_count + tag: convert_not_fetched_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.not_fetched_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.operational_users_count + tag: convert_operational_users_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.operational_users_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.total_users_count + tag: convert_total_users_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.total_users_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.event.hidden_for_gui + tag: convert_event_hidden_for_gui_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.event.hidden_for_gui + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.permissions + tag: convert_permissions_to_long + type: long + ignore_missing: true + on_failure: + - rename: + field: axonius.identity.permissions + target_field: axonius.identity.permissions_list + ignore_missing: true + - pipeline: + name: '{{ IngestPipeline "pipeline-account" }}' + tag: pipeline-account + if: >- + ctx.axonius?.identity?.asset_type.contains('accounts') + - pipeline: + name: '{{ IngestPipeline "pipeline-group" }}' + tag: pipeline-group + if: >- + ctx.axonius?.identity?.asset_type.contains('groups') + - pipeline: + name: '{{ IngestPipeline "pipeline-certificate" }}' + tag: pipeline-certificate + if: >- + ctx.axonius?.identity?.asset_type.contains('certificates') + - pipeline: + name: '{{ IngestPipeline "pipeline-user" }}' + tag: pipeline-user + if: >- + ctx.axonius?.identity?.asset_type.contains('users') + - remove: + field: + - axonius.identity.event.accurate_for_datetime + - axonius.identity.display_name + - axonius.identity.application_id + - axonius.identity.application_name + - axonius.identity.created_date + - axonius.identity.email + - axonius.identity.begins_on + - axonius.identity.expires_on + - axonius.identity.issuer.common_name + - axonius.identity.issuer.country_name + - axonius.identity.issuer.organization + - axonius.identity.serial_number + - axonius.identity.subject.common_name + - axonius.identity.subject.country_name + - axonius.identity.subject.locality + - axonius.identity.subject.organization + - axonius.identity.subject.state + - axonius.identity.aws_iam_identity_type + - axonius.identity.azure_account_id + - axonius.identity.cloud_provider + - axonius.identity.mail + - axonius.identity.user_created + - axonius.identity.user_full_name + - axonius.identity.username + tag: remove_custom_duplicate_fields + ignore_missing: true + if: ctx.tags == null || !ctx.tags.contains('preserve_duplicate_custom_fields') + - remove: + field: json + tag: remove_json + ignore_missing: true + - script: + tag: script_to_drop_null_values + lang: painless + description: This script processor iterates over the whole document to remove fields with null values. + source: |- + void handleMap(Map map) { + map.values().removeIf(v -> { + if (v instanceof Map) { + handleMap(v); + } else if (v instanceof List) { + handleList(v); + } + return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0) + }); + } + void handleList(List list) { + list.removeIf(v -> { + if (v instanceof Map) { + handleMap(v); + } else if (v instanceof List) { + handleList(v); + } + return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0) + }); + } + handleMap(ctx); + - set: + field: event.kind + tag: set_pipeline_error_into_event_kind + value: pipeline_error + if: ctx.error?.message != null + - append: + field: tags + value: preserve_original_event + allow_duplicates: false + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: |- + Processor '{{{ _ingest.on_failure_processor_type }}}' + {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' + {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}' + - set: + field: event.kind + tag: set_pipeline_error_to_event_kind + value: pipeline_error + - append: + field: tags + value: preserve_original_event + allow_duplicates: false diff --git a/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-account.yml b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-account.yml new file mode 100644 index 00000000000..dc826b00394 --- /dev/null +++ b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-account.yml @@ -0,0 +1,325 @@ +--- +description: Pipeline for processing account logs. +processors: + - rename: + field: axonius.identity.roles + target_field: axonius.identity.roles_accounts + ignore_missing: true + - convert: + field: axonius.identity.active_users + tag: convert_active_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.active_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.admin_non_operational_users + tag: convert_admin_non_operational_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.admin_non_operational_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.admin_operational_active_users + tag: convert_admin_operational_active_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.admin_operational_active_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.admin_operational_inactive_users + tag: convert_admin_operational_inactive_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.admin_operational_inactive_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.admin_operational_users + tag: convert_admin_operational_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.admin_operational_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.admins + tag: convert_admins_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.admins + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: service.id + tag: set_service_id_from_identity_application_id + copy_from: axonius.identity.application_id + ignore_empty_value: true + - set: + field: service.name + tag: set_service_name_from_identity_application_name + copy_from: axonius.identity.application_name + ignore_empty_value: true + - date: + field: axonius.identity.created_date + tag: date_created_date + target_field: axonius.identity.created_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.created_date != null && ctx.axonius.identity.created_date != '' + on_failure: + - remove: + field: axonius.identity.created_date + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.created + tag: set_event_created_from_identity_created_date + copy_from: axonius.identity.created_date + ignore_empty_value: true + - convert: + field: axonius.identity.deleted_users + tag: convert_deleted_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.deleted_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.direct_not_sso_users + tag: convert_direct_not_sso_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.direct_not_sso_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: user.email + tag: set_user_email_from_identity_email + copy_from: axonius.identity.email + ignore_empty_value: true + - dissect: + tag: dissect_user_email + if: ctx.user?.email != null && ctx.user.email.contains('@') + field: user.email + pattern: '%{}@%{user.domain}' + - append: + field: related.user + tag: append_identity_email_into_related_user + value: '{{{axonius.identity.email}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.email != null + - convert: + field: axonius.identity.external_users + tag: convert_external_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.external_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.inactive_users + tag: convert_inactive_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.inactive_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.is_managed_by_direct_app + tag: convert_is_managed_by_direct_app_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.is_managed_by_direct_app + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.last_enrichment_run + tag: date_last_enrichment_run + target_field: axonius.identity.last_enrichment_run + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.last_enrichment_run != null && ctx.axonius.identity.last_enrichment_run != '' + on_failure: + - remove: + field: axonius.identity.last_enrichment_run + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.managed_non_operational_users + tag: convert_managed_non_operational_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.managed_non_operational_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.managed_operational_users + tag: convert_managed_operational_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.managed_operational_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.managed_users + tag: convert_managed_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.managed_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.managed_users_by_app + tag: convert_managed_users_by_app_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.managed_users_by_app + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.managed_users_by_sso + tag: convert_managed_users_by_sso_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.managed_users_by_sso + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.orphaned_users + tag: convert_orphaned_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.orphaned_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.paid_users + tag: convert_paid_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.paid_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.suspended_users + tag: convert_suspended_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.suspended_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.unlinked_users + tag: convert_unlinked_users_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.unlinked_users + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' +on_failure: + - append: + field: error.message + value: |- + Processor '{{{ _ingest.on_failure_processor_type }}}' + {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' + {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}' + - set: + field: event.kind + tag: set_pipeline_error_to_event_kind + value: pipeline_error + - append: + field: tags + value: preserve_original_event + allow_duplicates: false diff --git a/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-certificate.yml b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-certificate.yml new file mode 100644 index 00000000000..22f9729f771 --- /dev/null +++ b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-certificate.yml @@ -0,0 +1,125 @@ +--- +description: Pipeline for processing certificate logs. +processors: + - date: + field: axonius.identity.begins_on + tag: date_begins_on + target_field: axonius.identity.begins_on + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.begins_on != null && ctx.axonius.identity.begins_on != '' + on_failure: + - remove: + field: axonius.identity.begins_on + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.start + tag: set_event_start_from_identity_begins_on + copy_from: axonius.identity.begins_on + ignore_empty_value: true + - convert: + field: axonius.identity.bit_size + tag: convert_bit_size_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.bit_size + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.expires_on + tag: date_expires_on + target_field: axonius.identity.expires_on + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.expires_on != null && ctx.axonius.identity.expires_on != '' + on_failure: + - remove: + field: axonius.identity.expires_on + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.end + tag: set_event_end_from_identity_expires_on + copy_from: axonius.identity.expires_on + ignore_empty_value: true + - append: + field: file.x509.issuer.common_name + tag: append_axonius_identity_issuer_common_name_into_file_x509_issuer_common_name + value: '{{{axonius.identity.issuer.common_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.issuer?.common_name != null + - append: + field: file.x509.issuer.country + tag: append_axonius_identity_issuer_country_name_into_file_x509_issuer_country + value: '{{{axonius.identity.issuer.country_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.issuer?.country_name != null + - append: + field: file.x509.issuer.organization + tag: append_axonius_identity_issuer_organization_into_file_x509_issuer_organization + value: '{{{axonius.identity.issuer.organization}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.issuer?.organization != null + - set: + field: file.x509.serial_number + tag: set_file_x509_serial_number_from_identity_serial_number + copy_from: axonius.identity.serial_number + ignore_empty_value: true + - append: + field: file.x509.subject.common_name + tag: append_axonius_identity_subject_common_name_into_file_x509_subject_common_name + value: '{{{axonius.identity.subject.common_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.subject?.common_name != null + - append: + field: file.x509.subject.country + tag: append_axonius_identity_subject_country_name_into_file_x509_subject_country + value: '{{{axonius.identity.subject.country_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.subject?.country_name != null + - append: + field: file.x509.subject.locality + tag: append_axonius_identity_subject_locality_into_file_x509_subject_locality + value: '{{{axonius.identity.subject.locality}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.subject?.locality != null + - append: + field: file.x509.subject.organization + tag: append_axonius_identity_subject_organization_into_file_x509_subject_organization + value: '{{{axonius.identity.subject.organization}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.subject?.organization != null + - append: + field: file.x509.subject.state_or_province + tag: append_axonius_identity_subject_state_into_file_x509_subject_state_or_province + value: '{{{axonius.identity.subject.state}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.subject?.state != null +on_failure: + - append: + field: error.message + value: |- + Processor '{{{ _ingest.on_failure_processor_type }}}' + {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' + {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}' + - set: + field: event.kind + tag: set_pipeline_error_to_event_kind + value: pipeline_error + - append: + field: tags + value: preserve_original_event + allow_duplicates: false diff --git a/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-group.yml b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-group.yml new file mode 100644 index 00000000000..6133d4ef490 --- /dev/null +++ b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-group.yml @@ -0,0 +1,112 @@ +--- +description: Pipeline for processing group logs. +processors: + - date: + field: axonius.identity.first_seen + tag: date_first_seen + target_field: axonius.identity.first_seen + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.first_seen != null && ctx.axonius.identity.first_seen != '' + on_failure: + - remove: + field: axonius.identity.first_seen + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_count_link + tag: foreach_user_count_link_bracketWeight + if: ctx.axonius?.identity?.user_count_link instanceof List + processor: + convert: + field: _ingest._value.bracketWeight + tag: convert_user_count_link_bracketWeight_to_double + type: double + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.bracketWeight + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_count_link + tag: foreach_user_count_link_leftBracket + if: ctx.axonius?.identity?.user_count_link instanceof List + processor: + convert: + field: _ingest._value.leftBracket + tag: convert_user_count_link_leftBracket_to_double + type: double + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.leftBracket + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_count_link + tag: foreach_user_count_link_not + if: ctx.axonius?.identity?.user_count_link instanceof List + processor: + convert: + field: _ingest._value.not + tag: convert_user_count_link_not_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.not + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_count_link + tag: foreach_user_count_link_rightBracket + if: ctx.axonius?.identity?.user_count_link instanceof List + processor: + convert: + field: _ingest._value.rightBracket + tag: convert_user_count_link_rightBracket_to_double + type: double + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.rightBracket + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_count_link + tag: foreach_user_count_link_value + if: ctx.axonius?.identity?.user_count_link instanceof List + processor: + append: + field: related.user + tag: append_user_count_link_value_into_related_user + value: '{{{_ingest._value.value}}}' + allow_duplicates: false +on_failure: + - append: + field: error.message + value: |- + Processor '{{{ _ingest.on_failure_processor_type }}}' + {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' + {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}' + - set: + field: event.kind + tag: set_pipeline_error_to_event_kind + value: pipeline_error + - append: + field: tags + value: preserve_original_event + allow_duplicates: false diff --git a/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-user.yml b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-user.yml new file mode 100644 index 00000000000..d0fc2065bbc --- /dev/null +++ b/packages/axonius/data_stream/identity/elasticsearch/ingest_pipeline/pipeline-user.yml @@ -0,0 +1,1209 @@ +--- +description: Pipeline for processing user logs. +processors: + - convert: + field: axonius.identity.account_disabled + tag: convert_account_disabled_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.account_disabled + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.associated_devices + tag: foreach_associated_devices_device_id + if: ctx.axonius?.identity?.associated_devices instanceof List + processor: + foreach: + field: _ingest._value.device_id + tag: foreach_associated_devices_device_id + ignore_missing: true + processor: + append: + field: device.id + tag: append_associated_devices_device_id_into_device_id + value: '{{{_ingest._value}}}' + allow_duplicates: false + - foreach: + field: axonius.identity.associated_devices + tag: foreach_associated_devices_device_model + if: ctx.axonius?.identity?.associated_devices instanceof List + processor: + foreach: + field: _ingest._value.device_model + tag: foreach_associated_devices_device_model + ignore_missing: true + processor: + append: + field: device.model.name + tag: append_associated_devices_device_model_into_device_model_name + value: '{{{_ingest._value}}}' + allow_duplicates: false + - foreach: + field: axonius.identity.associated_devices + tag: foreach_associated_devices_device_serial + if: ctx.axonius?.identity?.associated_devices instanceof List + processor: + foreach: + field: _ingest._value.device_serial + tag: foreach_associated_devices_device_serial + ignore_missing: true + processor: + append: + field: device.serial_number + tag: append_associated_devices_device_serial_into_device_serial_number + value: '{{{_ingest._value}}}' + allow_duplicates: false + - foreach: + field: axonius.identity.associated_devices + tag: foreach_associated_devices_device_preferred_mac_address + if: ctx.axonius?.identity?.associated_devices instanceof List + processor: + foreach: + field: _ingest._value.device_preferred_mac_address + tag: foreach_associated_devices_device_preferred_mac_address + ignore_missing: true + processor: + gsub: + field: _ingest._value + tag: gsub_device_preferred_mac_address + pattern: ':' + replacement: '-' + ignore_missing: true + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.associated_devices + tag: foreach_associated_devices_device_preferred_mac_address + if: ctx.axonius?.identity?.associated_devices instanceof List + processor: + foreach: + field: _ingest._value.device_preferred_mac_address + tag: foreach_associated_devices_device_preferred_mac_address + ignore_missing: true + processor: + uppercase: + field: _ingest._value + tag: uppercase_device_preferred_mac_address + ignore_missing: true + - foreach: + field: axonius.identity.associated_employees + tag: foreach_associated_employees_username + if: ctx.axonius?.identity?.associated_employees instanceof List + processor: + foreach: + field: _ingest._value.username + tag: foreach_associated_employees_username + ignore_missing: true + processor: + append: + field: related.user + tag: append_associated_employees_username_into_related_user + value: '{{{_ingest._value}}}' + allow_duplicates: false + - foreach: + field: axonius.identity.associated_groups + tag: foreach_associated_groups_display_name + if: ctx.axonius?.identity?.associated_groups instanceof List + processor: + append: + field: related.user + tag: append_associated_groups_display_name_into_related_user + value: '{{{_ingest._value.display_name}}}' + allow_duplicates: false + - set: + field: cloud.service.name + tag: set_cloud_service_name_from_identity_aws_iam_identity_type + copy_from: axonius.identity.aws_iam_identity_type + ignore_empty_value: true + - set: + field: cloud.account.id + tag: set_cloud_account_id_from_identity_azure_account_id + copy_from: axonius.identity.azure_account_id + ignore_empty_value: true + - foreach: + field: axonius.identity.breaches_data + tag: foreach_breaches_data_added_date + if: ctx.axonius?.identity?.breaches_data instanceof List + processor: + date: + field: _ingest._value.added_date + tag: date_breaches_data_added_date + target_field: _ingest._value.added_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + on_failure: + - remove: + field: _ingest._value.added_date + ignore_missing: true + - foreach: + field: axonius.identity.breaches_data + tag: foreach_breaches_data_added_date + if: ctx.axonius?.identity?.breaches_data instanceof List + processor: + append: + field: threat.enrichments.indicator.first_seen + tag: append_breaches_data_added_date_into_threat_enrichments_indicator_first_seen + value: '{{{_ingest._value.added_date}}}' + allow_duplicates: false + - foreach: + field: axonius.identity.breaches_data + tag: foreach_breaches_data_breach_date + if: ctx.axonius?.identity?.breaches_data instanceof List + processor: + date: + field: _ingest._value.breach_date + tag: date_breaches_data_breach_date + target_field: _ingest._value.breach_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + on_failure: + - remove: + field: _ingest._value.breach_date + ignore_missing: true + - script: + tag: script_convert_breaches_data_booleans + lang: painless + description: Converts all breaches_data boolean fields in each array element, removing values that cannot be converted. + if: ctx.axonius?.identity?.breaches_data instanceof List + source: |- + def fields = ['is_fabricated', 'is_retired', 'is_sensitive', 'is_spam_list', 'is_verified']; + for (def item : ctx.axonius.identity.breaches_data) { + for (def f : fields) { + if (!item.containsKey(f) || item[f] == null) { + continue; + } + def v = item[f]; + if (v instanceof Boolean) { + continue; + } + def s = v.toString().toLowerCase(); + if (s == 'true' || s == '1') { + item[f] = true; + } else if (s == 'false' || s == '0' || s == '') { + item[f] = false; + } else { + item.remove(f); + } + } + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.breaches_data + tag: foreach_breaches_data_modified_date + if: ctx.axonius?.identity?.breaches_data instanceof List + processor: + date: + field: _ingest._value.modified_date + tag: date_breaches_data_modified_date + target_field: _ingest._value.modified_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + on_failure: + - remove: + field: _ingest._value.modified_date + ignore_missing: true + - foreach: + field: axonius.identity.breaches_data + tag: foreach_breaches_data_pwn_count + if: ctx.axonius?.identity?.breaches_data instanceof List + processor: + convert: + field: _ingest._value.pwn_count + tag: convert_breaches_data_pwn_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.pwn_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: cloud.provider + tag: set_cloud_provider_from_identity_cloud_provider + copy_from: axonius.identity.cloud_provider + ignore_empty_value: true + - convert: + field: axonius.identity.distinct_associated_devices_count + tag: convert_distinct_associated_devices_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.distinct_associated_devices_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.email_activity.is_deleted + tag: convert_email_activity_is_deleted_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_activity.is_deleted + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.email_activity.read_count + tag: convert_email_activity_read_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_activity.read_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.email_activity.receive_count + tag: convert_email_activity_receive_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_activity.receive_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.email_activity.report_date + tag: date_email_activity_report_date + target_field: axonius.identity.email_activity.report_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.email_activity?.report_date != null && ctx.axonius.identity.email_activity.report_date != '' + on_failure: + - remove: + field: axonius.identity.email_activity.report_date + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.user_pass_last_used + tag: date_user_pass_last_used + target_field: axonius.identity.user_pass_last_used + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.user_pass_last_used != null && ctx.axonius.identity.user_pass_last_used != '' + on_failure: + - remove: + field: axonius.identity.user_pass_last_used + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + + - convert: + field: axonius.identity.email_activity.report_period + tag: convert_email_activity_report_period_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_activity.report_period + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.email_activity.send_count + tag: convert_email_activity_send_count_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_activity.send_count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.email_notification.alternative_host_reminder + tag: convert_email_notification_alternative_host_reminder_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_notification.alternative_host_reminder + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.email_notification.cancel_meeting_reminder + tag: convert_email_notification_cancel_meeting_reminder_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_notification.cancel_meeting_reminder + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.email_notification.jbh_reminder + tag: convert_email_notification_jbh_reminder_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.email_notification.jbh_reminder + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - script: + tag: script_convert_feature_fields + lang: painless + description: Converts feature boolean and long fields, removing values that cannot be converted. + if: ctx.axonius?.identity?.feature != null + source: |- + def boolFields = ['cn_meeting', 'in_meeting', 'large_meeting', 'webinar', 'zoom_phone']; + def longFields = ['meeting_capacity']; + def obj = ctx.axonius.identity.feature; + for (def f : boolFields) { + if (!obj.containsKey(f) || obj[f] == null) { + continue; + } + def v = obj[f]; + if (v instanceof Boolean) { + continue; + } + def s = v.toString().toLowerCase(); + if (s == 'true' || s == '1') { + obj[f] = true; + } else if (s == 'false' || s == '0' || s == '') { + obj[f] = false; + } else { + obj.remove(f); + } + } + for (def f : longFields) { + if (!obj.containsKey(f) || obj[f] == null) { + continue; + } + def v = obj[f]; + if (v instanceof Long || v instanceof Integer) { + continue; + } + try { + obj[f] = Long.parseLong(v.toString()); + } catch (NumberFormatException e) { + obj.remove(f); + } + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - append: + field: related.user + tag: append_identity_first_name_into_related_user + value: '{{{axonius.identity.first_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.first_name != null + - date: + field: axonius.identity.hire_date + tag: date_hire_date + target_field: axonius.identity.hire_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.hire_date != null && ctx.axonius.identity.hire_date != '' + on_failure: + - remove: + field: axonius.identity.hire_date + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - script: + tag: script_convert_in_meeting_booleans + lang: painless + description: Converts all in_meeting boolean fields, removing values that cannot be converted. + if: ctx.axonius?.identity?.in_meeting != null + source: |- + def fields = [ + 'allow_live_streaming', 'annotation', 'attendee_on_hold', 'auto_saving_chat', + 'breakout_room', 'chat', 'closed_caption', 'co_host', 'e2e_encryption', + 'entry_exit_chime', 'far_end_camera_control', 'feedback', 'group_hd', + 'non_verbal_feedback', 'polling', 'private_chat', 'record_play_voice', + 'remote_control', 'remote_support', 'share_dual_camera', + 'show_meeting_control_toolbar', 'virtual_background', 'waiting_room', + 'workplace_by_facebook' + ]; + def obj = ctx.axonius.identity.in_meeting; + for (def f : fields) { + if (!obj.containsKey(f) || obj[f] == null) { + continue; + } + def v = obj[f]; + if (v instanceof Boolean) { + continue; + } + def s = v.toString().toLowerCase(); + if (s == 'true' || s == '1') { + obj[f] = true; + } else if (s == 'false' || s == '0' || s == '') { + obj[f] = false; + } else { + obj.remove(f); + } + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - script: + tag: script_convert_identity_top_level_booleans + lang: painless + description: Converts top-level identity boolean fields, removing values that cannot be converted. + if: ctx.axonius?.identity != null + source: |- + def fields = [ + 'internal_is_admin', 'is_active', 'is_delegated_admin', 'is_from_sso_provider', + 'is_latest_last_seen', 'is_managed_by_application', 'is_mfa_enforced', + 'is_mfa_enrolled', 'is_non_editable', 'is_paid', 'is_permission_adapter', + 'is_saas_user', 'is_user_active', 'is_user_deleted', 'is_user_external', + 'is_user_inactive', 'is_user_suspended' + ]; + def obj = ctx.axonius.identity; + for (def f : fields) { + if (!obj.containsKey(f) || obj[f] == null) { + continue; + } + def v = obj[f]; + if (v instanceof Boolean) { + continue; + } + def s = v.toString().toLowerCase(); + if (s == 'true' || s == '1') { + obj[f] = true; + } else if (s == 'false' || s == '0' || s == '') { + obj[f] = false; + } else { + obj.remove(f); + } + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.last_login_attempt + tag: date_last_login_attempt + target_field: axonius.identity.last_login_attempt + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.last_login_attempt != null && ctx.axonius.identity.last_login_attempt != '' + on_failure: + - remove: + field: axonius.identity.last_login_attempt + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.last_logon + tag: date_last_logon + target_field: axonius.identity.last_logon + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.last_logon != null && ctx.axonius.identity.last_logon != '' + on_failure: + - remove: + field: axonius.identity.last_logon + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - append: + field: related.user + tag: append_identity_last_name_into_related_user + value: '{{{axonius.identity.last_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.last_name != null + - date: + field: axonius.identity.last_password_change + tag: date_last_password_change + target_field: axonius.identity.last_password_change + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.last_password_change != null && ctx.axonius.identity.last_password_change != '' + on_failure: + - remove: + field: axonius.identity.last_password_change + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.last_seen + tag: date_last_seen + target_field: axonius.identity.last_seen + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.last_seen != null && ctx.axonius.identity.last_seen != '' + on_failure: + - remove: + field: axonius.identity.last_seen + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: user.email + tag: set_user_email_from_identity_mail + copy_from: axonius.identity.mail + ignore_empty_value: true + - dissect: + tag: dissect_user_email + if: ctx.user?.email != null && ctx.user.email.contains('@') + field: user.email + pattern: '%{}@%{user.domain}' + - append: + field: related.user + tag: append_identity_mail_into_related_user + value: '{{{axonius.identity.mail}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.mail != null + - append: + field: related.user + tag: append_identity_manager_id_into_related_user + value: '{{{axonius.identity.manager_id}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.manager_id != null + - date: + field: axonius.identity.max_added_date + tag: date_max_added_date + target_field: axonius.identity.max_added_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.max_added_date != null && ctx.axonius.identity.max_added_date != '' + on_failure: + - remove: + field: axonius.identity.max_added_date + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.max_breach_date + tag: date_max_breach_date + target_field: axonius.identity.max_breach_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.max_breach_date != null && ctx.axonius.identity.max_breach_date != '' + on_failure: + - remove: + field: axonius.identity.max_breach_date + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.max_modified_date + tag: date_max_modified_date + target_field: axonius.identity.max_modified_date + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.max_modified_date != null && ctx.axonius.identity.max_modified_date != '' + on_failure: + - remove: + field: axonius.identity.max_modified_date + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.nested_grants_managers_last_updated + tag: date_nested_grants_managers_last_updated + target_field: axonius.identity.nested_grants_managers_last_updated + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.nested_grants_managers_last_updated != null && ctx.axonius.identity.nested_grants_managers_last_updated != '' + on_failure: + - remove: + field: axonius.identity.nested_grants_managers_last_updated + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_managers + tag: foreach_nested_managers_value + if: ctx.axonius?.identity?.nested_managers instanceof List + processor: + append: + field: related.user + tag: append_nested_managers_value_into_related_user + value: '{{{_ingest._value.value}}}' + allow_duplicates: false + - foreach: + field: axonius.identity.nested_permissions + tag: foreach_nested_permissions_has_administrative_permissions + if: ctx.axonius?.identity?.nested_permissions instanceof List + processor: + convert: + field: _ingest._value.has_administrative_permissions + tag: convert_nested_permissions_has_administrative_permissions_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.has_administrative_permissions + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.nested_permissions + tag: foreach_nested_permissions_is_admin + if: ctx.axonius?.identity?.nested_permissions instanceof List + processor: + convert: + field: _ingest._value.is_admin + tag: convert_nested_permissions_is_admin_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_admin + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.oracle_cloud_cis_incompliant + tag: foreach_oracle_cloud_cis_incompliant_rule_section + if: ctx.axonius?.identity?.oracle_cloud_cis_incompliant instanceof List + processor: + convert: + field: _ingest._value.rule_section + tag: convert_oracle_cloud_cis_incompliant_rule_section_to_float + type: string + ignore_missing: true + - foreach: + field: axonius.identity.oracle_cloud_cis_incompliant + tag: foreach_oracle_cloud_cis_incompliant_rule_cis_version + if: ctx.axonius?.identity?.oracle_cloud_cis_incompliant instanceof List + processor: + convert: + field: _ingest._value.rule_cis_version + tag: convert_oracle_cloud_cis_incompliant_rule_cis_version_to_float + type: float + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.rule_cis_version + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.password_never_expires + tag: convert_password_never_expires_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.password_never_expires + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.password_not_required + tag: convert_password_not_required_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.password_not_required + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.pmi + tag: convert_pmi_into_keyword + type: string + ignore_missing: true + - script: + tag: script_convert_recording_booleans + lang: painless + description: Converts all recording boolean fields, removing values that cannot be converted. + if: ctx.axonius?.identity?.recording != null + source: |- + def fields = [ + 'auto_delete_cmr', 'auto_delete_cmr_days', 'auto_recording', 'cloud_recording', + 'host_pause_stop_recording', 'local_recording', 'record_audio_file', + 'record_gallery_view', 'record_speaker_view', 'recording_audio_transcript', + 'save_chat_text', 'show_timestamp' + ]; + def obj = ctx.axonius.identity.recording; + for (def f : fields) { + if (!obj.containsKey(f) || obj[f] == null) { + continue; + } + def v = obj[f]; + if (v instanceof Boolean) { + continue; + } + def s = v.toString().toLowerCase(); + if (s == 'true' || s == '1') { + obj[f] = true; + } else if (s == 'false' || s == '0' || s == '') { + obj[f] = false; + } else { + obj.remove(f); + } + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.recovery_question_set + tag: convert_recovery_question_set_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.recovery_question_set + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - script: + tag: script_convert_schedule_meeting_booleans + lang: painless + description: Converts all schedule_meeting boolean fields, removing values that cannot be converted. + if: ctx.axonius?.identity?.schedule_meeting != null + source: |- + def fields = [ + 'force_pmi_jbh_password', 'host_video', 'join_before_host', 'participants_video', + 'pstn_password_protected', 'require_password_for_instant_meetings', + 'require_password_for_pmi_meetings', 'require_password_for_scheduled_meetings', + 'require_password_for_scheduling_new_meetings', 'use_pmi_for_instant_meetings', + 'use_pmi_for_scheduled_meetings' + ]; + def obj = ctx.axonius.identity.schedule_meeting; + for (def f : fields) { + if (!obj.containsKey(f) || obj[f] == null) { + continue; + } + def v = obj[f]; + if (v instanceof Boolean) { + continue; + } + def s = v.toString().toLowerCase(); + if (s == 'true' || s == '1') { + obj[f] = true; + } else if (s == 'false' || s == '0' || s == '') { + obj[f] = false; + } else { + obj.remove(f); + } + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - append: + field: related.user + tag: append_identity_snow_full_name_into_related_user + value: '{{{axonius.identity.snow_full_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.snow_full_name != null + - date: + field: axonius.identity.status_changed + tag: date_status_changed + target_field: axonius.identity.status_changed + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.status_changed != null && ctx.axonius.identity.status_changed != '' + on_failure: + - remove: + field: axonius.identity.status_changed + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.telephony.show_international_numbers_link + tag: convert_telephony_show_international_numbers_link_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.telephony.show_international_numbers_link + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.telephony.third_party_audio + tag: convert_telephony_third_party_audio_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.telephony.third_party_audio + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.tsp.call_out + tag: convert_tsp_call_out_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.tsp.call_out + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.tsp.show_international_numbers_link + tag: convert_tsp_show_international_numbers_link_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.tsp.show_international_numbers_link + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: axonius.identity.u_vip + tag: convert_u_vip_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.u_vip + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: axonius.identity.updated_on + tag: date_updated_on + target_field: axonius.identity.updated_on + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.updated_on != null && ctx.axonius.identity.updated_on != '' + on_failure: + - remove: + field: axonius.identity.updated_on + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - script: + tag: script_convert_user_apps_booleans + lang: painless + description: Converts all user_apps boolean fields in each array element, removing values that cannot be converted. + if: ctx.axonius?.identity?.user_apps instanceof List + source: |- + def fields = [ + 'active_from_direct_adapter', 'is_from_direct_adapter', 'is_managed', + 'is_saas_application', 'is_unmanaged_extension', 'is_user_deleted', + 'is_user_external', 'is_user_paid', 'is_user_suspended' + ]; + for (def item : ctx.axonius.identity.user_apps) { + for (def f : fields) { + if (!item.containsKey(f) || item[f] == null) { + continue; + } + def v = item[f]; + if (v instanceof Boolean) { + continue; + } + def s = v.toString().toLowerCase(); + if (s == 'true' || s == '1') { + item[f] = true; + } else if (s == 'false' || s == '0' || s == '') { + item[f] = false; + } else { + item.remove(f); + } + } + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_apps + tag: foreach_user_apps_last_access + if: ctx.axonius?.identity?.user_apps instanceof List + processor: + date: + field: _ingest._value.last_access + tag: date_user_apps_last_access + target_field: _ingest._value.last_access + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + on_failure: + - remove: + field: _ingest._value.last_access + ignore_missing: true + - date: + field: axonius.identity.user_created + tag: date_user_created + target_field: axonius.identity.user_created + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + if: ctx.axonius?.identity?.user_created != null && ctx.axonius.identity.user_created != '' + on_failure: + - remove: + field: axonius.identity.user_created + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.created + tag: set_event_created_from_identity_user_created + copy_from: axonius.identity.user_created + ignore_empty_value: true + - foreach: + field: axonius.identity.user_factors + tag: foreach_user_factors_created + if: ctx.axonius?.identity?.user_factors instanceof List + processor: + date: + field: _ingest._value.created + tag: date_user_factors_created + target_field: _ingest._value.created + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + on_failure: + - remove: + field: _ingest._value.created + ignore_missing: true + - foreach: + field: axonius.identity.user_factors + tag: foreach_user_factors_is_enabled + if: ctx.axonius?.identity?.user_factors instanceof List + processor: + convert: + field: _ingest._value.is_enabled + tag: convert_user_factors_is_enabled_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_enabled + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_factors + tag: foreach_user_factors_last_updated + if: ctx.axonius?.identity?.user_factors instanceof List + processor: + date: + field: _ingest._value.last_updated + tag: date_user_factors_last_updated + target_field: _ingest._value.last_updated + formats: + - EEE, dd MMM yyyy HH:mm:ss 'GMT' + - yyyy-MM-dd + - EEE,dd MMM yyyy HH:mm:ss 'GMT' + on_failure: + - remove: + field: _ingest._value.last_updated + ignore_missing: true + - set: + field: user.full_name + tag: set_user_full_name_from_identity_user_full_name + copy_from: axonius.identity.user_full_name + ignore_empty_value: true + - append: + field: related.user + tag: append_identity_user_full_name_into_related_user + value: '{{{axonius.identity.user_full_name}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.user_full_name != null + - convert: + field: axonius.identity.user_is_password_enabled + tag: convert_user_is_password_enabled_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.user_is_password_enabled + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - append: + field: related.user + tag: append_identity_user_manager_into_related_user + value: '{{{axonius.identity.user_manager}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.user_manager != null + - append: + field: related.user + tag: append_identity_user_manager_mail_into_related_user + value: '{{{axonius.identity.user_manager_mail}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.user_manager_mail != null + - foreach: + field: axonius.identity.user_permissions + tag: foreach_user_permissions_is_admin + if: ctx.axonius?.identity?.user_permissions instanceof List + processor: + convert: + field: _ingest._value.is_admin + tag: convert_user_permissions_is_admin_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.is_admin + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.user_related_resources + tag: foreach_user_related_resources_id + if: ctx.axonius?.identity?.user_related_resources instanceof List + processor: + convert: + field: _ingest._value.id + tag: convert_user_related_resources_id_into_keyword + type: string + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.id + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: user.name + tag: set_user_name_from_identity_username + copy_from: axonius.identity.username + ignore_empty_value: true + - append: + field: related.user + tag: append_identity_username_into_related_user + value: '{{{axonius.identity.username}}}' + allow_duplicates: false + if: ctx.axonius?.identity?.username != null + - convert: + field: axonius.identity.verified + tag: convert_verified_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: axonius.identity.verified + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: axonius.identity.associated_devices + tag: foreach_axonius_identity_associated_devices + if: ctx.axonius?.identity?.associated_devices instanceof List + processor: + remove: + field: + - _ingest._value.device_id + - _ingest._value.device_model + - _ingest._value.device_serial + tag: remove_custom_duplicate_fields_from_axonius_identity_associated_devices + ignore_missing: true + if: ctx.tags == null || !ctx.tags.contains('preserve_duplicate_custom_fields') + - foreach: + field: axonius.identity.breaches_data + tag: foreach_axonius_identity_breaches_data + if: ctx.axonius?.identity?.breaches_data instanceof List + processor: + remove: + field: _ingest._value.added_date + tag: remove_custom_duplicate_fields_from_axonius_identity_breaches_data + ignore_missing: true + if: ctx.tags == null || !ctx.tags.contains('preserve_duplicate_custom_fields') +on_failure: + - append: + field: error.message + value: |- + Processor '{{{ _ingest.on_failure_processor_type }}}' + {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' + {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}' + - set: + field: event.kind + tag: set_pipeline_error_to_event_kind + value: pipeline_error + - append: + field: tags + value: preserve_original_event + allow_duplicates: false diff --git a/packages/axonius/data_stream/identity/fields/base-fields.yml b/packages/axonius/data_stream/identity/fields/base-fields.yml new file mode 100644 index 00000000000..919b224d09f --- /dev/null +++ b/packages/axonius/data_stream/identity/fields/base-fields.yml @@ -0,0 +1,16 @@ +- name: data_stream.dataset + external: ecs +- name: data_stream.namespace + external: ecs +- name: data_stream.type + external: ecs +- name: event.dataset + type: constant_keyword + external: ecs + value: axonius.identity +- name: event.module + type: constant_keyword + external: ecs + value: axonius +- name: '@timestamp' + external: ecs diff --git a/packages/axonius/data_stream/identity/fields/beats.yml b/packages/axonius/data_stream/identity/fields/beats.yml new file mode 100644 index 00000000000..4084f1dc7f5 --- /dev/null +++ b/packages/axonius/data_stream/identity/fields/beats.yml @@ -0,0 +1,6 @@ +- name: input.type + type: keyword + description: Type of filebeat input. +- name: log.offset + type: long + description: Log offset. diff --git a/packages/axonius/data_stream/identity/fields/ecs.yml b/packages/axonius/data_stream/identity/fields/ecs.yml new file mode 100644 index 00000000000..e1d89be8ab4 --- /dev/null +++ b/packages/axonius/data_stream/identity/fields/ecs.yml @@ -0,0 +1,5 @@ +# Define ECS constant fields as constant_keyword +- name: observer.vendor + external: ecs + type: constant_keyword + value: Axonius diff --git a/packages/axonius/data_stream/identity/fields/fields.yml b/packages/axonius/data_stream/identity/fields/fields.yml new file mode 100644 index 00000000000..28f0aa82021 --- /dev/null +++ b/packages/axonius/data_stream/identity/fields/fields.yml @@ -0,0 +1,1413 @@ +- name: axonius + type: group + fields: + - name: identity + type: group + fields: + - name: adapter_list_length + type: long + description: How many adapters contributed to this asset. + - name: adapters + type: keyword + description: List of adapters that created this asset. + - name: asset_type + type: keyword + description: The type of asset. + - name: account_disabled + type: boolean + description: Indicates whether the user account is disabled. + - name: accurate_for_datetime + type: date + description: Timestamp indicating when this asset information was accurate. + - name: active + type: keyword + description: The active status of the identity. + - name: active_users + type: long + description: Number of active users in the account. + - name: active_users_saved_query_id + type: keyword + description: Saved query ID for the active users metric. + - name: admin_non_operational_users + type: long + description: Number of admin users that are non-operational. + - name: admin_non_operational_users_saved_query_id + type: keyword + description: Saved query ID for the admin non-operational users metric. + - name: admin_operational_active_users + type: long + description: Number of admin users that are both operational and active. + - name: admin_operational_active_users_saved_query_id + type: keyword + description: Saved query ID for the admin operational active users metric. + - name: admin_operational_inactive_users + type: long + description: Number of admin users that are operational but inactive. + - name: admin_operational_inactive_users_saved_query_id + type: keyword + description: Saved query ID for the admin operational inactive users metric. + - name: admin_operational_users + type: long + description: Number of admin users that are operational. + - name: admin_operational_users_saved_query_id + type: keyword + description: Saved query ID for the admin operational users metric. + - name: admin_roles + type: group + fields: + - name: display_name + type: keyword + description: Display name of the admin role. + - name: id + type: keyword + description: Unique identifier of the admin role. + - name: admins + type: long + description: Total number of administrators in the account. + - name: admins_saved_query_id + type: keyword + description: Saved query ID for the admins metric. + - name: alt_names + type: group + fields: + - name: name + type: keyword + description: The alternative name value. + - name: name_type + type: keyword + description: The type of alternative name (e.g., DNS, IP). + - name: application_and_account_name + type: keyword + description: The application and account name associated with the asset. + - name: application_id + type: keyword + description: Unique identifier of the application. + - name: application_name + type: keyword + description: Name of the application associated with this identity. + - name: asset_entity_info + type: keyword + description: Information about the asset entity and its properties. + - name: data_asset_type + type: keyword + description: The asset type from identity event data, distinguishing from root asset_type. + - name: description + type: text + description: The description of the asset. + - name: associated_devices + type: group + fields: + - name: device_associated_saas_apps_names + type: keyword + description: Names of SaaS applications associated with the device. + - name: device_caption + type: keyword + description: Caption or display name of the associated device. + - name: device_id + type: keyword + description: Unique identifier of the associated device. + - name: device_labels + type: keyword + description: Labels or tags assigned to the associated device. + - name: device_model + type: keyword + description: Model name of the associated device. + - name: device_os_distribution + type: keyword + description: Operating system distribution of the associated device. + - name: device_os_edition + type: keyword + description: Operating system edition of the associated device. + - name: device_os_end_of_life + type: keyword + description: End-of-life date of the device operating system. + - name: device_os_type + type: keyword + description: Operating system type of the associated device. + - name: device_os_version + type: keyword + description: Operating system version of the associated device. + - name: device_preferred_mac_address + type: keyword + description: Preferred MAC address of the associated device. + - name: device_serial + type: keyword + description: Serial number of the associated device. + - name: internal_axon_id + type: keyword + description: Internal Axonius ID of the associated device. + - name: associated_employees + type: group + fields: + - name: internal_axon_id + type: keyword + description: Internal Axonius ID of the associated employee. + - name: username + type: keyword + description: Username of the associated employee. + - name: associated_groups + type: group + fields: + - name: display_name + type: keyword + description: Display name of the associated group. + - name: remote_id + type: keyword + description: Remote identifier of the associated group. + - name: associated_licenses + type: group + fields: + - name: adapter_connection_label + type: keyword + description: Label of the adapter connection for the license. + - name: internal_axon_id + type: keyword + description: Internal Axonius ID of the license. + - name: license_name + type: keyword + description: Name of the license. + - name: pricing_unit + type: keyword + description: Pricing unit of the license. + - name: related_vendor_name + type: keyword + description: Vendor name associated with the license. + - name: unit_price + type: keyword + description: Unit price of the license. + - name: aws_arn + type: keyword + description: Amazon Web Services ARN (Amazon Resource Name) for this identity. + - name: aws_iam_identity_type + type: keyword + description: AWS IAM identity type (e.g., user, role, group). + - name: azure_account_id + type: keyword + description: Azure account identifier associated with this identity. + - name: begins_on + type: date + description: Start date of the certificate validity period. + - name: bit_size + type: long + description: Key bit size of the certificate. + - name: breaches_data + type: group + fields: + - name: added_date + type: date + description: Date when the breach was added to the database. + - name: breach_date + type: date + description: Date when the breach occurred. + - name: data_classes + type: keyword + description: Types of data exposed in the breach. + - name: domain + type: keyword + description: Domain affected by the breach. + - name: is_fabricated + type: boolean + description: Indicates if the breach data is fabricated. + - name: is_retired + type: boolean + description: Indicates if the breach record has been retired. + - name: is_sensitive + type: boolean + description: Indicates if the breach contains sensitive data. + - name: is_spam_list + type: boolean + description: Indicates if the breach is from a spam list. + - name: is_verified + type: boolean + description: Indicates if the breach has been verified. + - name: logo_path + type: keyword + description: Path to the logo of the breached service. + - name: modified_date + type: date + description: Date when the breach record was last modified. + - name: name + type: keyword + description: Name of the breach. + - name: pwn_count + type: long + description: Number of accounts affected by the breach. + - name: title + type: keyword + description: Title of the breach. + - name: class_name + type: keyword + description: The class name or system classification of this asset. + - name: cloud_provider + type: keyword + description: The cloud provider associated with this identity. + - name: connected_assets + type: keyword + description: Other assets connected to or associated with this identity asset. + - name: connection_label + type: keyword + description: Label of the adapter connection used to collect this identity data. + - name: created_date + type: date + description: Date when this identity record was created. + - name: deleted_users + type: long + description: Number of deleted users in the account. + - name: deleted_users_saved_query_id + type: keyword + description: Saved query ID for the deleted users metric. + - name: direct_not_sso_users + type: long + description: Number of users with direct access who are not using SSO. + - name: direct_not_sso_users_saved_query_id + type: keyword + description: Saved query ID for the direct non-SSO users metric. + - name: display_name + type: keyword + description: Display name of the identity. + - name: distinct_associated_devices_count + type: long + description: Number of distinct devices associated with this identity. + - name: domains + type: group + fields: + - name: name + type: keyword + description: Name of the domain. + - name: email + type: keyword + description: Email address of the identity. + - name: email_activity + type: group + fields: + - name: is_deleted + type: boolean + description: Indicates if the email activity record has been deleted. + - name: product_license + type: keyword + description: Product license associated with email activity. + - name: read_count + type: long + description: Number of emails read during the report period. + - name: receive_count + type: long + description: Number of emails received during the report period. + - name: report_date + type: date + description: Date of the email activity report. + - name: report_period + type: long + description: Reporting period in days for the email activity. + - name: send_count + type: long + description: Number of emails sent during the report period. + - name: email_notification + type: group + fields: + - name: alternative_host_reminder + type: boolean + description: Indicates if alternative host reminder emails are enabled. + - name: cancel_meeting_reminder + type: boolean + description: Indicates if meeting cancellation reminder emails are enabled. + - name: jbh_reminder + type: boolean + description: Indicates if join-before-host reminder emails are enabled. + - name: employee_id + type: keyword + description: Employee identifier assigned to this identity. + - name: employee_number + type: keyword + description: Employee number assigned to this identity. + - name: employee_type + type: keyword + description: Type of employee (e.g., full-time, contractor). + - name: expires_on + type: date + description: Expiration date of the certificate validity period. + - name: external_users + type: long + description: Number of external users in the account. + - name: external_users_saved_query_id + type: keyword + description: Saved query ID for the external users metric. + - name: feature + type: group + fields: + - name: cn_meeting + type: boolean + description: Indicates if China meeting feature is enabled. + - name: in_meeting + type: boolean + description: Indicates if in-meeting feature is enabled. + - name: large_meeting + type: boolean + description: Indicates if large meeting feature is enabled. + - name: meeting_capacity + type: long + description: Maximum meeting capacity for this identity. + - name: webinar + type: boolean + description: Indicates if webinar feature is enabled. + - name: zoom_phone + type: boolean + description: Indicates if Zoom Phone feature is enabled. + - name: fetch_time + type: date + description: The date and time when the identity data was last fetched. + - name: first_fetch_time + type: date + description: The date and time when this identity asset was first fetched. + - name: first_name + type: keyword + description: First name of the identity. + - name: first_seen + type: date + description: The date and time when this identity was first observed. + - name: from_last_fetch + type: boolean + description: Indicates whether this identity asset was modified since the last fetch. + - name: gce_account_id + type: keyword + description: Google Cloud Engine account ID associated with this identity. + - name: groups + type: group + fields: + - name: display_name + type: keyword + description: Display name of the group. + - name: name + type: keyword + description: Name of the group. + - name: remote_id + type: keyword + description: Remote identifier of the group. + - name: has_administrative_permissions + type: boolean + description: Indicates whether this identity has administrative permissions. + - name: hire_date + type: date + description: Date when the employee was hired. + - name: hr_employment_status + type: keyword + description: Human resources employment status of the identity. + - name: id + type: keyword + description: Unique identifier for the identity asset. + - name: id_raw + type: keyword + description: Raw unique identifier for the identity asset. + - name: in_meeting + type: group + fields: + - name: allow_live_streaming + type: boolean + description: Indicates if live streaming is allowed during meetings. + - name: annotation + type: boolean + description: Indicates if annotation is enabled during meetings. + - name: attendee_on_hold + type: boolean + description: Indicates if attendee-on-hold feature is enabled. + - name: auto_saving_chat + type: boolean + description: Indicates if auto-saving chat is enabled. + - name: breakout_room + type: boolean + description: Indicates if breakout rooms are enabled. + - name: chat + type: boolean + description: Indicates if chat is enabled during meetings. + - name: closed_caption + type: boolean + description: Indicates if closed captions are enabled. + - name: co_host + type: boolean + description: Indicates if co-host feature is enabled. + - name: data_center_regions + type: keyword + description: Data center regions configured for meetings. + - name: e2e_encryption + type: boolean + description: Indicates if end-to-end encryption is enabled. + - name: entry_exit_chime + type: boolean + description: Indicates if entry/exit chime is enabled. + - name: far_end_camera_control + type: boolean + description: Indicates if far-end camera control is enabled. + - name: feedback + type: boolean + description: Indicates if feedback feature is enabled. + - name: group_hd + type: boolean + description: Indicates if group HD video is enabled. + - name: non_verbal_feedback + type: boolean + description: Indicates if non-verbal feedback is enabled. + - name: polling + type: boolean + description: Indicates if polling is enabled during meetings. + - name: private_chat + type: boolean + description: Indicates if private chat is enabled during meetings. + - name: record_play_voice + type: boolean + description: Indicates if record and play voice is enabled. + - name: remote_control + type: boolean + description: Indicates if remote control is enabled. + - name: remote_support + type: boolean + description: Indicates if remote support is enabled. + - name: share_dual_camera + type: boolean + description: Indicates if dual camera sharing is enabled. + - name: show_meeting_control_toolbar + type: boolean + description: Indicates if meeting control toolbar is shown. + - name: virtual_background + type: boolean + description: Indicates if virtual background is enabled. + - name: waiting_room + type: boolean + description: Indicates if waiting room is enabled. + - name: workplace_by_facebook + type: boolean + description: Indicates if Workplace by Facebook integration is enabled. + - name: inactive_users + type: long + description: Number of inactive users in the account. + - name: inactive_users_saved_query_id + type: keyword + description: Saved query ID for the inactive users metric. + - name: internal_is_admin + type: boolean + description: Internal flag indicating if this identity has admin privileges. + - name: is_active + type: boolean + description: Indicates whether this identity is currently active. + - name: is_admin + type: boolean + description: Indicates whether this identity has administrator privileges. + - name: is_built_in + type: boolean + description: Indicates whether this is a built-in system account. + - name: is_delegated_admin + type: boolean + description: Indicates whether this identity is a delegated administrator. + - name: is_fetched_from_adapter + type: boolean + description: Indicates whether this identity data was fetched from an adapter. + - name: is_from_sso_provider + type: boolean + description: Indicates whether this identity originates from a Single Sign-On provider. + - name: is_latest_last_seen + type: boolean + description: Indicates if this is the latest recorded last-seen timestamp. + - name: is_managed_by_application + type: boolean + description: Indicates whether this identity is managed by an application. + - name: is_managed_by_direct_app + type: boolean + description: Indicates whether this identity is managed by a direct application. + - name: is_managed_by_sso + type: boolean + description: Indicates whether this identity is managed through SSO. + - name: is_mfa_enforced + type: boolean + description: Indicates whether multi-factor authentication is enforced. + - name: is_mfa_enrolled + type: boolean + description: Indicates whether this identity is enrolled in multi-factor authentication. + - name: is_non_editable + type: boolean + description: Indicates whether this identity record is non-editable. + - name: is_paid + type: boolean + description: Indicates whether this identity has a paid license or account. + - name: is_permission_adapter + type: boolean + description: Indicates whether this identity was collected by a permission adapter. + - name: is_privileged + type: boolean + description: Indicates whether this identity has privileged access. + - name: is_saas_user + type: boolean + description: Indicates whether this identity is a SaaS application user. + - name: is_user_active + type: boolean + description: Indicates whether the user account is active. + - name: is_user_deleted + type: boolean + description: Indicates whether the user account has been deleted. + - name: is_user_external + type: boolean + description: Indicates whether this is an external user. + - name: is_user_inactive + type: boolean + description: Indicates whether the user account is inactive. + - name: is_user_suspended + type: boolean + description: Indicates whether the user account is suspended. + - name: issuer + type: group + fields: + - name: common_name + type: keyword + description: Common name of the certificate issuer. + - name: country_name + type: keyword + description: Country name of the certificate issuer. + - name: organization + type: keyword + description: Organization name of the certificate issuer. + - name: last_client_version + type: keyword + description: Version of the last client used by this identity. + - name: last_enrichment_run + type: date + description: Date of the last enrichment run for this identity. + - name: last_fetch_connection_id + type: keyword + description: The connection ID of the adapter that last fetched this data. + - name: last_fetch_connection_label + type: keyword + description: The label of the connection that last fetched this identity data. + - name: last_login_attempt + type: date + description: Date and time of the last login attempt. + - name: last_logon + type: date + description: Date and time of the last successful logon. + - name: last_name + type: keyword + description: Last name of the identity. + - name: last_password_change + type: date + description: Date and time when the password was last changed. + - name: last_seen + type: date + description: The date and time when this identity was last observed. + - name: mail + type: keyword + description: Email address (mail attribute) of the identity. + - name: managed_non_operational_users + type: long + description: Number of managed users that are non-operational. + - name: managed_non_operational_users_saved_query_id + type: keyword + description: Saved query ID for the managed non-operational users metric. + - name: managed_operational_users + type: long + description: Number of managed users that are operational. + - name: managed_operational_users_saved_query_id + type: keyword + description: Saved query ID for the managed operational users metric. + - name: managed_users + type: long + description: Total number of managed users in the account. + - name: managed_users_by_app + type: long + description: Number of users managed by a direct application. + - name: managed_users_by_app_saved_query_id + type: keyword + description: Saved query ID for the managed-by-app users metric. + - name: managed_users_by_sso + type: long + description: Number of users managed through SSO. + - name: managed_users_by_sso_saved_query_id + type: keyword + description: Saved query ID for the managed-by-SSO users metric. + - name: managed_users_saved_query_id + type: keyword + description: Saved query ID for the managed users metric. + - name: manager_id + type: keyword + description: Identifier of the manager of this identity. + - name: max_added_date + type: date + description: Most recent date a breach was added across all breaches for this identity. + - name: max_breach_date + type: date + description: Most recent breach date across all breaches for this identity. + - name: max_modified_date + type: date + description: Most recent modified date across all breaches for this identity. + - name: name + type: keyword + description: The name or identifier of the identity asset. + - name: nested_applications + type: group + fields: + - name: active_from_direct_adapter + type: boolean + description: Indicates if active status is from a direct adapter. + - name: app_accounts + type: group + fields: + - name: name + type: keyword + description: Name of the application account. + - name: app_display_name + type: keyword + description: Display name of the application. + - name: app_links + type: keyword + description: Links or URLs associated with the application. + - name: assignment_type + type: keyword + description: How the application was assigned (e.g., direct, group). + - name: extension_type + type: keyword + description: Type of extension for the application. + - name: has_administrative_permissions + type: boolean + description: Indicates if the identity has admin permissions in this application. + - name: is_deleted + type: boolean + description: Indicates if the application assignment has been deleted. + - name: is_from_direct_adapter + type: boolean + description: Indicates if the data is from a direct adapter. + - name: is_managed + type: boolean + description: Indicates if the application is managed. + - name: is_suspended + type: boolean + description: Indicates if the application access is suspended. + - name: is_unmanaged_extension + type: boolean + description: Indicates if this is an unmanaged browser extension. + - name: is_user_external + type: boolean + description: Indicates if the user is external in this application. + - name: is_user_paid + type: boolean + description: Indicates if the user has a paid license in this application. + - name: last_access + type: date + description: Date and time of the last access to the application. + - name: last_access_count + type: long + description: Total number of accesses to the application. + - name: last_access_count_60_days + type: long + description: Number of accesses to the application in the last 60 days. + - name: last_access_count_90_days + type: long + description: Number of accesses to the application in the last 90 days. + - name: name + type: keyword + description: Name of the application. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: permissions + type: group + fields: + - name: name + type: keyword + description: Name of the permission. + - name: relation_direct_name + type: keyword + description: Name of the direct relationship to the application. + - name: relation_discovery_name + type: keyword + description: Name of the discovered relationship to the application. + - name: relation_extension_name + type: keyword + description: Name of the extension-based relationship to the application. + - name: relation_sso_name + type: keyword + description: Name of the SSO-based relationship to the application. + - name: source_application + type: keyword + description: Source application that provided this data. + - name: value + type: keyword + description: Value or identifier of the application. + - name: vendor_category + type: keyword + description: Vendor category of the application. + - name: nested_associated_devices + type: keyword + description: Flattened list of nested associated device identifiers. + - name: nested_grants_last_updated + type: date + description: Date when nested grants were last updated. + - name: nested_grants_managers_last_updated + type: date + description: Date when nested grants managers were last updated. + - name: nested_groups + type: group + fields: + - name: assignment_type + type: keyword + description: How the group was assigned (e.g., direct, inherited). + - name: group_name + type: keyword + description: Name of the group. + - name: name + type: keyword + description: Display name of the group entry. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the group. + - name: nested_managers + type: group + fields: + - name: assignment_type + type: keyword + description: How the manager was assigned. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the manager. + - name: nested_permissions + type: group + fields: + - name: has_administrative_permissions + type: boolean + description: Indicates if the identity has administrative permissions. + - name: is_admin + type: boolean + description: Indicates if the identity has admin privileges. + - name: assignment_type + type: keyword + description: How the permission was assigned (e.g., direct, inherited). + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the permission. + - name: nested_resources + type: group + fields: + - name: assignment_type + type: keyword + description: How the resource was assigned. + - name: name + type: keyword + description: Name of the resource. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the resource. + - name: nested_roles + type: group + fields: + - name: assignment_type + type: keyword + description: How the role was assigned (e.g., direct, inherited). + - name: name + type: keyword + description: Name of the role. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the role. + - name: not_fetched_count + type: long + description: The number of times this identity asset failed to be fetched. + - name: operational_users_count + type: long + description: Total number of operational users in the account. + - name: oracle_cloud_cis_incompliant + type: group + fields: + - name: rule_cis_version + type: float + description: CIS benchmark version of the incompliant rule. + - name: rule_section + type: keyword + description: Section number of the incompliant CIS rule. + - name: orphaned_users + type: long + description: Number of orphaned users in the account. + - name: orphaned_users_saved_query_id + type: keyword + description: Saved query ID for the orphaned users metric. + - name: paid_users + type: long + description: Number of paid users in the account. + - name: paid_users_saved_query_id + type: keyword + description: Saved query ID for the paid users metric. + - name: password_never_expires + type: boolean + description: Indicates whether the password is set to never expire. + - name: password_not_required + type: boolean + description: Indicates whether a password is not required for this account. + - name: permissions_list + type: group + fields: + - name: name + type: keyword + description: Name of the permission. + - name: permissions + type: long + description: Total number of permissions assigned to the identity. + - name: pmi + type: keyword + description: Personal Meeting ID (Zoom). + - name: pretty_id + type: keyword + description: A human-readable identifier for the identity asset. + - name: project_ids + type: keyword + description: Cloud project IDs associated with this identity. + - name: project_tags + type: group + fields: + - name: inherited + type: keyword + description: Indicates if the tag is inherited from a parent resource. + - name: key + type: keyword + description: Tag key. + - name: namespaced_tag_key + type: keyword + description: Namespaced version of the tag key. + - name: namespaced_tag_value + type: keyword + description: Namespaced version of the tag value. + - name: value + type: keyword + description: Tag value. + - name: projects_roles + type: group + fields: + - name: project_id + type: keyword + description: Identifier of the project. + - name: role_name + type: keyword + description: Name of the role in the project. + - name: roles + type: group + fields: + - name: display_name + type: keyword + description: Display Name of the role. + - name: remote_id + type: keyword + description: Remote ID of the role. + - name: roles_accounts + type: keyword + description: Account roles. + - name: provider_name + type: keyword + description: Name of the identity provider. + - name: provider_type + type: keyword + description: Type of the identity provider. + - name: recording + type: group + fields: + - name: auto_delete_cmr + type: boolean + description: Indicates if cloud meeting recordings are auto-deleted. + - name: auto_delete_cmr_days + type: boolean + description: Indicates if auto-delete days for cloud recordings is configured. + - name: auto_recording + type: boolean + description: Indicates if auto-recording is enabled. + - name: cloud_recording + type: boolean + description: Indicates if cloud recording is enabled. + - name: host_pause_stop_recording + type: boolean + description: Indicates if host can pause or stop recording. + - name: local_recording + type: boolean + description: Indicates if local recording is enabled. + - name: record_audio_file + type: boolean + description: Indicates if a separate audio file is recorded. + - name: record_gallery_view + type: boolean + description: Indicates if gallery view is recorded. + - name: record_speaker_view + type: boolean + description: Indicates if speaker view is recorded. + - name: recording_audio_transcript + type: boolean + description: Indicates if audio transcript is generated for recordings. + - name: save_chat_text + type: boolean + description: Indicates if chat text is saved with recordings. + - name: show_timestamp + type: boolean + description: Indicates if timestamp is shown in recordings. + - name: recovery_question_set + type: boolean + description: Indicates whether a recovery question has been set for this identity. + - name: relatable_ids + type: keyword + description: IDs used to relate this identity to other assets. + - name: remote_account_id + type: keyword + description: Remote account identifier for this identity. + - name: remote_id + type: keyword + description: Remote identifier for this identity in the source system. + - name: schedule_meeting + type: group + fields: + - name: audio_type + type: keyword + description: Audio type configured for scheduled meetings. + - name: force_pmi_jbh_password + type: boolean + description: Indicates if PMI join-before-host password is forced. + - name: host_video + type: boolean + description: Indicates if host video is on when joining a meeting. + - name: join_before_host + type: boolean + description: Indicates if participants can join before the host. + - name: participants_video + type: boolean + description: Indicates if participant video is on when joining a meeting. + - name: pstn_password_protected + type: boolean + description: Indicates if PSTN dial-in is password protected. + - name: require_password_for_instant_meetings + type: boolean + description: Indicates if password is required for instant meetings. + - name: require_password_for_pmi_meetings + type: boolean + description: Indicates if password is required for PMI meetings. + - name: require_password_for_scheduled_meetings + type: boolean + description: Indicates if password is required for scheduled meetings. + - name: require_password_for_scheduling_new_meetings + type: boolean + description: Indicates if password is required when scheduling new meetings. + - name: use_pmi_for_instant_meetings + type: boolean + description: Indicates if PMI is used for instant meetings. + - name: use_pmi_for_scheduled_meetings + type: boolean + description: Indicates if PMI is used for scheduled meetings. + - name: serial_number + type: keyword + description: Serial number of the certificate. + - name: shirt_size + type: keyword + description: Shirt size of the employee (HR attribute). + - name: sm_entity_type + type: keyword + description: SaaS management entity type for this identity. + - name: snow_full_name + type: keyword + description: Full name of the identity from ServiceNow. + - name: snow_location + type: keyword + description: Location of the identity from ServiceNow. + - name: source_application + type: keyword + description: The source application that provided this identity data. + - name: status + type: keyword + description: Current status of the identity account. + - name: status_changed + type: date + description: Date and time when the account status was last changed. + - name: subject + type: group + fields: + - name: common_name + type: keyword + description: Common name of the certificate subject. + - name: country_name + type: keyword + description: Country name of the certificate subject. + - name: locality + type: keyword + description: Locality (city) of the certificate subject. + - name: organization + type: keyword + description: Organization name of the certificate subject. + - name: state + type: keyword + description: State or province of the certificate subject. + - name: suspended_users + type: long + description: Number of suspended users in the account. + - name: suspended_users_saved_query_id + type: keyword + description: Saved query ID for the suspended users metric. + - name: telephony + type: group + fields: + - name: show_international_numbers_link + type: boolean + description: Indicates if international numbers link is shown. + - name: third_party_audio + type: boolean + description: Indicates if third-party audio is enabled. + - name: tenant_number + type: long + description: Tenant number associated with this identity. + - name: timezone + type: keyword + description: Timezone configured for this identity. + - name: total_users_count + type: long + description: Total number of users in the account. + - name: tsp + type: group + fields: + - name: call_out + type: boolean + description: Indicates if TSP call-out is enabled. + - name: show_international_numbers_link + type: boolean + description: Indicates if international numbers link is shown for TSP. + - name: type + type: keyword + description: The type or classification of the identity entity. + - name: u_department + type: keyword + description: Department of the identity from ServiceNow. + - name: u_vip + type: boolean + description: Indicates whether this identity is flagged as a VIP in ServiceNow. + - name: unlinked_users + type: long + description: Number of unlinked users in the account. + - name: unlinked_users_saved_query_id + type: keyword + description: Saved query ID for the unlinked users metric. + - name: updated_on + type: date + description: Date and time when this identity record was last updated. + - name: user_apps + type: group + fields: + - name: active_from_direct_adapter + type: boolean + description: Indicates if active status is from a direct adapter. + - name: app_accounts + type: group + fields: + - name: name + type: keyword + description: Name of the application account. + - name: app_display_name + type: keyword + description: Display name of the application. + - name: app_id + type: keyword + description: Unique identifier of the application. + - name: app_links + type: keyword + description: Links or URLs associated with the application. + - name: app_name + type: keyword + description: Name of the application. + - name: extension_type + type: keyword + description: Type of extension for the application. + - name: is_from_direct_adapter + type: boolean + description: Indicates if the data is from a direct adapter. + - name: is_managed + type: boolean + description: Indicates if the application is managed. + - name: is_saas_application + type: boolean + description: Indicates if this is a SaaS application. + - name: is_unmanaged_extension + type: boolean + description: Indicates if this is an unmanaged browser extension. + - name: is_user_deleted + type: boolean + description: Indicates if the user has been deleted in the application. + - name: is_user_external + type: boolean + description: Indicates if the user is external in the application. + - name: is_user_paid + type: boolean + description: Indicates if the user has a paid license in the application. + - name: is_user_suspended + type: boolean + description: Indicates if the user is suspended in the application. + - name: last_access + type: date + description: Date and time of the last access to the application. + - name: permissions + type: group + fields: + - name: name + type: keyword + description: Name of the permission. + - name: relation_direct_name + type: keyword + description: Name of the direct relationship to the application. + - name: relation_discovery_name + type: keyword + description: Name of the discovered relationship to the application. + - name: relation_extension_name + type: keyword + description: Name of the extension-based relationship to the application. + - name: relation_sso_name + type: keyword + description: Name of the SSO-based relationship to the application. + - name: source_application + type: keyword + description: Source application that provided this data. + - name: vendor_category + type: keyword + description: Vendor category of the application. + - name: user_count + type: long + description: Number of users in the application. + - name: user_count_link + type: group + fields: + - name: bracketWeight + type: double + description: Weight of the bracket in the query expression. + - name: compOp + type: keyword + description: Comparison operator used in the query. + - name: field + type: keyword + description: Field name used in the query filter. + - name: leftBracket + type: double + description: Left bracket position in the query expression. + - name: logicOp + type: keyword + description: Logical operator (e.g., AND, OR) in the query. + - name: not + type: boolean + description: Indicates if the query condition is negated. + - name: rightBracket + type: double + description: Right bracket position in the query expression. + - name: value + type: keyword + description: Value used in the query filter. + - name: user_country + type: keyword + description: Country of the user. + - name: user_created + type: date + description: Date and time when the user account was created. + - name: user_department + type: keyword + description: Department the user belongs to. + - name: user_factors + type: group + fields: + - name: created + type: date + description: Date when the MFA factor was created. + - name: factor_status + type: keyword + description: Current status of the MFA factor. + - name: factor_type + type: keyword + description: Type of the MFA factor (e.g., push, TOTP, SMS). + - name: is_enabled + type: boolean + description: Indicates if the MFA factor is enabled. + - name: last_updated + type: date + description: Date when the MFA factor was last updated. + - name: name + type: keyword + description: Name of the MFA factor. + - name: provider + type: keyword + description: Provider of the MFA factor. + - name: strength + type: keyword + description: Strength rating of the MFA factor. + - name: vendor_name + type: keyword + description: Vendor name of the MFA factor. + - name: user_full_name + type: keyword + description: Full name of the user. + - name: user_is_password_enabled + type: boolean + description: Indicates whether password authentication is enabled for this user. + - name: user_manager + type: keyword + description: Name or identifier of the user's manager. + - name: user_manager_mail + type: keyword + description: Email address of the user's manager. + - name: user_pass_last_used + type: date + description: Date or timestamp when the user's password was last used. + - name: user_path + type: keyword + description: Path of the user in the directory (e.g., AWS IAM path). + - name: user_permissions + type: group + fields: + - name: is_admin + type: boolean + description: Indicates if the user has admin privileges for this permission. + - name: name + type: keyword + description: Name of the permission. + - name: user_related_resources + type: group + fields: + - name: id + type: keyword + description: Identifier of the related resource. + - name: name + type: keyword + description: Name of the related resource. + - name: type + type: keyword + description: Type of the related resource. + - name: user_remote_id + type: keyword + description: Remote identifier of the user in the source system. + - name: user_sid + type: keyword + description: Security Identifier (SID) of the user (Windows/AD). + - name: user_status + type: keyword + description: Current status of the user account. + - name: user_telephone_number + type: keyword + description: Telephone number of the user. + - name: user_title + type: keyword + description: Job title of the user. + - name: user_type + type: keyword + description: Type of user account (e.g., member, guest, service). + - name: username + type: keyword + description: Username of the identity. + - name: verified + type: boolean + description: Indicates whether this identity has been verified. + - name: version + type: keyword + description: Version of the certificate or identity record. + - name: event + type: group + fields: + - name: accurate_for_datetime + type: date + description: Timestamp indicating when the event data was accurate. + - name: action_if_exists + type: keyword + description: Action associated with the identity event, if it exists. + - name: adapter_categories + type: keyword + description: List of adapter categories that this event belongs to. + - name: associated_adapter_plugin_name + type: keyword + description: The associated plugin name that created or processed the event. + - name: association_type + type: keyword + description: The type of association between the event and related entities. + - name: client_used + type: keyword + description: The client identifier that was used to process the event. + - name: entity + type: keyword + description: The entity type or category this event relates to. + - name: hidden_for_gui + type: boolean + description: Indicates if this event should be hidden in the GUI. + - name: initial_plugin_unique_name + type: keyword + description: The initial plugin name that created or processed the event. + - name: name + type: keyword + description: The name of the event. + - name: plugin_name + type: keyword + description: The name of the plugin that processed the event. + - name: plugin_type + type: keyword + description: The type or category of the plugin that processed the event. + - name: plugin_unique_name + type: keyword + description: The unique identifier of the plugin instance that processed the event. + - name: quick_id + type: keyword + description: A quick reference identifier combining plugin and entity information. + - name: type + type: keyword + description: The type or classification of the event data. + - name: internal_axon_id + type: keyword + description: Internal ID of this asset. This ID may change in the future. + - name: transform_unique_id + type: keyword + description: Unique identifier for this asset in the transformation process. diff --git a/packages/axonius/data_stream/identity/fields/is-transform-source-true.yml b/packages/axonius/data_stream/identity/fields/is-transform-source-true.yml new file mode 100644 index 00000000000..367ed8d40c6 --- /dev/null +++ b/packages/axonius/data_stream/identity/fields/is-transform-source-true.yml @@ -0,0 +1,4 @@ +- name: labels.is_transform_source + type: constant_keyword + description: Distinguishes between documents that are a source for a transform and documents that are an output of a transform, to facilitate easier filtering. + value: 'true' diff --git a/packages/axonius/data_stream/identity/lifecycle.yml b/packages/axonius/data_stream/identity/lifecycle.yml new file mode 100644 index 00000000000..f7b0d98d5aa --- /dev/null +++ b/packages/axonius/data_stream/identity/lifecycle.yml @@ -0,0 +1 @@ +data_retention: '30d' diff --git a/packages/axonius/data_stream/identity/manifest.yml b/packages/axonius/data_stream/identity/manifest.yml new file mode 100644 index 00000000000..024a925447c --- /dev/null +++ b/packages/axonius/data_stream/identity/manifest.yml @@ -0,0 +1,92 @@ +title: Identity +type: logs +ilm_policy: logs-axonius.identity-default_policy +streams: + - input: cel + title: Identity + description: Collect Identity logs from Axonius. + template_path: cel.yml.hbs + vars: + - name: interval + type: text + title: Interval + description: Duration between requests to the Axonius API. Supported units for this parameter are h/m/s. + multi: false + required: true + show_user: true + default: 24h + - name: batch_size + type: integer + title: Batch Size + description: Batch size for the response of the Axonius API. The batch size can range from a minimum of 1 to a maximum of 2000. + default: 2000 + multi: false + required: true + show_user: true + - name: asset_type_list + type: text + title: Asset Types + description: List of asset types to collect from the Axonius API. Do not modify. + multi: true + required: true + show_user: false + default: + - users + - groups + - security_roles + - organizational_units + - accounts + - certificates + - permissions + - latest_rules + - profiles + - job_titles + - access_review_campaign_instances + - access_review_approval_items + - name: enable_request_tracer + type: bool + title: Enable request tracing + multi: false + default: false + required: false + show_user: false + description: >- + The request tracer logs requests and responses to the agent's local file-system for debugging configurations. + Enabling this request tracing compromises security and should only be used for debugging. Disabling the request + tracer will delete any stored traces. + See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-cel.html#_resource_tracer_enable) + for details. + - name: tags + type: text + title: Tags + description: Tags for the data-stream. + multi: true + required: true + show_user: false + default: + - forwarded + - axonius-identity + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. Supported time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 5m + - name: preserve_duplicate_custom_fields + required: false + title: Preserve duplicate custom fields + description: Preserve axonius.identity.* fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + show_user: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. diff --git a/packages/axonius/data_stream/identity/sample_event.json b/packages/axonius/data_stream/identity/sample_event.json new file mode 100644 index 00000000000..49ecc84c6ed --- /dev/null +++ b/packages/axonius/data_stream/identity/sample_event.json @@ -0,0 +1,170 @@ +{ + "@timestamp": "2025-12-09T12:02:11.000Z", + "agent": { + "ephemeral_id": "9f03aa73-e75b-478f-a4b6-03207451caa1", + "id": "2009affa-84ca-46da-9030-1686d937a20d", + "name": "elastic-agent-92850", + "type": "filebeat", + "version": "8.18.0" + }, + "axonius": { + "identity": { + "account_disabled": true, + "accurate_for_datetime": "2025-12-09T12:02:11.000Z", + "adapter_list_length": 12, + "adapters": [ + "aws_adapter", + "zoom_adapter" + ], + "application_and_account_name": "microsoft/azure_ad-demo", + "asset_type": "users", + "associated_groups": [ + { + "display_name": "developers-group", + "remote_id": "a3e70162" + } + ], + "azure_account_id": "c8103abe-eda9-472b-894a-6260bb2ba8cc", + "cloud_provider": "Azure", + "email_activity": { + "is_deleted": false, + "product_license": "MICROSOFT FABRIC (FREE)+MICROSOFT TEAMS PHONE STANDARD+MICROSOFT DEFENDER FOR OFFICE365 (PLAN 2)+MICROSOFT 365 AUDIO CONFERENCING+ENTERPRISE MOBILITY + SECURITY E3+OFFICE365 E3+MICROSOFT 365 E3 EXTRA FEATURES", + "read_count": 2321, + "receive_count": 6965, + "report_date": "2025-01-10T20:34:43.000Z", + "report_period": 90, + "send_count": 3030 + }, + "event": { + "accurate_for_datetime": "2025-12-09T12:02:11.000Z", + "adapter_categories": [ + "Directory", + "IAM", + "SaaS Management" + ], + "client_used": "67fd09bbfe1c8e812a176bb5", + "initial_plugin_unique_name": "azure_ad_adapter_0", + "plugin_name": "azure_ad_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "azure_ad_adapter_0", + "quick_id": "azure_ad_adapter_0!c8103abe-eda9-472b-894a-6260bb2ba8cc", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T12:02:03.000Z", + "first_fetch_time": "2025-04-14T13:27:00.000Z", + "from_last_fetch": true, + "has_administrative_permissions": true, + "id": "c8103abe-eda9-472b-894a-6260bb2ba8cc", + "internal_axon_id": "bc11b2989fc0f69708b6865d172a49fe", + "internal_is_admin": false, + "is_admin": false, + "is_fetched_from_adapter": true, + "is_latest_last_seen": true, + "is_managed_by_application": true, + "is_permission_adapter": true, + "is_saas_user": true, + "is_user_external": false, + "last_fetch_connection_id": "67fd09bbfe1c8e812a176bb5", + "last_fetch_connection_label": "azure_ad-demo", + "last_logon": "2025-11-30T18:50:39.000Z", + "last_seen": "2025-11-10T22:18:25.000Z", + "mail": "helen.jordan@demo.local", + "nested_applications": [ + { + "app_display_name": "Calendly", + "assignment_type": "Direct", + "extension_type": "User Consent", + "is_managed": false, + "is_unmanaged_extension": true, + "name": "Calendly", + "permissions": [ + { + "name": "openid" + } + ], + "relation_extension_name": "Calendly", + "source_application": "Microsoft", + "value": "2E2a2e7c9f758BDcC0E2", + "vendor_category": "Productivity" + } + ], + "nested_grants_last_updated": "2025-12-09T12:10:06.000Z", + "nested_grants_managers_last_updated": "2025-12-09T12:10:10.000Z", + "nested_groups": [ + { + "assignment_type": "Direct", + "name": "Office365 Users", + "value": "d8e66837" + } + ], + "not_fetched_count": 0, + "sm_entity_type": "saas_user", + "source_application": "Microsoft", + "tenant_number": [ + 2 + ], + "transform_unique_id": "N8G3qDAOmSElCdviQ3d6FpD76pE=", + "user_created": "2024-06-28T08:49:28.000Z", + "user_permissions": [ + { + "is_admin": false, + "name": "OnlineMeetings.ReadWrite" + } + ], + "user_remote_id": "63d52bb0-7ce0-4467-9004-2b19c06b86ae", + "user_type": "Member", + "username": "helen.jordan@demo.local" + } + }, + "cloud": { + "account": { + "id": "c8103abe-eda9-472b-894a-6260bb2ba8cc" + }, + "provider": "Azure" + }, + "data_stream": { + "dataset": "axonius.identity", + "namespace": "42638", + "type": "logs" + }, + "ecs": { + "version": "9.2.0" + }, + "elastic_agent": { + "id": "2009affa-84ca-46da-9030-1686d937a20d", + "snapshot": false, + "version": "8.18.0" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "iam" + ], + "created": "2024-06-28T08:49:28.000Z", + "dataset": "axonius.identity", + "ingested": "2026-04-21T12:29:16Z", + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "cel" + }, + "related": { + "user": [ + "developers-group", + "helen.jordan@demo.local" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields", + "forwarded", + "axonius-identity" + ], + "user": { + "domain": "demo.local", + "email": "helen.jordan@demo.local", + "name": "helen.jordan@demo.local" + } +} diff --git a/packages/axonius/docs/README.md b/packages/axonius/docs/README.md index bfb14767a32..7041ab76311 100644 --- a/packages/axonius/docs/README.md +++ b/packages/axonius/docs/README.md @@ -50,6 +50,26 @@ This integration collects log messages of the following type: - nat_rules (endpoint: `/api/v2/nat_rules`) - network_routes (endpoint: `/api/v2/network_routes`) +- `Identity`: Collect details of all identity assets including: + - users (endpoint: `/api/v2/users`) + - groups (endpoint: `/api/v2/groups`) + - security_roles (endpoint: `/api/v2/security_roles`) + - organizational_units (endpoint: `/api/v2/organizational_units`) + - accounts (endpoint: `/api/v2/accounts`) + - certificates (endpoint: `/api/v2/certificates`) + - permissions (endpoint: `/api/v2/permissions`) + - latest_rules (endpoint: `/api/v2/latest_rules`) + - profiles (endpoint: `/api/v2/profiles`) + - job_titles (endpoint: `/api/v2/job_titles`) + - access_review_campaign_instances (endpoint: `/api/v2/access_review_campaign_instances`) + - access_review_approval_items (endpoint: `/api/v2/access_review_approval_items`) + +### Supported use cases + +Integrating the Axonius Identity Datastream with Elastic SIEM provides a unified view of users, groups, roles, organizational units, accounts, permissions, certificates, profiles, and access review activity. Metrics and breakdowns help teams quickly assess identity posture by highlighting active, inactive, suspended, and external users, as well as patterns across user types and departments. + +Tables showing top email addresses and cloud providers add context into frequently used identities and their sources. These insights help security and IAM teams detect identity anomalies, validate account hygiene, and maintain strong visibility into access across the organization. + ### Supported use cases Integrating the Axonius Adapter, User, Gateway, Exposure, Alert, Incident, Storage, Ticket, and Network data streams with Elastic SIEM provides centralized, end-to-end visibility across data ingestion, identity posture, network configuration, vulnerability exposure, security events, storage assets, ticketing, and network activity. Together, these data streams help analysts understand how data flows into the platform, how it maps to users and access, how gateways and network assets operate, where risks and exposures exist, and how alerts evolve into incidents and tracked issues. @@ -136,12 +156,14 @@ Destinations indices are aliased to `logs-axonius_latest.`. | `logs-axonius.gateway-*` | `logs-axonius_latest.dest_gateway-*` | `logs-axonius_latest.gateway` | | `logs-axonius.incident-*` | `logs-axonius_latest.dest_incident-*` | `logs-axonius_latest.incident` | | `logs-axonius.user-*` | `logs-axonius_latest.dest_user-*` | `logs-axonius_latest.user` | -| `logs-axonius.storage-*` | `logs-axonius_latest.dest_storage-*` | `logs-axonius_latest.storage` | -| `logs-axonius.ticket-*` | `logs-axonius_latest.dest_ticket-*` | `logs-axonius_latest.ticket` -| `logs-axonius.network-*` | `logs-axonius_latest.dest_network-*` | `logs-axonius_latest.network` +| `logs-axonius.storage-*` | `logs-axonius_latest.dest_storage-*` | `logs-axonius_latest.storage` | +| `logs-axonius.ticket-*` | `logs-axonius_latest.dest_ticket-*` | `logs-axonius_latest.ticket` | +| `logs-axonius.network-*` | `logs-axonius_latest.dest_network-*` | `logs-axonius_latest.network` | +| `logs-axonius.identity-*` | `logs-axonius_latest.dest_identity-*` | `logs-axonius_latest.identity` | + **Note:** Assets deleted from Axonius may reappear in a future discovery cycle if they are still present in connected data sources and get re-detected. Because the exact duration for which a deleted asset may remain dormant before being rediscovered is unknown, the transform retention period is set to **90 days** to reduce the risk of data loss for such assets. This means deleted assets will continue to appear in dashboards for up to 90 days after deletion. -The network destination index is a content-based deduplicated view, not an entity-level latest-state view like the other data streams (for example `user` and `gateway`), which rely on a unique entity identifier and reflect the latest state of each entity. +The network and identity destination indices are a content-based deduplicated view, not an entity-level latest-state view like the other data streams (for example `user` and `gateway`), which rely on a unique entity identifier and reflect the latest state of each entity. ## Troubleshooting @@ -2046,6 +2068,626 @@ An example event for `network` looks as following: } ``` +### Identity + +The `identity` data stream provides identity asset logs from axonius. + +#### identity fields + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events. | date | +| axonius.identity.account_disabled | Indicates whether the user account is disabled. | boolean | +| axonius.identity.accurate_for_datetime | Timestamp indicating when this asset information was accurate. | date | +| axonius.identity.active | The active status of the identity. | keyword | +| axonius.identity.active_users | Number of active users in the account. | long | +| axonius.identity.active_users_saved_query_id | Saved query ID for the active users metric. | keyword | +| axonius.identity.adapter_list_length | How many adapters contributed to this asset. | long | +| axonius.identity.adapters | List of adapters that created this asset. | keyword | +| axonius.identity.admin_non_operational_users | Number of admin users that are non-operational. | long | +| axonius.identity.admin_non_operational_users_saved_query_id | Saved query ID for the admin non-operational users metric. | keyword | +| axonius.identity.admin_operational_active_users | Number of admin users that are both operational and active. | long | +| axonius.identity.admin_operational_active_users_saved_query_id | Saved query ID for the admin operational active users metric. | keyword | +| axonius.identity.admin_operational_inactive_users | Number of admin users that are operational but inactive. | long | +| axonius.identity.admin_operational_inactive_users_saved_query_id | Saved query ID for the admin operational inactive users metric. | keyword | +| axonius.identity.admin_operational_users | Number of admin users that are operational. | long | +| axonius.identity.admin_operational_users_saved_query_id | Saved query ID for the admin operational users metric. | keyword | +| axonius.identity.admin_roles.display_name | Display name of the admin role. | keyword | +| axonius.identity.admin_roles.id | Unique identifier of the admin role. | keyword | +| axonius.identity.admins | Total number of administrators in the account. | long | +| axonius.identity.admins_saved_query_id | Saved query ID for the admins metric. | keyword | +| axonius.identity.alt_names.name | The alternative name value. | keyword | +| axonius.identity.alt_names.name_type | The type of alternative name (e.g., DNS, IP). | keyword | +| axonius.identity.application_and_account_name | The application and account name associated with the asset. | keyword | +| axonius.identity.application_id | Unique identifier of the application. | keyword | +| axonius.identity.application_name | Name of the application associated with this identity. | keyword | +| axonius.identity.asset_entity_info | Information about the asset entity and its properties. | keyword | +| axonius.identity.asset_type | The type of asset. | keyword | +| axonius.identity.associated_devices.device_associated_saas_apps_names | Names of SaaS applications associated with the device. | keyword | +| axonius.identity.associated_devices.device_caption | Caption or display name of the associated device. | keyword | +| axonius.identity.associated_devices.device_id | Unique identifier of the associated device. | keyword | +| axonius.identity.associated_devices.device_labels | Labels or tags assigned to the associated device. | keyword | +| axonius.identity.associated_devices.device_model | Model name of the associated device. | keyword | +| axonius.identity.associated_devices.device_os_distribution | Operating system distribution of the associated device. | keyword | +| axonius.identity.associated_devices.device_os_edition | Operating system edition of the associated device. | keyword | +| axonius.identity.associated_devices.device_os_end_of_life | End-of-life date of the device operating system. | keyword | +| axonius.identity.associated_devices.device_os_type | Operating system type of the associated device. | keyword | +| axonius.identity.associated_devices.device_os_version | Operating system version of the associated device. | keyword | +| axonius.identity.associated_devices.device_preferred_mac_address | Preferred MAC address of the associated device. | keyword | +| axonius.identity.associated_devices.device_serial | Serial number of the associated device. | keyword | +| axonius.identity.associated_devices.internal_axon_id | Internal Axonius ID of the associated device. | keyword | +| axonius.identity.associated_employees.internal_axon_id | Internal Axonius ID of the associated employee. | keyword | +| axonius.identity.associated_employees.username | Username of the associated employee. | keyword | +| axonius.identity.associated_groups.display_name | Display name of the associated group. | keyword | +| axonius.identity.associated_groups.remote_id | Remote identifier of the associated group. | keyword | +| axonius.identity.associated_licenses.adapter_connection_label | Label of the adapter connection for the license. | keyword | +| axonius.identity.associated_licenses.internal_axon_id | Internal Axonius ID of the license. | keyword | +| axonius.identity.associated_licenses.license_name | Name of the license. | keyword | +| axonius.identity.associated_licenses.pricing_unit | Pricing unit of the license. | keyword | +| axonius.identity.associated_licenses.related_vendor_name | Vendor name associated with the license. | keyword | +| axonius.identity.associated_licenses.unit_price | Unit price of the license. | keyword | +| axonius.identity.aws_arn | Amazon Web Services ARN (Amazon Resource Name) for this identity. | keyword | +| axonius.identity.aws_iam_identity_type | AWS IAM identity type (e.g., user, role, group). | keyword | +| axonius.identity.azure_account_id | Azure account identifier associated with this identity. | keyword | +| axonius.identity.begins_on | Start date of the certificate validity period. | date | +| axonius.identity.bit_size | Key bit size of the certificate. | long | +| axonius.identity.breaches_data.added_date | Date when the breach was added to the database. | date | +| axonius.identity.breaches_data.breach_date | Date when the breach occurred. | date | +| axonius.identity.breaches_data.data_classes | Types of data exposed in the breach. | keyword | +| axonius.identity.breaches_data.domain | Domain affected by the breach. | keyword | +| axonius.identity.breaches_data.is_fabricated | Indicates if the breach data is fabricated. | boolean | +| axonius.identity.breaches_data.is_retired | Indicates if the breach record has been retired. | boolean | +| axonius.identity.breaches_data.is_sensitive | Indicates if the breach contains sensitive data. | boolean | +| axonius.identity.breaches_data.is_spam_list | Indicates if the breach is from a spam list. | boolean | +| axonius.identity.breaches_data.is_verified | Indicates if the breach has been verified. | boolean | +| axonius.identity.breaches_data.logo_path | Path to the logo of the breached service. | keyword | +| axonius.identity.breaches_data.modified_date | Date when the breach record was last modified. | date | +| axonius.identity.breaches_data.name | Name of the breach. | keyword | +| axonius.identity.breaches_data.pwn_count | Number of accounts affected by the breach. | long | +| axonius.identity.breaches_data.title | Title of the breach. | keyword | +| axonius.identity.class_name | The class name or system classification of this asset. | keyword | +| axonius.identity.cloud_provider | The cloud provider associated with this identity. | keyword | +| axonius.identity.connected_assets | Other assets connected to or associated with this identity asset. | keyword | +| axonius.identity.connection_label | Label of the adapter connection used to collect this identity data. | keyword | +| axonius.identity.created_date | Date when this identity record was created. | date | +| axonius.identity.data_asset_type | The asset type from identity event data, distinguishing from root asset_type. | keyword | +| axonius.identity.deleted_users | Number of deleted users in the account. | long | +| axonius.identity.deleted_users_saved_query_id | Saved query ID for the deleted users metric. | keyword | +| axonius.identity.description | The description of the asset. | text | +| axonius.identity.direct_not_sso_users | Number of users with direct access who are not using SSO. | long | +| axonius.identity.direct_not_sso_users_saved_query_id | Saved query ID for the direct non-SSO users metric. | keyword | +| axonius.identity.display_name | Display name of the identity. | keyword | +| axonius.identity.distinct_associated_devices_count | Number of distinct devices associated with this identity. | long | +| axonius.identity.domains.name | Name of the domain. | keyword | +| axonius.identity.email | Email address of the identity. | keyword | +| axonius.identity.email_activity.is_deleted | Indicates if the email activity record has been deleted. | boolean | +| axonius.identity.email_activity.product_license | Product license associated with email activity. | keyword | +| axonius.identity.email_activity.read_count | Number of emails read during the report period. | long | +| axonius.identity.email_activity.receive_count | Number of emails received during the report period. | long | +| axonius.identity.email_activity.report_date | Date of the email activity report. | date | +| axonius.identity.email_activity.report_period | Reporting period in days for the email activity. | long | +| axonius.identity.email_activity.send_count | Number of emails sent during the report period. | long | +| axonius.identity.email_notification.alternative_host_reminder | Indicates if alternative host reminder emails are enabled. | boolean | +| axonius.identity.email_notification.cancel_meeting_reminder | Indicates if meeting cancellation reminder emails are enabled. | boolean | +| axonius.identity.email_notification.jbh_reminder | Indicates if join-before-host reminder emails are enabled. | boolean | +| axonius.identity.employee_id | Employee identifier assigned to this identity. | keyword | +| axonius.identity.employee_number | Employee number assigned to this identity. | keyword | +| axonius.identity.employee_type | Type of employee (e.g., full-time, contractor). | keyword | +| axonius.identity.event.accurate_for_datetime | Timestamp indicating when the event data was accurate. | date | +| axonius.identity.event.action_if_exists | Action associated with the identity event, if it exists. | keyword | +| axonius.identity.event.adapter_categories | List of adapter categories that this event belongs to. | keyword | +| axonius.identity.event.associated_adapter_plugin_name | The associated plugin name that created or processed the event. | keyword | +| axonius.identity.event.association_type | The type of association between the event and related entities. | keyword | +| axonius.identity.event.client_used | The client identifier that was used to process the event. | keyword | +| axonius.identity.event.entity | The entity type or category this event relates to. | keyword | +| axonius.identity.event.hidden_for_gui | Indicates if this event should be hidden in the GUI. | boolean | +| axonius.identity.event.initial_plugin_unique_name | The initial plugin name that created or processed the event. | keyword | +| axonius.identity.event.name | The name of the event. | keyword | +| axonius.identity.event.plugin_name | The name of the plugin that processed the event. | keyword | +| axonius.identity.event.plugin_type | The type or category of the plugin that processed the event. | keyword | +| axonius.identity.event.plugin_unique_name | The unique identifier of the plugin instance that processed the event. | keyword | +| axonius.identity.event.quick_id | A quick reference identifier combining plugin and entity information. | keyword | +| axonius.identity.event.type | The type or classification of the event data. | keyword | +| axonius.identity.expires_on | Expiration date of the certificate validity period. | date | +| axonius.identity.external_users | Number of external users in the account. | long | +| axonius.identity.external_users_saved_query_id | Saved query ID for the external users metric. | keyword | +| axonius.identity.feature.cn_meeting | Indicates if China meeting feature is enabled. | boolean | +| axonius.identity.feature.in_meeting | Indicates if in-meeting feature is enabled. | boolean | +| axonius.identity.feature.large_meeting | Indicates if large meeting feature is enabled. | boolean | +| axonius.identity.feature.meeting_capacity | Maximum meeting capacity for this identity. | long | +| axonius.identity.feature.webinar | Indicates if webinar feature is enabled. | boolean | +| axonius.identity.feature.zoom_phone | Indicates if Zoom Phone feature is enabled. | boolean | +| axonius.identity.fetch_time | The date and time when the identity data was last fetched. | date | +| axonius.identity.first_fetch_time | The date and time when this identity asset was first fetched. | date | +| axonius.identity.first_name | First name of the identity. | keyword | +| axonius.identity.first_seen | The date and time when this identity was first observed. | date | +| axonius.identity.from_last_fetch | Indicates whether this identity asset was modified since the last fetch. | boolean | +| axonius.identity.gce_account_id | Google Cloud Engine account ID associated with this identity. | keyword | +| axonius.identity.groups.display_name | Display name of the group. | keyword | +| axonius.identity.groups.name | Name of the group. | keyword | +| axonius.identity.groups.remote_id | Remote identifier of the group. | keyword | +| axonius.identity.has_administrative_permissions | Indicates whether this identity has administrative permissions. | boolean | +| axonius.identity.hire_date | Date when the employee was hired. | date | +| axonius.identity.hr_employment_status | Human resources employment status of the identity. | keyword | +| axonius.identity.id | Unique identifier for the identity asset. | keyword | +| axonius.identity.id_raw | Raw unique identifier for the identity asset. | keyword | +| axonius.identity.in_meeting.allow_live_streaming | Indicates if live streaming is allowed during meetings. | boolean | +| axonius.identity.in_meeting.annotation | Indicates if annotation is enabled during meetings. | boolean | +| axonius.identity.in_meeting.attendee_on_hold | Indicates if attendee-on-hold feature is enabled. | boolean | +| axonius.identity.in_meeting.auto_saving_chat | Indicates if auto-saving chat is enabled. | boolean | +| axonius.identity.in_meeting.breakout_room | Indicates if breakout rooms are enabled. | boolean | +| axonius.identity.in_meeting.chat | Indicates if chat is enabled during meetings. | boolean | +| axonius.identity.in_meeting.closed_caption | Indicates if closed captions are enabled. | boolean | +| axonius.identity.in_meeting.co_host | Indicates if co-host feature is enabled. | boolean | +| axonius.identity.in_meeting.data_center_regions | Data center regions configured for meetings. | keyword | +| axonius.identity.in_meeting.e2e_encryption | Indicates if end-to-end encryption is enabled. | boolean | +| axonius.identity.in_meeting.entry_exit_chime | Indicates if entry/exit chime is enabled. | boolean | +| axonius.identity.in_meeting.far_end_camera_control | Indicates if far-end camera control is enabled. | boolean | +| axonius.identity.in_meeting.feedback | Indicates if feedback feature is enabled. | boolean | +| axonius.identity.in_meeting.group_hd | Indicates if group HD video is enabled. | boolean | +| axonius.identity.in_meeting.non_verbal_feedback | Indicates if non-verbal feedback is enabled. | boolean | +| axonius.identity.in_meeting.polling | Indicates if polling is enabled during meetings. | boolean | +| axonius.identity.in_meeting.private_chat | Indicates if private chat is enabled during meetings. | boolean | +| axonius.identity.in_meeting.record_play_voice | Indicates if record and play voice is enabled. | boolean | +| axonius.identity.in_meeting.remote_control | Indicates if remote control is enabled. | boolean | +| axonius.identity.in_meeting.remote_support | Indicates if remote support is enabled. | boolean | +| axonius.identity.in_meeting.share_dual_camera | Indicates if dual camera sharing is enabled. | boolean | +| axonius.identity.in_meeting.show_meeting_control_toolbar | Indicates if meeting control toolbar is shown. | boolean | +| axonius.identity.in_meeting.virtual_background | Indicates if virtual background is enabled. | boolean | +| axonius.identity.in_meeting.waiting_room | Indicates if waiting room is enabled. | boolean | +| axonius.identity.in_meeting.workplace_by_facebook | Indicates if Workplace by Facebook integration is enabled. | boolean | +| axonius.identity.inactive_users | Number of inactive users in the account. | long | +| axonius.identity.inactive_users_saved_query_id | Saved query ID for the inactive users metric. | keyword | +| axonius.identity.internal_axon_id | Internal ID of this asset. This ID may change in the future. | keyword | +| axonius.identity.internal_is_admin | Internal flag indicating if this identity has admin privileges. | boolean | +| axonius.identity.is_active | Indicates whether this identity is currently active. | boolean | +| axonius.identity.is_admin | Indicates whether this identity has administrator privileges. | boolean | +| axonius.identity.is_built_in | Indicates whether this is a built-in system account. | boolean | +| axonius.identity.is_delegated_admin | Indicates whether this identity is a delegated administrator. | boolean | +| axonius.identity.is_fetched_from_adapter | Indicates whether this identity data was fetched from an adapter. | boolean | +| axonius.identity.is_from_sso_provider | Indicates whether this identity originates from a Single Sign-On provider. | boolean | +| axonius.identity.is_latest_last_seen | Indicates if this is the latest recorded last-seen timestamp. | boolean | +| axonius.identity.is_managed_by_application | Indicates whether this identity is managed by an application. | boolean | +| axonius.identity.is_managed_by_direct_app | Indicates whether this identity is managed by a direct application. | boolean | +| axonius.identity.is_managed_by_sso | Indicates whether this identity is managed through SSO. | boolean | +| axonius.identity.is_mfa_enforced | Indicates whether multi-factor authentication is enforced. | boolean | +| axonius.identity.is_mfa_enrolled | Indicates whether this identity is enrolled in multi-factor authentication. | boolean | +| axonius.identity.is_non_editable | Indicates whether this identity record is non-editable. | boolean | +| axonius.identity.is_paid | Indicates whether this identity has a paid license or account. | boolean | +| axonius.identity.is_permission_adapter | Indicates whether this identity was collected by a permission adapter. | boolean | +| axonius.identity.is_privileged | Indicates whether this identity has privileged access. | boolean | +| axonius.identity.is_saas_user | Indicates whether this identity is a SaaS application user. | boolean | +| axonius.identity.is_user_active | Indicates whether the user account is active. | boolean | +| axonius.identity.is_user_deleted | Indicates whether the user account has been deleted. | boolean | +| axonius.identity.is_user_external | Indicates whether this is an external user. | boolean | +| axonius.identity.is_user_inactive | Indicates whether the user account is inactive. | boolean | +| axonius.identity.is_user_suspended | Indicates whether the user account is suspended. | boolean | +| axonius.identity.issuer.common_name | Common name of the certificate issuer. | keyword | +| axonius.identity.issuer.country_name | Country name of the certificate issuer. | keyword | +| axonius.identity.issuer.organization | Organization name of the certificate issuer. | keyword | +| axonius.identity.last_client_version | Version of the last client used by this identity. | keyword | +| axonius.identity.last_enrichment_run | Date of the last enrichment run for this identity. | date | +| axonius.identity.last_fetch_connection_id | The connection ID of the adapter that last fetched this data. | keyword | +| axonius.identity.last_fetch_connection_label | The label of the connection that last fetched this identity data. | keyword | +| axonius.identity.last_login_attempt | Date and time of the last login attempt. | date | +| axonius.identity.last_logon | Date and time of the last successful logon. | date | +| axonius.identity.last_name | Last name of the identity. | keyword | +| axonius.identity.last_password_change | Date and time when the password was last changed. | date | +| axonius.identity.last_seen | The date and time when this identity was last observed. | date | +| axonius.identity.mail | Email address (mail attribute) of the identity. | keyword | +| axonius.identity.managed_non_operational_users | Number of managed users that are non-operational. | long | +| axonius.identity.managed_non_operational_users_saved_query_id | Saved query ID for the managed non-operational users metric. | keyword | +| axonius.identity.managed_operational_users | Number of managed users that are operational. | long | +| axonius.identity.managed_operational_users_saved_query_id | Saved query ID for the managed operational users metric. | keyword | +| axonius.identity.managed_users | Total number of managed users in the account. | long | +| axonius.identity.managed_users_by_app | Number of users managed by a direct application. | long | +| axonius.identity.managed_users_by_app_saved_query_id | Saved query ID for the managed-by-app users metric. | keyword | +| axonius.identity.managed_users_by_sso | Number of users managed through SSO. | long | +| axonius.identity.managed_users_by_sso_saved_query_id | Saved query ID for the managed-by-SSO users metric. | keyword | +| axonius.identity.managed_users_saved_query_id | Saved query ID for the managed users metric. | keyword | +| axonius.identity.manager_id | Identifier of the manager of this identity. | keyword | +| axonius.identity.max_added_date | Most recent date a breach was added across all breaches for this identity. | date | +| axonius.identity.max_breach_date | Most recent breach date across all breaches for this identity. | date | +| axonius.identity.max_modified_date | Most recent modified date across all breaches for this identity. | date | +| axonius.identity.name | The name or identifier of the identity asset. | keyword | +| axonius.identity.nested_applications.active_from_direct_adapter | Indicates if active status is from a direct adapter. | boolean | +| axonius.identity.nested_applications.app_accounts.name | Name of the application account. | keyword | +| axonius.identity.nested_applications.app_display_name | Display name of the application. | keyword | +| axonius.identity.nested_applications.app_links | Links or URLs associated with the application. | keyword | +| axonius.identity.nested_applications.assignment_type | How the application was assigned (e.g., direct, group). | keyword | +| axonius.identity.nested_applications.extension_type | Type of extension for the application. | keyword | +| axonius.identity.nested_applications.has_administrative_permissions | Indicates if the identity has admin permissions in this application. | boolean | +| axonius.identity.nested_applications.is_deleted | Indicates if the application assignment has been deleted. | boolean | +| axonius.identity.nested_applications.is_from_direct_adapter | Indicates if the data is from a direct adapter. | boolean | +| axonius.identity.nested_applications.is_managed | Indicates if the application is managed. | boolean | +| axonius.identity.nested_applications.is_suspended | Indicates if the application access is suspended. | boolean | +| axonius.identity.nested_applications.is_unmanaged_extension | Indicates if this is an unmanaged browser extension. | boolean | +| axonius.identity.nested_applications.is_user_external | Indicates if the user is external in this application. | boolean | +| axonius.identity.nested_applications.is_user_paid | Indicates if the user has a paid license in this application. | boolean | +| axonius.identity.nested_applications.last_access | Date and time of the last access to the application. | date | +| axonius.identity.nested_applications.last_access_count | Total number of accesses to the application. | long | +| axonius.identity.nested_applications.last_access_count_60_days | Number of accesses to the application in the last 60 days. | long | +| axonius.identity.nested_applications.last_access_count_90_days | Number of accesses to the application in the last 90 days. | long | +| axonius.identity.nested_applications.name | Name of the application. | keyword | +| axonius.identity.nested_applications.parents.name | Name of the parent entity. | keyword | +| axonius.identity.nested_applications.parents.value | Value or identifier of the parent entity. | keyword | +| axonius.identity.nested_applications.permissions.name | Name of the permission. | keyword | +| axonius.identity.nested_applications.relation_direct_name | Name of the direct relationship to the application. | keyword | +| axonius.identity.nested_applications.relation_discovery_name | Name of the discovered relationship to the application. | keyword | +| axonius.identity.nested_applications.relation_extension_name | Name of the extension-based relationship to the application. | keyword | +| axonius.identity.nested_applications.relation_sso_name | Name of the SSO-based relationship to the application. | keyword | +| axonius.identity.nested_applications.source_application | Source application that provided this data. | keyword | +| axonius.identity.nested_applications.value | Value or identifier of the application. | keyword | +| axonius.identity.nested_applications.vendor_category | Vendor category of the application. | keyword | +| axonius.identity.nested_associated_devices | Flattened list of nested associated device identifiers. | keyword | +| axonius.identity.nested_grants_last_updated | Date when nested grants were last updated. | date | +| axonius.identity.nested_grants_managers_last_updated | Date when nested grants managers were last updated. | date | +| axonius.identity.nested_groups.assignment_type | How the group was assigned (e.g., direct, inherited). | keyword | +| axonius.identity.nested_groups.group_name | Name of the group. | keyword | +| axonius.identity.nested_groups.name | Display name of the group entry. | keyword | +| axonius.identity.nested_groups.parents.name | Name of the parent entity. | keyword | +| axonius.identity.nested_groups.parents.parent_type | Type of the parent entity. | keyword | +| axonius.identity.nested_groups.parents.value | Value or identifier of the parent entity. | keyword | +| axonius.identity.nested_groups.value | Value or identifier of the group. | keyword | +| axonius.identity.nested_managers.assignment_type | How the manager was assigned. | keyword | +| axonius.identity.nested_managers.parents.name | Name of the parent entity. | keyword | +| axonius.identity.nested_managers.parents.parent_type | Type of the parent entity. | keyword | +| axonius.identity.nested_managers.parents.value | Value or identifier of the parent entity. | keyword | +| axonius.identity.nested_managers.value | Value or identifier of the manager. | keyword | +| axonius.identity.nested_permissions.assignment_type | How the permission was assigned (e.g., direct, inherited). | keyword | +| axonius.identity.nested_permissions.has_administrative_permissions | Indicates if the identity has administrative permissions. | boolean | +| axonius.identity.nested_permissions.is_admin | Indicates if the identity has admin privileges. | boolean | +| axonius.identity.nested_permissions.parents.name | Name of the parent entity. | keyword | +| axonius.identity.nested_permissions.parents.parent_type | Type of the parent entity. | keyword | +| axonius.identity.nested_permissions.parents.value | Value or identifier of the parent entity. | keyword | +| axonius.identity.nested_permissions.value | Value or identifier of the permission. | keyword | +| axonius.identity.nested_resources.assignment_type | How the resource was assigned. | keyword | +| axonius.identity.nested_resources.name | Name of the resource. | keyword | +| axonius.identity.nested_resources.parents.name | Name of the parent entity. | keyword | +| axonius.identity.nested_resources.parents.value | Value or identifier of the parent entity. | keyword | +| axonius.identity.nested_resources.value | Value or identifier of the resource. | keyword | +| axonius.identity.nested_roles.assignment_type | How the role was assigned (e.g., direct, inherited). | keyword | +| axonius.identity.nested_roles.name | Name of the role. | keyword | +| axonius.identity.nested_roles.parents.name | Name of the parent entity. | keyword | +| axonius.identity.nested_roles.parents.parent_type | Type of the parent entity. | keyword | +| axonius.identity.nested_roles.parents.value | Value or identifier of the parent entity. | keyword | +| axonius.identity.nested_roles.value | Value or identifier of the role. | keyword | +| axonius.identity.not_fetched_count | The number of times this identity asset failed to be fetched. | long | +| axonius.identity.operational_users_count | Total number of operational users in the account. | long | +| axonius.identity.oracle_cloud_cis_incompliant.rule_cis_version | CIS benchmark version of the incompliant rule. | float | +| axonius.identity.oracle_cloud_cis_incompliant.rule_section | Section number of the incompliant CIS rule. | keyword | +| axonius.identity.orphaned_users | Number of orphaned users in the account. | long | +| axonius.identity.orphaned_users_saved_query_id | Saved query ID for the orphaned users metric. | keyword | +| axonius.identity.paid_users | Number of paid users in the account. | long | +| axonius.identity.paid_users_saved_query_id | Saved query ID for the paid users metric. | keyword | +| axonius.identity.password_never_expires | Indicates whether the password is set to never expire. | boolean | +| axonius.identity.password_not_required | Indicates whether a password is not required for this account. | boolean | +| axonius.identity.permissions | Total number of permissions assigned to the identity. | long | +| axonius.identity.permissions_list.name | Name of the permission. | keyword | +| axonius.identity.pmi | Personal Meeting ID (Zoom). | keyword | +| axonius.identity.pretty_id | A human-readable identifier for the identity asset. | keyword | +| axonius.identity.project_ids | Cloud project IDs associated with this identity. | keyword | +| axonius.identity.project_tags.inherited | Indicates if the tag is inherited from a parent resource. | keyword | +| axonius.identity.project_tags.key | Tag key. | keyword | +| axonius.identity.project_tags.namespaced_tag_key | Namespaced version of the tag key. | keyword | +| axonius.identity.project_tags.namespaced_tag_value | Namespaced version of the tag value. | keyword | +| axonius.identity.project_tags.value | Tag value. | keyword | +| axonius.identity.projects_roles.project_id | Identifier of the project. | keyword | +| axonius.identity.projects_roles.role_name | Name of the role in the project. | keyword | +| axonius.identity.provider_name | Name of the identity provider. | keyword | +| axonius.identity.provider_type | Type of the identity provider. | keyword | +| axonius.identity.recording.auto_delete_cmr | Indicates if cloud meeting recordings are auto-deleted. | boolean | +| axonius.identity.recording.auto_delete_cmr_days | Indicates if auto-delete days for cloud recordings is configured. | boolean | +| axonius.identity.recording.auto_recording | Indicates if auto-recording is enabled. | boolean | +| axonius.identity.recording.cloud_recording | Indicates if cloud recording is enabled. | boolean | +| axonius.identity.recording.host_pause_stop_recording | Indicates if host can pause or stop recording. | boolean | +| axonius.identity.recording.local_recording | Indicates if local recording is enabled. | boolean | +| axonius.identity.recording.record_audio_file | Indicates if a separate audio file is recorded. | boolean | +| axonius.identity.recording.record_gallery_view | Indicates if gallery view is recorded. | boolean | +| axonius.identity.recording.record_speaker_view | Indicates if speaker view is recorded. | boolean | +| axonius.identity.recording.recording_audio_transcript | Indicates if audio transcript is generated for recordings. | boolean | +| axonius.identity.recording.save_chat_text | Indicates if chat text is saved with recordings. | boolean | +| axonius.identity.recording.show_timestamp | Indicates if timestamp is shown in recordings. | boolean | +| axonius.identity.recovery_question_set | Indicates whether a recovery question has been set for this identity. | boolean | +| axonius.identity.relatable_ids | IDs used to relate this identity to other assets. | keyword | +| axonius.identity.remote_account_id | Remote account identifier for this identity. | keyword | +| axonius.identity.remote_id | Remote identifier for this identity in the source system. | keyword | +| axonius.identity.roles.display_name | Display Name of the role. | keyword | +| axonius.identity.roles.remote_id | Remote ID of the role. | keyword | +| axonius.identity.roles_accounts | Account roles. | keyword | +| axonius.identity.schedule_meeting.audio_type | Audio type configured for scheduled meetings. | keyword | +| axonius.identity.schedule_meeting.force_pmi_jbh_password | Indicates if PMI join-before-host password is forced. | boolean | +| axonius.identity.schedule_meeting.host_video | Indicates if host video is on when joining a meeting. | boolean | +| axonius.identity.schedule_meeting.join_before_host | Indicates if participants can join before the host. | boolean | +| axonius.identity.schedule_meeting.participants_video | Indicates if participant video is on when joining a meeting. | boolean | +| axonius.identity.schedule_meeting.pstn_password_protected | Indicates if PSTN dial-in is password protected. | boolean | +| axonius.identity.schedule_meeting.require_password_for_instant_meetings | Indicates if password is required for instant meetings. | boolean | +| axonius.identity.schedule_meeting.require_password_for_pmi_meetings | Indicates if password is required for PMI meetings. | boolean | +| axonius.identity.schedule_meeting.require_password_for_scheduled_meetings | Indicates if password is required for scheduled meetings. | boolean | +| axonius.identity.schedule_meeting.require_password_for_scheduling_new_meetings | Indicates if password is required when scheduling new meetings. | boolean | +| axonius.identity.schedule_meeting.use_pmi_for_instant_meetings | Indicates if PMI is used for instant meetings. | boolean | +| axonius.identity.schedule_meeting.use_pmi_for_scheduled_meetings | Indicates if PMI is used for scheduled meetings. | boolean | +| axonius.identity.serial_number | Serial number of the certificate. | keyword | +| axonius.identity.shirt_size | Shirt size of the employee (HR attribute). | keyword | +| axonius.identity.sm_entity_type | SaaS management entity type for this identity. | keyword | +| axonius.identity.snow_full_name | Full name of the identity from ServiceNow. | keyword | +| axonius.identity.snow_location | Location of the identity from ServiceNow. | keyword | +| axonius.identity.source_application | The source application that provided this identity data. | keyword | +| axonius.identity.status | Current status of the identity account. | keyword | +| axonius.identity.status_changed | Date and time when the account status was last changed. | date | +| axonius.identity.subject.common_name | Common name of the certificate subject. | keyword | +| axonius.identity.subject.country_name | Country name of the certificate subject. | keyword | +| axonius.identity.subject.locality | Locality (city) of the certificate subject. | keyword | +| axonius.identity.subject.organization | Organization name of the certificate subject. | keyword | +| axonius.identity.subject.state | State or province of the certificate subject. | keyword | +| axonius.identity.suspended_users | Number of suspended users in the account. | long | +| axonius.identity.suspended_users_saved_query_id | Saved query ID for the suspended users metric. | keyword | +| axonius.identity.telephony.show_international_numbers_link | Indicates if international numbers link is shown. | boolean | +| axonius.identity.telephony.third_party_audio | Indicates if third-party audio is enabled. | boolean | +| axonius.identity.tenant_number | Tenant number associated with this identity. | long | +| axonius.identity.timezone | Timezone configured for this identity. | keyword | +| axonius.identity.total_users_count | Total number of users in the account. | long | +| axonius.identity.transform_unique_id | Unique identifier for this asset in the transformation process. | keyword | +| axonius.identity.tsp.call_out | Indicates if TSP call-out is enabled. | boolean | +| axonius.identity.tsp.show_international_numbers_link | Indicates if international numbers link is shown for TSP. | boolean | +| axonius.identity.type | The type or classification of the identity entity. | keyword | +| axonius.identity.u_department | Department of the identity from ServiceNow. | keyword | +| axonius.identity.u_vip | Indicates whether this identity is flagged as a VIP in ServiceNow. | boolean | +| axonius.identity.unlinked_users | Number of unlinked users in the account. | long | +| axonius.identity.unlinked_users_saved_query_id | Saved query ID for the unlinked users metric. | keyword | +| axonius.identity.updated_on | Date and time when this identity record was last updated. | date | +| axonius.identity.user_apps.active_from_direct_adapter | Indicates if active status is from a direct adapter. | boolean | +| axonius.identity.user_apps.app_accounts.name | Name of the application account. | keyword | +| axonius.identity.user_apps.app_display_name | Display name of the application. | keyword | +| axonius.identity.user_apps.app_id | Unique identifier of the application. | keyword | +| axonius.identity.user_apps.app_links | Links or URLs associated with the application. | keyword | +| axonius.identity.user_apps.app_name | Name of the application. | keyword | +| axonius.identity.user_apps.extension_type | Type of extension for the application. | keyword | +| axonius.identity.user_apps.is_from_direct_adapter | Indicates if the data is from a direct adapter. | boolean | +| axonius.identity.user_apps.is_managed | Indicates if the application is managed. | boolean | +| axonius.identity.user_apps.is_saas_application | Indicates if this is a SaaS application. | boolean | +| axonius.identity.user_apps.is_unmanaged_extension | Indicates if this is an unmanaged browser extension. | boolean | +| axonius.identity.user_apps.is_user_deleted | Indicates if the user has been deleted in the application. | boolean | +| axonius.identity.user_apps.is_user_external | Indicates if the user is external in the application. | boolean | +| axonius.identity.user_apps.is_user_paid | Indicates if the user has a paid license in the application. | boolean | +| axonius.identity.user_apps.is_user_suspended | Indicates if the user is suspended in the application. | boolean | +| axonius.identity.user_apps.last_access | Date and time of the last access to the application. | date | +| axonius.identity.user_apps.permissions.name | Name of the permission. | keyword | +| axonius.identity.user_apps.relation_direct_name | Name of the direct relationship to the application. | keyword | +| axonius.identity.user_apps.relation_discovery_name | Name of the discovered relationship to the application. | keyword | +| axonius.identity.user_apps.relation_extension_name | Name of the extension-based relationship to the application. | keyword | +| axonius.identity.user_apps.relation_sso_name | Name of the SSO-based relationship to the application. | keyword | +| axonius.identity.user_apps.source_application | Source application that provided this data. | keyword | +| axonius.identity.user_apps.vendor_category | Vendor category of the application. | keyword | +| axonius.identity.user_count | Number of users in the application. | long | +| axonius.identity.user_count_link.bracketWeight | Weight of the bracket in the query expression. | double | +| axonius.identity.user_count_link.compOp | Comparison operator used in the query. | keyword | +| axonius.identity.user_count_link.field | Field name used in the query filter. | keyword | +| axonius.identity.user_count_link.leftBracket | Left bracket position in the query expression. | double | +| axonius.identity.user_count_link.logicOp | Logical operator (e.g., AND, OR) in the query. | keyword | +| axonius.identity.user_count_link.not | Indicates if the query condition is negated. | boolean | +| axonius.identity.user_count_link.rightBracket | Right bracket position in the query expression. | double | +| axonius.identity.user_count_link.value | Value used in the query filter. | keyword | +| axonius.identity.user_country | Country of the user. | keyword | +| axonius.identity.user_created | Date and time when the user account was created. | date | +| axonius.identity.user_department | Department the user belongs to. | keyword | +| axonius.identity.user_factors.created | Date when the MFA factor was created. | date | +| axonius.identity.user_factors.factor_status | Current status of the MFA factor. | keyword | +| axonius.identity.user_factors.factor_type | Type of the MFA factor (e.g., push, TOTP, SMS). | keyword | +| axonius.identity.user_factors.is_enabled | Indicates if the MFA factor is enabled. | boolean | +| axonius.identity.user_factors.last_updated | Date when the MFA factor was last updated. | date | +| axonius.identity.user_factors.name | Name of the MFA factor. | keyword | +| axonius.identity.user_factors.provider | Provider of the MFA factor. | keyword | +| axonius.identity.user_factors.strength | Strength rating of the MFA factor. | keyword | +| axonius.identity.user_factors.vendor_name | Vendor name of the MFA factor. | keyword | +| axonius.identity.user_full_name | Full name of the user. | keyword | +| axonius.identity.user_is_password_enabled | Indicates whether password authentication is enabled for this user. | boolean | +| axonius.identity.user_manager | Name or identifier of the user's manager. | keyword | +| axonius.identity.user_manager_mail | Email address of the user's manager. | keyword | +| axonius.identity.user_pass_last_used | Date or timestamp when the user's password was last used. | date | +| axonius.identity.user_path | Path of the user in the directory (e.g., AWS IAM path). | keyword | +| axonius.identity.user_permissions.is_admin | Indicates if the user has admin privileges for this permission. | boolean | +| axonius.identity.user_permissions.name | Name of the permission. | keyword | +| axonius.identity.user_related_resources.id | Identifier of the related resource. | keyword | +| axonius.identity.user_related_resources.name | Name of the related resource. | keyword | +| axonius.identity.user_related_resources.type | Type of the related resource. | keyword | +| axonius.identity.user_remote_id | Remote identifier of the user in the source system. | keyword | +| axonius.identity.user_sid | Security Identifier (SID) of the user (Windows/AD). | keyword | +| axonius.identity.user_status | Current status of the user account. | keyword | +| axonius.identity.user_telephone_number | Telephone number of the user. | keyword | +| axonius.identity.user_title | Job title of the user. | keyword | +| axonius.identity.user_type | Type of user account (e.g., member, guest, service). | keyword | +| axonius.identity.username | Username of the identity. | keyword | +| axonius.identity.verified | Indicates whether this identity has been verified. | boolean | +| axonius.identity.version | Version of the certificate or identity record. | keyword | +| data_stream.dataset | The field can contain anything that makes sense to signify the source of the data. Examples include `nginx.access`, `prometheus`, `endpoint` etc. For data streams that otherwise fit, but that do not have dataset set we use the value "generic" for the dataset value. `event.dataset` should have the same value as `data_stream.dataset`. Beyond the Elasticsearch data stream naming criteria noted above, the `dataset` value has additional restrictions: \* Must not contain `-` \* No longer than 100 characters | constant_keyword | +| data_stream.namespace | A user defined namespace. Namespaces are useful to allow grouping of data. Many users already organize their indices this way, and the data stream naming scheme now provides this best practice as a default. Many users will populate this field with `default`. If no value is used, it falls back to `default`. Beyond the Elasticsearch index naming criteria noted above, `namespace` value has the additional restrictions: \* Must not contain `-` \* No longer than 100 characters | constant_keyword | +| data_stream.type | An overarching type for the data stream. Currently allowed values are "logs" and "metrics". We expect to also add "traces" and "synthetics" in the near future. | constant_keyword | +| event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | constant_keyword | +| event.module | Name of the module this data is coming from. If your monitoring agent supports the concept of modules or plugins to process events of a given source (e.g. Apache logs), `event.module` should contain the name of this module. | constant_keyword | +| input.type | Type of filebeat input. | keyword | +| labels.is_transform_source | Distinguishes between documents that are a source for a transform and documents that are an output of a transform, to facilitate easier filtering. | constant_keyword | +| log.offset | Log offset. | long | +| observer.vendor | Vendor name of the observer. | constant_keyword | + + +An example event for `identity` looks as following: + +```json +{ + "@timestamp": "2025-12-09T12:02:11.000Z", + "agent": { + "ephemeral_id": "9f03aa73-e75b-478f-a4b6-03207451caa1", + "id": "2009affa-84ca-46da-9030-1686d937a20d", + "name": "elastic-agent-92850", + "type": "filebeat", + "version": "8.18.0" + }, + "axonius": { + "identity": { + "account_disabled": true, + "accurate_for_datetime": "2025-12-09T12:02:11.000Z", + "adapter_list_length": 12, + "adapters": [ + "aws_adapter", + "zoom_adapter" + ], + "application_and_account_name": "microsoft/azure_ad-demo", + "asset_type": "users", + "associated_groups": [ + { + "display_name": "developers-group", + "remote_id": "a3e70162" + } + ], + "azure_account_id": "c8103abe-eda9-472b-894a-6260bb2ba8cc", + "cloud_provider": "Azure", + "email_activity": { + "is_deleted": false, + "product_license": "MICROSOFT FABRIC (FREE)+MICROSOFT TEAMS PHONE STANDARD+MICROSOFT DEFENDER FOR OFFICE365 (PLAN 2)+MICROSOFT 365 AUDIO CONFERENCING+ENTERPRISE MOBILITY + SECURITY E3+OFFICE365 E3+MICROSOFT 365 E3 EXTRA FEATURES", + "read_count": 2321, + "receive_count": 6965, + "report_date": "2025-01-10T20:34:43.000Z", + "report_period": 90, + "send_count": 3030 + }, + "event": { + "accurate_for_datetime": "2025-12-09T12:02:11.000Z", + "adapter_categories": [ + "Directory", + "IAM", + "SaaS Management" + ], + "client_used": "67fd09bbfe1c8e812a176bb5", + "initial_plugin_unique_name": "azure_ad_adapter_0", + "plugin_name": "azure_ad_adapter", + "plugin_type": "Adapter", + "plugin_unique_name": "azure_ad_adapter_0", + "quick_id": "azure_ad_adapter_0!c8103abe-eda9-472b-894a-6260bb2ba8cc", + "type": "entitydata" + }, + "fetch_time": "2025-12-09T12:02:03.000Z", + "first_fetch_time": "2025-04-14T13:27:00.000Z", + "from_last_fetch": true, + "has_administrative_permissions": true, + "id": "c8103abe-eda9-472b-894a-6260bb2ba8cc", + "internal_axon_id": "bc11b2989fc0f69708b6865d172a49fe", + "internal_is_admin": false, + "is_admin": false, + "is_fetched_from_adapter": true, + "is_latest_last_seen": true, + "is_managed_by_application": true, + "is_permission_adapter": true, + "is_saas_user": true, + "is_user_external": false, + "last_fetch_connection_id": "67fd09bbfe1c8e812a176bb5", + "last_fetch_connection_label": "azure_ad-demo", + "last_logon": "2025-11-30T18:50:39.000Z", + "last_seen": "2025-11-10T22:18:25.000Z", + "mail": "helen.jordan@demo.local", + "nested_applications": [ + { + "app_display_name": "Calendly", + "assignment_type": "Direct", + "extension_type": "User Consent", + "is_managed": false, + "is_unmanaged_extension": true, + "name": "Calendly", + "permissions": [ + { + "name": "openid" + } + ], + "relation_extension_name": "Calendly", + "source_application": "Microsoft", + "value": "2E2a2e7c9f758BDcC0E2", + "vendor_category": "Productivity" + } + ], + "nested_grants_last_updated": "2025-12-09T12:10:06.000Z", + "nested_grants_managers_last_updated": "2025-12-09T12:10:10.000Z", + "nested_groups": [ + { + "assignment_type": "Direct", + "name": "Office365 Users", + "value": "d8e66837" + } + ], + "not_fetched_count": 0, + "sm_entity_type": "saas_user", + "source_application": "Microsoft", + "tenant_number": [ + 2 + ], + "transform_unique_id": "N8G3qDAOmSElCdviQ3d6FpD76pE=", + "user_created": "2024-06-28T08:49:28.000Z", + "user_permissions": [ + { + "is_admin": false, + "name": "OnlineMeetings.ReadWrite" + } + ], + "user_remote_id": "63d52bb0-7ce0-4467-9004-2b19c06b86ae", + "user_type": "Member", + "username": "helen.jordan@demo.local" + } + }, + "cloud": { + "account": { + "id": "c8103abe-eda9-472b-894a-6260bb2ba8cc" + }, + "provider": "Azure" + }, + "data_stream": { + "dataset": "axonius.identity", + "namespace": "42638", + "type": "logs" + }, + "ecs": { + "version": "9.2.0" + }, + "elastic_agent": { + "id": "2009affa-84ca-46da-9030-1686d937a20d", + "snapshot": false, + "version": "8.18.0" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "iam" + ], + "created": "2024-06-28T08:49:28.000Z", + "dataset": "axonius.identity", + "ingested": "2026-04-21T12:29:16Z", + "kind": "event", + "type": [ + "info" + ] + }, + "input": { + "type": "cel" + }, + "related": { + "user": [ + "developers-group", + "helen.jordan@demo.local" + ] + }, + "tags": [ + "preserve_duplicate_custom_fields", + "forwarded", + "axonius-identity" + ], + "user": { + "domain": "demo.local", + "email": "helen.jordan@demo.local", + "name": "helen.jordan@demo.local" + } +} +``` + ### Inputs used These inputs can be used with this integration: @@ -2105,7 +2747,20 @@ These APIs are used with this integration: * firewalls (endpoint: `/api/v2/firewalls`) * nat_rules (endpoint: `/api/v2/nat_rules`) * network_routes (endpoint: `/api/v2/network_routes`) +* Identity: + * users (endpoint: `/api/v2/users`) + * groups (endpoint: `/api/v2/groups`) + * security_roles (endpoint: `/api/v2/security_roles`) + * organizational_units (endpoint: `/api/v2/organizational_units`) + * accounts (endpoint: `/api/v2/accounts`) + * certificates (endpoint: `/api/v2/certificates`) + * permissions (endpoint: `/api/v2/permissions`) + * latest_rules (endpoint: `/api/v2/latest_rules`) + * profiles (endpoint: `/api/v2/profiles`) + * job_titles (endpoint: `/api/v2/job_titles`) + * access_review_campaign_instances (endpoint: `/api/v2/access_review_campaign_instances`) + * access_review_approval_items (endpoint: `/api/v2/access_review_approval_items`) ### ILM Policy -To facilitate adapter, user, gateway and assets data including exposures, alert findings, incidents, storage and ticket source data stream-backed indices `.ds-logs-axonius.adapter-*`, `.ds-logs-axonius.user-*`, `.ds-logs-axonius.gateway-*`, `.ds-logs-axonius.exposure-*`, `.ds-logs-axonius.alert_finding-*`, `.ds-logs-axonius.incident-*`, `.ds-logs-axonius.storage-*` and `.ds-logs-axonius.ticket-*` respectively are allowed to contain duplicates from each polling interval. ILM policies `logs-axonius.adapter-default_policy`, `logs-axonius.user-default_policy`, `logs-axonius.gateway-default_policy`, `logs-axonius.exposure-default_policy`, `logs-axonius.alert_finding-default_policy`, `logs-axonius.incident-default_policy`, `logs-axonius.storage-default_policy` and `logs-axonius.ticket-default_policy` are added to these source indices, so it doesn't lead to unbounded growth. This means that in these source indices data will be deleted after `30 days` from ingested date. +To facilitate adapter, user, gateway and assets data including exposures, alert findings, incidents, storage and ticket, network and identity source data stream-backed indices `.ds-logs-axonius.adapter-*`, `.ds-logs-axonius.user-*`, `.ds-logs-axonius.gateway-*`, `.ds-logs-axonius.exposure-*`, `.ds-logs-axonius.alert_finding-*`, `.ds-logs-axonius.incident-*`, `.ds-logs-axonius.storage-*`, `.ds-logs-axonius.ticket-*`, `.ds-logs-axonius.network-*` and `.ds-logs-axonius.identity-*` respectively are allowed to contain duplicates from each polling interval. ILM policies `logs-axonius.adapter-default_policy`, `logs-axonius.user-default_policy`, `logs-axonius.gateway-default_policy`, `logs-axonius.exposure-default_policy`, `logs-axonius.alert_finding-default_policy`, `logs-axonius.incident-default_policy`, `logs-axonius.storage-default_policy`, `logs-axonius.ticket-default_policy`, `logs-axonius.network-default_policy` and `logs-axonius.identity-default_policy` are added to these source indices, so it doesn't lead to unbounded growth. This means that in these source indices data will be deleted after `30 days` from ingested date. diff --git a/packages/axonius/elasticsearch/transform/latest_identity/fields/base-fields.yml b/packages/axonius/elasticsearch/transform/latest_identity/fields/base-fields.yml new file mode 100644 index 00000000000..919b224d09f --- /dev/null +++ b/packages/axonius/elasticsearch/transform/latest_identity/fields/base-fields.yml @@ -0,0 +1,16 @@ +- name: data_stream.dataset + external: ecs +- name: data_stream.namespace + external: ecs +- name: data_stream.type + external: ecs +- name: event.dataset + type: constant_keyword + external: ecs + value: axonius.identity +- name: event.module + type: constant_keyword + external: ecs + value: axonius +- name: '@timestamp' + external: ecs diff --git a/packages/axonius/elasticsearch/transform/latest_identity/fields/beats.yml b/packages/axonius/elasticsearch/transform/latest_identity/fields/beats.yml new file mode 100644 index 00000000000..d5fd38748ba --- /dev/null +++ b/packages/axonius/elasticsearch/transform/latest_identity/fields/beats.yml @@ -0,0 +1,6 @@ +- name: input.type + type: keyword + description: Type of Filebeat input. +- name: log.offset + type: long + description: Log offset. diff --git a/packages/axonius/elasticsearch/transform/latest_identity/fields/ecs.yml b/packages/axonius/elasticsearch/transform/latest_identity/fields/ecs.yml new file mode 100644 index 00000000000..72c2bde2adb --- /dev/null +++ b/packages/axonius/elasticsearch/transform/latest_identity/fields/ecs.yml @@ -0,0 +1,74 @@ +- external: ecs + name: agent.ephemeral_id +- external: ecs + name: agent.id +- external: ecs + name: agent.name +- external: ecs + name: agent.type +- external: ecs + name: agent.version +- external: ecs + name: cloud.account.id +- external: ecs + name: cloud.provider +- external: ecs + name: cloud.service.name +- external: ecs + name: device.id +- external: ecs + name: device.model.name +- external: ecs + name: device.serial_number +- external: ecs + name: ecs.version +- external: ecs + name: error.code +- external: ecs + name: error.id +- external: ecs + name: error.message +- external: ecs + name: event.action +- external: ecs + name: event.created +- external: ecs + name: event.end +- external: ecs + name: event.id +- external: ecs + name: event.start +- external: ecs + name: file.x509.issuer.common_name +- external: ecs + name: file.x509.issuer.country +- external: ecs + name: file.x509.issuer.organization +- external: ecs + name: file.x509.serial_number +- external: ecs + name: file.x509.subject.common_name +- external: ecs + name: file.x509.subject.country +- external: ecs + name: file.x509.subject.locality +- external: ecs + name: file.x509.subject.organization +- external: ecs + name: file.x509.subject.state_or_province +- external: ecs + name: related.user +- external: ecs + name: service.id +- external: ecs + name: service.name +- external: ecs + name: threat.enrichments.indicator.first_seen +- external: ecs + name: user.domain +- external: ecs + name: user.email +- external: ecs + name: user.full_name +- external: ecs + name: user.name diff --git a/packages/axonius/elasticsearch/transform/latest_identity/fields/fields.yml b/packages/axonius/elasticsearch/transform/latest_identity/fields/fields.yml new file mode 100644 index 00000000000..28f0aa82021 --- /dev/null +++ b/packages/axonius/elasticsearch/transform/latest_identity/fields/fields.yml @@ -0,0 +1,1413 @@ +- name: axonius + type: group + fields: + - name: identity + type: group + fields: + - name: adapter_list_length + type: long + description: How many adapters contributed to this asset. + - name: adapters + type: keyword + description: List of adapters that created this asset. + - name: asset_type + type: keyword + description: The type of asset. + - name: account_disabled + type: boolean + description: Indicates whether the user account is disabled. + - name: accurate_for_datetime + type: date + description: Timestamp indicating when this asset information was accurate. + - name: active + type: keyword + description: The active status of the identity. + - name: active_users + type: long + description: Number of active users in the account. + - name: active_users_saved_query_id + type: keyword + description: Saved query ID for the active users metric. + - name: admin_non_operational_users + type: long + description: Number of admin users that are non-operational. + - name: admin_non_operational_users_saved_query_id + type: keyword + description: Saved query ID for the admin non-operational users metric. + - name: admin_operational_active_users + type: long + description: Number of admin users that are both operational and active. + - name: admin_operational_active_users_saved_query_id + type: keyword + description: Saved query ID for the admin operational active users metric. + - name: admin_operational_inactive_users + type: long + description: Number of admin users that are operational but inactive. + - name: admin_operational_inactive_users_saved_query_id + type: keyword + description: Saved query ID for the admin operational inactive users metric. + - name: admin_operational_users + type: long + description: Number of admin users that are operational. + - name: admin_operational_users_saved_query_id + type: keyword + description: Saved query ID for the admin operational users metric. + - name: admin_roles + type: group + fields: + - name: display_name + type: keyword + description: Display name of the admin role. + - name: id + type: keyword + description: Unique identifier of the admin role. + - name: admins + type: long + description: Total number of administrators in the account. + - name: admins_saved_query_id + type: keyword + description: Saved query ID for the admins metric. + - name: alt_names + type: group + fields: + - name: name + type: keyword + description: The alternative name value. + - name: name_type + type: keyword + description: The type of alternative name (e.g., DNS, IP). + - name: application_and_account_name + type: keyword + description: The application and account name associated with the asset. + - name: application_id + type: keyword + description: Unique identifier of the application. + - name: application_name + type: keyword + description: Name of the application associated with this identity. + - name: asset_entity_info + type: keyword + description: Information about the asset entity and its properties. + - name: data_asset_type + type: keyword + description: The asset type from identity event data, distinguishing from root asset_type. + - name: description + type: text + description: The description of the asset. + - name: associated_devices + type: group + fields: + - name: device_associated_saas_apps_names + type: keyword + description: Names of SaaS applications associated with the device. + - name: device_caption + type: keyword + description: Caption or display name of the associated device. + - name: device_id + type: keyword + description: Unique identifier of the associated device. + - name: device_labels + type: keyword + description: Labels or tags assigned to the associated device. + - name: device_model + type: keyword + description: Model name of the associated device. + - name: device_os_distribution + type: keyword + description: Operating system distribution of the associated device. + - name: device_os_edition + type: keyword + description: Operating system edition of the associated device. + - name: device_os_end_of_life + type: keyword + description: End-of-life date of the device operating system. + - name: device_os_type + type: keyword + description: Operating system type of the associated device. + - name: device_os_version + type: keyword + description: Operating system version of the associated device. + - name: device_preferred_mac_address + type: keyword + description: Preferred MAC address of the associated device. + - name: device_serial + type: keyword + description: Serial number of the associated device. + - name: internal_axon_id + type: keyword + description: Internal Axonius ID of the associated device. + - name: associated_employees + type: group + fields: + - name: internal_axon_id + type: keyword + description: Internal Axonius ID of the associated employee. + - name: username + type: keyword + description: Username of the associated employee. + - name: associated_groups + type: group + fields: + - name: display_name + type: keyword + description: Display name of the associated group. + - name: remote_id + type: keyword + description: Remote identifier of the associated group. + - name: associated_licenses + type: group + fields: + - name: adapter_connection_label + type: keyword + description: Label of the adapter connection for the license. + - name: internal_axon_id + type: keyword + description: Internal Axonius ID of the license. + - name: license_name + type: keyword + description: Name of the license. + - name: pricing_unit + type: keyword + description: Pricing unit of the license. + - name: related_vendor_name + type: keyword + description: Vendor name associated with the license. + - name: unit_price + type: keyword + description: Unit price of the license. + - name: aws_arn + type: keyword + description: Amazon Web Services ARN (Amazon Resource Name) for this identity. + - name: aws_iam_identity_type + type: keyword + description: AWS IAM identity type (e.g., user, role, group). + - name: azure_account_id + type: keyword + description: Azure account identifier associated with this identity. + - name: begins_on + type: date + description: Start date of the certificate validity period. + - name: bit_size + type: long + description: Key bit size of the certificate. + - name: breaches_data + type: group + fields: + - name: added_date + type: date + description: Date when the breach was added to the database. + - name: breach_date + type: date + description: Date when the breach occurred. + - name: data_classes + type: keyword + description: Types of data exposed in the breach. + - name: domain + type: keyword + description: Domain affected by the breach. + - name: is_fabricated + type: boolean + description: Indicates if the breach data is fabricated. + - name: is_retired + type: boolean + description: Indicates if the breach record has been retired. + - name: is_sensitive + type: boolean + description: Indicates if the breach contains sensitive data. + - name: is_spam_list + type: boolean + description: Indicates if the breach is from a spam list. + - name: is_verified + type: boolean + description: Indicates if the breach has been verified. + - name: logo_path + type: keyword + description: Path to the logo of the breached service. + - name: modified_date + type: date + description: Date when the breach record was last modified. + - name: name + type: keyword + description: Name of the breach. + - name: pwn_count + type: long + description: Number of accounts affected by the breach. + - name: title + type: keyword + description: Title of the breach. + - name: class_name + type: keyword + description: The class name or system classification of this asset. + - name: cloud_provider + type: keyword + description: The cloud provider associated with this identity. + - name: connected_assets + type: keyword + description: Other assets connected to or associated with this identity asset. + - name: connection_label + type: keyword + description: Label of the adapter connection used to collect this identity data. + - name: created_date + type: date + description: Date when this identity record was created. + - name: deleted_users + type: long + description: Number of deleted users in the account. + - name: deleted_users_saved_query_id + type: keyword + description: Saved query ID for the deleted users metric. + - name: direct_not_sso_users + type: long + description: Number of users with direct access who are not using SSO. + - name: direct_not_sso_users_saved_query_id + type: keyword + description: Saved query ID for the direct non-SSO users metric. + - name: display_name + type: keyword + description: Display name of the identity. + - name: distinct_associated_devices_count + type: long + description: Number of distinct devices associated with this identity. + - name: domains + type: group + fields: + - name: name + type: keyword + description: Name of the domain. + - name: email + type: keyword + description: Email address of the identity. + - name: email_activity + type: group + fields: + - name: is_deleted + type: boolean + description: Indicates if the email activity record has been deleted. + - name: product_license + type: keyword + description: Product license associated with email activity. + - name: read_count + type: long + description: Number of emails read during the report period. + - name: receive_count + type: long + description: Number of emails received during the report period. + - name: report_date + type: date + description: Date of the email activity report. + - name: report_period + type: long + description: Reporting period in days for the email activity. + - name: send_count + type: long + description: Number of emails sent during the report period. + - name: email_notification + type: group + fields: + - name: alternative_host_reminder + type: boolean + description: Indicates if alternative host reminder emails are enabled. + - name: cancel_meeting_reminder + type: boolean + description: Indicates if meeting cancellation reminder emails are enabled. + - name: jbh_reminder + type: boolean + description: Indicates if join-before-host reminder emails are enabled. + - name: employee_id + type: keyword + description: Employee identifier assigned to this identity. + - name: employee_number + type: keyword + description: Employee number assigned to this identity. + - name: employee_type + type: keyword + description: Type of employee (e.g., full-time, contractor). + - name: expires_on + type: date + description: Expiration date of the certificate validity period. + - name: external_users + type: long + description: Number of external users in the account. + - name: external_users_saved_query_id + type: keyword + description: Saved query ID for the external users metric. + - name: feature + type: group + fields: + - name: cn_meeting + type: boolean + description: Indicates if China meeting feature is enabled. + - name: in_meeting + type: boolean + description: Indicates if in-meeting feature is enabled. + - name: large_meeting + type: boolean + description: Indicates if large meeting feature is enabled. + - name: meeting_capacity + type: long + description: Maximum meeting capacity for this identity. + - name: webinar + type: boolean + description: Indicates if webinar feature is enabled. + - name: zoom_phone + type: boolean + description: Indicates if Zoom Phone feature is enabled. + - name: fetch_time + type: date + description: The date and time when the identity data was last fetched. + - name: first_fetch_time + type: date + description: The date and time when this identity asset was first fetched. + - name: first_name + type: keyword + description: First name of the identity. + - name: first_seen + type: date + description: The date and time when this identity was first observed. + - name: from_last_fetch + type: boolean + description: Indicates whether this identity asset was modified since the last fetch. + - name: gce_account_id + type: keyword + description: Google Cloud Engine account ID associated with this identity. + - name: groups + type: group + fields: + - name: display_name + type: keyword + description: Display name of the group. + - name: name + type: keyword + description: Name of the group. + - name: remote_id + type: keyword + description: Remote identifier of the group. + - name: has_administrative_permissions + type: boolean + description: Indicates whether this identity has administrative permissions. + - name: hire_date + type: date + description: Date when the employee was hired. + - name: hr_employment_status + type: keyword + description: Human resources employment status of the identity. + - name: id + type: keyword + description: Unique identifier for the identity asset. + - name: id_raw + type: keyword + description: Raw unique identifier for the identity asset. + - name: in_meeting + type: group + fields: + - name: allow_live_streaming + type: boolean + description: Indicates if live streaming is allowed during meetings. + - name: annotation + type: boolean + description: Indicates if annotation is enabled during meetings. + - name: attendee_on_hold + type: boolean + description: Indicates if attendee-on-hold feature is enabled. + - name: auto_saving_chat + type: boolean + description: Indicates if auto-saving chat is enabled. + - name: breakout_room + type: boolean + description: Indicates if breakout rooms are enabled. + - name: chat + type: boolean + description: Indicates if chat is enabled during meetings. + - name: closed_caption + type: boolean + description: Indicates if closed captions are enabled. + - name: co_host + type: boolean + description: Indicates if co-host feature is enabled. + - name: data_center_regions + type: keyword + description: Data center regions configured for meetings. + - name: e2e_encryption + type: boolean + description: Indicates if end-to-end encryption is enabled. + - name: entry_exit_chime + type: boolean + description: Indicates if entry/exit chime is enabled. + - name: far_end_camera_control + type: boolean + description: Indicates if far-end camera control is enabled. + - name: feedback + type: boolean + description: Indicates if feedback feature is enabled. + - name: group_hd + type: boolean + description: Indicates if group HD video is enabled. + - name: non_verbal_feedback + type: boolean + description: Indicates if non-verbal feedback is enabled. + - name: polling + type: boolean + description: Indicates if polling is enabled during meetings. + - name: private_chat + type: boolean + description: Indicates if private chat is enabled during meetings. + - name: record_play_voice + type: boolean + description: Indicates if record and play voice is enabled. + - name: remote_control + type: boolean + description: Indicates if remote control is enabled. + - name: remote_support + type: boolean + description: Indicates if remote support is enabled. + - name: share_dual_camera + type: boolean + description: Indicates if dual camera sharing is enabled. + - name: show_meeting_control_toolbar + type: boolean + description: Indicates if meeting control toolbar is shown. + - name: virtual_background + type: boolean + description: Indicates if virtual background is enabled. + - name: waiting_room + type: boolean + description: Indicates if waiting room is enabled. + - name: workplace_by_facebook + type: boolean + description: Indicates if Workplace by Facebook integration is enabled. + - name: inactive_users + type: long + description: Number of inactive users in the account. + - name: inactive_users_saved_query_id + type: keyword + description: Saved query ID for the inactive users metric. + - name: internal_is_admin + type: boolean + description: Internal flag indicating if this identity has admin privileges. + - name: is_active + type: boolean + description: Indicates whether this identity is currently active. + - name: is_admin + type: boolean + description: Indicates whether this identity has administrator privileges. + - name: is_built_in + type: boolean + description: Indicates whether this is a built-in system account. + - name: is_delegated_admin + type: boolean + description: Indicates whether this identity is a delegated administrator. + - name: is_fetched_from_adapter + type: boolean + description: Indicates whether this identity data was fetched from an adapter. + - name: is_from_sso_provider + type: boolean + description: Indicates whether this identity originates from a Single Sign-On provider. + - name: is_latest_last_seen + type: boolean + description: Indicates if this is the latest recorded last-seen timestamp. + - name: is_managed_by_application + type: boolean + description: Indicates whether this identity is managed by an application. + - name: is_managed_by_direct_app + type: boolean + description: Indicates whether this identity is managed by a direct application. + - name: is_managed_by_sso + type: boolean + description: Indicates whether this identity is managed through SSO. + - name: is_mfa_enforced + type: boolean + description: Indicates whether multi-factor authentication is enforced. + - name: is_mfa_enrolled + type: boolean + description: Indicates whether this identity is enrolled in multi-factor authentication. + - name: is_non_editable + type: boolean + description: Indicates whether this identity record is non-editable. + - name: is_paid + type: boolean + description: Indicates whether this identity has a paid license or account. + - name: is_permission_adapter + type: boolean + description: Indicates whether this identity was collected by a permission adapter. + - name: is_privileged + type: boolean + description: Indicates whether this identity has privileged access. + - name: is_saas_user + type: boolean + description: Indicates whether this identity is a SaaS application user. + - name: is_user_active + type: boolean + description: Indicates whether the user account is active. + - name: is_user_deleted + type: boolean + description: Indicates whether the user account has been deleted. + - name: is_user_external + type: boolean + description: Indicates whether this is an external user. + - name: is_user_inactive + type: boolean + description: Indicates whether the user account is inactive. + - name: is_user_suspended + type: boolean + description: Indicates whether the user account is suspended. + - name: issuer + type: group + fields: + - name: common_name + type: keyword + description: Common name of the certificate issuer. + - name: country_name + type: keyword + description: Country name of the certificate issuer. + - name: organization + type: keyword + description: Organization name of the certificate issuer. + - name: last_client_version + type: keyword + description: Version of the last client used by this identity. + - name: last_enrichment_run + type: date + description: Date of the last enrichment run for this identity. + - name: last_fetch_connection_id + type: keyword + description: The connection ID of the adapter that last fetched this data. + - name: last_fetch_connection_label + type: keyword + description: The label of the connection that last fetched this identity data. + - name: last_login_attempt + type: date + description: Date and time of the last login attempt. + - name: last_logon + type: date + description: Date and time of the last successful logon. + - name: last_name + type: keyword + description: Last name of the identity. + - name: last_password_change + type: date + description: Date and time when the password was last changed. + - name: last_seen + type: date + description: The date and time when this identity was last observed. + - name: mail + type: keyword + description: Email address (mail attribute) of the identity. + - name: managed_non_operational_users + type: long + description: Number of managed users that are non-operational. + - name: managed_non_operational_users_saved_query_id + type: keyword + description: Saved query ID for the managed non-operational users metric. + - name: managed_operational_users + type: long + description: Number of managed users that are operational. + - name: managed_operational_users_saved_query_id + type: keyword + description: Saved query ID for the managed operational users metric. + - name: managed_users + type: long + description: Total number of managed users in the account. + - name: managed_users_by_app + type: long + description: Number of users managed by a direct application. + - name: managed_users_by_app_saved_query_id + type: keyword + description: Saved query ID for the managed-by-app users metric. + - name: managed_users_by_sso + type: long + description: Number of users managed through SSO. + - name: managed_users_by_sso_saved_query_id + type: keyword + description: Saved query ID for the managed-by-SSO users metric. + - name: managed_users_saved_query_id + type: keyword + description: Saved query ID for the managed users metric. + - name: manager_id + type: keyword + description: Identifier of the manager of this identity. + - name: max_added_date + type: date + description: Most recent date a breach was added across all breaches for this identity. + - name: max_breach_date + type: date + description: Most recent breach date across all breaches for this identity. + - name: max_modified_date + type: date + description: Most recent modified date across all breaches for this identity. + - name: name + type: keyword + description: The name or identifier of the identity asset. + - name: nested_applications + type: group + fields: + - name: active_from_direct_adapter + type: boolean + description: Indicates if active status is from a direct adapter. + - name: app_accounts + type: group + fields: + - name: name + type: keyword + description: Name of the application account. + - name: app_display_name + type: keyword + description: Display name of the application. + - name: app_links + type: keyword + description: Links or URLs associated with the application. + - name: assignment_type + type: keyword + description: How the application was assigned (e.g., direct, group). + - name: extension_type + type: keyword + description: Type of extension for the application. + - name: has_administrative_permissions + type: boolean + description: Indicates if the identity has admin permissions in this application. + - name: is_deleted + type: boolean + description: Indicates if the application assignment has been deleted. + - name: is_from_direct_adapter + type: boolean + description: Indicates if the data is from a direct adapter. + - name: is_managed + type: boolean + description: Indicates if the application is managed. + - name: is_suspended + type: boolean + description: Indicates if the application access is suspended. + - name: is_unmanaged_extension + type: boolean + description: Indicates if this is an unmanaged browser extension. + - name: is_user_external + type: boolean + description: Indicates if the user is external in this application. + - name: is_user_paid + type: boolean + description: Indicates if the user has a paid license in this application. + - name: last_access + type: date + description: Date and time of the last access to the application. + - name: last_access_count + type: long + description: Total number of accesses to the application. + - name: last_access_count_60_days + type: long + description: Number of accesses to the application in the last 60 days. + - name: last_access_count_90_days + type: long + description: Number of accesses to the application in the last 90 days. + - name: name + type: keyword + description: Name of the application. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: permissions + type: group + fields: + - name: name + type: keyword + description: Name of the permission. + - name: relation_direct_name + type: keyword + description: Name of the direct relationship to the application. + - name: relation_discovery_name + type: keyword + description: Name of the discovered relationship to the application. + - name: relation_extension_name + type: keyword + description: Name of the extension-based relationship to the application. + - name: relation_sso_name + type: keyword + description: Name of the SSO-based relationship to the application. + - name: source_application + type: keyword + description: Source application that provided this data. + - name: value + type: keyword + description: Value or identifier of the application. + - name: vendor_category + type: keyword + description: Vendor category of the application. + - name: nested_associated_devices + type: keyword + description: Flattened list of nested associated device identifiers. + - name: nested_grants_last_updated + type: date + description: Date when nested grants were last updated. + - name: nested_grants_managers_last_updated + type: date + description: Date when nested grants managers were last updated. + - name: nested_groups + type: group + fields: + - name: assignment_type + type: keyword + description: How the group was assigned (e.g., direct, inherited). + - name: group_name + type: keyword + description: Name of the group. + - name: name + type: keyword + description: Display name of the group entry. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the group. + - name: nested_managers + type: group + fields: + - name: assignment_type + type: keyword + description: How the manager was assigned. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the manager. + - name: nested_permissions + type: group + fields: + - name: has_administrative_permissions + type: boolean + description: Indicates if the identity has administrative permissions. + - name: is_admin + type: boolean + description: Indicates if the identity has admin privileges. + - name: assignment_type + type: keyword + description: How the permission was assigned (e.g., direct, inherited). + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the permission. + - name: nested_resources + type: group + fields: + - name: assignment_type + type: keyword + description: How the resource was assigned. + - name: name + type: keyword + description: Name of the resource. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the resource. + - name: nested_roles + type: group + fields: + - name: assignment_type + type: keyword + description: How the role was assigned (e.g., direct, inherited). + - name: name + type: keyword + description: Name of the role. + - name: parents + type: group + fields: + - name: name + type: keyword + description: Name of the parent entity. + - name: parent_type + type: keyword + description: Type of the parent entity. + - name: value + type: keyword + description: Value or identifier of the parent entity. + - name: value + type: keyword + description: Value or identifier of the role. + - name: not_fetched_count + type: long + description: The number of times this identity asset failed to be fetched. + - name: operational_users_count + type: long + description: Total number of operational users in the account. + - name: oracle_cloud_cis_incompliant + type: group + fields: + - name: rule_cis_version + type: float + description: CIS benchmark version of the incompliant rule. + - name: rule_section + type: keyword + description: Section number of the incompliant CIS rule. + - name: orphaned_users + type: long + description: Number of orphaned users in the account. + - name: orphaned_users_saved_query_id + type: keyword + description: Saved query ID for the orphaned users metric. + - name: paid_users + type: long + description: Number of paid users in the account. + - name: paid_users_saved_query_id + type: keyword + description: Saved query ID for the paid users metric. + - name: password_never_expires + type: boolean + description: Indicates whether the password is set to never expire. + - name: password_not_required + type: boolean + description: Indicates whether a password is not required for this account. + - name: permissions_list + type: group + fields: + - name: name + type: keyword + description: Name of the permission. + - name: permissions + type: long + description: Total number of permissions assigned to the identity. + - name: pmi + type: keyword + description: Personal Meeting ID (Zoom). + - name: pretty_id + type: keyword + description: A human-readable identifier for the identity asset. + - name: project_ids + type: keyword + description: Cloud project IDs associated with this identity. + - name: project_tags + type: group + fields: + - name: inherited + type: keyword + description: Indicates if the tag is inherited from a parent resource. + - name: key + type: keyword + description: Tag key. + - name: namespaced_tag_key + type: keyword + description: Namespaced version of the tag key. + - name: namespaced_tag_value + type: keyword + description: Namespaced version of the tag value. + - name: value + type: keyword + description: Tag value. + - name: projects_roles + type: group + fields: + - name: project_id + type: keyword + description: Identifier of the project. + - name: role_name + type: keyword + description: Name of the role in the project. + - name: roles + type: group + fields: + - name: display_name + type: keyword + description: Display Name of the role. + - name: remote_id + type: keyword + description: Remote ID of the role. + - name: roles_accounts + type: keyword + description: Account roles. + - name: provider_name + type: keyword + description: Name of the identity provider. + - name: provider_type + type: keyword + description: Type of the identity provider. + - name: recording + type: group + fields: + - name: auto_delete_cmr + type: boolean + description: Indicates if cloud meeting recordings are auto-deleted. + - name: auto_delete_cmr_days + type: boolean + description: Indicates if auto-delete days for cloud recordings is configured. + - name: auto_recording + type: boolean + description: Indicates if auto-recording is enabled. + - name: cloud_recording + type: boolean + description: Indicates if cloud recording is enabled. + - name: host_pause_stop_recording + type: boolean + description: Indicates if host can pause or stop recording. + - name: local_recording + type: boolean + description: Indicates if local recording is enabled. + - name: record_audio_file + type: boolean + description: Indicates if a separate audio file is recorded. + - name: record_gallery_view + type: boolean + description: Indicates if gallery view is recorded. + - name: record_speaker_view + type: boolean + description: Indicates if speaker view is recorded. + - name: recording_audio_transcript + type: boolean + description: Indicates if audio transcript is generated for recordings. + - name: save_chat_text + type: boolean + description: Indicates if chat text is saved with recordings. + - name: show_timestamp + type: boolean + description: Indicates if timestamp is shown in recordings. + - name: recovery_question_set + type: boolean + description: Indicates whether a recovery question has been set for this identity. + - name: relatable_ids + type: keyword + description: IDs used to relate this identity to other assets. + - name: remote_account_id + type: keyword + description: Remote account identifier for this identity. + - name: remote_id + type: keyword + description: Remote identifier for this identity in the source system. + - name: schedule_meeting + type: group + fields: + - name: audio_type + type: keyword + description: Audio type configured for scheduled meetings. + - name: force_pmi_jbh_password + type: boolean + description: Indicates if PMI join-before-host password is forced. + - name: host_video + type: boolean + description: Indicates if host video is on when joining a meeting. + - name: join_before_host + type: boolean + description: Indicates if participants can join before the host. + - name: participants_video + type: boolean + description: Indicates if participant video is on when joining a meeting. + - name: pstn_password_protected + type: boolean + description: Indicates if PSTN dial-in is password protected. + - name: require_password_for_instant_meetings + type: boolean + description: Indicates if password is required for instant meetings. + - name: require_password_for_pmi_meetings + type: boolean + description: Indicates if password is required for PMI meetings. + - name: require_password_for_scheduled_meetings + type: boolean + description: Indicates if password is required for scheduled meetings. + - name: require_password_for_scheduling_new_meetings + type: boolean + description: Indicates if password is required when scheduling new meetings. + - name: use_pmi_for_instant_meetings + type: boolean + description: Indicates if PMI is used for instant meetings. + - name: use_pmi_for_scheduled_meetings + type: boolean + description: Indicates if PMI is used for scheduled meetings. + - name: serial_number + type: keyword + description: Serial number of the certificate. + - name: shirt_size + type: keyword + description: Shirt size of the employee (HR attribute). + - name: sm_entity_type + type: keyword + description: SaaS management entity type for this identity. + - name: snow_full_name + type: keyword + description: Full name of the identity from ServiceNow. + - name: snow_location + type: keyword + description: Location of the identity from ServiceNow. + - name: source_application + type: keyword + description: The source application that provided this identity data. + - name: status + type: keyword + description: Current status of the identity account. + - name: status_changed + type: date + description: Date and time when the account status was last changed. + - name: subject + type: group + fields: + - name: common_name + type: keyword + description: Common name of the certificate subject. + - name: country_name + type: keyword + description: Country name of the certificate subject. + - name: locality + type: keyword + description: Locality (city) of the certificate subject. + - name: organization + type: keyword + description: Organization name of the certificate subject. + - name: state + type: keyword + description: State or province of the certificate subject. + - name: suspended_users + type: long + description: Number of suspended users in the account. + - name: suspended_users_saved_query_id + type: keyword + description: Saved query ID for the suspended users metric. + - name: telephony + type: group + fields: + - name: show_international_numbers_link + type: boolean + description: Indicates if international numbers link is shown. + - name: third_party_audio + type: boolean + description: Indicates if third-party audio is enabled. + - name: tenant_number + type: long + description: Tenant number associated with this identity. + - name: timezone + type: keyword + description: Timezone configured for this identity. + - name: total_users_count + type: long + description: Total number of users in the account. + - name: tsp + type: group + fields: + - name: call_out + type: boolean + description: Indicates if TSP call-out is enabled. + - name: show_international_numbers_link + type: boolean + description: Indicates if international numbers link is shown for TSP. + - name: type + type: keyword + description: The type or classification of the identity entity. + - name: u_department + type: keyword + description: Department of the identity from ServiceNow. + - name: u_vip + type: boolean + description: Indicates whether this identity is flagged as a VIP in ServiceNow. + - name: unlinked_users + type: long + description: Number of unlinked users in the account. + - name: unlinked_users_saved_query_id + type: keyword + description: Saved query ID for the unlinked users metric. + - name: updated_on + type: date + description: Date and time when this identity record was last updated. + - name: user_apps + type: group + fields: + - name: active_from_direct_adapter + type: boolean + description: Indicates if active status is from a direct adapter. + - name: app_accounts + type: group + fields: + - name: name + type: keyword + description: Name of the application account. + - name: app_display_name + type: keyword + description: Display name of the application. + - name: app_id + type: keyword + description: Unique identifier of the application. + - name: app_links + type: keyword + description: Links or URLs associated with the application. + - name: app_name + type: keyword + description: Name of the application. + - name: extension_type + type: keyword + description: Type of extension for the application. + - name: is_from_direct_adapter + type: boolean + description: Indicates if the data is from a direct adapter. + - name: is_managed + type: boolean + description: Indicates if the application is managed. + - name: is_saas_application + type: boolean + description: Indicates if this is a SaaS application. + - name: is_unmanaged_extension + type: boolean + description: Indicates if this is an unmanaged browser extension. + - name: is_user_deleted + type: boolean + description: Indicates if the user has been deleted in the application. + - name: is_user_external + type: boolean + description: Indicates if the user is external in the application. + - name: is_user_paid + type: boolean + description: Indicates if the user has a paid license in the application. + - name: is_user_suspended + type: boolean + description: Indicates if the user is suspended in the application. + - name: last_access + type: date + description: Date and time of the last access to the application. + - name: permissions + type: group + fields: + - name: name + type: keyword + description: Name of the permission. + - name: relation_direct_name + type: keyword + description: Name of the direct relationship to the application. + - name: relation_discovery_name + type: keyword + description: Name of the discovered relationship to the application. + - name: relation_extension_name + type: keyword + description: Name of the extension-based relationship to the application. + - name: relation_sso_name + type: keyword + description: Name of the SSO-based relationship to the application. + - name: source_application + type: keyword + description: Source application that provided this data. + - name: vendor_category + type: keyword + description: Vendor category of the application. + - name: user_count + type: long + description: Number of users in the application. + - name: user_count_link + type: group + fields: + - name: bracketWeight + type: double + description: Weight of the bracket in the query expression. + - name: compOp + type: keyword + description: Comparison operator used in the query. + - name: field + type: keyword + description: Field name used in the query filter. + - name: leftBracket + type: double + description: Left bracket position in the query expression. + - name: logicOp + type: keyword + description: Logical operator (e.g., AND, OR) in the query. + - name: not + type: boolean + description: Indicates if the query condition is negated. + - name: rightBracket + type: double + description: Right bracket position in the query expression. + - name: value + type: keyword + description: Value used in the query filter. + - name: user_country + type: keyword + description: Country of the user. + - name: user_created + type: date + description: Date and time when the user account was created. + - name: user_department + type: keyword + description: Department the user belongs to. + - name: user_factors + type: group + fields: + - name: created + type: date + description: Date when the MFA factor was created. + - name: factor_status + type: keyword + description: Current status of the MFA factor. + - name: factor_type + type: keyword + description: Type of the MFA factor (e.g., push, TOTP, SMS). + - name: is_enabled + type: boolean + description: Indicates if the MFA factor is enabled. + - name: last_updated + type: date + description: Date when the MFA factor was last updated. + - name: name + type: keyword + description: Name of the MFA factor. + - name: provider + type: keyword + description: Provider of the MFA factor. + - name: strength + type: keyword + description: Strength rating of the MFA factor. + - name: vendor_name + type: keyword + description: Vendor name of the MFA factor. + - name: user_full_name + type: keyword + description: Full name of the user. + - name: user_is_password_enabled + type: boolean + description: Indicates whether password authentication is enabled for this user. + - name: user_manager + type: keyword + description: Name or identifier of the user's manager. + - name: user_manager_mail + type: keyword + description: Email address of the user's manager. + - name: user_pass_last_used + type: date + description: Date or timestamp when the user's password was last used. + - name: user_path + type: keyword + description: Path of the user in the directory (e.g., AWS IAM path). + - name: user_permissions + type: group + fields: + - name: is_admin + type: boolean + description: Indicates if the user has admin privileges for this permission. + - name: name + type: keyword + description: Name of the permission. + - name: user_related_resources + type: group + fields: + - name: id + type: keyword + description: Identifier of the related resource. + - name: name + type: keyword + description: Name of the related resource. + - name: type + type: keyword + description: Type of the related resource. + - name: user_remote_id + type: keyword + description: Remote identifier of the user in the source system. + - name: user_sid + type: keyword + description: Security Identifier (SID) of the user (Windows/AD). + - name: user_status + type: keyword + description: Current status of the user account. + - name: user_telephone_number + type: keyword + description: Telephone number of the user. + - name: user_title + type: keyword + description: Job title of the user. + - name: user_type + type: keyword + description: Type of user account (e.g., member, guest, service). + - name: username + type: keyword + description: Username of the identity. + - name: verified + type: boolean + description: Indicates whether this identity has been verified. + - name: version + type: keyword + description: Version of the certificate or identity record. + - name: event + type: group + fields: + - name: accurate_for_datetime + type: date + description: Timestamp indicating when the event data was accurate. + - name: action_if_exists + type: keyword + description: Action associated with the identity event, if it exists. + - name: adapter_categories + type: keyword + description: List of adapter categories that this event belongs to. + - name: associated_adapter_plugin_name + type: keyword + description: The associated plugin name that created or processed the event. + - name: association_type + type: keyword + description: The type of association between the event and related entities. + - name: client_used + type: keyword + description: The client identifier that was used to process the event. + - name: entity + type: keyword + description: The entity type or category this event relates to. + - name: hidden_for_gui + type: boolean + description: Indicates if this event should be hidden in the GUI. + - name: initial_plugin_unique_name + type: keyword + description: The initial plugin name that created or processed the event. + - name: name + type: keyword + description: The name of the event. + - name: plugin_name + type: keyword + description: The name of the plugin that processed the event. + - name: plugin_type + type: keyword + description: The type or category of the plugin that processed the event. + - name: plugin_unique_name + type: keyword + description: The unique identifier of the plugin instance that processed the event. + - name: quick_id + type: keyword + description: A quick reference identifier combining plugin and entity information. + - name: type + type: keyword + description: The type or classification of the event data. + - name: internal_axon_id + type: keyword + description: Internal ID of this asset. This ID may change in the future. + - name: transform_unique_id + type: keyword + description: Unique identifier for this asset in the transformation process. diff --git a/packages/axonius/elasticsearch/transform/latest_identity/fields/is-transform-source-false.yml b/packages/axonius/elasticsearch/transform/latest_identity/fields/is-transform-source-false.yml new file mode 100644 index 00000000000..759b444efd7 --- /dev/null +++ b/packages/axonius/elasticsearch/transform/latest_identity/fields/is-transform-source-false.yml @@ -0,0 +1,4 @@ +- name: labels.is_transform_source + type: constant_keyword + description: Distinguishes between documents that are a source for a transform and documents that are an output of a transform, to facilitate easier filtering. + value: 'false' diff --git a/packages/axonius/elasticsearch/transform/latest_identity/manifest.yml b/packages/axonius/elasticsearch/transform/latest_identity/manifest.yml new file mode 100644 index 00000000000..24e9e926793 --- /dev/null +++ b/packages/axonius/elasticsearch/transform/latest_identity/manifest.yml @@ -0,0 +1,11 @@ +start: true +destination_index_template: + mappings: + dynamic: true + dynamic_templates: + - strings_as_keyword: + match_mapping_type: string + mapping: + ignore_above: 1024 + type: keyword + date_detection: true diff --git a/packages/axonius/elasticsearch/transform/latest_identity/transform.yml b/packages/axonius/elasticsearch/transform/latest_identity/transform.yml new file mode 100644 index 00000000000..4cdbfccddc2 --- /dev/null +++ b/packages/axonius/elasticsearch/transform/latest_identity/transform.yml @@ -0,0 +1,42 @@ +# Use of '*' to use all namespaces defined. +source: + index: + - 'logs-axonius.identity-*' +dest: + index: 'logs-axonius_latest.dest_identity-1' + aliases: + - alias: 'logs-axonius_latest.identity' + move_on_creation: true +# Unique key for latest transform: +# - The Axonius API does not provide a stable unique identifier for identity assets. +# - axonius.identity.transform_unique_id is a fingerprint of event.original (the full raw document). +# - Per the vendor, identity assets are only created or deleted, not updated in place. +# - If the API response shape changes (field ordering, new fields, etc.), the fingerprint can differ and duplicate rows may appear in the destination index. +latest: + unique_key: + - event.dataset + - axonius.identity.transform_unique_id + sort: '@timestamp' +description: >- + Deduplicates identity asset documents produced by repeated Axonius API polling: identical payloads (same fingerprint in axonius.identity.transform_unique_id) collapse to one row per event.dataset, keeping the document with the latest @timestamp. This is content-based deduplication, not tracking mutable entity state per the vendor, assets are created and deleted rather than updated in place. +frequency: 5m +settings: + # This is required to prevent the transform from clobbering the Fleet-managed mappings. + deduce_mappings: false + unattended: true +sync: + time: + field: 'event.ingested' + # Updated to 120s because of refresh delay in Serverless. With default 60s, + # sometimes transform wouldn't process all documents. + delay: 120s +retention_policy: + time: + field: '@timestamp' + max_age: 90d +_meta: + managed: false + # Bump this version to delete, reinstall, and restart the transform during + # package installation. + fleet_transform_version: 0.1.0 + run_as_kibana_system: false diff --git a/packages/axonius/img/axonius-identity-dashboard.png b/packages/axonius/img/axonius-identity-dashboard.png new file mode 100644 index 00000000000..65d45b9ee94 Binary files /dev/null and b/packages/axonius/img/axonius-identity-dashboard.png differ diff --git a/packages/axonius/img/axonius-logo.svg b/packages/axonius/img/axonius-logo.svg index 4a7fe63c0f5..9700c30034b 100644 --- a/packages/axonius/img/axonius-logo.svg +++ b/packages/axonius/img/axonius-logo.svg @@ -1,3 +1,3 @@ - - - + + + \ No newline at end of file diff --git a/packages/axonius/kibana/dashboard/axonius-60472232-ca7b-45e6-9fa6-72e6efc41a8e.json b/packages/axonius/kibana/dashboard/axonius-60472232-ca7b-45e6-9fa6-72e6efc41a8e.json new file mode 100644 index 00000000000..6339297c8ab --- /dev/null +++ b/packages/axonius/kibana/dashboard/axonius-60472232-ca7b-45e6-9fa6-72e6efc41a8e.json @@ -0,0 +1,2343 @@ +{ + "attributes": { + "controlGroupInput": { + "chainingSystem": "HIERARCHICAL", + "controlStyle": "oneLine", + "ignoreParentSettingsJSON": { + "ignoreFilters": false, + "ignoreQuery": false, + "ignoreTimerange": false, + "ignoreValidations": false + }, + "panelsJSON": {}, + "showApplySelections": false + }, + "description": "Overview of Identity assets including users, groups, accounts and more in Axonius.", + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "labels.is_transform_source", + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.filter[1].meta.index", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + } + } + }, + "optionsJSON": { + "hidePanelTitles": false, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false, + "useMargins": true + }, + "panelsJSON": [ + { + "embeddableConfig": { + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "savedVis": { + "data": { + "aggs": [], + "searchSource": { + "filter": [], + "query": { + "language": "kuery", + "query": "" + } + } + }, + "description": "", + "id": "", + "params": { + "fontSize": 12, + "markdown": "**Overview**\n\nThis dashboard provides a comprehensive view of Identity Assets collected from Axonius.\n\nIt provides clear visibility into the identity landscape, highlighting counts of active, inactive, suspended, and external users to help teams quickly assess overall account posture. Breakdowns by user status, type, and department reveal how identities are distributed across the organization and make it easier to spot unusual patterns or high-risk groups.\n\nTables showing top email addresses and cloud providers add further context, helping analysts identify heavily used accounts and understand where identities originate. These insights enable teams to monitor user activity, detect anomalies, and maintain strong identity hygiene across the environment.\n\n**Note:** Assets deleted from Axonius may reappear in a future discovery cycle if they are still present in connected data sources and get re-detected. Because the exact duration for which a deleted asset may remain dormant before being rediscovered is unknown, the transform retention period is set to **90 days** to reduce the risk of data loss for such assets. This means deleted assets will continue to appear in dashboards for up to 90 days after deletion.\n\n**[Integration Page](/app/integrations/detail/axonius)**\n\n", + "openLinksInNewTab": false + }, + "title": "", + "type": "markdown", + "uiState": {} + } + }, + "gridData": { + "h": 34, + "i": "81193bb4-dd3c-480a-be42-294bdc0ea866", + "w": 12, + "x": 0, + "y": 0 + }, + "panelIndex": "81193bb4-dd3c-480a-be42-294bdc0ea866", + "type": "visualization" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "2ee7aed8-ca20-47ad-b404-5795f39d9ffa", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "92ddbbb3-5b76-406f-94fa-5d6403f357c9": { + "columnOrder": [ + "85f5d170-59ca-4644-a4cf-86731e716c59" + ], + "columns": { + "85f5d170-59ca-4644-a4cf-86731e716c59": { + "customLabel": true, + "dataType": "number", + "filter": { + "language": "kuery", + "query": "axonius.identity.is_user_suspended : true " + }, + "isBucketed": false, + "label": "Suspended Users", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "2ee7aed8-ca20-47ad-b404-5795f39d9ffa", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#6092C0", + "layerId": "92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "layerType": "data", + "metricAccessor": "85f5d170-59ca-4644-a4cf-86731e716c59" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 8, + "i": "26f08be7-2fb8-467a-a083-b1ad005319aa", + "w": 9, + "x": 39, + "y": 0 + }, + "panelIndex": "26f08be7-2fb8-467a-a083-b1ad005319aa", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "39f9a1ea-9b71-4e3d-bde8-6c4f01c6ee6b", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "92ddbbb3-5b76-406f-94fa-5d6403f357c9": { + "columnOrder": [ + "85f5d170-59ca-4644-a4cf-86731e716c59" + ], + "columns": { + "85f5d170-59ca-4644-a4cf-86731e716c59": { + "customLabel": true, + "dataType": "number", + "filter": { + "language": "kuery", + "query": "axonius.identity.is_user_external : true " + }, + "isBucketed": false, + "label": "External Users", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "39f9a1ea-9b71-4e3d-bde8-6c4f01c6ee6b", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#6092C0", + "layerId": "92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "layerType": "data", + "metricAccessor": "85f5d170-59ca-4644-a4cf-86731e716c59" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 8, + "i": "3e391701-e157-420b-951a-d7100ae6a606", + "w": 9, + "x": 30, + "y": 0 + }, + "panelIndex": "3e391701-e157-420b-951a-d7100ae6a606", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "d9ff7f56-a595-45c7-85ee-0eb26c6c8ed3", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "92ddbbb3-5b76-406f-94fa-5d6403f357c9": { + "columnOrder": [ + "85f5d170-59ca-4644-a4cf-86731e716c59" + ], + "columns": { + "85f5d170-59ca-4644-a4cf-86731e716c59": { + "customLabel": true, + "dataType": "number", + "filter": { + "language": "kuery", + "query": "axonius.identity.is_user_inactive : true " + }, + "isBucketed": false, + "label": "Inactive Users", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "d9ff7f56-a595-45c7-85ee-0eb26c6c8ed3", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#6092C0", + "layerId": "92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "layerType": "data", + "metricAccessor": "85f5d170-59ca-4644-a4cf-86731e716c59" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 8, + "i": "6ad81d2b-ed86-4f33-8b69-f53709f63008", + "w": 9, + "x": 21, + "y": 0 + }, + "panelIndex": "6ad81d2b-ed86-4f33-8b69-f53709f63008", + "title": "", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "3e95f6ea-740a-4d7f-9d54-5ba9aa5919c3", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "92ddbbb3-5b76-406f-94fa-5d6403f357c9": { + "columnOrder": [ + "85f5d170-59ca-4644-a4cf-86731e716c59" + ], + "columns": { + "85f5d170-59ca-4644-a4cf-86731e716c59": { + "customLabel": true, + "dataType": "number", + "filter": { + "language": "kuery", + "query": "axonius.identity.is_user_active : true " + }, + "isBucketed": false, + "label": "Active Users", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "3e95f6ea-740a-4d7f-9d54-5ba9aa5919c3", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#6092C0", + "layerId": "92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "layerType": "data", + "metricAccessor": "85f5d170-59ca-4644-a4cf-86731e716c59" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 8, + "i": "e6ecc072-ac29-41db-9114-13dae61a9b12", + "w": 9, + "x": 12, + "y": 0 + }, + "panelIndex": "e6ecc072-ac29-41db-9114-13dae61a9b12", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-de0785d4-371f-4a45-ac37-2186b2edcf67", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "813ed89d-a9c0-4aa9-b7cc-5b18e16754dd", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "de0785d4-371f-4a45-ac37-2186b2edcf67": { + "columnOrder": [ + "74985650-cdb9-4a10-ba7a-b894a621146b", + "023570e2-b88f-44fc-a677-c18fede19957" + ], + "columns": { + "023570e2-b88f-44fc-a677-c18fede19957": { + "customLabel": true, + "dataType": "number", + "filter": { + "language": "kuery", + "query": "axonius.identity.asset_type : \"users\" " + }, + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "74985650-cdb9-4a10-ba7a-b894a621146b": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "User Status", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "023570e2-b88f-44fc-a677-c18fede19957", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "axonius.identity.user_status" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "813ed89d-a9c0-4aa9-b7cc-5b18e16754dd", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "colorMapping": { + "assignments": [], + "colorMode": { + "type": "categorical" + }, + "paletteId": "eui_amsterdam_color_blind", + "specialAssignments": [ + { + "color": { + "type": "loop" + }, + "rule": { + "type": "other" + }, + "touched": false + } + ] + }, + "layerId": "de0785d4-371f-4a45-ac37-2186b2edcf67", + "layerType": "data", + "legendDisplay": "show", + "metrics": [ + "023570e2-b88f-44fc-a677-c18fede19957" + ], + "nestedLegend": false, + "numberDisplay": "percent", + "primaryGroups": [ + "74985650-cdb9-4a10-ba7a-b894a621146b" + ], + "truncateLegend": false + } + ], + "shape": "pie" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 13, + "i": "091d0600-2142-4fed-99b2-cf8b3fd41f4f", + "w": 12, + "x": 12, + "y": 8 + }, + "panelIndex": "091d0600-2142-4fed-99b2-cf8b3fd41f4f", + "title": "Users by Status", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-de0785d4-371f-4a45-ac37-2186b2edcf67", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "ea142db6-7dc7-40fb-9dc5-9645e6541f82", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "de0785d4-371f-4a45-ac37-2186b2edcf67": { + "columnOrder": [ + "54671554-f13e-4033-ae23-a5a3b3611f70", + "023570e2-b88f-44fc-a677-c18fede19957" + ], + "columns": { + "023570e2-b88f-44fc-a677-c18fede19957": { + "customLabel": true, + "dataType": "number", + "filter": { + "language": "kuery", + "query": "axonius.identity.asset_type : \"users\" " + }, + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "54671554-f13e-4033-ae23-a5a3b3611f70": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "User Department", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "023570e2-b88f-44fc-a677-c18fede19957", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "axonius.identity.user_department" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "ea142db6-7dc7-40fb-9dc5-9645e6541f82", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "axisTitlesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "fittingFunction": "Linear", + "gridlinesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "labelsOrientation": { + "x": 0, + "yLeft": 0, + "yRight": 0 + }, + "layers": [ + { + "accessors": [ + "023570e2-b88f-44fc-a677-c18fede19957" + ], + "colorMapping": { + "assignments": [], + "colorMode": { + "type": "categorical" + }, + "paletteId": "eui_amsterdam_color_blind", + "specialAssignments": [ + { + "color": { + "type": "loop" + }, + "rule": { + "type": "other" + }, + "touched": false + } + ] + }, + "layerId": "de0785d4-371f-4a45-ac37-2186b2edcf67", + "layerType": "data", + "seriesType": "bar_horizontal", + "xAccessor": "54671554-f13e-4033-ae23-a5a3b3611f70" + } + ], + "legend": { + "isVisible": true, + "position": "right", + "shouldTruncate": false + }, + "preferredSeriesType": "bar_percentage_stacked", + "tickLabelsVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 13, + "i": "6951a3ed-dc02-4bfa-afa9-39c2832d7f1b", + "w": 24, + "x": 24, + "y": 8 + }, + "panelIndex": "6951a3ed-dc02-4bfa-afa9-39c2832d7f1b", + "title": "Users by Department", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-de0785d4-371f-4a45-ac37-2186b2edcf67", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "9578e7d9-00be-46e4-918c-d9c1fa412741", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "de0785d4-371f-4a45-ac37-2186b2edcf67": { + "columnOrder": [ + "54671554-f13e-4033-ae23-a5a3b3611f70", + "023570e2-b88f-44fc-a677-c18fede19957" + ], + "columns": { + "023570e2-b88f-44fc-a677-c18fede19957": { + "customLabel": true, + "dataType": "number", + "filter": { + "language": "kuery", + "query": "axonius.identity.asset_type : \"users\" " + }, + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "54671554-f13e-4033-ae23-a5a3b3611f70": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "User Type", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "023570e2-b88f-44fc-a677-c18fede19957", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "axonius.identity.user_type" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "9578e7d9-00be-46e4-918c-d9c1fa412741", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "axisTitlesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "fittingFunction": "Linear", + "gridlinesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "labelsOrientation": { + "x": 0, + "yLeft": 0, + "yRight": 0 + }, + "layers": [ + { + "accessors": [ + "023570e2-b88f-44fc-a677-c18fede19957" + ], + "colorMapping": { + "assignments": [], + "colorMode": { + "type": "categorical" + }, + "paletteId": "eui_amsterdam_color_blind", + "specialAssignments": [ + { + "color": { + "type": "loop" + }, + "rule": { + "type": "other" + }, + "touched": false + } + ] + }, + "layerId": "de0785d4-371f-4a45-ac37-2186b2edcf67", + "layerType": "data", + "seriesType": "bar_horizontal", + "xAccessor": "54671554-f13e-4033-ae23-a5a3b3611f70" + } + ], + "legend": { + "isVisible": true, + "position": "right", + "shouldTruncate": false + }, + "preferredSeriesType": "bar_percentage_stacked", + "tickLabelsVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 13, + "i": "c1ee3a0e-4c34-482f-b05f-4e7688cade1e", + "w": 36, + "x": 12, + "y": 21 + }, + "panelIndex": "c1ee3a0e-4c34-482f-b05f-4e7688cade1e", + "title": "Users by Type", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-d8a5a798-7079-4ee6-ac52-6fa6465a663f", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "0d3b0169-f18d-4606-8433-4a3cb429d180", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "d8a5a798-7079-4ee6-ac52-6fa6465a663f": { + "columnOrder": [ + "5703741c-7c01-443c-a659-9504618d9b66", + "8e4741cc-61b9-4853-822c-faacd3e44b9a" + ], + "columns": { + "5703741c-7c01-443c-a659-9504618d9b66": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "User email", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "8e4741cc-61b9-4853-822c-faacd3e44b9a", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "user.email" + }, + "8e4741cc-61b9-4853-822c-faacd3e44b9a": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "0d3b0169-f18d-4606-8433-4a3cb429d180", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "columns": [ + { + "columnId": "8e4741cc-61b9-4853-822c-faacd3e44b9a", + "isMetric": true, + "isTransposed": false + }, + { + "columnId": "5703741c-7c01-443c-a659-9504618d9b66", + "isMetric": false, + "isTransposed": false + } + ], + "layerId": "d8a5a798-7079-4ee6-ac52-6fa6465a663f", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 17, + "i": "66ca2230-6ac6-4f1d-a8d0-7e00a907e562", + "w": 23, + "x": 0, + "y": 34 + }, + "panelIndex": "66ca2230-6ac6-4f1d-a8d0-7e00a907e562", + "title": "Top Email Addresses", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-d8a5a798-7079-4ee6-ac52-6fa6465a663f", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "009cf6e6-9a5a-4eb6-a90a-842d200df069", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "d8a5a798-7079-4ee6-ac52-6fa6465a663f": { + "columnOrder": [ + "5703741c-7c01-443c-a659-9504618d9b66", + "8e4741cc-61b9-4853-822c-faacd3e44b9a" + ], + "columns": { + "5703741c-7c01-443c-a659-9504618d9b66": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Cloud Provider", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "8e4741cc-61b9-4853-822c-faacd3e44b9a", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "cloud.provider" + }, + "8e4741cc-61b9-4853-822c-faacd3e44b9a": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "009cf6e6-9a5a-4eb6-a90a-842d200df069", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "columns": [ + { + "columnId": "8e4741cc-61b9-4853-822c-faacd3e44b9a", + "isMetric": true, + "isTransposed": false + }, + { + "columnId": "5703741c-7c01-443c-a659-9504618d9b66", + "isMetric": false, + "isTransposed": false + } + ], + "layerId": "d8a5a798-7079-4ee6-ac52-6fa6465a663f", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "logs-*", + "negate": false, + "params": [ + { + "meta": { + "disabled": false, + "field": "data_stream.dataset", + "index": "logs-*", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "axonius.identity" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "axonius.identity" + } + } + }, + { + "meta": { + "disabled": false, + "field": "labels.is_transform_source", + "index": "logs-*", + "key": "labels.is_transform_source", + "negate": false, + "params": { + "query": "false" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "labels.is_transform_source": "false" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 17, + "i": "c6710940-9337-4981-9769-1fc79e4ece1a", + "w": 25, + "x": 23, + "y": 34 + }, + "panelIndex": "c6710940-9337-4981-9769-1fc79e4ece1a", + "title": "Top Cloud Providers", + "type": "lens" + } + ], + "timeRestore": false, + "title": "[Logs Axonius] Identity", + "version": 3 + }, + "coreMigrationVersion": "8.8.0", + "created_at": "2026-04-20T18:21:58.952Z", + "id": "axonius-60472232-ca7b-45e6-9fa6-72e6efc41a8e", + "references": [ + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[1].meta.index", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "26f08be7-2fb8-467a-a083-b1ad005319aa:indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "26f08be7-2fb8-467a-a083-b1ad005319aa:2ee7aed8-ca20-47ad-b404-5795f39d9ffa", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "3e391701-e157-420b-951a-d7100ae6a606:indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "3e391701-e157-420b-951a-d7100ae6a606:39f9a1ea-9b71-4e3d-bde8-6c4f01c6ee6b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "6ad81d2b-ed86-4f33-8b69-f53709f63008:indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "6ad81d2b-ed86-4f33-8b69-f53709f63008:d9ff7f56-a595-45c7-85ee-0eb26c6c8ed3", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e6ecc072-ac29-41db-9114-13dae61a9b12:indexpattern-datasource-layer-92ddbbb3-5b76-406f-94fa-5d6403f357c9", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e6ecc072-ac29-41db-9114-13dae61a9b12:3e95f6ea-740a-4d7f-9d54-5ba9aa5919c3", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "091d0600-2142-4fed-99b2-cf8b3fd41f4f:indexpattern-datasource-layer-de0785d4-371f-4a45-ac37-2186b2edcf67", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "091d0600-2142-4fed-99b2-cf8b3fd41f4f:813ed89d-a9c0-4aa9-b7cc-5b18e16754dd", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "6951a3ed-dc02-4bfa-afa9-39c2832d7f1b:indexpattern-datasource-layer-de0785d4-371f-4a45-ac37-2186b2edcf67", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "6951a3ed-dc02-4bfa-afa9-39c2832d7f1b:ea142db6-7dc7-40fb-9dc5-9645e6541f82", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c1ee3a0e-4c34-482f-b05f-4e7688cade1e:indexpattern-datasource-layer-de0785d4-371f-4a45-ac37-2186b2edcf67", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c1ee3a0e-4c34-482f-b05f-4e7688cade1e:9578e7d9-00be-46e4-918c-d9c1fa412741", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "66ca2230-6ac6-4f1d-a8d0-7e00a907e562:indexpattern-datasource-layer-d8a5a798-7079-4ee6-ac52-6fa6465a663f", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "66ca2230-6ac6-4f1d-a8d0-7e00a907e562:0d3b0169-f18d-4606-8433-4a3cb429d180", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c6710940-9337-4981-9769-1fc79e4ece1a:indexpattern-datasource-layer-d8a5a798-7079-4ee6-ac52-6fa6465a663f", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c6710940-9337-4981-9769-1fc79e4ece1a:009cf6e6-9a5a-4eb6-a90a-842d200df069", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[1].meta.index", + "type": "index-pattern" + } + ], + "type": "dashboard", + "typeMigrationVersion": "10.2.0", + "updated_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0" +} \ No newline at end of file diff --git a/packages/axonius/manifest.yml b/packages/axonius/manifest.yml index 605314ae55f..84b5764e26f 100644 --- a/packages/axonius/manifest.yml +++ b/packages/axonius/manifest.yml @@ -51,6 +51,10 @@ screenshots: title: Network Assets Dashboard size: 600x600 type: image/png + - src: /img/axonius-identity-dashboard.png + title: Identity Assets Dashboard + size: 600x600 + type: image/png icons: - src: /img/axonius-logo.svg title: Axonius Logo