Skip to content
This repository has been archived by the owner on Mar 28, 2024. It is now read-only.

Commit

Permalink
[Security Solution] Elastic Security Assistant (elastic#156933)
Browse files Browse the repository at this point in the history
## [Security Solution] Elastic Security Assistant

The _Elastic Security Assistant_ has entered the chat, integrating generative AI and large language models (LLMs) into the workflows of Elastic Security users.

Bring your alerts, events, rules, and data quality checks into the conversation.

<https://github.com/elastic/kibana/assets/2946766/31d65c78-5692-4817-b726-820c5df0801c>

This PR merges a feature branch developed by @spong and @andrew-goldstein , seeded by @jamesspi 's prototype of the assistant. Connectivity to LLMs is provided the [Generative AI Connector](<elastic#157228>) , developed by @stephmilovic . This PR includes:

- A new reusable Kibana package containing the assistant: `x-pack/packages/kbn-elastic-assistant`
  - See the `How to embed the Assistant in other parts of Kibana` for details
- Assistant integration into Elastic Security Solution workflows (e.g. alerts, cases, Timeline, rules, data quality)

### An assistant trained on the Elastic stack and Elastic Security

The [Generative AI Connector](<elastic#157228>) connects the assistant to OpenAI and Azure OpenAI models trained with knowledge of the Elastic stack and the Elastic Security solution, including:

- The Elastic open [Detection Rules](https://github.com/elastic/detection-rules)
- The [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html)
- Elastic query languages, including [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), [EQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html), and the [Elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)
- [Elasticsearch API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rest-apis.html)

This training enables the assistant to offer fully interactive chat experiences that include:

- alert summarization
- interactive query generation
- workflow suggestions
- generating ingestion configurations that conform to the Elastic Common Schema
- your imagination

using context from Elastic Security.

### Take action from your conversations

The Actions (from assistant response):

- Send KQL to Timeline
- Send EQL to Timeline
- Send Elasticsearch DSL to Timeline
- Send Note to timeline
- Create new case
- Add to existing case
- Copy to clipboard

### Components architecture diagram

![Untitled Diagram drawio (1)](https://github.com/elastic/kibana/assets/55110838/8f446313-629f-4646-b44d-0cb0ca74aeaf)

### How to embed the Assistant in other parts of Kibana

Follow the general instructions in `x-pack/packages/kbn-elastic-assistant/index.ts` to integrate the assistant into a Kibana app.

#### Step 1 - Wrap your Kibana app in the `AssistantProvider` component

```ts
// Step 1: Wrap your Kibana app in the `AssistantProvider` component. This typically
// happens in the root of your app. Optionally provide a custom title for the assistant:

/** provides context (from the app) to the assistant, and injects Kibana services, like `http` */
export { AssistantProvider } from './impl/assistant_context';
```

#### Step 2: Add the `AssistantOverlay` component to your app

```ts
// Step 2: Add the `AssistantOverlay` component to your app. This component displays the assistant
// overlay in a modal, bound to a shortcut key:

/** modal overlay for Elastic Assistant conversations */
export { AssistantOverlay } from './impl/assistant/assistant_overlay';

// In addition to the `AssistantOverlay`, or as an alternative, you may use the `Assistant` component
// to display the assistant without the modal overlay:

/** this component renders the Assistant without the modal overlay to, for example, render it in a Timeline tab */
export { Assistant } from './impl/assistant';
```

#### Step 3: Wherever you want to bring context into the assistant, use the any combination of the following

```ts
// Step 3: Wherever you want to bring context into the assistant, use the any combination of the following
// components and hooks:
// - `NewChat` component
// - `NewChatById` component
// - `useAssistantOverlay` hook

/**
 * `NewChat` displays a _New chat_ icon button, providing all the context
 * necessary to start a new chat. You may optionally style the button icon,
 * or override the default _New chat_ text with custom content, like `🪄✨`
 *
 * USE THIS WHEN: All the data necessary to start a new chat is available
 * in the same part of the React tree as the _New chat_ button.
 */
export { NewChat } from './impl/new_chat';

/**
 * `NewChatByID` displays a _New chat_ icon button by providing only the `promptContextId`
 * of a context that was (already) registered by the `useAssistantOverlay` hook. You may
 * optionally style the button icon, or override the default _New chat_ text with custom
 * content, like {'🪄✨'}
 *
 * USE THIS WHEN: all the data necessary to start a new chat is NOT available
 * in the same part of the React tree as the _New chat_ button. When paired
 * with the `useAssistantOverlay` hook, this option enables context to be be
 * registered where the data is available, and then the _New chat_ button can be displayed
 * in another part of the tree.
 */
export { NewChatById } from './impl/new_chat_by_id';

/**
 * `useAssistantOverlay` is a hook that registers context with the assistant overlay, and
 * returns an optional `showAssistantOverlay` function to display the assistant overlay.
 * As an alterative to using the `showAssistantOverlay` returned from this hook, you may
 * use the `NewChatById` component and pass it the `promptContextId` returned by this hook.
 *
 * USE THIS WHEN: You want to register context in one part of the tree, and then show
 * a _New chat_ button in another part of the tree without passing around the data, or when
 * you want to build a custom `New chat` button with features not not provided by the
 * `NewChat` component.
 */
export { useAssistantOverlay } from './impl/assistant/use_assistant_overlay';
```

Co-authored-by: Garrett Spong <garrett.spong@elastic.co>
Co-authored-by: Andrew Macri <andrew.macri@elastic.co>
  • Loading branch information
3 people authored and Sloane Perrault committed Jun 6, 2023
1 parent 02d2288 commit ce0cbf3
Show file tree
Hide file tree
Showing 147 changed files with 7,871 additions and 94 deletions.
5 changes: 5 additions & 0 deletions .eslintrc.js
Expand Up @@ -982,6 +982,7 @@ module.exports = {
// front end and common typescript and javascript files only
files: [
'x-pack/plugins/ecs_data_quality_dashboard/common/**/*.{js,mjs,ts,tsx}',
'x-pack/packages/kbn-elastic-assistant/**/*.{js,mjs,ts,tsx}',
'x-pack/packages/security-solution/**/*.{js,mjs,ts,tsx}',
'x-pack/plugins/security_solution/public/**/*.{js,mjs,ts,tsx}',
'x-pack/plugins/security_solution/common/**/*.{js,mjs,ts,tsx}',
Expand Down Expand Up @@ -1009,13 +1010,15 @@ module.exports = {
// This should be a very small set as most linter rules are useful for tests as well.
files: [
'x-pack/plugins/ecs_data_quality_dashboard/**/*.{ts,tsx}',
'x-pack/packages/kbn-elastic-assistant/**/*.{ts,tsx}',
'x-pack/packages/security-solution/**/*.{ts,tsx}',
'x-pack/plugins/security_solution/**/*.{ts,tsx}',
'x-pack/plugins/timelines/**/*.{ts,tsx}',
'x-pack/plugins/cases/**/*.{ts,tsx}',
],
excludedFiles: [
'x-pack/plugins/ecs_data_quality_dashboard/**/*.{test,mock,test_helper}.{ts,tsx}',
'x-pack/packages/kbn-elastic-assistant/**/*.{test,mock,test_helper}.{ts,tsx}',
'x-pack/packages/security-solution/**/*.{test,mock,test_helper}.{ts,tsx}',
'x-pack/plugins/security_solution/**/*.{test,mock,test_helper}.{ts,tsx}',
'x-pack/plugins/timelines/**/*.{test,mock,test_helper}.{ts,tsx}',
Expand All @@ -1029,6 +1032,7 @@ module.exports = {
// typescript only for front and back end
files: [
'x-pack/plugins/ecs_data_quality_dashboard/**/*.{ts,tsx}',
'x-pack/packages/kbn-elastic-assistant/**/*.{ts,tsx}',
'x-pack/packages/security-solution/**/*.{ts,tsx}',
'x-pack/plugins/security_solution/**/*.{ts,tsx}',
'x-pack/plugins/timelines/**/*.{ts,tsx}',
Expand Down Expand Up @@ -1061,6 +1065,7 @@ module.exports = {
// typescript and javascript for front and back end
files: [
'x-pack/plugins/ecs_data_quality_dashboard/**/*.{js,mjs,ts,tsx}',
'x-pack/packages/kbn-elastic-assistant/**/*.{js,mjs,ts,tsx}',
'x-pack/packages/security-solution/**/*.{js,mjs,ts,tsx}',
'x-pack/plugins/security_solution/**/*.{js,mjs,ts,tsx}',
'x-pack/plugins/timelines/**/*.{js,mjs,ts,tsx}',
Expand Down
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Expand Up @@ -321,6 +321,7 @@ packages/kbn-ebt-tools @elastic/kibana-core
packages/kbn-ecs @elastic/kibana-core @elastic/security-threat-hunting-investigations
x-pack/packages/security-solution/ecs_data_quality_dashboard @elastic/security-threat-hunting-investigations
x-pack/plugins/ecs_data_quality_dashboard @elastic/security-threat-hunting-investigations
x-pack/packages/kbn-elastic-assistant @elastic/security-solution
test/plugin_functional/plugins/elasticsearch_client_plugin @elastic/kibana-core
x-pack/test/plugin_api_integration/plugins/elasticsearch_client @elastic/kibana-core
x-pack/plugins/embeddable_enhanced @elastic/kibana-presentation
Expand Down
1 change: 1 addition & 0 deletions package.json
Expand Up @@ -361,6 +361,7 @@
"@kbn/ecs": "link:packages/kbn-ecs",
"@kbn/ecs-data-quality-dashboard": "link:x-pack/packages/security-solution/ecs_data_quality_dashboard",
"@kbn/ecs-data-quality-dashboard-plugin": "link:x-pack/plugins/ecs_data_quality_dashboard",
"@kbn/elastic-assistant": "link:x-pack/packages/kbn-elastic-assistant",
"@kbn/elasticsearch-client-plugin": "link:test/plugin_functional/plugins/elasticsearch_client_plugin",
"@kbn/elasticsearch-client-xpack-plugin": "link:x-pack/test/plugin_api_integration/plugins/elasticsearch_client",
"@kbn/embeddable-enhanced-plugin": "link:x-pack/plugins/embeddable_enhanced",
Expand Down
1 change: 1 addition & 0 deletions packages/kbn-babel-preset/styled_components_files.js
Expand Up @@ -16,6 +16,7 @@ module.exports = {
/src[\/\\]plugins[\/\\](kibana_react)[\/\\]/,
/x-pack[\/\\]plugins[\/\\](apm|beats_management|cases|fleet|infra|lists|observability|observability_shared|exploratory_view|osquery|security_solution|timelines|synthetics|ux)[\/\\]/,
/x-pack[\/\\]test[\/\\]plugin_functional[\/\\]plugins[\/\\]resolver_test[\/\\]/,
/x-pack[\/\\]packages[\/\\]elastic_assistant[\/\\]/,
/x-pack[\/\\]packages[\/\\]security-solution[\/\\]ecs_data_quality_dashboard[\/\\]/,
],
};
2 changes: 2 additions & 0 deletions tsconfig.base.json
Expand Up @@ -636,6 +636,8 @@
"@kbn/ecs-data-quality-dashboard/*": ["x-pack/packages/security-solution/ecs_data_quality_dashboard/*"],
"@kbn/ecs-data-quality-dashboard-plugin": ["x-pack/plugins/ecs_data_quality_dashboard"],
"@kbn/ecs-data-quality-dashboard-plugin/*": ["x-pack/plugins/ecs_data_quality_dashboard/*"],
"@kbn/elastic-assistant": ["x-pack/packages/kbn-elastic-assistant"],
"@kbn/elastic-assistant/*": ["x-pack/packages/kbn-elastic-assistant/*"],
"@kbn/elasticsearch-client-plugin": ["test/plugin_functional/plugins/elasticsearch_client_plugin"],
"@kbn/elasticsearch-client-plugin/*": ["test/plugin_functional/plugins/elasticsearch_client_plugin/*"],
"@kbn/elasticsearch-client-xpack-plugin": ["x-pack/test/plugin_api_integration/plugins/elasticsearch_client"],
Expand Down
1 change: 1 addition & 0 deletions x-pack/.i18nrc.json
Expand Up @@ -21,6 +21,7 @@
"xpack.dashboard": "plugins/dashboard_enhanced",
"xpack.discover": "plugins/discover_enhanced",
"xpack.crossClusterReplication": "plugins/cross_cluster_replication",
"xpack.elasticAssistant": "packages/kbn-elastic-assistant",
"xpack.embeddableEnhanced": "plugins/embeddable_enhanced",
"xpack.endpoint": "plugins/endpoint",
"xpack.enterpriseSearch": "plugins/enterprise_search",
Expand Down
20 changes: 20 additions & 0 deletions x-pack/packages/kbn-elastic-assistant/README.md
@@ -0,0 +1,20 @@
# @kbn/elastic-assistant

The `Elastic Assistant` is a user interface for interacting with generative AIs, like `ChatGPT`.

This package provides:

- Components for rendering the `Elastic Assistant`
- Hooks for passing context (for example, fields in an alert) to the `Elastic Assistant`, enabling users to include this content in their queries

## Maintainers

Maintained by the Security Solution team

## Running unit tests with code coverage

To (interactively) run unit tests with code coverage, run the following command:

```sh
cd $KIBANA_HOME && node scripts/jest --watch x-pack/packages/kbn-elastic-assistant --coverage
```
84 changes: 84 additions & 0 deletions x-pack/packages/kbn-elastic-assistant/impl/assistant/api.tsx
@@ -0,0 +1,84 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { OpenAiProviderType } from '@kbn/stack-connectors-plugin/public/common';

import { HttpSetup } from '@kbn/core-http-browser';
import type { Message } from '../assistant_context/types';
import { Conversation } from '../assistant_context/types';
import { API_ERROR } from './translations';

export interface FetchConnectorExecuteAction {
apiConfig: Conversation['apiConfig'];
http: HttpSetup;
messages: Message[];
signal?: AbortSignal | undefined;
}

export const fetchConnectorExecuteAction = async ({
http,
messages,
apiConfig,
signal,
}: FetchConnectorExecuteAction): Promise<string> => {
const outboundMessages = messages.map((msg) => ({
role: msg.role,
content: msg.content,
}));

const body =
apiConfig?.provider === OpenAiProviderType.OpenAi
? {
model: 'gpt-3.5-turbo',
messages: outboundMessages,
n: 1,
stop: null,
temperature: 0.2,
}
: {
messages: outboundMessages,
};

const requestBody = {
params: {
subActionParams: {
body: JSON.stringify(body),
},
subAction: 'test',
},
};

try {
// TODO: Find return type for this API
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const response = await http.fetch<any>(
`/api/actions/connector/${apiConfig?.connectorId}/_execute`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(requestBody),
signal,
}
);

const data = response.data;
if (response.status !== 'ok') {
return API_ERROR;
}

if (data.choices && data.choices.length > 0 && data.choices[0].message.content) {
const result = data.choices[0].message.content.trim();
return result;
} else {
return API_ERROR;
}
} catch (error) {
return API_ERROR;
}
};
@@ -0,0 +1,90 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import React, { useCallback, useEffect, useState } from 'react';
import { EuiModal } from '@elastic/eui';

import useEvent from 'react-use/lib/useEvent';
// eslint-disable-next-line @kbn/eslint/module_migration
import styled from 'styled-components';
import { ShowAssistantOverlayProps, useAssistantContext } from '../../assistant_context';
import { Assistant } from '..';
import { WELCOME_CONVERSATION_TITLE } from '../use_conversation/translations';

const isMac = navigator.platform.toLowerCase().indexOf('mac') >= 0;

const StyledEuiModal = styled(EuiModal)`
min-width: 1200px;
max-height: 100%;
height: 100%;
`;

/**
* Modal container for Security Assistant conversations, receiving the page contents as context, plus whatever
* component currently has focus and any specific context it may provide through the SAssInterface.
*/
export const AssistantOverlay: React.FC = React.memo(() => {
const [isModalVisible, setIsModalVisible] = useState(false);
const [conversationId, setConversationId] = useState<string | undefined>(
WELCOME_CONVERSATION_TITLE
);
const [promptContextId, setPromptContextId] = useState<string | undefined>();
const { setShowAssistantOverlay } = useAssistantContext();

// Bind `showAssistantOverlay` in SecurityAssistantContext to this modal instance
const showOverlay = useCallback(
() =>
({
showOverlay: so,
promptContextId: pid,
conversationId: cid,
}: ShowAssistantOverlayProps) => {
setIsModalVisible(so);
setPromptContextId(pid);
setConversationId(cid);
},
[setIsModalVisible]
);
useEffect(() => {
setShowAssistantOverlay(showOverlay);
}, [setShowAssistantOverlay, showOverlay]);

// Register keyboard listener to show the modal when cmd + ; is pressed
const onKeyDown = useCallback(
(event: KeyboardEvent) => {
if (event.key === ';' && (isMac ? event.metaKey : event.ctrlKey)) {
event.preventDefault();
setIsModalVisible(!isModalVisible);
}
},
[isModalVisible]
);
useEvent('keydown', onKeyDown);

// Modal control functions
const cleanupAndCloseModal = useCallback(() => {
setIsModalVisible(false);
setPromptContextId(undefined);
setConversationId(conversationId);
}, [conversationId]);

const handleCloseModal = useCallback(() => {
cleanupAndCloseModal();
}, [cleanupAndCloseModal]);

return (
<>
{isModalVisible && (
<StyledEuiModal onClose={handleCloseModal}>
<Assistant conversationId={conversationId} promptContextId={promptContextId} />
</StyledEuiModal>
)}
</>
);
});

AssistantOverlay.displayName = 'AssistantOverlay';
@@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { i18n } from '@kbn/i18n';

export const CANCEL_BUTTON = i18n.translate(
'xpack.elasticAssistant.assistant.overlay.CancelButton',
{
defaultMessage: 'Cancel',
}
);

0 comments on commit ce0cbf3

Please sign in to comment.