From 5de947efa143dff291402dcacaf5aff9ebb0dda4 Mon Sep 17 00:00:00 2001 From: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> Date: Fri, 13 Dec 2024 12:56:32 -0500 Subject: [PATCH 1/2] adds note to LLM perf matrix (#6297) (cherry picked from commit a74bc3140815445a2e58b09f13e3e548efd492b0) # Conflicts: # docs/serverless/AI-for-security/llm-performance-matrix.asciidoc --- .../llm-performance-matrix.asciidoc | 3 ++- .../llm-performance-matrix.asciidoc | 21 +++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 docs/serverless/AI-for-security/llm-performance-matrix.asciidoc diff --git a/docs/AI-for-security/llm-performance-matrix.asciidoc b/docs/AI-for-security/llm-performance-matrix.asciidoc index c8f9e845c3..abed9dfdb2 100644 --- a/docs/AI-for-security/llm-performance-matrix.asciidoc +++ b/docs/AI-for-security/llm-performance-matrix.asciidoc @@ -13,4 +13,5 @@ This table describes the performance of various large language models (LLMs) for | *Assistant - Knowledge retrieval* | Good | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Excellent | *Attack Discovery* | Great | Great | Excellent | Poor | Poor | Great | Poor | Excellent | Poor |=== - \ No newline at end of file + +NOTE: `Excellent` is the best rating, followed by `Great`, then by `Good`, and finally by `Poor`. \ No newline at end of file diff --git a/docs/serverless/AI-for-security/llm-performance-matrix.asciidoc b/docs/serverless/AI-for-security/llm-performance-matrix.asciidoc new file mode 100644 index 0000000000..95a428dfe1 --- /dev/null +++ b/docs/serverless/AI-for-security/llm-performance-matrix.asciidoc @@ -0,0 +1,21 @@ +[[security-llm-performance-matrix]] += Large language model performance matrix + +// :description: Learn how different models perform on different tasks in {elastic-sec}. +// :keywords: security, overview, get-started + +This table describes the performance of various large language models (LLMs) for different use cases in {elastic-sec}, based on our internal testing. To learn more about these use cases, refer to <> or <>. + + +[cols="1,1,1,1,1,1,1,1,1,1", options="header"] +|=== +| *Feature* | *Model* | | | | | | | | +| | *Claude 3: Opus*| *Claude 3.5: Sonnet v2* | *Claude 3.5: Sonnet* | *Claude 3.5: Haiku*| *Claude 3: Haiku* | *GPT-4o* | *GPT-4o-mini* | **Gemini 1.5 Pro 002** | **Gemini 1.5 Flash 002** +| *Assistant - General* | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent +| *Assistant - {esql} generation*| Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Poor +| *Assistant - Alert questions* | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Good +| *Assistant - Knowledge retrieval* | Good | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Excellent +| *Attack Discovery* | Great | Great | Excellent | Poor | Poor | Great | Poor | Excellent | Poor +|=== + +NOTE: `Excellent` is the best rating, followed by `Great`, then by `Good`, and finally by `Poor`. \ No newline at end of file From 694ccbb9b6b677cad011d65302301b3370b3add8 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Fri, 13 Dec 2024 17:59:46 +0000 Subject: [PATCH 2/2] Delete docs/serverless directory and its contents --- .../llm-performance-matrix.asciidoc | 21 ------------------- 1 file changed, 21 deletions(-) delete mode 100644 docs/serverless/AI-for-security/llm-performance-matrix.asciidoc diff --git a/docs/serverless/AI-for-security/llm-performance-matrix.asciidoc b/docs/serverless/AI-for-security/llm-performance-matrix.asciidoc deleted file mode 100644 index 95a428dfe1..0000000000 --- a/docs/serverless/AI-for-security/llm-performance-matrix.asciidoc +++ /dev/null @@ -1,21 +0,0 @@ -[[security-llm-performance-matrix]] -= Large language model performance matrix - -// :description: Learn how different models perform on different tasks in {elastic-sec}. -// :keywords: security, overview, get-started - -This table describes the performance of various large language models (LLMs) for different use cases in {elastic-sec}, based on our internal testing. To learn more about these use cases, refer to <> or <>. - - -[cols="1,1,1,1,1,1,1,1,1,1", options="header"] -|=== -| *Feature* | *Model* | | | | | | | | -| | *Claude 3: Opus*| *Claude 3.5: Sonnet v2* | *Claude 3.5: Sonnet* | *Claude 3.5: Haiku*| *Claude 3: Haiku* | *GPT-4o* | *GPT-4o-mini* | **Gemini 1.5 Pro 002** | **Gemini 1.5 Flash 002** -| *Assistant - General* | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent -| *Assistant - {esql} generation*| Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Poor -| *Assistant - Alert questions* | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Good -| *Assistant - Knowledge retrieval* | Good | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Excellent -| *Attack Discovery* | Great | Great | Excellent | Poor | Poor | Great | Poor | Excellent | Poor -|=== - -NOTE: `Excellent` is the best rating, followed by `Great`, then by `Good`, and finally by `Poor`. \ No newline at end of file