Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions explore-analyze/query-filter.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@ applies_to:
serverless: ga
products:
- id: elasticsearch
- id: kibana
---

# Querying and filtering [search-analyze]

You can use {{es}} as a basic document store to retrieve documents and their metadata. However, the real power of {{es}} comes from its advanced search and analytics capabilities.
{{es}} is not only great at storing and retrieving documents and their metadata, it also offers powerful querying and analytics capabilities that let you search, filter, and analyze your data at scale. These same capabilities are available in {{kib}} applications to facilitate interactive data exploration and visualization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice


* **{{es}} makes JSON documents searchable and aggregatable.** The documents are stored in an [index](/manage-data/data-store/index-basics.md) or [data stream](/manage-data/data-store/data-streams.md), which represent one type of data.
* **Searchable** means that you can filter the documents for conditions.** For example, you can filter for data "within the last 7 days" or data that "contains the word {{kib}}". {{kib}} provides many ways for you to construct filters, which are also called queries or search terms.
* **Aggregatable** means that you can extract summaries from matching documents.** The simplest aggregation is **count**, and it is frequently used in combination with the **date histogram**, to see count over time. The **terms** aggregation shows the most frequent values.
* **Searchable means that you can find documents through multiple retrieval methods.** This includes filtering by yes/no conditions, keyword and full-text search with relevance scoring, and vector/semantic search to find content based on meaning rather than exact terms. {{kib}} provides many ways for you to construct these searches, from simple filters in dashboards to relevance-ranked queries in its search interfaces.
* **Aggregatable means that you can compute statistics and summaries from matching documents to reveal patterns and insights in your dataset.** The simplest aggregation is **count**, and it is frequently used in combination with the **date histogram**, to see count over time. The **terms** aggregation shows the most frequent values.

## Querying
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be out of scope, but I wonder if this section needs a little overhaul too given the focus in E&A is now Kib?

This section now feels a little disjointed:

  • talks about apis instead of defining what querying means
  • mixes kib and es concepts without clear separation
  • api-centric — emphasizes technical endpoints over actual query capabilities
  • omits how to query through kibana ui components

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we haven't made a cut yet. Instead of making this page about Kibana, I rather think it should stay as is and move once we clean this up


Expand Down
21 changes: 10 additions & 11 deletions explore-analyze/query-filter/languages.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,21 @@
applies_to:
stack: ga
serverless: ga
products:
- id: elasticsearch
- id: kibana
---

# Query languages [search-analyze-query-languages]

{{es}} provides a number of query languages for interacting with your data.
{{es}} provides a number of query languages for interacting with your data. You can use these languages programmatically when working with {{es}} and {{kib}} APIs in your application, or interactively using the {{kib}} UI.


| Name | Description | Use cases | API endpoint |
| --- | --- | --- | --- |
| [Query DSL](languages/querydsl.md) | The primary query language for {{es}}. A powerful and flexible JSON-style language that enables complex queries. | Full-text search, semantic search, keyword search, filtering, aggregations, and more. | [`_search`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) |
| [{{esql}}](elasticsearch://reference/query-languages/esql.md) | Introduced in **8.11**, the Elasticsearch Query Language ({{esql}}) is a piped query language language for filtering, transforming, and analyzing data. | Initially tailored towards working with time series data like logs and metrics. Robust integration with {{kib}} for querying, visualizing, and analyzing data. | [`_query`](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-esql) |
| [EQL](languages/eql.md) | Event Query Language (EQL) is a query language for event-based time series data. Data must contain the `@timestamp` field to use EQL. | Designed for the threat hunting security use case. | [`_eql`](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-eql) |
| [Elasticsearch SQL](languages/sql.md) | Allows native, real-time SQL-like querying against {{es}} data. JDBC and ODBC drivers are available for integration with business intelligence (BI) tools. | Enables users familiar with SQL to query {{es}} data using familiar syntax for BI and reporting. | [`_sql`](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-sql) |
| [Kibana Query Language (KQL)](languages/kql.md) | {{kib}} Query Language (KQL) is a text-based query language for filtering data when you access it through the {{kib}} UI. | Use KQL to filter documents where a value for a field exists, matches a given value, or is within a given range. | N/A |

:::{tip}
> {{esql}} does not yet support all the features of Query DSL. Look forward to new {{esql}} features and functionalities in each release. [Learn more about {{esql}}](elasticsearch://reference/query-languages/esql.md).
:::

| **Query DSL** | {{es}}'s primary, most powerful and flexible JSON-style language for complex queries.<br><br>[Full language reference](elasticsearch://reference/query-languages/querydsl.md)| Full-text search, semantic search, keyword search, filtering, aggregations, and more. <br><br>[Query DSL in Kibana](languages/querydsl.md) |[`_search`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) |
| **{{esql}}** | Introduced in **8.11**, the Elasticsearch Query Language ({{esql}}) is a piped query language for filtering, transforming, and analyzing data.<br><br>[Full language reference](elasticsearch://reference/query-languages/esql.md) | You can use {{esql}} in multiple {{kib}} applications for querying, visualizing, and analyzing data.<br><br>[{{esql}} in Kibana](languages/esql-kibana.md) | [`_query`](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-esql) |
| **EQL** | Event Query Language (EQL) is a query language for event-based time series data. Data must contain the `@timestamp` field to use EQL.<br><br>[Full language reference](elasticsearch://reference/query-languages/eql.md) | Designed for the threat hunting security use case.<br><br>[EQL in Kibana](languages/eql.md) | [`_eql`](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-eql) |
| **Elasticsearch SQL** | Allows native, real-time SQL-like querying against {{es}} data. JDBC and ODBC drivers are available for integration with business intelligence (BI) tools.<br><br>[Full language reference](elasticsearch://reference/query-languages/sql.md) | Query {{es}} data using a familiar SQL syntax for BI and reporting.<br><br>[Elasticsearch SQL in Kibana](languages/sql.md) | [`_sql`](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-sql) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you use SQL in Kibana somewhere? 🤷‍♂️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Yes, Elasticsearch SQL can be used in several other places within Kibana beyond Dev Tools and SQL Workbench:

Kibana Apps Supporting SQL

  1. Discover
    Use the KQL/Lucene toggle and switch to SQL mode
    Write SQL queries directly in the search bar
    Results display in the familiar Discover interface with filtering capabilities
  2. Canvas
    Use SQL as a data source for Canvas workpads
    Create custom visualizations with SQL queries
  3. Lens (Limited)
    Some Lens visualizations can accept SQL-generated data
    Primarily through saved queries that can be referenced
  4. Dashboard Panels
    Create dashboard panels using SQL queries through Canvas elements
    Use SQL-based saved searches in dashboard visualizations

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a comment on this issue #2563 (comment) to deal with this

| **Kibana Query Language (KQL)** | {{kib}} Query Language (KQL) is a text-based query language for filtering data when you access it through the {{kib}} UI. | Use KQL to filter documents where a value for a field exists, matches a given value, or is within a given range.<br><br>[KQL in Kibana](languages/kql.md) | N/A |
| **Lucene query syntax** | The original query syntax for {{es}}, based on Apache Lucene. Simple text-based syntax for basic searches and filtering. | Perform basic text searches, simple field queries, and wildcard searches. Useful for quick searches and simple filtering.<br><br>[Lucene syntax in Kibana](languages/lucene-query-syntax.md) | [`_search`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-search) |
3 changes: 0 additions & 3 deletions explore-analyze/query-filter/tools/console.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,6 @@ products:

# Run API requests with Console [console-kibana]

$$$import-export-console-requests$$$


**Console** is an interactive UI for sending requests to [{{es}} APIs](elasticsearch://reference/elasticsearch/rest-apis/index.md) and [{{kib}} APIs](https://www.elastic.co/docs/api) and viewing their responses.

:::{image} /explore-analyze/images/kibana-console.png
Expand Down