Skip to content

Commit

Permalink
Query documentation: Query playground and query converter.
Browse files Browse the repository at this point in the history
  • Loading branch information
@mspanik
mspanik committed Jan 12, 2024
1 parent 90db927 commit 707ec1e
Show file tree
Hide file tree
Showing 4 changed files with 117 additions and 92 deletions.
46 changes: 27 additions & 19 deletions docs/concepts/query/midpoint-query-language/index.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,44 +32,52 @@ image:advanced-query-select.png[Select Advanced query]
You can test the queries directly in GUI. Starting in All users view. It is easy and straightforward.
As an example, to select all users with given name starting with "J" just enter `givenName startsWith "J"` to search bar and click search button.


For starting with midPoint query, please check
xref:introduction.adoc[introduction] document.

Examples how to use the midPoint query in GUI can be found in xref:/midpoint/guides/gui-midpoint-query-examples[Advanced search - EXAMPLES].

MidPoint provides xref:../query-concepts/index.adoc#_query_playground[query playground] feature that allows administrator more advanced experimenting with midPoint queries.

=== Using in Configuration

Advanced query filters can be used in any configuration place, normal XML filters
would be used. Advanced query filter is wrapped inside `<text>`
element inside `<filter>` element.
Advanced query filters can be used in any configuration place. In XML configuration files it is wrapped inside `<text>` element inside `<filter>` element.

//TODO: sem dat, ze potrebujem vybrat objekt.
IMPORTANT: Midpoint performs queries over defined set of objects. In GUI, the set is defined by actually opened view. In configuration, the object must be specified.
For example query `givenName startsWith "J"` is stored in midPoint configuration this way.

.Example XML
.midPoint Query in XML configuration format
[source, xml]
----
<filter>
<text>roleMembershipRef matches (relation = manager)</text>
</filter>
<query>
<filter>
<text>givenName startsWith "J"</text>
</filter>
</query>
----

.Example YAML
.midPoint Query in YAML configuration format
[source, yaml]
----
filter:
text: roleMembershipRef matches (relation = manager)
text: givenName startsWith "J"
----

.midPoint Query in JSON configuration format
[source, json]
----
"query" : {
"filter" : {
"text" : "givenName startsWith \"J\""
}
}
----

=== Query Converter

You may need to convert existing configuration using XML queries to midPoint Query Language.
MidPoint provides #Query converter# interface for such helping you with query conversion.
To start with midPoint Query Language, please read xref:introduction.adoc[*Introduction to midPoint Query Language*].

=== Query Playground

To experiment with the query language, there is hardly a better place than the actual running midPoint.
MidPoint provides xref:./query-playground/index.adoc#_query_playground[query playground] feature that allows administrator more advanced experimenting with midPoint queries.

You may need to convert existing configuration using XML queries to midPoint Query Language.
There is xref:./query-playground/index.adoc#_query_converter[query converter] in midPoint that is helpful with such query conversion.


== Motivation, Origin and Future
Expand Down
9 changes: 8 additions & 1 deletion docs/concepts/query/midpoint-query-language/introduction.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@
:page-nav-title: Introduction
:page-display-order: 100
:page-toc: top
:toclevels: 3
:toclevels: 2
:sectnums:
:sectnumlevels: 3


This document provides introduction to the *midPoint Query Language*.
Some details are omitted, and there are some simplifications for clarity.

After completing this document, reader should be able to apply midPoint Query Language in configuration.
We encourage to test the queries using the xref:./query-playground/index.adoc#_query_playground[query playground], which provides convenient and interactive environment for experimenting with queries.

== Language Description

Expand Down Expand Up @@ -420,3 +422,8 @@ assignment/targetRef/@ matches (
. fullText "secret"
)
----


//=== Query in configuration object

//TODO - tu ako dat query do tasku na recompute vsetkych userov, ktori maju rolu "End user"
82 changes: 82 additions & 0 deletions docs/concepts/query/midpoint-query-language/query-playground/index.adoc
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
= Query Playground and Query Converter
:page-nav-title: Errors while querying
:page-display-order: 600

To experiment with the query language, there is hardly a better place than the actual running midPoint.
Log in the GUI as administrator and choose *Query playground* in the main menu on the left, all the way down, just above *About*.

[#_query_playground]
== Query playground
To test the query, you have to:
. Select the *Object type*,
** Object type defines type of objects the query will be selecting from. While searching in GUI the object type is defined by actually opened view. In query playground you need to define it manually.
** check *Distinct* if needed (first try without it),
. write the *query* into the text area,
. press *Translate and execute*.
Alternatively use an existing example from the selection box below.
You can either execute the query or just simulate it.
In both cases you will see the translated SQL query (or HQL for the old Generic repository) and the parameter values.
[NOTE]
Container value queries are not yet supported by the Query playground.
[IMPORTANT]
====
The *distinct* option is often essential to get the right count of objects when searching in the Generic repository.
This is caused by `LEFT JOIN` used even when traversing to multi-value containers.
E.g., filter on `assignment/targetRef` causes each object with multiple assignments to be returned multiple times.
Using Exists filter (see later) does not fix this in the Generic repository either, as it also uses `LEFT JOIN` internally.
The new xref:/midpoint/reference/repository/native-postgresql/[Native repository] does not suffer
by these problems as it always uses SQL `EXISTS` when traversing to multi-value containers and also for Exists filter.
Native repository actually *ignores distinct* if there is no `JOIN` in the final query, as the returned raws must be distinct.
Native repository uses `LEFT JOIN` only to traverse across single-valued containers and references and their targets,
so even then the distinct option is useless and, when honoured, can potentially hurt SQL performance.
====
=== Fluent API script translation
Query playground offers an option to translate queries from fluent API scripts.
This way you can debug the Groovy code for an expression directly in the GUI before using it.
To do this, follow these steps:
* Check *Translate from Query API script*, the expression text area will appear.
* Enter the code as an expression, e.g.:
+
[source,xml]
----
<expression>
<script>
<code>
import com.evolveum.midpoint.xml.ns._public.common.common_3.*;
prismContext.queryFor(FocusType.class)
.item(FocusType.F_NAME).startsWith("a").build();
</code>
</script>
</expression>
----
* Choose the *Object type*, just like for any other query.
Just use the type from `queryFor(...)` call, in our example `FocusType`.
If the query does not provide expected results, very likely the object type selection is not right.
* Press *Translate and execute*.
This will execute the query in the expression and also shows the query in the text area for
XML/JSON/YAML query - in the language currently chosen.
Using the expression requires proper imports - depending on the complexity of the script you
may need additional imports from packages like `com.evolveum.midpoint.schema`,
`com.evolveum.midpoint.prism.query` or others.
[#_query_converter]
== Query converter
72 changes: 0 additions & 72 deletions docs/concepts/query/query-concepts/index.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2386,77 +2386,5 @@ ObjectQuery query = prismContext
.build();
----

//== Java fluent API TODO how to start, how to end build/buildFilter

[#_query_playground]
== Query playground

To experiment with the query language, there is hardly a better place than the actual running midPoint.
Log in the GUI as administrator and choose *Query playground* in the main menu on the left, all the way down, just above *About*.

To test the query, you have to:

. Select the *Object type*,
** Object type defines type of objects the query will be selecting from. While searching in GUI the object type is defined by actually opened view. In query playground you need to define it manually.
** check *Distinct* if needed (first try without it),
. write the *query* into the text area,
. press *Translate and execute*.

Alternatively use an existing example from the selection box below.

You can either execute the query or just simulate it.
In both cases you will see the translated SQL query (or HQL for the old Generic repository) and the parameter values.

[NOTE]
Container value queries are not yet supported by the Query playground.

[IMPORTANT]
====
The *distinct* option is often essential to get the right count of objects when searching in the Generic repository.
This is caused by `LEFT JOIN` used even when traversing to multi-value containers.
E.g., filter on `assignment/targetRef` causes each object with multiple assignments to be returned multiple times.
Using Exists filter (see later) does not fix this in the Generic repository either, as it also uses `LEFT JOIN` internally.
The new xref:/midpoint/reference/repository/native-postgresql/[Native repository] does not suffer
by these problems as it always uses SQL `EXISTS` when traversing to multi-value containers and also for Exists filter.
Native repository actually *ignores distinct* if there is no `JOIN` in the final query, as the returned raws must be distinct.
Native repository uses `LEFT JOIN` only to traverse across single-valued containers and references and their targets,
so even then the distinct option is useless and, when honoured, can potentially hurt SQL performance.
====

=== Fluent API script translation

Query playground offers an option to translate queries from fluent API scripts.
This way you can debug the Groovy code for an expression directly in the GUI before using it.

To do this, follow these steps:

* Check *Translate from Query API script*, the expression text area will appear.
* Enter the code as an expression, e.g.:
+
[source,xml]
----
<expression>
<script>
<code>
import com.evolveum.midpoint.xml.ns._public.common.common_3.*;
prismContext.queryFor(FocusType.class)
.item(FocusType.F_NAME).startsWith("a").build();
</code>
</script>
</expression>
----
* Choose the *Object type*, just like for any other query.
Just use the type from `queryFor(...)` call, in our example `FocusType`.
If the query does not provide expected results, very likely the object type selection is not right.
* Press *Translate and execute*.

This will execute the query in the expression and also shows the query in the text area for
XML/JSON/YAML query - in the language currently chosen.

Using the expression requires proper imports - depending on the complexity of the script you
may need additional imports from packages like `com.evolveum.midpoint.schema`,
`com.evolveum.midpoint.prism.query` or others.

// TODO mention extensions, xref:/midpoint/reference/schema/custom-schema-extension/

0 comments on commit 707ec1e

Please sign in to comment.