Merge remote-tracking branch 'refs/remotes/origin/master'

docs/mark/event-marks.adoc

@@ -0,0 +1,110 @@
+= Event Marks
+:page-toc: top
+:page-since: "4.7"
+
+MidPoint can classify individual operations, or events, by attaching one or more marks to each of them.
+Examples of such a mark are "Focus activated", "Focus renamed", "Projection entitlement changed".
+Some marks are available as standard initial objects, so that they can be customized, supplemented by customer-defined ones, or even removed.
+
+Conceptually, they are related to xref:index.adoc[object marks], but the major difference is that while object marks are attached to individual _objects_ (currently, shadows), event marks are attached to _operations_ on objects (currently both focal objects and shadows).
+What is common is that both object and event marks are defined using xref:/midpoint/reference/roles-policies/policy-rules.adoc[policy rules].
+
+== List of Standard Event Marks
+
+As of 4.7, there are the following standard event marks.
+
+.Standard event marks
+[%autowidth]
+|===
+| Name | Description
+
+| Focus activated
+| Marks an operation that brought the focus into existence and being effectively enabled.
+(So, previously it was either non-existent or effectively disabled.)
+
+| Focus deactivated
+| Marks an operation that deleted or effectively disabled the focus.
+(So, previously it existed and was effectively enabled.)
+
+| Focus renamed
+| Marks an operation that changed the focus name.
+
+| Focus assignments changed
+| Marks an operation that changed one or more of the focus assignments.
+
+| Focus archetype changed
+| Marks an operation that changed the focus archetype or archetypes.
+
+| Focus parent organization reference changed
+| Marks an operation that changed the focus parent organization reference (`parentOrgRef` item).
+
+| Focus role membership changed
+| Marks an operation that changed the focus role membership (`roleMembershipRef` item).
+
+| Projection activated
+| Marks an operation that brought the projection into existence and being effectively enabled.
+(So, previously it was either non-existent or effectively disabled.)
+
+| Projection deactivated
+| Marks an operation that deleted or effectively disabled the projection.
+(So, previously it existed and was effectively enabled.)
+
+| Projection renamed
+| Marks an operation that changed the name (naming attribute) of a projection.
+
+| Projection identifier changed
+| Marks an operation that changed the identifier (primary or secondary) of a projection.
+
+| Projection entitlement changed
+| This is an operation that changed the entitlement of a projection.
+
+| Projection password changed
+| Marks an operation that changed the password of a projection.
+
+| Resource object affected
+| Marks an operation that made any change (addition, modification, deletion) to a resource object.
+
+| Shadow classification changed
+| Marks an operation that changed the classification (kind, intent) of a shadow.
+Experimental.
+
+| Shadow correlation state changed
+| Marks an operation that changed the correlation state of a shadow.
+The synchronization situation is considered to be part of that state.
+Experimental.
+|===
+
+== Use
+
+In 4.7, the event marks are used exclusively in connection with xref:../simulation/[simulations].
+In the future we plan to extend their use to operations that are really executed, so they might be found in audit logs or on operational dashboards.
+
+== Definition
+
+Event marks are defined using standard policy rules.
+
+.Listing 1. An example of event mark definition
+[source,xml]
+----
+<mark xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
+    oid="00000000-0000-0000-0000-000000000712">
+    <name>Focus renamed</name>
+    <description>Operation that changed the focus name</description>
+    <assignment>
+        <targetRef oid="00000000-0000-0000-0000-000000000700" type="ArchetypeType"/> <!--1-->
+    </assignment>
+    <policyRule>
+        <policyConstraints>
+            <modification> <!--2-->
+                <operation>modify</operation>
+                <item>name</item>
+            </modification>
+        </policyConstraints>
+    </policyRule>
+</mark>
+----
+<1> This is an archetype for event marks.
+The other one is for object marks.
+<2> This constraint triggers when it "sees" a modification of focus object that changes the `name` property.
+
+Please see the initial objects for more examples.

docs/misc/reports/configuration/index.adoc

@@ -94,7 +94,7 @@ Possible values: _csv_ and _html_.
 
 | csv
 | *CsvFileFormatType*
-| Configuration attribute for csv export.
+| Configuration attribute for CSV export.
 
 
 | html
@@ -167,14 +167,20 @@ Possible values is _all, allNonNull, minimal, nonNumeric_ and _none._
 
 | _createHeader_
 | *boolean*
-| Create header in csv output file.
+| Create header in CSV output file.
 | true
 
 
 | _encoding_
 | *string*
-| Encoding of csv file.
-| utf-8
+| Encoding of CSV file.
+Most common encodings (UTF-8, windows-125x, ISO-8859-x) are supported.
+
+See https://www.iana.org/assignments/character-sets/character-sets.xml[IANA list of charsets]
+for proper values - generally, both IANA and MIME names are supported as well as many aliases.
+Details of supported character sets depend on the concrete JDK used to run midPoint,
+but it is unlikely that your charset is not supported.
+| UTF-8
 
 
 |===

docs/simulation/index.adoc

@@ -49,13 +49,8 @@ We are looking for a better one.
 Here, no actions that could have persistent externally-visible effects are executed.
 Accordingly, nothing is written to the system audit log.
 
-The changes are computed, just like in <<Persistent-Effects Mode>> but instead of their application, they are written to so-called simulation result.
-This is a special type of objects in the midPoint repository.
-A simulation result collects information about:
-
-. objects seen by the activity (both midPoint repository objects and resource objects),
-. changes computed against these objects,
-. values of so-called _simulation metrics_ that provide high-level view of the activity being simulated.
+The changes are computed, just like in <<Persistent-Effects Mode>> but instead of their application, they are written to so-called xref:/midpoint/reference/simulation/results/[simulation result].
+It is a special kind of objects in midPoint repository, collecting both statistical and detailed information about changes that would be applied.
 
 == Production vs Development Configuration
 

docs/simulation/reports/items-changed.adoc

@@ -1,6 +1,7 @@
 = Items Changed
 :page-toc: top
 :page-since: "4.7"
+:page-display-order: 300
 
 This report lists items changed within individual processed objects.
 

docs/simulation/reports/objects-with-metrics.adoc

@@ -1,6 +1,7 @@
 = Objects with Metrics
 :page-toc: top
 :page-since: "4.7"
+:page-display-order: 250
 
 This report lists objects processed by given simulation along with the values of individual metrics for these objects.
 

docs/simulation/reports/objects.adoc

@@ -1,6 +1,7 @@
 = Objects
 :page-toc: top
 :page-since: "4.7"
+:page-display-order: 200
 
 This report lists objects processed by given simulation.
 One processed object corresponds to one report row.

docs/simulation/reports/results.adoc

@@ -1,6 +1,7 @@
 = Results
 :page-toc: top
 :page-since: "4.7"
+:page-display-order: 100
 
 This is the most high-level report on simulation results.
 It shows individual results, along with metric values, decomposed into individual partitions
@@ -53,11 +54,11 @@ _The following part of the report is instantiated once per each simulation xref:
 
 | Aggregation function
 | Aggregation function used for given metric.
-See xref:../results/aggregation-functions.adoc[].
+See xref:../results/metrics.adoc#_aggregation[].
 
 2+|
 {zwsp} +
-_The following part of the report is instantiated once per each xref:../results/partitions.adoc[metric partition]._ +
+_The following part of the report is instantiated once per each metric partition._ +
 {zwsp}
 
 | Scope: type

docs/simulation/reports/values-changed.adoc

@@ -1,6 +1,7 @@
 = Values Changed
 :page-toc: top
 :page-since: "4.7"
+:page-display-order: 400
 
 This report lists values added or deleted within individual items in processed objects.
 

docs/simulation/results/definition.adoc

@@ -1,3 +1,128 @@
 = Simulation Result Definition
+:page-toc: top
+:page-since: "4.7"
 
-#TODO#
+There is a possibility to configure the way of how xref:index.adoc[simulation result] is computed and stored.
+MidPoint maintains a global configuration for simulation results, but - if needed - it is possible to configure the simulation result creation for each simulation activity independently.
+
+== Global Configuration
+
+The global configuration is stored in `simulation` container in the system configuration.
+There are currently two sub-items there:
+
+.Items in the global configuration of simulations
+[%autowidth]
+|====
+| Container | Description
+
+| `metric`
+| Explicitly defined xref:metrics.adoc[metrics].
+
+| `simulation`
+| Library of xref:#_library[global simulation result definitions].
+|====
+
+Please follow the links above to learn more about the respective topics.
+Now let us have a look at how individual simulation result can be configured.
+
+== Simulation Result Definition
+
+Individual simulation result creation is configured using the following configuration items:
+
+.Configuration items for simulation result creation
+[%autowidth]
+|====
+| Property | Description | Default value
+| `identifier`
+| Identifier of the definition.
+Useful when extending this definition using `super` property, when determining the name of the simulation result, and for diagnostics.
+If used in the global library of simulation result definitions, it must be unique.
+| none
+
+| `super`
+| Identifier of the "super-definition", i.e. the definition the current one is extending.
+| none
+
+| `default`
+| Is this the default definition?
+| see <<_library>> below
+
+| `metrics`
+| How should be the (globally-defined) xref:metrics.adoc[metrics] used in this particular simulation type?
+| "as they are", without any changes
+
+| `useOwnPartitionForProcessedObjects`
+| Should this simulation use its own database partition for processed objects?
+| no
+
+|====
+
+[#_partitions]
+== Database Partitions for Processed Objects
+
+As described in the xref:index.adoc[overview], a simulation result consists of metrics and other overview data, plus detailed information about individual processed objects.
+The latter can occupy substantial place in the repository.
+Moreover, there can be use cases when we want to keep historic records of the metrics and overview data, but we are not so interested in the details.
+
+Hence, midPoint allows to quickly delete processed objects from a simulation by placing them into separate PostgreSQL database table partition - one partition per one simulation result with this feature enabled.
+
+Whether a partition should be created for given simulation result, is driven by `useOwnPartitionForProcessedObjects` property in the definition for that simulation result.
+
+== An Example of Definition Reuse
+
+Let us have the following two global simulation result definitions:
+
+.Listing 1. Sample global simulation result definitions
+[source,xml]
+----
+<simulation>
+    <metric><!-- some metrics are defined here --></metric>
+    <simulation>
+        <identifier>small</identifier>
+        <description>Simulation result definition suitable for simulations with small number of processed objects.</description>
+        <default>true</default>
+        <useOwnPartitionForProcessedObjects>false</useOwnPartitionForProcessedObjects>
+    </simulation>
+    <simulation>
+        <identifier>large</identifier>
+        <description>Simulation result definition suitable for simulations with large number of processed objects.</description>
+        <metrics><!-- some metrics are disabled or enabled here --></metrics>
+        <useOwnPartitionForProcessedObjects>true</useOwnPartitionForProcessedObjects>
+    </simulation>
+</simulation>
+----
+
+The `small` one is the default.
+The `large` is suitable in special cases when large numbers of objects are processed, so it uses a different configuration.
+Now let us assume you want to create a simulation activity that would use the `large` definition but with some minor adjustments.
+We can do that like this:
+
+.Listing 2. A simulation activity extending the `large` simulation result definition
+[source,xml]
+----
+<activity>
+    <!-- ... -->
+    <reporting>
+        <simulationResult>
+            <definition>
+                <super>large</super>
+                <useOwnPartitionForProcessedObjects>false</useOwnPartitionForProcessedObjects>
+            </definition>
+        </simulationResult>
+    </reporting>
+</activity>
+----
+
+This activity will use the `large` definition, with all of its metrics tailoring, but setting `useOwnPartitionForProcessedObjects` value to `false`.
+
+[#_library]
+== Library of Global Simulation Result Definitions
+
+The library of global simulation result definitions can be used when configuring individual activities.
+An activity can use or extend any of these global definitions, or - if nothing is explicitly specified for that activity - it can use the default definition.
+The default definition is determined like this:
+
+. If there is no global definition, the default definition is the empty one (i.e. with all properties having their default values).
+. If there is a single global definition, it is used as the default one.
+. If there are more definitions, and exactly one of them is marked as `default`, it is used as such.
+. Otherwise, a configuration exception is signalled.

docs/simulation/results/index.adoc

@@ -0,0 +1,30 @@
+= Simulation Result
+:page-since: "4.7"
+
+Each activity, regardless of whether it is running on foreground or on background, may create so-called _simulation result_.
+It is a special type of objects in the midPoint repository.
+A simulation result collects information about:
+
+. objects seen by the activity (both midPoint repository objects and resource objects),
+. changes computed against these objects,
+. values of so-called xref:metrics.adoc[simulation metrics] that provide high-level view of the activity being simulated.
+
+== Removing Details of Processed Objects
+
+Simulation results storage is modular in the sense that - in order to spare the space used - it is possible to leave out the storage of individual objects and changes projected against them.
+In such cases, only the metrics values are stored.
+
+This elimination of details can be done in various ways:
+
+. via GUI (see xref:#_figure1[] below),
+. using a clean-up task (planned for future),
+. via experimental midPoint Java API (`RepositoryService.deleteSimulatedProcessedObjects`),
+. right in the database, e.g. by deleting the whole xref:definition.adoc#_partitions[simulation result partition].
+
+[#_figure1]
+.Deleting processed objects
+image::deleting-processed-objects.png[Deleting processed objects]
+
+#TODO - authorizations in code#
+
+#TODO - deleting the partitions#