hooks, representation formats

docs/concepts/clockwork/hooks.adoc

@@ -0,0 +1,17 @@
+= Clockwork Hooks
+:page-nav-title: Hooks
+:page-wiki-name: Hooks
+
+Hooks are midPoint mechanism for injecting a code to or intercepting of normal midPoint flows.
+E.g. a hook can be used to inset custom notification after an operation is complete, modify the operation request before it is even processed, redirect processing to a workflow or trouble-ticket system or do any other kind of advanced customization magic.
+Hooks are invoked by wiki:Clockwork+and+Projector[Clockwork] component at appropriate stages of midPoint computation.
+
+Hooks are typically pieces of Java code that has to be compiled and assembled into midPoint.
+However there is also a lightweight way to wiki:Scripting+Hooks[create a hook using a scripting code].
+
+
+== See Also
+
+* wiki:Scripting+Hooks[Scripting Hooks]
+
+* wiki:Clockwork+and+Projector[Clockwork and Projector]

docs/concepts/clockwork/index.adoc

@@ -0,0 +1,6 @@
+= Clockwork-Related Concepts
+:page-liquid:
+
+++++
+{% children %}
+++++

docs/concepts/clockwork/scripting-hooks.adoc

@@ -0,0 +1,110 @@
+= Scripting Hooks
+:page-wiki-name: Scripting Hooks
+
+== Introduction
+
+xref:hooks.adoc[Hooks] are midPoint mechanism for injecting a code to or intercepting of normal midPoint flows.
+E.g. a hook can be used to inset custom notification after an operation is complete, modify the operation request before it is even processed, redirect processing to a workflow or trouble-ticket system or do any other kind of advanced customization magic.
+
+Hooks are typically pieces of Java code that has to be compiled and assembled into midPoint.
+However there is also a lightweight way to create a hook using a scripting code.
+Each hook is simply a piece of wiki:Script+Expression[script code] that is executed at the specified moment in the midPoint recompute and execute flow (see wiki:Clockwork+and+Projector[Clockwork and Projector]). The code can influence the recomputation and execution by manipulating the wiki:Model+Context[model context], it can invoke wiki:Script+Expression+Functions[script expression functions], invoke external functions or do almost any other thing.
+
+
+== Scripting Hook Definition
+
+Scripting hooks can be defined in wiki:System+Configuration+Object[System Configuration Object]. The model`Hooks` element is used the define both heavyweight Java hooks and lightweight scripting hooks.
+The following code block provides an example of Groovy scripting hook.
+
+[source,html/xml]
+----
+<systemConfiguration oid="00000000-0000-0000-0000-000000000001">
+    <name>SystemConfiguration</name>
+    <modelHooks>
+        <change>
+            <hook>
+                <name>Example scripting hook</name>
+                <state>primary</state>
+                <focusType>c:UserType</focusType>
+                <script>
+                    <code>
+                        import com.evolveum.midpoint.prism.delta.*;
+                        import com.evolveum.midpoint.xml.ns._public.common.common_3.*;
+                        //import com.evolveum.midpoint.model.intest.util.StaticHookRecorder;
+
+                        log.debug('ORG HOOK WAS HERE')
+
+                        UserType user = (UserType)focus;
+                        organizations = user.getOrganization();
+                        for (orgName in organizations) {
+                            org = midpoint.searchObjectByName(OrgType.class, orgName)
+                            if (org == null) {
+                                // Org does not exist, lets create it
+                                org = midpoint.createEmptyObjectWithName(OrgType.class, orgName);
+                                topOrgRef = new ObjectReferenceType();
+                                topOrgRef.setOid('80808080-8888-6666-0000-100000000001');
+                                org.getParentOrgRef().add(topOrgRef);
+                                midpoint.addObject(org);
+                            }
+                            if (!midpoint.isDirectlyAssigned(org)) {
+                                // The org is not assigned. Let's assign it.
+                                // We need to construct a delta to do this
+                                assignment = new AssignmentType();
+                                orgTarget = new ObjectReferenceType();
+                                orgTarget.setOid(org.getOid());
+                                orgTarget.setType(OrgType.COMPLEX_TYPE);
+                                assignment.setTargetRef(orgTarget);
+                                assignmentDelta = ContainerDelta.createModificationAdd(UserType.F_ASSIGNMENT, UserType.class, prismContext, assignment);
+                                modelContext.getFocusContext().swallowToPrimaryDelta(assignmentDelta);
+                            }
+                        }
+                    </code>
+                </script>
+            </hook>
+        </change>
+    </modelHooks>
+    ....
+</systemConfiguration>
+
+
+----
+
+
+=== Example 1
+
+Following scripting hook removes all assignments from disabled users. Please note usage of modelContext.rot()
+
+[source,html/xml]
+----
+<hook>
+    <name>Remove assignments from disabled users</name>
+    <state>secondary</state>
+    <focusType>c:UserType</focusType>
+    <script>
+       <code>
+            import com.evolveum.midpoint.prism.delta.*;
+            import com.evolveum.midpoint.xml.ns._public.common.common_3.*;
+
+            UserType user = (UserType) focus;
+            ActivationStatusType administrativeStatus = user.getActivation().getAdministrativeStatus();
+            if (administrativeStatus == ActivationStatusType.DISABLED) {     
+				changed = false;
+                for (AssignmentType assign : user.getAssignment()) {
+                    assignmentDelta = ContainerDelta.createModificationDelete(UserType.F_ASSIGNMENT, UserType.class, prismContext, assign.clone());
+                    log.debug('Removing assignment ' + assignmentDelta + ' from disabled user ' + user.getName());
+                    modelContext.getFocusContext().swallowToSecondaryDelta(assignmentDelta);
+					changed = true;
+                }
+				if (changed) {
+                    modelContext.rot(); // this makes Projector to recompute the model context
+                }
+            }
+        </code>
+    </script>
+</hook>
+----
+
+
+
+
+

docs/concepts/data-representation-formats.adoc

@@ -0,0 +1,132 @@
+= Data Representation Formats
+:page-wiki-name: Objects, XML, JSON and others
+:page-description: Use of XML, JSON, YAML and other formats in midPoint.
+
+midPoint is using objects composed of properties and containers, known as wiki:Prism+Objects[Prism Objects].
+MidPoint wiki:Data+Model[Data Model] specifies how the objects look like, what properties and containers they have, what are the types and multiplicities of the data and so on.
+This is an abstract data structure that is used in all parts of midPoint.
+This internal data structure are converted to _data representation formats_ such as JSON, YAML or XML when needed.
+
+The same midPoint objects can be represented in all the supported formats:
+
+[source,xml]
+----
+<user>
+  <name>jdoe</name>
+  <givenName>John</givenName>
+  <familyName>Doe</familyName>
+  <fullName>John Doe</fullName>
+  <activation>
+    <administrativeStatus>enabled</administrativeStatus>
+  </activation>
+</user>
+----
+
+[source,json]
+----
+{
+  "name" : "jdoe",
+  "givenName" : "John",
+  "familyName" : "Doe",
+  "fullName" : "John Doe",
+  "activation" : {
+    "administrativeStatus" : "enabled"
+  }
+}
+----
+
+[source,yaml]
+----
+name: jdoe
+givenName: John
+familyName: Doe
+fullName: John Doe
+activation:
+  administrativeStatus: enabled
+----
+
+All the supported data representation formats are equivalent.
+MidPoint user may choose any of them.
+
+
+== Objects Are Not Documents
+
+Although the midPoint objects are represented in XML, JSON and YAML, they must not be considered XML or JSON _documents_.
+MidPoint objects are composed of (potentially multi-valued) properties and containers (wiki:Prism+Objects[Prism Objects]), not XML elements or JSON hashmaps.
+There are also some subtle differences.
+E.g. midPoint does not maintain ordering of values, while XML and JSON do maintain the ordering.
+Ordering of elements is significant in XML, but it is not significant for midPoint.
+
+XML, JSON and YAML are just ways to represent midPoint objects in a readable and "serializable" form.
+All midPoint objects can be represented as XML, JSON and YAML documents.
+However, it does not work the other way around.
+Not every XML or JSON document is a valid midPoint object.
+
+
+== Extensibility
+
+MidPoint data model is designed to be extensible.
+Practically all midPoint objects have an `extension` section that can hold any custom items.
+Therefore, it is easy to extend existing object types such as User or Role.
+The extensions are identified by namespace URI, therefore there is a natural namespace allocation and minimized chance of naming conflicts even if several extensions are used at the same time.
+
+Existing objects types are easy to extends, however new object types are quite difficult to create.
+Luckily, there is usually no need to.
+MidPoint provides very good set of basic data types (User, Role, Org, Service).
+These types can be further specified by using wiki:Archetypes[archetypes].
+
+
+== Multiple Formats
+
+Why do we need to support many data presentation formats?
+Why is JSON not enough?
+
+JSON has lots of problems. For example:
+
+* JSON has no namespace support. Without namespacing mechanism it is extremely difficult to have extensibility and interoperability at the same time.
+
+* JSON Schema has most of the problems and limitations of XML Schema Definitions (XSD).
+We had to extend XSD to make it useful for midPoint, we would probably need the same to do with JSON schema.
+In fact, the limitations of both XSD and JSON Schema lead us to xref:/midpoint/devel/axiom[Axiom].
+
+* JSON ecosystem is still not mature enough. It is actually re-inventing XML and repeating some of the most severe XML mistakes.
+
+* JSON positions itself as a "pure data" format.
+E.g. there is no support for comments in JSON, and there is no plan to support that.
+That makes sense for such a "pure" format.
+However, that makes JSON unsuitable for configurations samples and similar data that needs to be human-readable and understandable.
+
+Other representation formats have their own advantages and disadvantages.
+They are also influenced by current technology and fashion trends.
+XML was really popular in 2000s, JSON is the king now, and who knows what will come in next couple of years.
+Making midPoint data model independent of the data representation gives us ability to adapt when we need to.
+
+== Why Is There So Much XML?
+
+The short answer is that XML was mature and popular when midPoint project was started.
+Originally, MidPoint was built on top of XML ecosystem.
+Even though we knew that this was not ideal, it was still the best choice at the time.
+
+MidPoint gained independence from data representation formats since then.
+However, project culture is difficult to chance, as is the volume of configuration samples and documentation.
+Besides, XML is still a very good choice for midPoint configuration.
+Most notably:
+
+* Unlike JSON, XML can include comments in the files.
+This is very useful for development of midPoint configurations in the team.
+
+* XML has native support for namespaces.
+Which may still be quite useful in complex configuration scenarios.
+
+* Albeit we are slowly moving to xref:/midpoint/devel/axiom[Axiom], majority of midPoint schema is still maintained in XSD.
+Therefore common XML tooling can still be used when working with XML, e.g. XML code completion in the development environment (IDE).
+
+However, we are slowly going towards full independence of representation format.
+Not only in code, but also in documentation, samples and tests.
+There will certainly be more JSON and YAML in the documentation, samples and tests.
+
+== See Also
+
+* xref:/midpoint/devel/axiom[Axiom]
+* wiki:Data+Model[Data Model]
+* wiki:Archetypes[]