docs: add missing page and misc enhancements #1924 (#1942)

docs/admin/developer/agama/faq.md

@@ -2,7 +2,6 @@
 
 ## Common errors
 
-
 ### Source code of flow ... has not been parsed yet or has errors
 
 TODO
@@ -15,20 +14,24 @@ A Java invocation of the form `Call package.className#new ...` is passing a numb
 
 A java invocation is attempting to call a method that is not part of the given class or the number of parameters passed for the method is not correct.
 
+### No Finish instruction was reached
+
+This occurs when no `Finish` statement has been found in the execution of a flow and there are no remaining instructions.
+
 ## Classes added on the fly
 
 ### A class does not "see" other classes in its own package
 
 This is a limitation of the scripting engine. Here, classes have to be imported even if they belong to the same package, or the fully qualified name used.
 
-### What Groovy and Java versions are supported?
-
-Groovy 4.0 and Java 11. The runtime is Amazon Corretto 11.
-
 ### How to append data to a flow's log directly?
 
 This can be done by calling method `log` of class `io.jans.agama.engine.script.LogUtils`. This method receives a variable number of arguments as DSL's `Log` does. Thus you can do `LogUtils.log("@w Today is Friday %th", 13)`, as in the logging [examples](./dsl-full.md#logging).
 
+### What Groovy and Java versions are supported?
+
+Groovy 4.0 and Java 11. The runtime is Amazon Corretto 11.
+
 ## Templates
 
 ### How to implement localization?
@@ -63,21 +66,37 @@ Not yet unfortunately. We plan to offer tools in the future to ease the developm
 
 We plan to offer a debugger in the future. In the meantime, you can do `printf`-like debugging using the `Log` instruction. See [Agama logging](./logging.md).
 
+## About Agama engine
+
+### Does it support AJAX?
+
+If you require a flow with no page refreshes, it could be implemented using AJAX calls as long as they align to the POST-REDIRECT-GET pattern, where a form is submitted, and as response a 302/303 HTTP redirection is obtained. Your Javascript code must also render UI elements in accordance with the data obtained by following the redirect (GET). Also, care must be taken in order to process server errors, timeouts, etc. In general, this requires a considerable amount of effort.
+
+If you require AJAX to consume a resource (service) residing in the same domain of your server, there is no restriction - the engine is not involved. Interaction with external domains may require to setup CORS configuration appropriately in the authentication server.  
+
+### I want/need to understand the internals, where to start?
+
+The quick start guide is a must, followed by [Hello world flow for project maintainers](./hello-world-maintainer.md). In the end a complete sweep over all the docs is needed.
+
 ## Miscellaneous
 
 ### Does flow execution times out?
 
-Yes. The maximum amount of time an end-user can take to fully complete a flow is driven by the configuration of the authentication server and can be constrained even more in the flow itself. Read about timeouts [here](./flows-lifecycle.md#timeouts). 
+Yes. The maximum amount of time an end-user can take to fully complete a flow is driven by the configuration of the authentication server and can be constrained even more in the flow itself. Read about timeouts [here](./flows-lifecycle.md#timeouts).
+
+### How to prevent launching a flow directly from the browser?
+
+Disable the flow. It will still be callable from other flows. 
 
 ### Why are the contents of a list or map logged partially?
 
 This is to avoid traversing big structures fully. You can increase the value of `maxItemsLoggedInCollections` in the [engine configuration](./engine-config.md).
 
-## How to add two numbers or compare numeric values in Agama?
+### How to add two numbers or compare numeric values in Agama?
 
 Agama only provides operators for boolean comparison in conditional statements. The structure of an authentication flow will rarely have to deal with computations/comparisons of numbers, strings, etc. In case this is needed, developers have to resort to Java.
 
-## How to concatenate strings in Agama?
+### How to concatenate strings in Agama?
 
 See the previous answer. A two-lines solution could be:
 
@@ -86,10 +105,10 @@ strings = [ s1, s2, ... ]
 Call java.lang.String#join "" strings
 ```
 
-## How to know the index of a given loop iteration?
+### How to know the index of a given loop iteration?
 
 See the examples in the Looping section of the DSL [full reference](./dsl-full.md#looping).
 
-## Can Agama code be called from Java?
+### Can Agama code be called from Java?
 
 No. These two languages are supposed to play roles that should not be mixed, check [here](./dsl.md#introduction) and [here](./lifecycle.md#design-and-code).

docs/admin/developer/agama/hello-world-closer.md

@@ -0,0 +1,65 @@
+# Hello world flow: a closer look
+
+This document revisits the hello world flow of the [quick start guide](./quick-start.md) using an approach that serves as an introduction to the Agama engine internals. The course of this flow, and generally of any other, can be studied in three stages:
+
+1. From authentication request to bridge script
+1. Within Agama engine
+1. From bridge script to authentication server
+
+## Stage 1
+
+In normal circumstances the flow is launched through a web browser via an [OIDC authentication request](./quick-start.md#craft-an-authentication-request) which in turn activates the Agama [bridge](./index.md#agama-engine) script. Its associated (facelet) page is `agama.xhtml` that invokes the `prepareForStep` method of that script. This method performs some initialization and makes a redirection to the URL `/jans-auth/fl/agama.fls` to hand the control to the Agama engine.
+
+## Stage 2
+
+### First request cycle
+
+The GET request for `agama.fls` is processed by [this servlet](https://github.com/JanssenProject/jans/blob/main/agama/engine/src/main/java/io/jans/agama/engine/servlet/ExecutionServlet.java). Most of logic takes place at method `startFlow` of [FlowService](
+https://github.com/JanssenProject/jans/blob/main/agama/engine/src/main/java/io/jans/agama/engine/service/FlowService.java) bean. Here, the associated data of the flow in question is retrieved from the database. Its transpiled code in addition to utility code (`util.js`) is loaded and the main entry point called - see invocation of `callFunctionWithContinuations`. This will throw a `ContinuationPending` exception once the equivalent instruction to `RRF` (or `RFAC`) is found in the transpiled code.
+
+The above means that at line 6 of [Hello world](./test), the DSL code execution is interrupted: method `processPause` collects the required data for template rendering, like template location and its associated data model. This is saved to permanent storage in addition to the "state of the program", that is the values of all variables defined so far in the flow - this is known as the "continuation".
+
+Finally the `ExecutionServlet` sends a response consisting of a redirect. This completes the first request/response of this flow.
+
+### Second request cycle
+
+As a consequence of the HTTP redirect received, the browser will request the server a URL like `/jans-auth/fl/hello/index.fls`. Note how this is correlated to the template path defined in the flow. Here the GET handler of `ExecutionServlet` takes control again, this time calling the `sendPageContents` method defined in the parent class [`BaseServlet`](https://github.com/JanssenProject/jans/blob/main/agama/engine/src/main/java/io/jans/agama/engine/servlet/BaseServlet.java). `sendPageContents` performs the actual page rendering - specifically for `hello/index.ftlh` in this case - and the produced markup is sent to browser. So far we have "covered" the first two letters of the RRF instruction.
+
+### Third cycle
+
+Once the browser displays the page contents and the user hits the submit button, a new request cycle begins, this time POSTing to `/jans-auth/fl/hello/index.fls`. The method `continueFlow` of `ExecutionServlet` is called by the POST handler. Here the parameters of the payload are parsed and the `continueFlow` method of `FlowService` called in turn.
+
+At `FlowService` the continuation is restored and the transpiled flow code resumed (see `resumeContinuation` usage). This means that statements after the `RRF` instruction are all executed until another `RRF` or `RFAC` call is reached. However in this case, there are only a couple of statements left: `Log` and `Finish`. The value supplied for the latter, namely `"john_doe"` - a shortcut for `{ success: true, data: { userId: "john_doe" } }` - is wrapped in the `NativeObject` instance resulting from the `resumeContinuation` call.
+
+!!! Note
+    If we had another `RRF` instruction in this flow, a `ContinuationPending` exception would have been thrown, as in the [first request](#first-request-cycle), and a redirection sent to browser.
+
+Finally, this result is persisted to database, and a finalization page sent to the browser, see `sendFinishPage` in `ExecutionServlet`. The page consists of an auto-submitting form that makes a POST to `/postlogin.htm`, a server URL employed to resume Jython-based authentication flows.
+
+This is how stage 2 ends. The engine now hands control back to the bridge.
+
+## Stage 3
+
+The POST to `/postlogin.htm` provokes a call to the `authenticate` method of the bridge. Here, if the process finished successfully, the user gets authenticated in the server, and the browser is taken to the `redirect_uri` supplied in the original authentication request. There, the requesting party (RP) can make further processing for the user to get access according to the OpenId Connect protocol.
+
+## Transpiled code
+
+The engine has some timers running in the background. One of them [transpiles code](https://github.com/JanssenProject/jans/blob/main/agama/engine/src/main/java/io/jans/agama/timer/Transpilation.java) when a change is detected in a given flow's source (written in Agama DSL). The transpilation process generates vanilla Javascript code runnable through [Mozilla Rhino](https://github.com/mozilla/rhino) by using a transformation chain like  (DSL) flow code -> (ANTLR4) parse tree -> (XML) abstract syntax tree -> JS. 
+
+The transformation chain guarantees that a flow written in Agama DSL cannot:
+
+- Access Java classes/instances not specified in the original flow code (i.e. the only bridge to Java world is via `Call`s)
+- Access/modify the standard javascript built-in objects directly
+- Conflict with javascript keywords
+
+**Notes**
+
+- You can find the (ANTLR4) DSL grammar [here](https://github.com/JanssenProject/jans/blob/main/agama/transpiler/src/main/antlr4/io/jans/agama/antlr/AuthnFlow.g4).
+- The last step of the transformation chain is carried out by means of [this](https://github.com/JanssenProject/jans/blob/main/agama/transpiler/src/main/resources/JSGenerator.ftl) Freemarker transformer
+
+## Additional notes
+
+- The engine does not use asynchronous paradigms: no events, callbacks, extra threads, etc. All computations remain in the classic request/response servlet lifecycle familiar to most Java developers
+- _Continuations_ allow to express a flow as if it were a straight sequence of commands despite there are actual pauses in the middle: the gaps between an HTTP response and the next request
+- Currently Mozilla Rhino seems to be the only mechanism that brings continuations into the Java language
+- In order to preserve server statelessness, continuations are persisted to storage at every flow pause. This way the proper state can be restored when the continuation is resumed in the upcoming HTTP request

docs/admin/developer/agama/registration/io.jans.flow.sample.registration

@@ -13,6 +13,6 @@ Repeat 3 times max
         RRF "confirmation.ftlh" o
         Finish true
 
-    Log "Error creating user %" o.error
+    Log "Error creating user %: %" o.name o.error
 
 Finish false

docs/admin/developer/agama/samples.md

@@ -130,7 +130,7 @@ Source code [here](./otp-email-registration/io.jans.flow.sample.otp.emailWithReg
 
 This flow is short and powerful. To start it launches the [email OTP authentication](#email-otp-authentication) flow (line 4). However, instead of the `login.ftlh` template used internally by the basic flow, a different version is used (line 5). The new template is located in this flow's basepath, i.e. `samples/otp-email-registration`, and the file name does not change.
 
-This new page simply adds a small link that reads "Don't have an account?" in a new HTML form. When clicked it will provoke the email-OTP flow to be aborted early and the condition at line 8 will be truthy. If that's not the case, the email-OTP flow will proceed as usual and the current flow will finish in the same way as email-OTP (line 16).
+This new page simply adds a small link that reads "Don't have an account?" in a new HTML form. When clicked it will provoke the email-OTP flow to be aborted early and the condition at line 7 will be truthy. If that's not the case, the email-OTP flow will proceed as usual and the current flow will finish in the same way as email-OTP (line 16).
 
 If email-OTP is aborted, the [registration flow](#registration) that we just saw is launched (line 9). Here the template `confirmation.ftlh` is overriden too (line 10). This new page changes the label of the button that is shown after an account has been created: originally the text is "Continue" but for the current flow, "Proceed to login" is a better fit.