initial-logging.adoc: complete update, required many factual changes

docs/diag/initial-logging.adoc

@@ -16,42 +16,123 @@ This initial logging setup may be insufficient for troubleshooting purposes,
 but there are ways how to adjust it when the need arises.
 This way it's possible to diagnose issues with low-level system initialization, database initialization, etc.
 
-The trick is to put `logback.xml` file in wiki:MidPoint+Home+Directory[MidPoint Home Directory]. This file will be applied during very early phases of system initialization.
+You can use one of these two options to adjust the initial logging:
+
+* Adding `logback.xml` file in xref:/midpoint/reference/deployment/midpoint-home-directory/[MidPoint Home Directory].
+This file will be applied during very early phases of system initialization.
 The default initial logging configuration will be *replaced* with the content of this file.
+* Or adding `logback-extra` inside midPoint home which has the same format, but adds to the default
+configuration instead of replacing it.
+This file is only considered when no `logback.xml` is found.
+
+Both files follow the link:http://logback.qos.ch/manual/configuration.html[Logback configuration] format.
+When `logback.xml` is used, don't forget to set appenders otherwise logs go nowhere.
+
+== Overriding initial logging
+
+Example of minimalistic `logback.xml` configuration writing logs to the standard output:
 
 .logback.xml
 [source,xml]
 ----
 <?xml version="1.0" encoding="UTF-8"?>
 <configuration>
-    <logger name="PROFILING" level="TRACE"/>
-    <logger name="com.evolveum.midpoint" level="TRACE" />
+    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
+        <encoder>
+            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
+        </encoder>
+    </appender>
+
+    <root level="DEBUG">
+        <appender-ref ref="STDOUT"/>
+    </root>
+
+    <logger name="com.evolveum.midpoint" level="DEBUG"/>
+    <logger name="com.evolveum.midpoint.init.StartupConfiguration" level="TRACE"/>
+</configuration>
+----
+
+Note that both `<appender>` and `<root>` sections are important to get logs visible.
+
+Instead of writing the whole `logback.xml` from scratch you can also start with the
+link:https://github.com/Evolveum/midpoint/blob/master/gui/admin-gui/src/main/resources/logback.xml[default configuration]
+as a reference (although some `<if>` sections can likely be omitted).
+
+== Amending initial logging
+
+Alternatively a `logback-extra.xml` file can be used to only change what is necessary without
+affecting existing appender setup.
+This is much better for minor changes like setting more detailed logs for startup classes.
+
+Useful `logback-extra.xml` can look simply like this:
+
+.logback-extra.xml
+[source,xml]
+----
+<?xml version="1.0" encoding="UTF-8"?>
+<configuration>
+    <logger name="com.evolveum.midpoint" level="DEBUG"/>
+    <logger name="com.evolveum.midpoint.init.StartupConfiguration" level="TRACE"/>
 </configuration>
 ----
 
-Alternatively a `logback-extra.xml` file can be used.
-This file has the same format, but instead of replacing the default configuration it will be appended to it.
-Any statements present in this file will be applied after the default (compiled-in) logging configuration is applied.
+This should help to diagnose most of the problems (`DEBUG`) with the focus to the startup
+configuration part (`TRACE`).
+All the logging goes to the files, or other outputs, where it would go before.
+
+== Ignoring system configuration logging setup
+
+The files above are useful but may be insufficient on their own if the problem happens after
+the normal logging is initialized.
+This logging is configured in the xref:/midpoint/reference/concepts/system-configuration-object/[system
+configuration object].
+If you can't log in or the GUI is otherwise unavailable, the problem may be reported on the level
+that does not make it to the log - but would with initial logging overridden as shown above!
+Technically it's still possible to overwrite the system configuration object in the repository.
+But this requires access to the database and also relies on readable serialized representation of
+the object - which is not necessarily true as it's an implementation detail.
+What you need then is a mechanism that ignores the logging configuration from the system configuration object.
 
-This should help to diagnose most of the problems.
-But there are still some tricky problems that need one more trick.
-The problem is that once the system goes to the point when the repository is initialized it will read the sytem configuration from the repository and apply it.
-Logging configuration is part of the system config.
-Therefore this logging configuration replaces the initial logging configuration from `logback.xml`. This may be a problem if you do not have access to the GUI and cannot change the logging configuration in the repository.
-Therefore there is a config.xml setting that forces midPoint to skip the application of logging configuration from the repository:
+This mechanism is available via `config.xml` setting:
 
 .config.xml
 [source,xml]
 ----
 <configuration>
-    ...
-    <system>
-        <skipRepositoryLoggingSettings>true</skipRepositoryLoggingSettings>
-    </system>
+    <midpoint>
+        ...
+
+        <internals>
+            <avoidLoggingChange>true</avoidLoggingChange>
+        </internals>
+    </midpoint>
 </configuration>
 ----
 
-// TODO -DmidpointPrintSensitiveValues
+Alternatively, JVM argument `-Dmidpoint.internals.avoidLoggingChange=true` can be used.
+This uses available mechanism to set/override the `config.xml` values without changing the file.
+
+[WARNING]
+Needless to say, this is not the production setup and makes logging setup via GUI impossible.
+This is intended only for troubleshooting, testing or development needs.
+The configuration entry (and related JVM argument) can change any time in the future.
+Hopefully this will be reflected in this place.
+
+== Sensitive values (passwords)
+
+++++
+{% include since.html since="4.4" %}
+++++
+
+Initial setup can log values from `config.xml` file if `com.evolveum.midpoint.init.StartupConfiguration` logger is set to level `DEBUG` or `TRACE` - which can be a security issue if it's a value of some password.
+To avoid this, values are replaced by a placeholder string.
+In rare troubleshooting situations this may be a problem and actual values must be investigated.
+
+To allow this JVM argument `-DmidpointPrintSensitiveValues` must be added to the command line,
+value is not required and is ignored.
+This may require modifying `JAVA_OPTS` environment variable or otherwise changing `midpoint.sh` if
+stating scripts are used.
+Be sure to revert this change after the troubleshooting is finished.
 
 == See Also