WebServerLog/ELFFields not working

[pwninja]

I am trying to configure ELFFields in Weblogic domain. using createDomain.sh, but it is not getting reflected.
snippet of topology configuration:

            WebServer:
                WebServerLog:
                    FileCount: 25
                    RotateLogOnStartup: true
                    LogFileFormat: extended
                    LogFileRotationDir: '{{ logs_directory }}/AdminServer/backup'
                    ELFFields: "date time cs-method sc-status"
                    DateFormatPattern: 'MMM d, yyyy h:mm:ss,SSS a z'
                    FileName: '{{ logs_directory }}/AdminServer/access_extended.log'
                    LogTimeInGMT: true

This is just an example. Actually I need to for the custom fields, but the above one still doesn't work.
Resulting config.xml section is:

    <web-server>
      <name>AdminServer</name>
      <web-server-log>
        <name>AdminServer</name>
        <date-format-pattern>MMM d, yyyy h:mm:ss,SSS a z</date-format-pattern>
        <file-name>(...)/AdminServer/access_extended.log</file-name>
        <file-count>25</file-count>
        <log-file-rotation-dir>(...)/AdminServer/backup</log-file-rotation-dir>
        <elf-fields>date time cs-method ctx-ecid ctx-rid cs-uri sc-status bytes</elf-fields>
        <log-file-format>extended</log-file-format>
        <log-time-in-gmt>true</log-time-in-gmt>
      </web-server-log>

It always defaults to date time cs-method ctx-ecid ctx-rid cs-uri sc-status bytes.
I didn't find any related information in the WDT log file.
Oracle Weblogic version is 12.2.1.4
WDT version is 1.9.17. I also tried 1.9.8 as it was mentioed to be working fine in another ticket (#817), but maybe it works only in the updateDomain.sh scenario.

[CarolynRountree]

I ran createDomain with two different values in th elffields and both were as expected in the config file. Do you have two different admin servers in the config.xml ? Your model snippet doesn't show the name of the server you are changing

[pwninja]

There is one AdminServer only and two OsbServers. ELFFields setting doesn't work for any of them.
I am 100% sure about the section I am changing as it properly reflects other fields - I can switch for example LogTimeInGMT and I can see the new value after domain creation.

[CarolynRountree]

What domain_type are you using?

[pwninja]

The issue is both with domain_type OSB & BPM

[CarolynRountree]

We don't support OSB and BPM. Could you provide either one of your domain typedef files? OSB or BPM. I would appreciate it.

[pwninja]

Is it anywhere documented that WDT behavior is such dependent on the domain_type?
Our typedef file for OSB is as below:

{
    "name": "OSB",
    "description": "OSB type domain definitions",
    "versions": {
        "12.2.1.4": "OSB_12214"
    },
    "definitions": {
        "OSB_12214": {
            "baseTemplate": "Basic WebLogic Server Domain",
            "extensionTemplates": ["Oracle Service Bus"],
            "customExtensionTemplates": [],           
            "serverGroupsToTarget": [ "JRF-MAN-SVR", "WSMPM-MAN-SVR", "OSB-MGD-SVRS-COMBINED", "OSB-MGD-SVRS-ONLY" ],
            "rcuSchemas": [ "STB", "WLS", "MDS", "IAU", "IAU_VIEWER", "IAU_APPEND", "OPSS", "UCSUMS", "SOAINFRA" ]
        }    
    }
}

[CarolynRountree]

No the comments just mean that I didn't have a typedef file on hand. I suspect since i cannot reproduce this with domain type WLS, that the template is changing the field after being set.

[robertpatrick]

@pwninja WDT is a wrapper around WLST so by definition, the behavior of createDomain.sh is heavily influenced by the domain_type because it defines what domain templates are applied when creating the domain. For example, the JRF typedef file has sections that look like this:

        "JRF_12CR2": {
            "baseTemplate": "Basic WebLogic Server Domain",
            "extensionTemplates": [
                "Oracle JRF WebServices Asynchronous services",
                "Oracle WSM Policy Manager",
                "Oracle Enterprise Manager"
            ],
            "customExtensionTemplates": [ ],
            "serverGroupsToTarget": [ "JRF-MAN-SVR", "WSMPM-MAN-SVR" ],
            "dynamicClusterServerGroupsToTarget" : [ "WSMPM-DYN-CLUSTER" ],
            "rcuSchemas": [ "MDS", "IAU", "IAU_VIEWER", "IAU_APPEND", "OPSS", "STB" ]
        }

Some of the domain templates have final.py file that get run while writing the domain. These final.py files can do almost anything, including overwriting values that were previously set prior to the initial domain save to disk.

Your issue is on my list to look at. However, as you might be aware, we do not officially support any upper stack FMW products. It is possible to create domains some upper stack products by creating your own typedef file but this is where the support gets blurry. While we want to make sure the core WDT product does not contain bugs that prevent users from making progress with custom typedefs, we cannot support these upper stack products without help from the product development teams that own these products.

[robertpatrick]

@pwninja The culprit for the log field being overwritten is the script/final.py file in the Oracle JRF Base extension template at $ORACLE_HOME/oracle_common/common/templates/wls/oracle.jrf_base_template.jar.

#!/usr/bin/python

# turn on diagnostic context enabled flag to admin server and all managed servers
# enable extended log format (ELF) on the adminserver and all managed servers
try:
  cd('/')
  servers = cmo.getServers()
  for server in servers:
    # turn on diagnostic context enabled flag to admin server and all managed servers
    server_name = server.getName()
    cd('/Server/' + server_name)
    create(server_name,"ServerDiagnosticConfig")
    cd('ServerDiagnosticConfig/' + server_name)
    set('DiagnosticContextEnabled','true')
    # enable extended log format (ELF) on the adminserver and all managed servers
    cd('/Server/' + server_name)
    create(server_name, 'WebServer')
    cd('WebServer/' + server_name)
    create(server_name, 'WebServerLog')
    cd('WebServerLog/' + server_name)
    set('LogFileFormat','extended')
    set('ELFFields', 'date time cs-method ctx-ecid ctx-rid cs-uri sc-status bytes')
except:
  print "Warning: failed to enable the diagnostic context enabled flag."
  print "Warning: failed to enable extended access log format."

In order for WDT to be able to set this field properly during createDomain, we would need to revise how the Server/ServerTemplate fields are being written so that they are written in "Phase 2" (after the domain is written to disk the first time). We will discuss this internally to determine the feasibility of making this type of change.

[textanalyticsman]

Hello,

Why is this closed? I tried WDT 2.1.1 and I still see the issue.

TAM

[robertpatrick]

@textanalyticsman we restored what we believed to be the previous behavior (because we believed you that this worked previously). With the current code base, the Phase 2 application of the model is still happening before the writeDomain() call (where the final.py code is invoked). The only fix is to add a Phase 3 that applies these changes again after writeDomain(), which will make Create Domain even slower when applying extension templates.

The only other alternative would be for you to run Update Domain with a sparse model to apply your changes again after Create Domain completes.

[robertpatrick]

Closing with PR #1115. This fix is available in main and will be distributed with the next version of WDT.