Kerberos Integration with Tomcat, arises errors in Opengrok Suggester

[Leo-Adlakha]

Describe the bug
I am not able to integrate Kerberos with Opengrok in Tomcat server, I get some error while integrating it.

If you provide the exact version of OpenGrok, JDK used, OS(and its version) used and Tomcat(or your webapp server) used it can help figuring out an environment issue.
Opengrok v1.7.33
Tomcat v10.0.22
JDK OpenJDK 11
OS RHEL7

To Reproduce
Added Kerberos Authentication to Tomcat, which shows an unexpected error, on the Opengrok Suggester side. It doesn't happen if I remove the Authentication filter from the web.xml of my webapp.

Expected behavior
Working state of Kerberos with Opengrok Instance

Screenshots
This is the error in tomcat logs.

Exception in thread "Thread-3" java.util.ConcurrentModificationException
        at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
        at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
        at org.apache.catalina.webresources.StandardRoot.getResourceInternal(StandardRoot.java:278)
        at org.apache.catalina.webresources.Cache.getResource(Cache.java:64)
        at org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:216)
        at org.apache.catalina.webresources.StandardRoot.getClassLoaderResource(StandardRoot.java:225)
        at org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2426)
        at org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:877)
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1413)
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1257)
        at org.opengrok.web.api.v1.suggester.provider.service.impl.SuggesterServiceImpl.getTimeToNextRebuild(SuggesterServiceImpl.java:394)
        at org.opengrok.web.api.v1.suggester.provider.service.impl.SuggesterServiceImpl.scheduleRebuild(SuggesterServiceImpl.java:364)
        at org.opengrok.web.api.v1.suggester.provider.service.impl.SuggesterServiceImpl.lambda$initSuggester$1(SuggesterServiceImpl.java:325)
        at java.base/java.lang.Thread.run(Thread.java:834

Additional context
If I remove the Authentication Filter, everything works fine with Simple HTTP Auth. For adding Auth, I have added the dependencies and added the Filter in web.xml and else everything on Kerberos side is taken care of by the system. Thats how its done for other applications. If this is due to the suggester, is there a way to turn off suggester in Opengrok ?

[Leo-Adlakha]

I have ~1100 repositories currently, and planning to add many more.

[vladak]

Suggester can be disabled via configuration - see https://github.com/oracle/opengrok/wiki/Suggester#enabled

[vladak]

Could you describe in detail the changes done to the OpenGrok instance ?

[vladak]

Also, when does the above exception happen ? During webapp deployement/startup or some other time ?

[vladak]

As for the actual exception, the call in

opengrok/opengrok-web/src/main/java/org/opengrok/web/api/v1/suggester/provider/service/impl/SuggesterServiceImpl.java

Line 396 in bf515fc

ExecutionTime executionTime = ExecutionTime.forCron(parser.parse(cronDefinition));

triggers Tomcat class loader, most likely to load the ExecutionTime.class from cron-utils-9.1.6.jar.

Inside the class loader, there is a list of web resources lists that is traversed: https://github.com/apache/tomcat/blob/ced2fe39f36d5da4cc13f2ff539a9abeacd503b9/java/org/apache/catalina/webresources/StandardRoot.java#L272-L281

It is for one of the sub-lists the problem happens: https://github.com/apache/tomcat/blob/ced2fe39f36d5da4cc13f2ff539a9abeacd503b9/java/org/apache/catalina/webresources/StandardRoot.java#L72-L76 and https://github.com/apache/tomcat/blob/ced2fe39f36d5da4cc13f2ff539a9abeacd503b9/java/org/apache/catalina/webresources/StandardRoot.java#L87

These lists are modified during startup via createWebResourceSet() method in https://github.com/apache/tomcat/blob/ced2fe39f36d5da4cc13f2ff539a9abeacd503b9/java/org/apache/catalina/webresources/StandardRoot.java#L376

Assuming this is not the trivial case of concurrent modification of the ArrayList, there must be some other thread working with one of these lists, hence my question above webapp deploy/startup. Given this is first such report, there must be something about the Kerberos modifications that lead to this problem, hence my question above about the exact changes done to the webapp config.

[Leo-Adlakha]

As for the changes that I do which leads to this are

• I add a authentication filter and map it to the endpoint /* for the webapp
• Second is I add the relevant dependencies of the Authentication Filter in the classpath variable in {CATALINA_HOME}/bin/setenv.sh

This doesn't allow the webapp to even start, and I get a 404. I don't get why the Auth Filter is responsible for the Exception.

[Leo-Adlakha]

    <filter>
        <filter-name>Authentication Filter</filter-name>
        <filter-class>............KerberosAuthenticationFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>Authentication Filter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

is the XML I add to $CATALINA_HOME/webapps/source/WEB_INF/web.xml

Dependencies I add are javax, slf4j and my authentication filter jar