Skip to content

Conversation

@mc1arke
Copy link
Member

@mc1arke mc1arke commented Jul 18, 2024

The tomcat-jdbc connection pool creates reflective proxies around Connections, Statements, and ResultSets to allow interception of 'close' calls, and consistency of traversing the link back from a child element to its parent - e.g. Statement to Connection. However, the link from ResultSet to Statement does not intercept the call so a non-proxied Statement is returned, as well as the equals method not handling the comparison against a non-proxied class correctly. To overcome this, an invocation handler is being created for wrapping a ResultSet returned from any of the relevant Statement methods, with that invocation handler intercepting the getStatement call and returning the proxied creating statement. The equals checks have also been altered to handle non-proxied instances being passed in to them.

https://bz.apache.org/bugzilla/show_bug.cgi?id=69206

The tomcat-jdbc connection pool creates reflective proxies around
Connections, Statements, and ResultSets to allow interception of 'close'
calls, and consistency of traversing the link back from a child element
to its parent - e.g. Statement to Connection. However, the link from
ResultSet to Statement does not intercept the call so a non-proxied
Statement is returned, as well as the `equals` method not handling the
comparison against a non-proxied class correctly. To overcome this, an
invocation handler is being created for wrapping a ResultSet returned
from any of the relevant Statement methods, with that invocation handler
intercepting the `getStatement` call and returning the proxied creating
statement. The `equals` checks have also been altered to handle
non-proxied instances being passed in to them.
@michael-o michael-o requested a review from markt-asf July 18, 2024 19:41
@markt-asf
Copy link
Contributor

Thanks for the PR. All looks reasonable to me. There are a few bits of potential clean-up I noticed both in the patch and the surrounding code. I'll do that after I've merged this PR.

@markt-asf markt-asf merged commit fad2a3f into apache:main Jul 23, 2024
@chrisw-s
Copy link

chrisw-s commented Aug 8, 2024

Unfortunately this caused some issues. Where we previously relied on catching SQLException a java.lang.reflect.UndeclaredThrowableException is now being thrown.

@mc1arke
Copy link
Member Author

mc1arke commented Aug 8, 2024

@chrisw-s Do you have a stacktrace that shows the exception you're getting or steps to reproduce the issue?

@chrisw-s
Copy link

chrisw-s commented Aug 8, 2024

Just this:

try (Statement statement = connection.createStatement();
     ResultSet resultSet = statement.executeQuery("sp_help_fulltext_catalogs")) {
      fullTextSearchable = resultSet.next();
    }
-->   catch (SQLException ex) {

This is with JTDS. It works fine in 10.1.26

@VRBogdanov
Copy link

Hello,

Same is observed on my side with the following strack trace below. With latest tomcat 9.091 is ok while after updating to tomcat 9.092/93 it fails. The problem is that DB initiation is delegated to liquibase and after that fails application initialisation.


java.lang.reflect.UndeclaredThrowableException
                                       liquibase.exception.LockException: java.lang.reflect.UndeclaredThrowableException
                                       	at liquibase.lockservice.StandardLockService.listLocks(StandardLockService.java:280)
                                       	at *********
                                       	at liquibase.Liquibase.update(Liquibase.java:194)
                                       	at liquibase.Liquibase.update(Liquibase.java:190)
                                       	at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:434)
                                       	at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:391)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:336)
                                       	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:334)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:209)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:209)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:209)
                                       	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
                                       	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391)
                                       	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
                                       	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:904)
                                       	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:781)
                                       	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:220)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:336)
                                       	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:334)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:209)
                                       	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
                                       	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391)
                                       	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
                                       	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:710)
                                       	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:693)
                                       	at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
                                       	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:408)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
                                       	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:336)
                                       	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:334)
                                       	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:209)
                                       	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
                                       	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
                                       	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:591)
                                       	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:399)
                                       	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:278)
                                       	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
                                       	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4007)
                                       	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4448)
                                       	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
                                       	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
                                       	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
                                       	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                                       	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
                                       	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
                                       	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
                                       	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:721)
                                       	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
                                       	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
                                       	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
                                       	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                                       	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
                                       	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
                                       	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
                                       	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:211)
                                       	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
                                       	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:415)
                                       	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
                                       	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:878)
                                       	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
                                       	at org.apache.catalina.startup.Catalina.start(Catalina.java:735)
                                       	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                       	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                                       	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                                       	at java.lang.reflect.Method.invoke(Method.java:498)
                                       	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345)
                                       	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:473)
                                       Caused by: java.lang.reflect.UndeclaredThrowableException: null
                                       	at com.sun.proxy.$Proxy60.executeQuery(Unknown Source)
                                       	at liquibase.executor.jvm.JdbcExecutor$QueryStatementCallback.doInStatement(JdbcExecutor.java:352)
                                       	at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:55)
                                       	at liquibase.executor.jvm.JdbcExecutor.query(JdbcExecutor.java:135)
                                       	at liquibase.executor.jvm.JdbcExecutor.query(JdbcExecutor.java:143)
                                       	at liquibase.executor.jvm.JdbcExecutor.queryForObject(JdbcExecutor.java:151)
                                       	at liquibase.executor.jvm.JdbcExecutor.queryForObject(JdbcExecutor.java:166)
                                       	at liquibase.executor.jvm.JdbcExecutor.queryForInt(JdbcExecutor.java:187)
                                       	at liquibase.executor.jvm.JdbcExecutor.queryForInt(JdbcExecutor.java:182)
                                       	at liquibase.snapshot.SnapshotGeneratorFactory.has(SnapshotGeneratorFactory.java:98)
                                       	at liquibase.snapshot.SnapshotGeneratorFactory.hasDatabaseChangeLogLockTable(SnapshotGeneratorFactory.java:190)
                                       	at liquibase.lockservice.StandardLockService.hasDatabaseChangeLogLockTable(StandardLockService.java:140)
                                       	at liquibase.lockservice.StandardLockService.listLocks(StandardLockService.java:259)
                                       	... 84 common frames omitted
                                       Caused by: java.lang.reflect.InvocationTargetException: null
                                       	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                       	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                                       	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                                       	at java.lang.reflect.Method.invoke(Method.java:498)
                                       	at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:130)
                                       	... 97 common frames omitted
                                       **Caused by: com.sap.db.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: [259] (at 53): invalid table name:  Could not find table/view DATABASECHANGELOGLOCK in schema USR_5Z7R7FZF9MR389P39W32COTNX: line 1 col 54 (at pos 53)**
                                       	at com.sap.db.jdbc.exceptions.SQLExceptionSapDB._newInstance(SQLExceptionSapDB.java:209)
                                       	at com.sap.db.jdbc.exceptions.SQLExceptionSapDB.newInstance(SQLExceptionSapDB.java:42)
                                       	at com.sap.db.jdbc.packet.HReplyPacket._buildExceptionChain(HReplyPacket.java:854)
                                       	at com.sap.db.jdbc.packet.HReplyPacket.getSQLExceptionChain(HReplyPacket.java:196)
                                       	at com.sap.db.jdbc.packet.HPartInfo.getSQLExceptionChain(HPartInfo.java:39)
                                       	at com.sap.db.jdbc.ConnectionSapDB._receive(ConnectionSapDB.java:5811)
                                       	at com.sap.db.jdbc.ConnectionSapDB.exchange(ConnectionSapDB.java:2701)
                                       	at com.sap.db.jdbc.StatementSapDB._executeDirect(StatementSapDB.java:2258)
                                       	at com.sap.db.jdbc.StatementSapDB._execute(StatementSapDB.java:2232)
                                       	at com.sap.db.jdbc.StatementSapDB._execute(StatementSapDB.java:2193)
                                       	at com.sap.db.jdbc.StatementSapDB._executeQuery(StatementSapDB.java:2147)
                                       	at com.sap.db.jdbc.StatementSapDB.executeQuery(StatementSapDB.java:217)
                                       	... 102 common frames omitted

@mc1arke
Copy link
Member Author

mc1arke commented Aug 8, 2024

Ah, ok. It's an exception being thrown whilst executing the underlying statement, which is happening outside the existing try/catch block that unwraps the exception. I'll raise a Bugzilla record and propose a PR to fix it.

@mc1arke
Copy link
Member Author

mc1arke commented Aug 8, 2024

Proposed fix in #744

@robertonr
Copy link

robertonr commented Nov 27, 2024

Hello, I think this change introduced an important performance penalty comparing my tests using 10.1.26 vs 10.1.27

10.1.27:
image

10.1.26:
image

Call is direct in 10.1.26 and I guess that is the root cause of the bug, but instead of using reflection we can just create a wrapper calling the real delegate

@markt-asf
Copy link
Contributor

You should test the latest version (although I suspect you'll see the same results).

Use of proxies rather than wrappers was a design decision for jdbc-pool so it could handle changes in the JDBC API.

@robertonr
Copy link

Hello proxies could be implemented dynamically using one of ASM libraries; Reflection is very expensive and taking into consideration the huge number of applications using Tomcat this change will generate a lot of CO2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants