NullPointerException during JPA deleteAll with lazy collections

[pwandl]

Since upgrading to Spring Boot 3.1.0 from 3.0.6 we are experiencing NPEs when calling .deleteAll() on Spring Data JPA repositories, but not for each entity. I suspect it is related to entities which contain lazy collections, but could not narrow it down further yet.

I have extracted a minimal sample application which shows the issue: https://github.com/pwandl/spring-hibernate-bug-demo

It seems to only occur if the transaction calling .deleteAll has not previously modified the to-be-deleted entities. I was not yet able to reproduce the issue using hibernate directly without any spring components, so I suspect that it might be related to some spring component, even though the issue does not occur when downgrading hibernate to 6.1.7

[wilkinsona]

Thanks for the sample. While I haven't managed to reproduce the problem without any usage of Spring, I have reproduced it without Spring Boot and Spring Data JPA. I've pushed a commit to my fork of your sample that just uses Spring Framework and Hibernate. It should fail with the same NPE.

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
May 24, 2023 5:25:06 PM org.hibernate.jpa.internal.util.LogHelper logPersistenceUnitInformation
INFO: HHH000204: Processing PersistenceUnitInfo [name: default]
May 24, 2023 5:25:06 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate ORM core version 6.2.2.Final
May 24, 2023 5:25:06 PM org.hibernate.cfg.Environment <clinit>
INFO: HHH000406: Using bytecode reflection optimizer
May 24, 2023 5:25:06 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 24, 2023 5:25:07 PM org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl logSelectedDialect
INFO: HHH035001: Using dialect: org.hibernate.dialect.H2Dialect, version: 2.1.214
May 24, 2023 5:25:07 PM org.hibernate.bytecode.internal.BytecodeProviderInitiator buildBytecodeProvider
INFO: HHH000021: Bytecode provider name : bytebuddy
May 24, 2023 5:25:07 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 8.0.0.Final
May 24, 2023 5:25:08 PM org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator initiateService
INFO: HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Hibernate: create table application_user (id bigint not null, createdBy_id bigint, primary key (id))
Hibernate: create table User_userRoles (User_id bigint not null, userRoles varchar(255) check (userRoles in ('USER','ADMIN','OPERATOR')))
Hibernate: create sequence application_user_SEQ start with 1 increment by 50
Hibernate: alter table if exists application_user add constraint FKiag4pd92c0slpoao47m5i793b foreign key (createdBy_id) references application_user
Hibernate: alter table if exists User_userRoles add constraint FKbuid0qjp6u03wg7qj2fnpf4e8 foreign key (User_id) references application_user
Hibernate: select next value for application_user_SEQ
Hibernate: select next value for application_user_SEQ
1
Hibernate: insert into application_user (createdBy_id,id) values (?,?)
Hibernate: insert into application_user (createdBy_id,id) values (?,?)
Hibernate: insert into User_userRoles (User_id,userRoles) values (?,?)
Hibernate: insert into User_userRoles (User_id,userRoles) values (?,?)
Hibernate: select u1_0.id,u1_0.createdBy_id from application_user u1_0
Hibernate: delete from User_userRoles where User_id=?
Hibernate: delete from User_userRoles where User_id=?
Exception in thread "main" org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:570)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:152)
	at com.example.demo.DemoApplication.deleteUsers(DemoApplication.java:47)
	at com.example.demo.DemoApplication.main(DemoApplication.java:41)
Caused by: jakarta.persistence.RollbackException: Error while committing the transaction
	at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:65)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:561)
	... 5 more
Caused by: jakarta.validation.ValidationException: HV000028: Unexpected exception during isValid call.
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:186)
	at org.hibernate.validator.internal.engine.constraintvalidation.SimpleConstraintTree.validateConstraints(SimpleConstraintTree.java:66)
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateConstraints(ConstraintTree.java:75)
	at org.hibernate.validator.internal.metadata.core.MetaConstraint.doValidateConstraint(MetaConstraint.java:130)
	at org.hibernate.validator.internal.metadata.core.MetaConstraint.validateConstraint(MetaConstraint.java:123)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateMetaConstraint(ValidatorImpl.java:555)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForSingleDefaultGroupElement(ValidatorImpl.java:518)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForDefaultGroup(ValidatorImpl.java:488)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateConstraintsForCurrentGroup(ValidatorImpl.java:450)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validateInContext(ValidatorImpl.java:400)
	at org.hibernate.validator.internal.engine.ValidatorImpl.validate(ValidatorImpl.java:172)
	at org.hibernate.boot.beanvalidation.BeanValidationEventListener.validate(BeanValidationEventListener.java:128)
	at org.hibernate.boot.beanvalidation.BeanValidationEventListener.onPreUpdate(BeanValidationEventListener.java:92)
	at org.hibernate.action.internal.EntityUpdateAction.preUpdate(EntityUpdateAction.java:329)
	at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:159)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
	at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:489)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:486)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:358)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1412)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:485)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2296)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1961)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:169)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:267)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
	... 6 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.hibernate.persister.collection.CollectionPersister.isExtraLazy()" because "persister" is null
	at org.hibernate.collection.spi.AbstractPersistentCollection.lambda$readSize$0(AbstractPersistentCollection.java:155)
	at org.hibernate.collection.spi.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:265)
	at org.hibernate.collection.spi.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:148)
	at org.hibernate.collection.spi.PersistentSet.size(PersistentSet.java:151)
	at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:37)
	at org.hibernate.validator.internal.constraintvalidators.bv.notempty.NotEmptyValidatorForCollection.isValid(NotEmptyValidatorForCollection.java:22)
	at org.hibernate.validator.internal.engine.constraintvalidation.ConstraintTree.validateSingleConstraint(ConstraintTree.java:180)
	... 35 more

Given that we can't (yet?) reproduce the problem without Spring Framework, I think we should get the Framework team to take a look. To that end, please open a Spring Framework issue referencing this issue and my updated version of your sample.

[quaff]

I think it's bug of hibernate 6.2, It works fine when downgrade to 6.1.7, you should report to hibernate team not spring team.

[wilkinsona]

@pwandl already said that in the opening comment:

I was not yet able to reproduce the issue using hibernate directly without any spring components, so I suspect that it might be related to some spring component, even though the issue does not occur when downgrading hibernate to 6.1.7

However, thus far we've been unable to reproduce the problem without Spring. If you have managed to do so, @quaff, please do share the code.

[quaff]

I would take a look if @pwandl share the code that works fine using hibernate directly without any spring components.

[jhoeller]

As far as I was able to reproduce this, neither Spring transaction management nor HibernateJpaVendorAdapter have any impact here since the problem remains even with plain EntityManagerFactory.createEntityManager / EntityManager.getTransaction().begin() etc and with HibernatePersistenceProvider configuration on our LocalContainerEntityManagerFactoryBean, coming close to plain Hibernate usage in practice.

The one thing that makes it pass is removing the Jakarta EL provider org.apache.tomcat.embed:tomcat-embed-el:10.1.8 from the classpath. Replacing it with org.glassfish:jakarta.el:4.0.2 breaks as well, whereas it still passes with the presence of the plain EL API in the form of jakarta.el:jakarta.el-api:4.0.0.

So it looks like something is reacting to the presence of a Jakarta EL provider here, registering some special handling that ultimately breaks persister availability for lazy collection introspection. I have a suspicion that this is related to Hibernate Validator but it might be in Hibernate ORM 6.2 itself since a plain downgrade to ORM 6.1.7 makes the problem go away.

In any case, this is definitely a Hibernate interaction problem. If someone has the cycles for it, please try to reproduce it with plain Hibernate (bringing a Jakarta EL provider onto the classpath) and report it to the Hibernate project.

[wilkinsona]

So it looks like something is reacting to the presence of a Jakarta EL provider here, registering some special handling that ultimately breaks persister availability for lazy collection introspection

That matches what I saw too. It's size-related validation that calls a method on AbstractPersistentCollection which breaks. Without an EL provider, Bean Validation backs off so the problem doesn't occur.

[jhoeller]

I originally could not reproduce this in the form of a test in spring-orm. It took a while to find the reason: namely that there was no EL provider on the classpath there, whereas it was declared in the standalone repro build. This difference unfortunately was not obvious at all.

You got a point there that without an EL provider, Hibernate ORM seems to back off from applying Hibernate Validator completely. A standalone Hibernate Validator setup fails with a proper exception if there is no EL provider on the classpath; I suppose Hibernate ORM silently swallows that and simply does not validate then.

In any case, I expect the AbstractPersistentCollection problem to be reproducible with plain Hibernate as long as there is a complete Hibernate Validator setup with an EL provider on the classpath. The CollectionPersister is meant to be present during flush but is somehow missing in action here, with nothing that we can do about it from the outside.