EAR application fails at deploy/redeploy due to WAR file inconsistencies [SPR-1435]

[spring-projects-issues]

Martti Tamm opened SPR-1435 and commented

I am having problems with (re)deploying my EAR appication with WebLogic 8.1 SP4 and SP5 in both Windows XP and Linux environment as sometimes there are exceptions while reading the contents of the WAR.

I may get different exceptions depending on the application I deploy, though it's always related to reading a file in the WAR. Exception says that the file in the WAR is either missing or corrupt, though it's not true. The hardest part is that usually my EAR gets deployed while sometimes I must restart WebLogic to do do that -- there's almost no rule when will it happen.

It was hard for me to track down the cause, but I managed to create a simple test-case that most likely produces the effect while trying to read a configuration XML file. It says that the XML file is invalid, though it's quite there and all XML files are saved in UTF-8.

I also tested creating Hibernate configuration similar to the test-case but without Spring. However, I didn't get this problem any more.

Stacktrace I receive on WebLogic's console is following:

org.dom4j.DocumentException: invalid stored block lengths Nested exception: invalid stored block lengths
at org.dom4j.io.SAXReader.read(SAXReader.java:484)
at org.hibernate.cfg.Configuration.addInputStream(Configuration.java:398)
at org.springframework.orm.hibernate3.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:631)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1058)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:363)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:226)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:147)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:269)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:318)
at org.springframework.web.context.support.AbstractRefreshableWebApplicationContext.refresh(AbstractRefreshableWebApplicationContext.java:134)
at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:230)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:156)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:48)
at weblogic.servlet.internal.WebAppServletContext$FireContextListenerAction.run(WebAppServletContext.java:7044)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.notifyCreated(WebAppServletContext.java:1753)
at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:3352)
at weblogic.servlet.internal.WebAppServletContext.setStarted(WebAppServletContext.java:6208)
at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:864)
at weblogic.j2ee.J2EEApplicationContainer.start(J2EEApplicationContainer.java:2134)
at weblogic.j2ee.J2EEApplicationContainer.activate(J2EEApplicationContainer.java:2175)
at weblogic.management.deploy.slave.SlaveDeployer$ComponentActivateTask.activateContainer(SlaveDeployer.java:2520)
at weblogic.management.deploy.slave.SlaveDeployer$ActivateTask.doCommit(SlaveDeployer.java:2438)
at weblogic.management.deploy.slave.SlaveDeployer$Task.commit(SlaveDeployer.java:2155)
at weblogic.management.deploy.slave.SlaveDeployer$Task.checkAutoCommit(SlaveDeployer.java:2254)
at weblogic.management.deploy.slave.SlaveDeployer$Task.prepare(SlaveDeployer.java:2149)
at weblogic.management.deploy.slave.SlaveDeployer$ActivateTask.prepare(SlaveDeployer.java:2401)
at weblogic.management.deploy.slave.SlaveDeployer.processPrepareTask(SlaveDeployer.java:883)
at weblogic.management.deploy.slave.SlaveDeployer.prepareDelta(SlaveDeployer.java:591)
at weblogic.management.deploy.slave.SlaveDeployer.prepareUpdate(SlaveDeployer.java:500)
at weblogic.drs.internal.SlaveCallbackHandler$1.execute(SlaveCallbackHandler.java:25)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
Nested exception:
java.util.zip.ZipException: invalid stored block lengths
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:140)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:105)
at java.io.FilterInputStream.read(FilterInputStream.java:66)
at weblogic.apache.xerces.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:3494)
at weblogic.apache.xerces.impl.XMLEntityManager.startEntity(XMLEntityManager.java:847)
at weblogic.apache.xerces.impl.XMLEntityManager.startDocumentEntity(XMLEntityManager.java:782)
at weblogic.apache.xerces.impl.XMLDocumentScannerImpl.setInputSource(XMLDocumentScannerImpl.java:260)
at weblogic.apache.xerces.parsers.DTDConfiguration.parse(DTDConfiguration.java:499)
at weblogic.apache.xerces.parsers.DTDConfiguration.parse(DTDConfiguration.java:581)
at weblogic.apache.xerces.parsers.XMLParser.parse(XMLParser.java:152)
at weblogic.apache.xerces.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1175)
at weblogic.xml.jaxp.WebLogicXMLReader.parse(WebLogicXMLReader.java:135)
at weblogic.xml.jaxp.RegistryXMLReader.parse(RegistryXMLReader.java:152)
at org.dom4j.io.SAXReader.read(SAXReader.java:465)
at org.hibernate.cfg.Configuration.addInputStream(Configuration.java:398)
at org.springframework.orm.hibernate3.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:631)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1058)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:363)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:226)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:147)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:269)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:318)
at org.springframework.web.context.support.AbstractRefreshableWebApplicationContext.refresh(AbstractRefreshableWebApplicationContext.java:134)
at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:230)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:156)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:48)
at weblogic.servlet.internal.WebAppServletContext$FireContextListenerAction.run(WebAppServletContext.java:7044)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.notifyCreated(WebAppServletContext.java:1753)
at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:3352)
at weblogic.servlet.internal.WebAppServletContext.setStarted(WebAppServletContext.java:6208)
at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:864)
at weblogic.j2ee.J2EEApplicationContainer.start(J2EEApplicationContainer.java:2134)
at weblogic.j2ee.J2EEApplicationContainer.activate(J2EEApplicationContainer.java:2175)
at weblogic.management.deploy.slave.SlaveDeployer$ComponentActivateTask.activateContainer(SlaveDeployer.java:2520)
at weblogic.management.deploy.slave.SlaveDeployer$ActivateTask.doCommit(SlaveDeployer.java:2438)
at weblogic.management.deploy.slave.SlaveDeployer$Task.commit(SlaveDeployer.java:2155)
at weblogic.management.deploy.slave.SlaveDeployer$Task.checkAutoCommit(SlaveDeployer.java:2254)
at weblogic.management.deploy.slave.SlaveDeployer$Task.prepare(SlaveDeployer.java:2149)
at weblogic.management.deploy.slave.SlaveDeployer$ActivateTask.prepare(SlaveDeployer.java:2401)
at weblogic.management.deploy.slave.SlaveDeployer.processPrepareTask(SlaveDeployer.java:883)
at weblogic.management.deploy.slave.SlaveDeployer.prepareDelta(SlaveDeployer.java:591)
at weblogic.management.deploy.slave.SlaveDeployer.prepareUpdate(SlaveDeployer.java:500)
at weblogic.drs.internal.SlaveCallbackHandler$1.execute(SlaveCallbackHandler.java:25)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)

From the log I can also see that Spring can successfully find the Test.hbm.xml file and all settings, but the problem starts when Hibernate's org.hibernate.cfg.Configuration class starts to use these settings and begins to read this Test.hbm.cfg file.

Business effect for the issue: since this behaviour occurs in development and production mode, it's a cause for development slowdown and customer dissatisfaction.

Does anybody have an idea? Is Spring doing anything with the JAR files like mapping-files or could it be connected with how WebLogic handles compressed files during deploy time?

Thanks for any help!

---

Affects: 1.2.4, 1.2.5

Attachments:

• testcase.zip (3.76 MB)

[spring-projects-issues]

Martti Tamm commented

This is the test-case I mentioned. Instructions are inside.

[spring-projects-issues]

Juergen Hoeller commented

This is strange; Spring does not influence Hibernate's parsing of the mapping file. That rather looks like a side effect between Hibernate's parsing and WebLogic's jar file handling.

Could you please try whether specifying the resources in your "hibernate.cfg.xml" file (with that file specified as "configLocation" on LocalSessionFactoryBean) changes the behavior?

Juergen

[spring-projects-issues]

Martti Tamm commented

I tested again by providing Hibernate configuration through hibernate.cfg.xml. LocalSessionFactoryBean was created like that:

<bean id="sessionFactory"
	class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">

	<property name="configLocation" value="classpath:hibernate.cfg.xml" />

</bean>

hibernate.cfg.xml specified the same attributes as the test-case scenario together with creating the datasource:

<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE hibernate-configuration PUBLIC
"-//Hibernate/Hibernate Configuration DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>

<session-factory>

    <!-- Data source -->
    <property name="hibernate.connection.datasource">jdbc/test</property>

    <!-- Database connection settings -->
    <property name="hibernate.dialect">org.hibernate.dialect.OracleDialect</property>

    <property name="hibernate.show_sql">false</property>

    <property name="hibernate.query.factory_class">org.hibernate.hql.classic.ClassicQueryTranslatorFactory</property>

    <property name="hibernate.cache.use_second_level_cache">false</property>

    <mapping resource="mappings/Test.hbm.xml"/>

</session-factory>

</hibernate-configuration>

From the console log I saw that Hibernate successfully retrieved all its parameters and parsed the mapping file. I couldn't produce an error.

Martti

[spring-projects-issues]

Juergen Hoeller commented

So this means that you get the error only when specifying "mappingLocations" on LocalSessionFactoryBean but not when specifying "mapping resource" tags in hibernate.cfg.xml? This would mean that the problem is indeed in LocalSessionFactoryBean's loading of mapping locations...

Juergen

[spring-projects-issues]

Martti Tamm commented

Yes, it seems so, indeed. I've also checked that the problem occurs when I try to deploy packaged/zipped file like *.ear or *.war. It doesn't occur when I just deploy an exploded directory.

There's a similar problem with a bigger application I have (I couldn't create a smaller testcase for it). Here's a stacktrace from there. Maybe it helps a bit:

at java.util.zip.ZipFile.getEntry(Native Method)
at java.util.zip.ZipFile.getEntry(ZipFile.java:146)
- locked <0x13126700> (a sun.net.www.protocol.jar.URLJarFile)
at java.util.jar.JarFile.getEntry(JarFile.java:194)
at sun.net.www.protocol.jar.URLJarFile.getEntry(URLJarFile.java:89)
at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:95)
at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:107)
at java.net.URL.openStream(URL.java:913)
at org.springframework.core.io.UrlResource.getInputStream(UrlResource.java:66)
at org.springframework.orm.hibernate3.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:631)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1058

)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:363)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:226)

• locked <0x13df1400> (a java.util.HashMap)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:147)
  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:176)
  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:105)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:10

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:823)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:345)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:226)

• locked <0x13df1400> (a java.util.HashMap)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:147)
  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:176)
  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:105)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:10

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:823)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:345)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:226)

• locked <0x13df1400> (a java.util.HashMap)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:147)
  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:176)
  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:105)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:10

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:823)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:345)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:226)

• locked <0x13df1400> (a java.util.HashMap)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:151)
  at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:557)
  at org.springframework.context.support.AbstractApplicationContext.initMessageSource(AbstractApplicationContext.java:417)
  at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:306)
  at org.springframework.web.context.support.AbstractRefreshableWebApplicationContext.refresh(AbstractRefreshableWebApplicationContext.java:134)
  at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:230)
  at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:156)
  at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:48)
  at weblogic.servlet.internal.WebAppServletContext$FireContextListenerAction.run(WebAppServletContext.java:7044)
  at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
  at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
  at weblogic.servlet.internal.WebAppServletContext.notifyCreated(WebAppServletContext.java:1753)
  at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:3352)
• locked <0x13cbb148> (a weblogic.servlet.internal.WebAppServletContext)
  at weblogic.servlet.internal.WebAppServletContext.setStarted(WebAppServletContext.java:6208)
  at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:864)
  at weblogic.j2ee.J2EEApplicationContainer.start(J2EEApplicationContainer.java:2134)
  at weblogic.j2ee.J2EEApplicationContainer.activate(J2EEApplicationContainer.java:2175)
  at weblogic.management.deploy.slave.SlaveDeployer$ComponentActivateTask.activateContainer(SlaveDeployer.java:2520)
  at weblogic.management.deploy.slave.SlaveDeployer$ActivateTask.doCommit(SlaveDeployer.java:2438)
  at weblogic.management.deploy.slave.SlaveDeployer$Task.commit(SlaveDeployer.java:2155)
  at weblogic.management.deploy.slave.SlaveDeployer$Task.checkAutoCommit(SlaveDeployer.java:2254)
  at weblogic.management.deploy.slave.SlaveDeployer$Task.prepare(SlaveDeployer.java:2149)
  at weblogic.management.deploy.slave.SlaveDeployer$ActivateTask.prepare(SlaveDeployer.java:2401)
  at weblogic.management.deploy.slave.SlaveDeployer.processPrepareTask(SlaveDeployer.java:883)
  at weblogic.management.deploy.slave.SlaveDeployer.prepareDelta(SlaveDeployer.java:591)
  at weblogic.management.deploy.slave.SlaveDeployer.prepareUpdate(SlaveDeployer.java:500)
  at weblogic.drs.internal.SlaveCallbackHandler$1.execute(SlaveCallbackHandler.java:25)
  at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
  at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)

(This trace is the Java thread that WebLogic outputs as "An unexpected exception has been detected in native code outside the VM." when I try to redeploy.)

This happens after ServletContextResourcePatternResolver has discovered all *.hbm.xml files (correctly) and then tries to add them as input streams to org.hibernate.cfg.Configuration.addInputStream(InputStream) method as a parameter.

All in all, to me it seems that the problem could be there, where Spring sets the input streams, because different sources need different handling (e.g whether the file is in JAR or not). In the above stacktrace it seems that UrlResource is used to read the file in WAR. Is this the correct way? Why not use, for example, ServletContextResource?

Martti

[spring-projects-issues]

Rob Harrop commented

Modified the path matching behavior to use Resource.createRelative for JAR resources instead of reconstructing URLs. Tested redeploy 20 times with the new version - works fine. Tested with Spring 1.2.5 release fails about 1/3rd of the time.

Rob/Juergen

[spring-projects-issues]

Juergen Hoeller commented

Note that this fix only applies to "classpath:" patterns, not to "classpath*:" patterns. Due to the ClassLoader API, we can only use ClassLoader.getResourceAsStream() for the former but not for the latter, where we have to use ClassLoader.getResources().

Juergen