Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DefaultMessageListenerContainer failover to work with Weblogic JMS and security credentials [SPR-4720] #9397

Closed
spring-projects-issues opened this issue Apr 17, 2008 · 41 comments
Assignees
Labels
in: messaging Issues in messaging modules (jms, messaging) type: enhancement A general enhancement
Milestone

Comments

@spring-projects-issues
Copy link
Collaborator

spring-projects-issues commented Apr 17, 2008

John Baker opened SPR-4720 and commented

Weblogic has a strange 'feature' where by an InitialContext created in one thread can not be used in another when security credentials have been provided. I personally think this design is a little silly, but BEA are unlikely to listen. The default message listener container does not recover a connection when it's failed, and the exception suggests it's due to this 'feature'. Here is the exception:

java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[eventfetcher]
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:195)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:338)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:252)
at weblogic.jndi.internal.ServerNamingNode_921_WLStub.lookup(Unknown Source)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:374)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:362)
at weblogic.rmi.cluster.BasicReplicaHandler.refreshReplicaList(BasicReplicaHandler.java:506)
at weblogic.rmi.cluster.BasicReplicaHandler.failOver(BasicReplicaHandler.java:206)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:257)
at weblogic.jms.frontend.FEConnectionFactoryImpl_922_WLStub.connectionCreateRequest(Unknown Source)
at weblogic.jms.client.JMSConnectionFactory.setupJMSConnection(JMSConnectionFactory.java:238)
at weblogic.jms.client.JMSConnectionFactory.createConnectionInternal(JMSConnectionFactory.java:299)
at weblogic.jms.client.JMSConnectionFactory.createConnection(JMSConnectionFactory.java:205)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContaine
r.java:401)
at com.db.websso.jms.WeblogicMessageListenerContainer.createSharedConnection(WeblogicMessageListenerContainer.java:41
)
at org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContain
er.java:386)
at org.springframework.jms.listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful(DefaultMessageLi
stenerContainer.java:782)
at org.springframework.jms.listener.DefaultMessageListenerContainer.recoverAfterListenerSetupFailure(DefaultMessageLi
stenerContainer.java:764)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageLis
tenerContainer.java:892)

I had implemented a hack to partly solve the problem by extending the listener, but it's not completely effective. The result is that some threads seem to recover, while others continue to fail (and I can not currently explain why, given my hack re-creates the JNDI in the current thread!). Here's the hack:

protected Connection createSharedConnection()
throws JMSException
{
// Associate JNDI variables (user and password) with this thread for the benefit
// of the WL drivers.
try
{ InitialContext ic = new InitialContext(jndiTemplate.getEnvironment()); }
catch (NamingException e)
{ logger.warn(e.getMessage()); }
return super.createSharedConnection();
}

It's not very nice and it would be better to find a long term decent solution to this problem. Until a solution is in place, this object (and other JMS functionality, such as multiple threads and the JmsTemplate) will fail to work correctly with Weblogic.

I'm happy to test possible solutions if someone with indepth experience of the Spring JMS code can put forward suggestions.

Thanks,

John


Affects: 2.5.3

Attachments:

Sub-tasks:

Issue Links:

1 votes, 3 watchers

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Apr 17, 2008

John Baker commented

Hello,

Annoyingly, my fix sometimes works and sometimes does not - I can't work out why! Look at this stack trace and you'll see it's passing through the com.db.websso object (my extension of the default message listener that recreates the InitialContext) yet it still fails to reconnect...

18-04 05:36:03,233 jms.messageListenerContainer.events.ms1-19 INFO WeblogicMessageListenerContainer.refreshConnectionUntilSu
ccessful:793 - Could not refresh JMS Connection - retrying in 5000 ms
java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[eventfetcher]
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:195)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:338)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:252)
at weblogic.jndi.internal.ServerNamingNode_921_WLStub.lookup(Unknown Source)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:374)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:362)
at weblogic.rmi.cluster.BasicReplicaHandler.refreshReplicaList(BasicReplicaHandler.java:506)
at weblogic.rmi.cluster.BasicReplicaHandler.failOver(BasicReplicaHandler.java:206)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:257)
at weblogic.jms.frontend.FEConnectionFactoryImpl_922_WLStub.connectionCreateRequest(Unknown Source)
at weblogic.jms.client.JMSConnectionFactory.setupJMSConnection(JMSConnectionFactory.java:238)
at weblogic.jms.client.JMSConnectionFactory.createConnectionInternal(JMSConnectionFactory.java:299)
at weblogic.jms.client.JMSConnectionFactory.createConnection(JMSConnectionFactory.java:205)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContaine
r.java:401)
at com.db.websso.jms.WeblogicMessageListenerContainer.createSharedConnection(WeblogicMessageListenerContainer.java:41
)
at org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContain
er.java:386)

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Apr 17, 2008

John Baker commented

After further research I have concluded that the randomness can be explained by the way in which we restart our Weblogic instance.

The hack works perfectly acceptable 100% of the time for a start restart of Weblogic. Therefore, this needs including in the next release of Spring (but perhaps in a cleaner form!).

The randomness seems to be something to do with a restart of WL that involves a full rebuild of the configuration between the stop and start. While this is only a theory, the JMS client drivers may be caching something from the configuration (such as a salt) and using this cached value to encrypt the user credentials. Hence, after the configuration is rebuilt, the client drivers can not connect, because Weblogic server can not correctly decrypt the credentials. But we can live with this as it's clearly a fault in the JMS drivers.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Apr 24, 2008

Juergen Hoeller commented

We actually had the underlying issue reported before (see #7627). Our presently recommended solution is to implement a custom ConnectionFactory decorator (e.g. deriving from the Spring-provided DelegatingConnectionFactory) where you re-initialize your JNDI environment for every "createConnection()" call. This should work fine for WebLogic's purposes and doesn't require a DefaultMessageListenerContainer subclass or the like.

We may provide such a WebLogic-specific ConnectionFactory decorator ourselves at some point. I'll reconsider this for a later Spring 2.5.x release.

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 8, 2008

Juergen Hoeller commented

Actually, given how this WebLogic JNDI arrangement works, you could also try defining your ConnectionFactory lookup as JndiObjectFactoryBean with proxyInterface="javax.jms.ConnectionFactory" and cache="false"... (or the equivalent <jee:jndi-lookup jndi-name="..." proxy-interface="javax.jms.ConnectionFactory" cache="false"/>). This would perform a fresh JNDI lookup for every createConnection call, which shouldn't be too bad in terms of overhead - and naturally reinitializes the JNDI InitialContext for every such call...

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 11, 2008

John Baker commented

Hello,

I've removed my hack and defined the JndiObjectFactoryBean as discussed above and the listener does connect correctly with user credentials. But interestingly, this appears in the logs:

12-05 04:13:47,434 jms.messageListenerContainer.events.ms1-1779 INFO DefaultMessageListenerContainer.handleListenerSetupFailure:748 - Setup of JMS message listener invoker failed - trying to recover: weblogic.jms.common.JMSSecurityException: Access denied to resource: type=<jms>, application=Reporting, destinationType=queue, resource=EventQueue, action=receive
12-05 04:13:47,450 jms.messageListenerContainer.events.ms1-1779 INFO DefaultMessageListenerContainer.refreshConnectionUntilSuccessful:788 - Successfully refreshed JMS Connection

This happens on every connection attempt, but it shouldn't be failing then working.

I've read the linked bug and I do understand WL's JMS implementation. While I think it's really quite poor, the username & password in the initial context is bound to the thread, so the only safe way to connect is to create the initial context before each connection attempt. This is why a cached copy, accessed by different threads, fails to work.

I am still chasing another problem and while I'm 99% convinced it's not Spring (but WL's JMS), if I can see a way to improve Spring then I'll update this ticket. In the meantime, what is the official Spring recommended route to fixing this problem? I think a non-cached JndiObjectFactoryBean is the safest, although the above debug has to be addressed.

This should be documented somewhere in Spring given WL JMS is common and others will come across this problem.

John

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 14, 2008

Juergen Hoeller commented

This looks like you cannot use a shared JMS Connection at all in such a scenario then... I assume you're setting up your DefaultMessageListenerContainer without JTA transactions (i.e. without "transactionManager" property), in which case it will be caching a shared Connection by default... Try setting DMLC's "cacheLevelName" property to "CACHE_NONE"; this might make that setup failure log disappear.

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 15, 2008

John Baker commented

Hiya,

I still see this:

15-05 10:50:29,656 jms.messageListenerContainer.events.ms2-271 INFO DefaultMessageListenerContainer.handleListenerSetupFailure:748 - Setup of JMS message listener invoker failed - trying to recover: weblogic.jms.common.JMSSecurityException: Access denied to resource: type=<jms>, application=Reporting, destinationType=queue, resource=EventQueue, action=receive

When the DFML is setup as follows:

<bean id="jms.messageListenerContainer.events.ms2"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="concurrentConsumers" value="5" />
<property name="connectionFactory" ref="jms.queueConnectionFactory.ms2" />
<property name="destination" ref="jms.destination.events.ms2" />
<property name="messageListener" ref="jms.messageListener" />
<property name="cacheLevelName" value="CACHE_NONE" />
</bean>

John

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 15, 2008

John Baker commented

In fact, it's quite a serious problem because the listener never seems to pick up messages again - it simply prints out that debug, announces that it's recovered the connection and does no more. So setting CACHE_NONE makes things somewhat worse!

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 15, 2008

John Baker commented

I'm concerned that this solution isn't particularly efficient. We don't want to be opening connections to WLS every time the listener refreshes - we simply want a new Initial context created every time a new connection is created. Therefore, despite CACHE_NONE not working as described above, I think caching of the connection/session does have to happen inside the DFML.

So if I forget about CACHE_NONE and assume DFML should be caching the connection/session, how do I get rid of the "invoker failed" error? I' m currently debugging the source but it's fairly complicated...

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 15, 2008

Juergen Hoeller commented

FWIW, I would prefer caching to remain active there as well, in particular for non-XA scenarios. CACHE_NONE is usually used in conjunction with connection pools. Reopening connections all the time is certainly not desirable for the general case. However, reobtaining connections for each receive attempt is necessary in case of XA-aware pools in order to get proper XA enlistment.

So where do those invoker failures come from? It seems that for some reason the invoker threads fail when using the shared Connection... That should have affected your original workaround as well, since you were just setting the InitialContext for the thread that recovered the shared Connection. Did you see any such invoker failure logging at that point? We need to track down when exactly the security credentials are accepted and when not (leading to JMSSecurityExceptions)...

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 15, 2008

John Baker commented

I didn't see the failure in my original workaround:

public class WeblogicMessageListenerContainer extends DefaultMessageListenerContainer
{
private JndiTemplate jndiTemplate;

public void setJndiTemplate(JndiTemplate jndiTemplate)
{ this.jndiTemplate = jndiTemplate; }

protected void wlFix()
{
// Associate JNDI variables (user and password) with this thread for the benefit
// of the WL drivers.
try
{ InitialContext ic = new InitialContext(jndiTemplate.getEnvironment()); }
catch (NamingException e)
{ logger.warn(e.getMessage()); }
}

protected Connection createSharedConnection()
throws JMSException
{
wlFix();
return super.createSharedConnection();
}
}

I simply recreated the InitialContext (in the current thread) and let Spring use the InitialContext from the JndiObjectFactoryBean - the important point is that my hack associated the credentials with the current thread. Ironically, I think my workaround actually worked, but we have been trying to diagnose other WLS errors (which I believe are unrelated to Spring) and I wanted to remove it.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 15, 2008

John Baker commented

So with the cache mode set to CONSUMER (i.e. I haven't set it), the error is thrown when this is called:

return session.createConsumer(destination, getMessageSelector());

in AbstractPollingMessageListenerContainer.createConsumer. The exception suggests a different thread created the connection, but I've only got one thread running.

I've written a test program to connect, create a session, create a message consumer, close it and create another, and that works correctly. So I don't think the exception is being thrown because the consumer is being creaetd twice from the same session (I put nothing past WLS!).

I shall carry on debugging...

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 15, 2008

John Baker commented

Are there multiple threads sharing the same connection?

You need to tell me more about how this works. All I currently have is a set of evidence to suggest that multiple threads are sharing the same connection.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 18, 2008

hisaak commented

I've followed this issue for few days as I face the similar problem. I came to the same workaround but it didn't work for me as I use cluster address to my weblogic cluster like t3://192.168.42.58:8001,192.168.42.58:9001

If I use single node weblogic instance, it works just fine. When I use two nodes failover still works but only when I shutdown first node and then start it again. However when I try to shutdown the first node and migrate JMS server to the second node my Spring client doesn't failover. I get this exception:

May 18, 2008 9:14:37 PM org.springframework.jms.listener.DefaultMessageListenerContainer handleListenerSetupFailure
SEVERE: Setup of JMS message listener invoker failed - trying to recover
weblogic.jms.common.JMSException: Destination not found
at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:110)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:45)
at weblogic.jms.client.JMSSession.consumerCreate(JMSSession.java:2673)
at weblogic.jms.client.JMSSession.setupConsumer(JMSSession.java:2446)
at weblogic.jms.client.JMSSession.createConsumer(JMSSession.java:2393)
at weblogic.jms.client.JMSSession.createConsumer(JMSSession.java:2373)
at weblogic.jms.client.WLSessionImpl.createConsumer(WLSessionImpl.java:800)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.createConsumer(AbstractPollingMessageListenerContainer.java:437)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.createListenerConsumer(AbstractPollingMessageListenerContainer.java:216)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:297)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:254)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:870)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:810)
at java.lang.Thread.run(Thread.java:595)
Caused by: weblogic.jms.common.JMSException: Destination not found
at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:110)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:45)
at weblogic.jms.frontend.FEConsumer.<init>(FEConsumer.java:258)
at weblogic.jms.frontend.FESession$2.run(FESession.java:989)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at weblogic.jms.frontend.FESession.consumerCreate(FESession.java:985)
at weblogic.jms.frontend.FESession.invoke(FESession.java:2876)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:759)
at weblogic.messaging.dispatcher.DispatcherServerRef.invoke(DispatcherServerRef.java:276)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:141)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:36)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:112)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
Caused by: weblogic.jms.common.JMSException: Destination not found
at weblogic.jms.dispatcher.Request.handleThrowable(Request.java:63)
at weblogic.jms.dispatcher.Request.getResult(Request.java:51)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:895)
at weblogic.messaging.dispatcher.DispatcherImpl.dispatchSync(DispatcherImpl.java:178)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:43)
... 13 more
Caused by: weblogic.jms.common.JMSException: Destination not found
at weblogic.jms.dispatcher.InvocableManagerDelegate.invocableFind(InvocableManagerDelegate.java:224)
at weblogic.jms.backend.BESessionImpl.createBEConsumer(BESessionImpl.java:339)
at weblogic.jms.backend.BESessionImpl.createConsumer(BESessionImpl.java:412)
at weblogic.jms.backend.BESessionImpl.invoke(BESessionImpl.java:310)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:759)
... 15 more
Caused by: weblogic.jms.common.InvalidDestinationException: Destination not found
at weblogic.jms.dispatcher.InvocableManagerDelegate$MyInvocableManager.invocableFind(InvocableManagerDelegate.java:303)
at weblogic.jms.dispatcher.InvocableManagerDelegate.invocableFind(InvocableManagerDelegate.java:222)
... 19 more
Caused by: java.lang.Exception: Destination not found
at weblogic.messaging.dispatcher.InvocableManager.invocableFind(InvocableManager.java:107)
at weblogic.jms.dispatcher.InvocableManagerDelegate$MyInvocableManager.invocableFind(InvocableManagerDelegate.java:298)
... 20 more

My simple (no Spring) single-thread client works fine and fails over as I expect. Here comes my configuration:

<bean id="queueMDP" class="spring.QueueMDP" />

<bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
    <property name="environment">
        <props>
            <prop key="java.naming.factory.initial">weblogic.jndi.WLInitialContextFactory</prop>
            <prop key="java.naming.provider.url">t3://192.168.93.248:8001,192.168.93.248:9001</prop>
            <prop key="java.naming.security.principal">jmsuser</prop>
            <prop key="java.naming.security.credentials">xyz</prop>
            <prop key="java.naming.security.authentication">simple</prop>
        </props>
    </property>
</bean>

<bean id="queueConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiTemplate" ref="jndiTemplate" />
    <property name="jndiName" value="javax/jms/QueueConnectionFactory"/>
    <property name="cache" value="false"/>
    <property name="proxyInterface" value="javax.jms.ConnectionFactory"/>
</bean>

<bean id="queue1" class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiTemplate" ref="jndiTemplate" />
    <property name="jndiName" value="jms.queue1" />
</bean>

<bean id="queue1Listener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
    <property name="connectionFactory" ref="queueConnectionFactory" />
    <property name="destination" ref="queue1" />
    <property name="messageListener" ref="queueMDP" />
    <property name="concurrentConsumers" value="10" />
    <property name="cacheLevelName" value="CACHE_NONE" />
</bean>

I found out that my Spring-based client connects to only one node and I can't set it to use both nodes as defined in cluster address. Maybe there's something cached in queue1 (I think so because of "Destination not found" exception) bean but I don't know how to get over it.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 18, 2008

Juergen Hoeller commented

The destination failover problem is a common issue on WebLogic. Our recommendation is to not use JndiObjectFactoryBean for the lookup of Destination objects there but rather to specify the "destinationName" property as value "jms.queue1" (in your case) on DefaultMessageListenerContainer itself. For the latter to resolve destinations against JNDI, configuring a JndiDestinationResolver and pass it into DMLC's "destinationResolver" property as a bean reference:

<bean id="destinationResolver" class="org.springframework.jms.support.destination.JndiDestinationResolver"> 
    <property name="jndiTemplate" ref="jndiTemplate" /> 
</bean> 

<bean id="queue1Listener" class="org.springframework.jms.listener.DefaultMessageListenerContainer"> 
    <property name="connectionFactory" ref="queueConnectionFactory" /> 
    <property name="destinationResolver" ref="destinationResolver" /> 
    <property name="destinationName" value="jms.queue1" /> 
    <property name="messageListener" ref="queueMDP" /> 
    <property name="concurrentConsumers" value="10" /> 
</bean> 

This should work fine with the default cache level as well, actually, so I'm not sure CACHE_NONE is really needed in your scenario.

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 18, 2008

hisaak commented

Thanks, Juergen. It works great now even with cached consumer. However I still have to use the same initial context hack as mentioned by John Baker earlier in this issue to get over JMSSecurityException: Access denied.

So far I am quite happy with the solution because it fails over correctly and initial context is retrieved only when creating connection. Nonetheless I would appreciate to use clean pure Spring no-hack-needed design.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

Juergen Hoeller commented

DMLC's shared Connection is indeed used by multiple threads - it has to, since the very point of message listener containers is to manage asynchronous consumers.

However, it's unclear to me what exactly works and what doesn't now: Does the "createSharedConnection" overriding hack solve the user credentials problem completely, or is there some reproducible issue remaining in that case? Does it also work if you factor out that "hack" code into a ConnectionFactory decorator - i.e. leave DMLC as-is but link it to a custom ConnectonFactory decorator bean that has the JNDI initialization code in its "createConnection" implementation?

public class JndiInitializingConnectionFactory extends DelegatingConnectionFactory {

public Connection createConnection()) {
wlFix();
return super.createConnection();
}

protected void wlFix() {
// Associate JNDI variables (user and password) with this thread for the benefit
// of the WL drivers.
try {
InitialContext ic = new InitialContext(jndiTemplate.getEnvironment());
}
catch (NamingException e) {
logger.warn(e.getMessage());
}
}
}

<bean id="wlfixConnectionFactory" class="yourpackage.JndiInitializingConnectionFactory">
<property name="targetConnectionFactory" ref="queueConnectionFactory" />
</bean>

And your DefaultMessageListenerContainer bean definition's "connectionFactory" property in turn linking to that "wlfixConnectionFactory" bean.

Finally, does it also work if you pass in the same credentials not into a JNDI InitialContext but rather into "ConnectionFactory.createConnection(user, pw)"? That could for example be done in such a custom ConnectionFactory decorator as well, adapting a plain "createConnection()" call onto a "createConnection(user, pw)" call on the target ConnectionFactory.

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

Does the "createSharedConnection" overriding hack solve the user credentials problem completely, or is there some
reproducible issue remaining in that case?

This works, but I'd prefer not to extend DMLC. I think we should be able to address this within Spring. There should be a documented approach for using DMLC with Weblogic (given the 'associate user credentials with current thread' bug - I call it a bug as it's not worthy of being described as a feature!).

I'll get back to you on the other suggestions.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

Actually, it's going to work, isn't it? I think you now understand the problem, but the question is how best do we solve it throuh spring without a nasty hack!

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

Juergen Hoeller commented

I guess the factored-out ConnectionFactory decorator should work fine then, with no need to extend DMLC. I'm afraid there is no solution with less configuration effort, since DMLC itself doesn't deal with JNDI at all...

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

So we need to incorporate the factored out connection factory into Spring and document it properly. Weblogic is a popular JMS platform and while BEA are entirely at fault for this bizarre design, some documentation within Spring would be most helpful.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

Juergen Hoeller commented

Yes... FYI, we have a related problem in #6206: WebLogic seems to insist on thread-local JNDI context for EJB invocations as well, in case of that EJB having authorization requirements. Our idea is to solve this through automatic JNDI context opening/closing around EJB invocations automatically there, which solves the issue on WebLogic and shouldn't have any negative side effects. In the EJB accessor case, we have the JNDI environment settings locally specified already - so we can reuse them in a straightforward fashion. However, in the JMS message listener container case, there are no such local JNDI settings... which is why we'll need an external ConnectionFactory decorator, unfortunately to be configured explicitly.

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

Can I throw something else into the pot? Check out this exception coming from WLS:

ay 20, 2008 5:39:25 AM EDT> <Warning> <RMI> <BEA-080003> <RuntimeException thrown by rmi server: weblogic.jndi.internal.AdminRoleBasedDispatchServerRef@9, implementation: 'weblogic.jndi.internal.RootNamingNode@1f59bbd', oid: '9', implementationClassName: 'weblogic.jndi.internal.RootNamingNode'
java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[eventfetcher].
java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[eventfetcher]
at weblogic.security.service.SecurityServiceManager.seal(SecurityServiceManager.java:815)
at weblogic.security.service.SecurityServiceManager.getSealedSubjectFromWire(SecurityServiceManager.java:504)
at weblogic.rjvm.MsgAbbrevInputStream.getSubject(MsgAbbrevInputStream.java:316)
at weblogic.rmi.internal.BasicServerRef.acceptRequest(BasicServerRef.java:809)
at weblogic.rmi.internal.BasicServerRef.dispatch(BasicServerRef.java:298)
Truncated. see log file for complete stacktrace

We are seeing this over and over again and our client is the one we're discussing above. Do you have any ideas why WLS is doing this? We're seeing no obvious failure on the client side...

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

Juergen Hoeller commented

Hmm... that could be a related issue when doing RMI authorization. This may happen anywhere when operating on JNDI-obtained WebLogic handles... All a consequence of that odd design choice in WLS, I guess.

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

I'm having a read of this:

http://edocs.beasys.com/wls/docs81/jndi/jndi.html

It may prove useful.

As a matter of interest, is Spring calling jndiContext.close(); when a connection is closed?

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

The docs say: "You can use the same properties on either a client or a server. If you define the properties on a server-side object, a local Context is used. If you define the properties on a client or another WebLogic Server, the Context delegates to a remote Context running on the WebLogic Server specified by the Context.PROVIDER_URL property. A remote object bound to the server will not be serviced by peerGone, and will not be reachable if the client should fail."

Can you tell me more about the RMI problem? I've got some sample code running without Spring that I'm trying to use to nail this problem.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

Juergen,

On the related bug you've written "As for the JndiCallback solution: It should actually be sufficient to override "doInvoke" only there, since that encompasses "create" and "remove" already (at least in Spring 2.x). So executing "doInvoke" within one all-encompassing JndiCallback should result in equivalent behavior. Is this assumption correct? "

Could you write a couple lines of sample code and I'll integrate/play with it to see if it may help solve my problem?

J

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

Juergen Hoeller commented

The idea is essentially simply to do a "Context ctx = new InitialContext(...)" first, then perform your invocations on the target object, then close the Context object afterwards. WebLogic apparently expects all access to obtained objects to happen within an open InitialContext, at least for access that requires authorization...

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

That makes sense and ties in with what we suspect is "dangling proxies" (or whatever they'd be called) on the server. Where would you fit this in though? And would it involve this:

Listing 2-3 JNDI Context and Threads Wrapper Code

import java.security.PrivilegedAction;
import javax.security.auth.Subject;
import weblogic.security.Security;

public class client
{
  public static void main(String[] args)
  {
  Security.runAs(new Subject(),
    new PrivilegedAction() {
     public Object run() {
      //
      //If implementing in client code, main() goes here.
      //
     return null;
     }
   }); 
  }
}

I.e. do we need:

Security.runAs(new Subject(),
  new PrivilegedAction() {
   public Object run() {
    InitialContext ic = new InitialContext ( jndiTemplate.getEnvironment() );
    try {
      // Do stuff
    } finally { ic.close(); }
    return null;
   }
 }); 
}

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 19, 2008

John Baker commented

Which of course doesn't sound very efficient!

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 20, 2008

John Baker commented

I've tried extending the JndiTemplate and it did not solve the problem:

public class JndiTemplate extends org.springframework.jndi.JndiTemplate
{
Context context = null;

@Override
public Object execute(final JndiCallback contextCallback)
throws NamingException
{
return Security.runAs(new Subject(),
new PrivilegedAction() {
public Object run()
{
Context ctx = null;
try
{
logger.info("Creating context "+Thread.currentThread());
ctx = JndiTemplate.this.createInitialContext();
return contextCallback.doInContext(ctx);
}
catch (NamingException e)
{ logger.warn(e); }
finally
{
try
{ ctx.close(); }
catch (NamingException ex)
{ logger.debug("Could not close JNDI InitialContext", ex); }
}
return null;
}
});
}
}

However I've discovered something a little odd. The exception in WLS appears every five minutes to the second! Is there any kind of refresh connection code firing every five minutes?

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 20, 2008

John Baker commented

WLS is mad.

I've disconnected the client but the exceptions are still appearing in the server logs, every five minutes pretty much to the second! The client has been disconnected for 20 minutes now and it's still printing out the exception. We reckon there's some server side stub still running (!).

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented May 20, 2008

Juergen Hoeller commented

I've introduced an "exposeAccessContext" flag on JndiObjectFactoryBean and jee:jndi-lookup, for WebLogic resource factories that have authorization requirements on their connection retrieval methods. This should work fine for a JMS ConnectionFactory obtained that way, satisfying WebLogic's strange JNDI context requirements... with reasonably little configuration (just add expose-access-context="true" to your jee:jndi-lookup tag).

The same flag is also available on Spring's EJB accessors now, for access to WebLogic EJBs with authorization requirements.

I have no clue what those connection refresh messages are doing there every fine minutes, even for a disconnected client... I'm not aware of anything Spring might do wrong here. If you nevertheless find something we could do about it from Spring's side, let me know!

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jun 9, 2008

John Baker commented

Juergen,

I believe some of the strange WLS server side stack traces are due to multiple Weblogic servers connecting to one server resulting in some kind of cross domain problem. It's (sort of) documented in various locations when one Googles for the stack trace.

Could you expand on your fix for this problem? How does one configure the JndiObjectFactoryBean to work around this problem?

John

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jun 12, 2008

Juergen Hoeller commented

It's a simple "exposeAccessContext" flag on JndiObjectFactoryBean:

<bean id="..." class="org.springframework.jndi.JndiObjectFactoryBean">
<property ...>
<property name="exposeAccessContext" value="true"/>
</bean>

The equivalent is:

<jee:jndi-lookup ... expose-access-context="true"/>

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jun 17, 2009

Troy Harris commented

I want to make sure I understand correctly, the issue with Spring and WebLogic's need for credentials on the InitialContext has been resolved by the addition of "exposeAccessConext" on JndiObjectFactoryBean?

Rather than use a Decorator and init the InitialContext with every connection I was hoping to use a Spring provided resolution. The Decorator works with secure JMS but I removed it and tried the code shown in the last example by Juergen and I am now getting the following:

Cannot resolve reference to bean 'queueConnectionFactory' while setting bean prope
rty 'connectionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with n
ame 'queueConnectionFactory' defined in ServletContext resource [/WEB-INF/spring-config/infrastructure-config.xml]: Invocation o
f init method failed; nested exception is java.lang.IllegalAccessError: class weblogic.jms.client.$Proxy235 cannot access its su
perinterface weblogic.jms.client.Reconnectable.

Here is my config:

    <bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
           <property name="environment">
                   <props>
                           <prop key="java.naming.factory.initial">weblogic.jndi.WLInitialContextFactory</prop>
                           <prop key="java.naming.provider.url">t3://myserver.com</prop>
                           <prop key="java.naming.security.authentication">simple</prop>
                           <prop key="java.naming.security.principal">jmsuser<prop>
                           <prop key="java.naming.security.credentials">jmspassword</prop>
                   </props>
           </property>
    </bean>

    <bean id="queueConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
            <property name="exposeAccessContext" value="true"/>
            <property name="jndiTemplate" ref="jndiTemplate" />
            <property name="jndiName" value="my/jms/QueueConnectionFactory" />
    </bean>

    <bean id="testJmsEventSender" class="troy.jms.EvenSender">
            <property name="jmsTemplate">
                    <bean class="org.springframework.jms.core.JmsTemplate">
                            <property name="defaultDestination" ref="testDestination" />
                            <property name="connectionFactory" ref="queueConnectionFactory" />
                    </bean>
            </property>
    </bean>

Also, how can jee:jndi-lookup be used, I didn't think that lookup accepted the properties required to accept the credentials?
Thanks!

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Aug 6, 2009

John Baker commented

Hello,

I've had reason to review this problem today and I'm not sure the exposeAccessContext flag actually works. This is what happens to a working configuration when I switch it on:

<06-Aug-2009 16:31:11 o'clock BST> <Warning> <Deployer> <BEA-149078> <Stack trace for message 149004
weblogic.application.ModuleException:
at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:891)
at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:333)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:204)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:26)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:60)
Truncated. see log file for complete stacktrace
java.lang.IllegalAccessError: tried to access class weblogic/jms/client/Reconnectable from class weblogic/jms/client/$Proxy195
at java.lang.reflect.Proxy.defineClass0(Native Method)
at java.lang.reflect.Proxy.getProxyClass(Proxy.java:504)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:581)
at org.springframework.aop.framework.JdkDynamicAopProxy.getProxy(JdkDynamicAopProxy.java:117)
at org.springframework.aop.framework.ProxyFactory.getProxy(ProxyFactory.java:110)
Truncated. see log file for complete stacktrace

All I did was enable it in the JndiObjectFactoryBean. I'd been previously using my hacked connection factory to create the InitialContext each time a call was made (to a connection factory method), and I thought it may be interesting to switch it off and try the exposeAccessContext flag.

I've looked at the code and am not really sure how it's supposed to work either, given the only way to work around this Weblogic "design feature" is to create the InitialContext in the thread that makes a call to the connection factory.

John

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Aug 6, 2009

John Baker commented

Perhaps a longer stack trace would help:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jms.core.connectionFactory.y' defined in class path resource [jms-core-context.xml]: Invocation of init method failed; nested exception is java.lang.IllegalAccessError: tried to access class weblogic/jms/client/Reconnectable from class weblogic/jms/client/$Proxy195
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1338)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:423)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:255)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:199)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)
at weblogic.servlet.internal.EventsManager$FireContextListenerAction.run(EventsManager.java:376)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:82)
at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1616)
at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:2761)
at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:889)
at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:333)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:204)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:26)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:60)
at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:117)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:204)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:26)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:60)
at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:26)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:635)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:26)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:212)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:154)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:80)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:566)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:136)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:104)
at weblogic.deploy.internal.targetserver.operations.StartOperation.doCommit(StartOperation.java:139)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:320)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:815)
at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1222)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:433)
at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:161)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:181)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:12)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:67)
at weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
Caused by: java.lang.IllegalAccessError: tried to access class weblogic/jms/client/Reconnectable from class weblogic/jms/client/$Proxy195
at java.lang.reflect.Proxy.defineClass0(Native Method)
at java.lang.reflect.Proxy.getProxyClass(Proxy.java:504)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:581)
at org.springframework.aop.framework.JdkDynamicAopProxy.getProxy(JdkDynamicAopProxy.java:117)
at org.springframework.aop.framework.ProxyFactory.getProxy(ProxyFactory.java:110)
at org.springframework.jndi.JndiObjectFactoryBean$JndiObjectProxyFactory.createJndiObjectProxy(JndiObjectFactoryBean.java:303)
at org.springframework.jndi.JndiObjectFactoryBean$JndiObjectProxyFactory.access$000(JndiObjectFactoryBean.java:273)
at org.springframework.jndi.JndiObjectFactoryBean.afterPropertiesSet(JndiObjectFactoryBean.java:176)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1369)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1335)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:423)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:255)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:199)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)
at weblogic.servlet.internal.EventsManager$FireContextListenerAction.run(EventsManager.java:376)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:82)
at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1616)
at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:2761)
at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:889)

(And can we unfix this issue given it's not fixed! :)

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Aug 12, 2009

larry liang commented

Hi,

By reading the thread, it seems that people believe that this issue is caused by the fact that weblogic jndi security context is attached to the thread which opens the initial context, in other words, it is thread local. However I have experienced the same issue when trying to send a message to a queue, which I believe that everything happens in the same thread.

What I did is (not using any spring code here at all):

  1. Created a queue on weblogic server and secure it with certain groups.
  2. Created initial context using: new InitialContext(properties); Here I pass in the right usernamd and password.
  3. Successfully created the initalContext and sent the first message to weblogic server.
  4. Close the connection by calling context.close();
  5. Try to create a new connection from exsisting factory by calling connectionFactory.createConnection(); //no username and password provided.
  6. Create a new session and messageProducer and invoke messageProducer.send(message).
  7. At this point of time, I got the exception from weblogic saying that "Access denied to resource: type=<jms>, application=test, destinationType=queue, resource=securedQueue, action=send". This makes sense because the context has been closed.

I was wondering if Spring tries to close the jndi context some where (such as in JndiObjectFactoryBean) after creating the initialContext or after a successful lookup()?

I wonder if this explains why the jms client failed in the same thread that did open the initialContext.

Thanks,

Larry

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Feb 1, 2010

Klaus Kreuzwieser commented

After extensive testing I think this issue is fixed with WebLogic 10.3 and the usage of the following properties (please see also SPR-5869) in the bean definition of JndiObjectFactoryBean.
<property name="exposeAccessContext" value="true"/>
<property name="proxyInterface" value="javax.jms.QueueConnectionFactory"/>

I've created a small demo during the tests which I've attached to this issue.
We use 3 WebLogic instances: msg providing the JMS queues, appl2 as simulation of a remote WebLogic server (integrated into a backend system) and appl1 with the web application deployed.
The demo tests DefaultMessageListenerContainer (receiver) and JmsTemplate (sender).
During the tests we have restarted both msg and appl2 - everything works fine.

I've also tried to use a WebLogic 9.2 appl2 server, but here I still the following error after restart of appl2.
java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[weblogic, Administrators
].
java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[weblogic, Administrators]
at weblogic.security.service.SecurityServiceManager.seal(SecurityServiceManager.java:815)
at weblogic.security.service.SecurityServiceManager.getSealedSubjectFromWire(SecurityService
Manager.java:504)
at weblogic.rjvm.MsgAbbrevInputStream.getSubject(MsgAbbrevInputStream.java:316)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:
137)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:36)
Truncated. see log file for complete stacktrace

We have worked around that issue by using a plain old Java message putter instead of Spring.

@wudc
Copy link

wudc commented Nov 28, 2019

Hi, I have a similar issue. I wonder has this issue been resolved for good. I know it has been 10 years. I have Google around and couldn't find a solution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: messaging Issues in messaging modules (jms, messaging) type: enhancement A general enhancement
Projects
None yet
Development

No branches or pull requests

3 participants