Strange Service behavior after ECF upgrade

[phermsdorf]

Hi,

as mentioned in #23 we have stange service import / export behaviour after upgrading ECF with our latest release.

We updated ECF from 3.14.18 (2020-12) to 3.14.33 (2022-06) (i'm referring to the version of the org.eclipse.ecf.remoteservice.sdk feature).
In our setup we use the ecf.generic server to connect RCP clients via ecf.generic provider to one application server and calling remote services from that server.
The basic setup is similar to my consumer and provider example implementation here (but based on ecf generic server implementation) So we create an EDEF description and use that to import remote services from a known remote server.

In some setup's we now encounter the problem that remote services on the client side come and go after a few seconds (with IMPORT_UNREGISTRATION events).

Currently we can't reproduce that exactly, but we see that the first client connecting to the server somehow seems to work but others coming later don't work (and have that described behaviour). If we restart the client it also does not work anymore.

I'm adding some trace outputs from client and server i generated with the provided .options file.

Any hints what could be the problem are more than welcome! Maybe it's just something dumb we are not seeing.

Thanks.

Bye Peter

trace_server.txt
trace_client.txt
dot_options.txt

[phermsdorf]

I'm attaching a log as csv from one client filtered to one service which should show the disconnect events. The cause is still unclear.
Uploading log_result.csv…

In the client trace file from my previous post i see lot of messages like this:
(TRACE)[01/27/23;15:14:54:494]TRACING com.godyo.p5.util.remoting.internal.GenericAuthConsumerContainerSelector#selectExistingConsumerContainer(No match of existing remote service container id=LongID[0] endpointID=StringID[ecftcp://APPSRVP4.customer.local:8889/server] remoteSupportedConfigs=[ecf.generic.client])
but i don't know if that is somehow an "expected" message. Since we can't reproduce the behaviour it's currently not possible to debug that.

One time there is an exception in the trace (i tried to clean that up a bit since it seems to be mixed with other concurrent output):

(TRACE)[01/27/23;15:14:54:498]TRACING com.godyo.p5.util.remoting.internal.GenericAuthConsumerContainerSelector#selectExistingConsumerContainer(MATCH of existing remote service container id=StringID[1b0k86HR4k/T0wZ2R2803VNhdC0=] endpointID=StringID[ecftcp://APPSRVP4.customer.local:8889/server] remoteSupportedConfigs=[ecf.generic.client])
(TRACE)[01/27/23;15:14:54:498]TRACING com.godyo.p5.util.remoting.internal.GenericAuthConsumerContainerSelector#selectConsumerContainer(rsContainer selected=RemoteServiceContainer [containerID=StringID[1b0k86HR4k/T0wZ2R2803VNhdC0=], container=org.eclipse.ecf.provider.generic.TCPClientSOContainer@76d99766, containerAdapter=com.godyo.p5.util.remoting.internal.P4RegistrySharedObjectClient@d8bec76])
(TRACE)[01/27/23;15:14:54:534]TRACING org.eclipse.ecf.osgi.services.remoteserviceadmin.RemoteServiceAdmin#comparePackageVersions(bundle=com.godyo.p4.system,class=interface com.godyo.p5.server.api.IServerDatabaseInfoService,packageName=com.godyo.p5.server.api,remoteVersion=1.0.0,localVersion=1.0.0) java.lang.NullPointerException
        at org.eclipse.ecf.provider.remoteservice.generic.RegistrySharedObject.handleTargetConnected(RegistrySharedObject.java:876)
        at org.eclipse.ecf.provider.remoteservice.generic.RegistrySharedObject.handleContainerConnectedEvent(RegistrySharedObject.java:861)(TRACE)[01/27/23;15:14:54:997]TRACING org.eclipse.ecf.osgi.services.remoteserviceadmin.RemoteServiceAdmin#importService(importRegistration=ImportRegistration[importReference=ImportReference[importEndpoint=ImportEndpoint[proxyRegistration={com.godyo.p5.server.api.IServerDatabaseInfoService}={ecf.generic.server.port=8889, service.id=484, service.bundleid=211, service.scope=bundle, osgi.private=intent, service.imported.configs=[ecf.generic.client], endpoint.id=ecftcp://APPSRVP4.customer.local:8889/server, service.imported=org.eclipse.ecf.provider.remoteservice.generic.RemoteServiceImpl@7b9c39ea, osgi.basic=intent, osgi.async=intent, remote.service.description=com.godyo.p5.server.ui.impl.DatabaseInfoRemoteServiceDescription@1e35e238}rsReference=RemoteServiceReferenceImpl[registration=RemoteServiceRegistrationImpl[remoteServiceID=org.eclipse.ecf.remoteservice.RemoteServiceID[containerID=StringID[ecftcp://APPSRVP4.customer.local:8889/server];containerRelativeID=5];rserviceranking=0;classes=[com.godyo.p5.server.api.IServerDatabaseInfoService];state=0;properties={ordered=intent, ecf.generic.server.port=8889, osgi.private=intent, osgi.basic=intent, exactlyOnce=intent, osgi.async=intent, ecf.rsvc.id=5, passByValue=intent}]];];exception=null];closed=false])
        at org.eclipse.ecf.provider.remoteservice.generic.RegistrySharedObject$4.processEvent(RegistrySharedObject.java:453)(TRACE)[01/27/23;15:14:55:194]TRACING com.godyo.p5.util.remoting.internal.GenericAuthConsumerContainerSelector#selectExistingConsumerContainer(No match of existing remote service container id=LongID[2] endpointID=StringID[ecftcp://APPSRVP4.customer.local:8889/server] remoteSupportedConfigs=[ecf.generic.client])
        at org.eclipse.ecf.core.sharedobject.BaseSharedObject.fireEventProcessors(BaseSharedObject.java:194)
        at org.eclipse.ecf.core.sharedobject.BaseSharedObject.handleEvent(BaseSharedObject.java:128)
        at org.eclipse.ecf.provider.generic.SOWrapper.svc(SOWrapper.java:184)
        at org.eclipse.ecf.provider.generic.SOWrapper$2.run(SOWrapper.java:140)
        at java.base/java.lang.Thread.run(Thread.java:834)

[phermsdorf]

when debugging i see this call stack on client side creating an RemoteServiceUnregisteredEvent:

Thread [org.eclipse.ecf.remoteservice.IRemoteServiceContainerAdapter:run] (Suspended (breakpoint at line 1986 in RegistrySharedObject))	
	P4RegistrySharedObjectClient(RegistrySharedObject).createUnregisteredEvent(RemoteServiceRegistrationImpl) line: 1986	
	P4RegistrySharedObjectClient(RegistrySharedObject).handleTargetGoneEvent(ID) line: 772	
	P4RegistrySharedObjectClient(RegistrySharedObject).handleContainerDisconnectedEvent(IContainerDisconnectedEvent) line: 789	
	RegistrySharedObject$4.processEvent(Event) line: 455	
	P4RegistrySharedObjectClient(BaseSharedObject).fireEventProcessors(Event) line: 194	
	P4RegistrySharedObjectClient(BaseSharedObject).handleEvent(Event) line: 128	
	SOWrapper.svc(Event) line: 184	
	SOWrapper$2.run() line: 140	
	Thread.run() line: 834	

This seems to be caused by a ContainerDisconnectedEvent coming from:

Thread [StringID[E/M07RMcBfsm0dh8oNyOnOEpNFU=]:rcvr:62745:DELASP4APP02.kopla.local:8889] (Suspended (breakpoint at line 35 in ContainerDisconnectedEvent))	
	owns: SOContainerGMM  (id=10858)	
	owns: ClientSOContainer$Lock  (id=10859)	
	owns: Client  (id=10860)	
	owns: Object  (id=10861)	
	ContainerDisconnectedEvent.<init>(ID, ID) line: 35	
	SOWrapper.memberChanged(Member, boolean) line: 122	
	SOContainerGMM.notifyAllOfMemberChange(Member, TreeMap, boolean) line: 250	
	SOContainerGMM.update(Observable, Object) line: 256	
	GMMImpl(Observable).notifyObservers(Object) line: 173	
	GMMImpl.removeMember(Member) line: 38	
	SOContainerGMM.removeMember(Member) line: 73	
	SOContainerGMM.removeAllMembers(Member) line: 106	
	SOContainerGMM.removeNonLocalMembers() line: 93	
	TCPClientSOContainer(ClientSOContainer).handleLeave(ID, IConnection) line: 350	
	TCPClientSOContainer(ClientSOContainer).disconnect(Throwable) line: 458	
	TCPClientSOContainer(ClientSOContainer).processDisconnect(DisconnectEvent) line: 476	
	SOContainer$2.handleDisconnectEvent(DisconnectEvent) line: 204	
	Client.handleException(Throwable) line: 294	
	Client$4.run() line: 365	
	Thread.run() line: 834	

and down there in Client.handleException the exception is
java.lang.ClassCastException: class org.eclipse.ecf.core.sharedobject.SharedObjectMsgEvent cannot be cast to class org.eclipse.ecf.core.identity.ID (org.eclipse.ecf.core.sharedobject.SharedObjectMsgEvent is in unnamed module of loader org.eclipse.osgi.internal.loader.EquinoxClassLoader @5e1f08ca; org.eclipse.ecf.core.identity.ID is in unnamed module of loader org.eclipse.osgi.internal.loader.EquinoxClassLoader @546857d4

the ClassCastException comes from SOContainer.defaultDeserializeSharedObjectMessage:

protected Object defaultDeserializeSharedObjectMessage(byte[] bytes) throws IOException, ClassNotFoundException {
		try {
			return AccessController.doPrivileged((PrivilegedExceptionAction<Object>) () -> {
				final ByteArrayInputStream bins = new ByteArrayInputStream(bytes);
				final OSGIObjectInputStream oins = new OSGIObjectInputStream(ProviderPlugin.getDefault().getContext().getBundle(), bins);
				ID sharedObjectID = (ID) oins.readObject(); // <- produces ClassCastException

Looks like the data received does not fit to the expected data.
Any ideas how that could happen?
Do i have to continue looking at server side?

@scottslewis could you take a look at this?

Again: this does not seems to happen everytime on every system. It looks like something goes somehow wrong and then this happens. We can still not reproduce every time.

Thanks,
bye Peter

[scottslewis]

> and down there in Client.handleException the exception is `java.lang.ClassCastException: class org.eclipse.ecf.core.sharedobject.SharedObjectMsgEvent cannot be cast to class org.eclipse.ecf.core.identity.ID (org.eclipse.ecf.core.sharedobject.SharedObjectMsgEvent is in unnamed module of loader org.eclipse.osgi.internal.loader.EquinoxClassLoader @5e1f08ca; org.eclipse.ecf.core.identity.ID is in unnamed module of loader org.eclipse.osgi.internal.loader.EquinoxClassLoader 546857d4` > > the ClassCastException comes from SOContainer.defaultDeserializeSharedObjectMessage: > > ``` > protected Object defaultDeserializeSharedObjectMessage(byte[] bytes) throws IOException, ClassNotFoundException { > try { > return AccessController.doPrivileged((PrivilegedExceptionAction) () -> { > final ByteArrayInputStream bins = new ByteArrayInputStream(bytes); > final OSGIObjectInputStream oins = new OSGIObjectInputStream(ProviderPlugin.getDefault().getContext().getBundle(), bins); > ID sharedObjectID = (ID) oins.readObject(); // <- produces ClassCastException > ``` >

I agree that this is the root cause...as it results in client disconnection and everything follows from this it seems.

Looks like the data received does not fit to the expected data. Any ideas how that could happen? Do i have to continue looking at server side?

@scottslewis could you take a look at this?

Again: this does not seems to happen everytime on every system. It looks like something goes somehow wrong and then this happens. We can still not reproduce every time.

Is it possible that you are using a different version of ECF on the server than you are using on the clients? That might do this.

Scott

[phermsdorf]

Finally we could reproduce the problem:
the ClassCastException happens when an old client (with previous ecf version) connects to the server (which basically cannot work since there were protocol changes between these versions).

But e.g. before a old RCP client gets it's update from the update site it already tries to connect to the server which causes the ClassCastException on all other clients. Which means the server does not send the expected org.eclipse.ecf.core.identity.ID but a org.eclipse.ecf.core.sharedobject.SharedObjectMsgEvent.

I'm still trying to debug why that happens.
Peter

[phermsdorf]

I thought it maybe has to do something with the protocol versions in OSGIObjectOutputStream but wasn't able to find something that causes this problem.
I suspect some backward compatibility fallback which causes wrong messages to all other clients ...

[phermsdorf]

the ClassCastException happens on the Server as well on the connected clients (in case an old client connects to the server)
further debugging revealed where exactly the classcast happens:

• AsyncMessage is received
• AsyncMessage.data is deserialized to AsyncEvent
• AsyncEvent.data is deserialized to ContainerMessage
• ContainerMessage.data (=subMessage) is deserialized
• when trying to deserialize ContainerMessage.SharedObjectMessage.data the ClassCastException SharedObjectMsgEvent cannot be cast to class org.eclipse.ecf.core.identity.ID

When comparing the deserialization of the new/current and the our previous version shows:
org.eclipse.ecf.provider.generic.SOContainer.handleSharedObject

obj = (Serializable) deserializeSharedObjectMessage((byte[]) resp.getData());
// Actually deliver event to shared object asynchronously
sow.deliverSharedObjectMessage(fromID, obj);

stayed basically the same, but in the previous version deserializeSharedObjectMessage basically just does:

Object result = oins.readObject();
return result; 

now:

ID sharedObjectID = (ID) oins.readObject();
if (sharedObjectID != null) {
	ISharedObject so = getSharedObject(sharedObjectID);
	if (so != null) {
		oins.setClassLoader(so.getClass().getClassLoader());
		Object result = oins.readObject();
		oins.close();
		return result;

where the cast to ID leads to the ClasscastException.

Why the "wrong" object is there in place i still can't explain. Is this somehow from the "old client"? For the server side that would be an explanation, but how can it be that this object is passed to all other "new clients" then without the ID object in place?

We urgently need a resolution to this problem, otherwise we need to switch back to the older ECF release for now.

Thanks,
bye Peter

Bonus Question: why are there no release branches or tags in the github repository? I wanted to reference the code directly ...

[scottslewis]

Hi Peter.

> > Why the "wrong" object is there in place i still can't explain. Is this somehow from the "old client"?

Yes I think that's right...it's from the old client...which is using an old version of SOContainer to call the serializeSharedObject method resulting in failure on attempted deserialize on server or new clients. Here is the for SOContainer

122fa9b#diff-5ac082ab76c9a31a4d18cb11593fbd3597e487f562c42aacd18486bec32c5bbf

Both the defaultSerializeSharedObjectMessage and the defaultDeserializeSharedObjectMessage were changed, and these different versions are run on both the clients and servers.

For the server side that would be an explanation, but how can it be that this object is passed to all other "new clients" then without the ID object in place?

It could be that the ContainerMessage is forwarded onto other clients before it's actually handled by the server container.

We urgently need a resolution to this problem, otherwise we need to >switch back to the older ECF release for now.

Do you need to support connection from both old (can't be updated?) and new clients with a single server? That's going to be very difficult.

The release of ECF that was prior to this change to SOContainer shared object serialization was version 3.14.22 (6-2022). I think you could probably work with 3.14.22 with both old and new clients. Here is is:

https://download.eclipse.org/rt/ecf/3.14.22/

I'm pretty sure that new clients and servers that use 3.14.22 will interoperate with your old clients. What version are they currently on BTW?

I'm sorry about this...I should have realized that the 'wire format' change at the SOContainer level would break old clients (assuming they can't be updated) but obviously I didn't.

BTW, if you end up needing to use an old version of ECF to get through this, I can probably create a new branch for it and apply some of the more recent fixes to that new branch. Eventually it would be nice to get onto version that has new SOContainer serialize/deserialize so please LMK if I can help with that.

Thanks, bye Peter

Bonus Question: why are there no release branches or tags in the github repository? I wanted to reference the code directly ...

ECF changed over to github hosting last year and apparently it broke our Jenkins builder (which was doing automatic tagging on release when git repo was at eclipse.org). I didn't realize it wasn't doing them properly since github changeover...and the tags didn't seem to make the changeover (done by the Eclipse Foundation folks). That's unfortunate. I'll look into the builder to see about fixing. Our Jenkins builder is here, btw:

https://ci.eclipse.org/ecf/

Eventually I would prefer to build ECF with github actions and do without the Jenkins builder, but haven't had a chance to make that changeover.

[phermsdorf]

Hi @scottslewis ,

Do you need to support connection from both old (can't be updated?) and new clients with a single server? That's going to be very difficult.

No that's not necessary. But we need to make sure that old clients do not disrupt the new ones.
Because the Clients are RCP applications they check for an update on startup, but at that time they already try to connect to the server or are already connected.
That leads to the situation that someone who was e.g. on holiday starts an old client and while doing the update kicks out all other already working clients etc. (in some installations there are clients that were not updated for several releases - for unknown reasons (e.g. problem accessing update site, left open -> no restart etc. ))

The release of ECF that was prior to this change to SOContainer shared object serialization was version 3.14.22 (6-2022). I think you could probably work with 3.14.22 with both old and new clients.

our "new clients" use this target definition

    <location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="slicer" includeSource="true" type="InstallableUnit">
      <unit id="org.eclipse.ecf.remoteservice.sdk.feature.feature.group" version="3.14.33.v20220224-2257"/>
      <unit id="org.eclipse.ecf.filetransfer.feature.feature.group" version="3.14.1800.v20220215-0126"/>
      <repository location="http://download.eclipse.org/releases/2022-06/"/>
    </location>

our old clients use

    <location includeAllPlatforms="false" includeConfigurePhase="false" includeMode="slicer" includeSource="true" type="InstallableUnit">
      <unit id="org.eclipse.ecf.remoteservice.sdk.feature.feature.group" version="3.14.18.v20201102-0420"/>
      <repository location="http://repository.godyo.int/mirror-2020-12/"/>
    </location>

I'm pretty sure that new clients and servers that use 3.14.22 will interoperate with your old clients. What version are they currently on BTW?

I'm sorry about this...I should have realized that the 'wire format' change at the SOContainer level would break old clients (assuming they can't be updated) but obviously I didn't.

no worries about that.

BTW, if you end up needing to use an old version of ECF to get through this, I can probably create a new branch for it and apply some of the more recent fixes to that new branch. Eventually it would be nice to get onto version that has new SOContainer serialize/deserialize so please LMK if I can help with that.

I don't know if we need this fixes - basically the old version worked for us. We just updated because we updated to the newer target platform and i now wanted to understand what caused this weird behavior.

Maybe it's the best to go back for now to the old version and continue to work and test the new TCPSocketDistribution provider and the switch to that one - which was planned anyway. (i know you did some changed but i haven't had time to try them - removing of the test service was missing i think...)

But the question is if that shouldn't be fixed despite of this switch. In a real world scenario this can be used as a kind of DoS attack...

Thanks for your comments.

Bye Peter

[phermsdorf]

so using the previous ECF version basically works but we don't get that deployed because we already have a lot of clients that updated to the newer version with org.eclipse.equinox.core.feature_1.13.600.v20220516-2155 and we are not able to remove/replace/downgrade to the required org.eclipse.ecf.core.feature_1.5.701.v20201027-0550 (including their transitive dependencies) ...
I tried to fiddle around with a p2.inf like
update.matchExp=providedCapabilities.exists(pc | pc.name == 'org.eclipse.ecf.core.feature.feature.group' || pc.name == 'com.godyo.p5.ecf.feature.feature.group')
but the already existing feature does not get uninstalled a so i get dependency problems when trying to update like:

Cannot complete the install because of a conflicting dependency.
Zu installierende Software: ECF Remote Service Feature 5.5.2309.202302061132 (com.godyo.p5.ecf.feature.feature.group 5.5.2309.202302061132)
Aktuell installierte Software: ECF Identity Core API 3.9.402.v20210409-2301 (org.eclipse.ecf.identity 3.9.402.v20210409-2301)
Only one of the following can be installed at once: 
ECF Identity Core API 3.9.401.v20201027-0550 (org.eclipse.ecf.identity 3.9.401.v20201027-0550)
ECF Identity Core API 3.9.402.v20210409-2301 (org.eclipse.ecf.identity 3.9.402.v20210409-2301)
Abhängigkeit nicht erfüllt: {0}
Von: ECF Remote Service Feature 5.5.2309.202302061132 (com.godyo.p5.ecf.feature.feature.group 5.5.2309.202302061132)
To: org.eclipse.equinox.p2.iu; org.eclipse.ecf.core.feature.feature.group [1.5.0,1.6.0)
Abhängigkeit nicht erfüllt: {0}
Von: ECF Core Feature 1.5.701.v20201027-0550 (org.eclipse.ecf.core.feature.feature.group 1.5.701.v20201027-0550)
To: org.eclipse.equinox.p2.iu; org.eclipse.ecf.identity [3.9.401.v20201027-0550,3.9.401.v20201027-0550]

(i know that's not an ECF issue ;) )

So my question is, is it possible to fix this ECF issue in a version we can upgrade to? (the idea would be to update to a newer ECF version that fixes this issue with the old clients killing new ones ...)
From MPOV an instanceof check to prevent the ClassCastException and then discard the message would be enough.

Thanks,
Bye Peter

PS:
the feature.xml requires block of com.godyo.p5.ecf.feature if anyone is interested in that: (idea was to pin the ecf.core.feature version to 1.5.x)

<requires>
      <import feature="org.eclipse.equinox.core.feature" version="1.2.102.v20160128-1406"/>
      <import feature="org.eclipse.ecf.discovery.feature" version="1.0" match="compatible"/>
      <import plugin="org.eclipse.ecf.osgi.services.distribution"/>
      <import feature="org.eclipse.ecf.core.feature" version="1.5.701.v20201027-0550" match="equivalent"/>
</requires>

[scottslewis]

Hi Peter,

So my question is, is it possible to fix this ECF issue in a version we can upgrade to? (the idea would be to update to a newer ECF version that fixes this issue with the old clients killing new ones ...)
From MPOV an instanceof check to prevent the ClassCastException and then discard the message would be enough.

I'm willing to do this and produce a new version, but I"m not exactly sure what you mean by this. Wouldn't old clients fail to work if the message were simply discarded?

One thought. The method defaultSerialize/DeserializeSharedObjectMessage are only called to do ISharedObjectMessage ser/deser if the SOContainer.setSharedObjectMessageSerializer(ISharedObjectMessageSerializer) has not been called. See SOContainer line 161 for setting the default serializer/deserializer for the SOContainer (both client and server) and SOContainer:176 for the setSharedObjectMessageSerializer method.

Wouldn't it be possible to have create a serializer/deserializer (with my help if you wish) that could handle both old and new clients on your server? Perhaps some info passed by the new clients on connect that which kind of shared object ser/deser was appropriate? I haven't thought through whether this would deal with your use case (wanting to support old and new clients), but I think it could.

[phermsdorf]

Hi Scott,

I'm willing to do this and produce a new version, but I"m not exactly sure what you mean by this. Wouldn't old clients fail to work if the message were simply discarded?

yes, but that would be absolutely OK (see my previous answers): it is ok if old clients to not work or fail. They should just not disrupt the new clients - which is the case here.

One thought. The method defaultSerialize/DeserializeSharedObjectMessage are only called to do ISharedObjectMessage ser/deser if the SOContainer.setSharedObjectMessageSerializer(ISharedObjectMessageSerializer) has not been called. See SOContainer line 161 for setting the default serializer/deserializer for the SOContainer (both client and server) and SOContainer:176 for the setSharedObjectMessageSerializer method.

Thanks for this tip! I should have seen that before. I added a custom ISharedObjectMessageSerializer with this implementation:

public class P4SharedObjectSerializer implements ISharedObjectMessageSerializer {

	private final Bundle bundle;

	public P4SharedObjectSerializer() {
		bundle = FrameworkUtil.getBundle(getClass());
	}

	@Override
	public byte[] serializeMessage(final ID sharedObjectID, final Object message) throws IOException {
		try (final ByteArrayOutputStream bouts = new ByteArrayOutputStream();
				final ObjectOutputStream oos = new OSGIObjectOutputStream(bouts)) {
			oos.writeObject(message);
			return bouts.toByteArray();
		}
	}

	@Override
	public Object deserializeMessage(final byte[] bytes) throws IOException, ClassNotFoundException {
		try (final ByteArrayInputStream bins = new ByteArrayInputStream(bytes);
				final OSGIObjectInputStream oins = new OSGIObjectInputStream(bundle, bins)) {
			return oins.readObject();
		}
	}
}

now everything works again, we can update to the new version and are even backward compatible to the old clients.
Thanks for your support!

But i still think this should be fixed (eg. add instanceof). As i wrote above: one old/faulty/attacker client can disconnect/kill all other clients connected to a server...

Bye Peter

BTW: the package org.eclipse.ecf.core.util is exported by bundle org.eclipse.ecf and org.eclipse.ecf.identity. That prevents usage of package import in MANIFEST.MF

[scottslewis]

But i still think this should be fixed (eg. add instanceof). As i wrote above: one old/faulty/attacker client can disconnect/kill all other clients connected to a server...

Hi Peter. I want to implement something like you describe, but first I want to understand what exactly what you were seeing so that I can provide more robustness for your use case and others.

I understand that the default ISharedObjectMessageSerializer was throwing a ClassCastException. In the ECF code, the only place that the ISharedObjectMessageSerializer is called is SOContainer.deserializeSharedObjectMessage and this method allows the Exception to get to the caller of deserializeSharedObjectMessage (which is SOContainer.handleSharedObjectMessage). The exception handlers in SOContainer.handleSharedObjectMessage currently catch any ClassCastException (or IOExcepiton) and simply logs it: (this is from SOContainer.handleSharedObjectMessage):

				} catch (final ClassNotFoundException e) {
					String message = "shared object message ClassNotFoundException.  sharedObjectID=" + sharedObjectID + " fromContainerID=" + fromID; //$NON-NLS-1$ //$NON-NLS-2$
					ProviderPlugin.getDefault().log(new Status(IStatus.ERROR, ProviderPlugin.PLUGIN_ID, message, e));
					printToSystemError(message, e);

For the default servers and clients (any subclasses of SOContainer that don't override handleSharedObjectMessage or deserializeSharedObjectMessage the behavior should be to simply log the exception on the server or the client.

But this is obviously not what you were seeing. In looking at your description again above it was just your old clients? having disconnect? And if that's right (old clients), I don't understand what you are suggesting with fixing/preventing those old clients from disconnecting. Thanksinadvance for any thoughts.

[scottslewis]

I think this was effectively dealt with. If not, please reopen.