Shared subprogram access yields too many connection instances in instance model

[AaronGreenhouse]

This example is derived from the example in Issue #1941. When the system below is instantiated, the process instance MyP has three connection instances, two of which should not be generated. Furthermore, it should have an additional connection instance that is missing.

package ExtraConnections
public
	subprogram ComputeAverage
	end ComputeAverage;

	thread T1
		features
			shared_sub: provides subprogram access ComputeAverage;
	end T1;

	thread implementation T1.impl
		subcomponents
			CompAvg: subprogram ComputeAverage;
		connections
			ac1: subprogram access CompAvg <-> shared_sub;
	end T1.impl;

	thread T2
		features
			ext_sub: requires subprogram access ComputeAverage;
	end T2;

	thread implementation T2.impl
	end T2.impl;

	process p1
	end p1;

	process implementation p1.impl
		subcomponents
			MyT1: thread T1.impl;
			MyT2: thread T2.impl;
		connections
			ac2: subprogram access MyT1.shared_sub <-> MyT2.ext_sub;
	end p1.impl;

	system SubprogramExample19
	end SubprogramExample19;

	system implementation SubprogramExample19.impl
		subcomponents
			MyP: process p1.impl;
	end SubprogramExample19.impl;
end ExtraConnections;

The instance model has the connection instances (with the given connection references):

Only the second connection instance MyT1.CompAvg -> MyT2.ext_sub should exist. The other two connections are incomplete nonsense.

Furthermore, the connection between CompAvg and ext_sub should be bidirectional, so there should be a connection instances MyT2.ext_sub -> MyT1.CompAvg, but there currently is not one.

[AaronGreenhouse]

Further testing indicates the same problems for shared data connections, but that shared bus connections work as desired:

package WithDataConnections
public
	subprogram ComputeAverage
	end ComputeAverage;

	data D
	end D;
	
	bus B
	end B;

	system s1
		features
			shared_bus: provides bus access B;
	end s1;

	system implementation s1.impl
		subcomponents
			myBus: bus B;
		connections
			bc1: bus access myBus <-> shared_bus;
	end s1.impl;
	
	system s2
		features
			ext_bus: requires bus access B;
	end s2;
	
	system implementation s2.impl
	end s2.impl;
	
	system MySystem
	end MySystem;
	
	system implementation MySystem.impl
		subcomponents
			myS1: system s1.impl;
			myS2: system s2.impl;
		connections
			bc2: bus access myS1.shared_bus <-> myS2.ext_bus;
	end MySystem.impl;
	
	thread T1
		features
			shared_sub: provides subprogram access ComputeAverage;
			shared_data: provides data access D;
	end T1;

	thread implementation T1.impl
		subcomponents
			CompAvg: subprogram ComputeAverage;
			myData: data D;
		connections
			ac1: subprogram access CompAvg <-> shared_sub;
			dc1: data access myData <-> shared_data;
	end T1.impl;

	thread T2
		features
			ext_sub: requires subprogram access ComputeAverage;
			ext_data: requires data access D;
	end T2;

	thread implementation T2.impl
	end T2.impl;

	process p1
	end p1;

	process implementation p1.impl
		subcomponents
			MyT1: thread T1.impl;
			MyT2: thread T2.impl;
		connections
			ac2: subprogram access MyT1.shared_sub <-> MyT2.ext_sub;
			dc2: data access MyT1.shared_data <-> MyT2.ext_data;
	end p1.impl;

	system SubprogramExample19
	end SubprogramExample19;

	system implementation SubprogramExample19.impl
		subcomponents
			MyP: process p1.impl;
			MyS: system MySystem.impl;
	end SubprogramExample19.impl;
end WithDataConnections;

[AaronGreenhouse]

More tests:

• subprogram groups don't work (like subprogram and data)
• virtual buses do work (like bus)

Complete test system:

package TestAccessConnections
public
	subprogram ComputeAverage
	end ComputeAverage;

	subprogram group SubGroup
	end SubGroup;

	data D
	end D;
	
	bus B
	end B;
	
	virtual bus VB
	end VB;

	system s1
		features
			shared_bus: provides bus access B;
			shared_VB: provides virtual bus access VB;
	end s1;

	system implementation s1.impl
		subcomponents
			myBus: bus B;
			myVB: virtual bus VB;
		connections
			bc1: bus access myBus <-> shared_bus;
			vbc1: virtual bus access myVB <-> shared_VB;
	end s1.impl;
	
	system s2
		features
			ext_bus: requires bus access B;
			ext_VB: requires virtual bus access VB;
	end s2;
	
	system implementation s2.impl
	end s2.impl;
	
	system MySystem
	end MySystem;
	
	system implementation MySystem.impl
		subcomponents
			myS1: system s1.impl;
			myS2: system s2.impl;
		connections
			bc2: bus access myS1.shared_bus <-> myS2.ext_bus;
			vbc2: virtual bus access myS1.shared_vb <-> myS2.ext_vb;
	end MySystem.impl;
	
	thread T1
		features
			shared_sub: provides subprogram access ComputeAverage;
			shared_subg: provides subprogram group access SubGroup;
			shared_data: provides data access D;
	end T1;

	thread implementation T1.impl
		subcomponents
			CompAvg: subprogram ComputeAverage;
			SubG: subprogram group SubGroup;
			myData: data D;
		connections
			ac1: subprogram access CompAvg <-> shared_sub;
			sgc1: subprogram group access SubG <-> shared_subg;
			dc1: data access myData <-> shared_data;
	end T1.impl;

	thread T2
		features
			ext_sub: requires subprogram access ComputeAverage;
			ext_subg: requires subprogram group access SubGroup;
			ext_data: requires data access D;
	end T2;

	thread implementation T2.impl
	end T2.impl;

	process p1
	end p1;

	process implementation p1.impl
		subcomponents
			MyT1: thread T1.impl;
			MyT2: thread T2.impl;
		connections
			ac2: subprogram access MyT1.shared_sub <-> MyT2.ext_sub;
			sgc2: subprogram group access MyT1.shared_subg <-> MyT2.ext_subg;
			dc2: data access MyT1.shared_data <-> MyT2.ext_data;
	end p1.impl;

	system SubprogramExample19
	end SubprogramExample19;

	system implementation SubprogramExample19.impl
		subcomponents
			MyP: process p1.impl;
			MyS: system MySystem.impl;
	end SubprogramExample19.impl;
end TestAccessConnections;

[AaronGreenhouse]

There are two main places in CreateConnectionsSwitch that need to be update, and the correspond with the two different directions of the connection. I found these places pretty quickly by just searching for places where there are tests for BUS.

1. In method instantiateConnections() there is the block of code

					if (hasOutgoingFeatureSubcomponents
							&& ((cat != THREAD && cat != PROCESSOR && cat != DEVICE && cat != VIRTUAL_PROCESSOR)
									// in case of a provides bus access we want to
									// start from the bus.
									|| ((cat == PROCESSOR || cat == DEVICE || cat == MEMORY)
											&& feature instanceof BusAccess
											&& ((BusAccess) feature).getKind() == AccessType.PROVIDES)) {
						connectedInside = isConnectionEnd(insideSubConns, feature);
						destinationFromInside = isDestination(insideSubConns, feature);
					}

This clearly captures all the possibilities of provides bus access features. (This is the case that currently works correctly.) So this needs to be expanded to capture the other provides acces features.

2. In the method appendSegment() there is the block of code

							if (((toImpl instanceof ProcessorImplementation || toImpl instanceof DeviceImplementation
									|| toImpl instanceof MemoryImplementation)
									&& !(toEnd instanceof BusAccess
											&& ((BusAccess) toEnd).getKind() == AccessType.PROVIDES))) {
								final ConnectionInfo clone = connInfo.cloneInfo();
								clone.complete = true;
								finalizeConnectionInstance(ci, clone, toFi);
							}

Again, this filters out the case of provides bus access features. This also needs to be updated to handle the other provides access features.

I started by trying to fix data access connections. It is definitely true that these two locations must be updated, and I have done so as

					if (hasOutgoingFeatureSubcomponents
							&& ((cat != THREAD && cat != PROCESSOR && cat != DEVICE && cat != VIRTUAL_PROCESSOR)
									// in case of a provides bus access we want to
									// start from the bus.
									|| ((cat == PROCESSOR || cat == DEVICE || cat == MEMORY)
											&& feature instanceof BusAccess
											&& ((BusAccess) feature).getKind() == AccessType.PROVIDES)
									// in case of a provides data access we want to
									// start from the data.
									|| ((cat == DATA || cat == THREAD || cat == THREAD_GROUP || cat == PROCESS)
											&& feature instanceof DataAccess
											&& ((DataAccess) feature).getKind() == AccessType.PROVIDES))) {
						connectedInside = isConnectionEnd(insideSubConns, feature);
						destinationFromInside = isDestination(insideSubConns, feature);
					}

and

							if (((toImpl instanceof ProcessorImplementation || toImpl instanceof DeviceImplementation
									|| toImpl instanceof MemoryImplementation)
									&& !(toEnd instanceof BusAccess
											&& ((BusAccess) toEnd).getKind() == AccessType.PROVIDES))
									|| ((toImpl instanceof DataImplementation || toImpl instanceof ThreadImplementation
											|| toImpl instanceof ThreadGroup || toImpl instanceof ProcessImplementation)
											&& !(toEnd instanceof DataAccess
													&& ((DataAccess) toEnd).getKind() == AccessType.PROVIDES))) {
								final ConnectionInfo clone = connInfo.cloneInfo();
								clone.complete = true;
								finalizeConnectionInstance(ci, clone, toFi);
							}

but this is not sufficient because the second block of code is guarded by an extremely complicated chain of conditions. Towards the start of this chain, there is test of the local variable finalComponent which gets its value from isConnectionEndingComponent(toCtx), which is true if the test component is a thread or device subcomponent. This has the effect of preventing the connection processing from going back "in" a thread component to find the nested data (or other subcomponent) being provided, event though this connection is subcomponent is connected and allowed to go "out" of the thread when the connection instance is generated in the other direction.

There is a test early on in this conditional chain

				if (toEnd instanceof Parameter
						|| finalComponent && !(toEnd instanceof FeatureGroup)) {

If I change it to

				if (toEnd instanceof Parameter
						|| finalComponent && !(toEnd instanceof FeatureGroup || toEnd instanceof Access)) {

Then data access connections work as required. As I have used the super type Access this test should be sufficient for subprogram and subprogram group accesses as well. However, I am concerned this change could cause unexpected problems in other cases because I don't at all understand everything happening in this method because it is so extremely complicated. I need to keep an eye on the unit tests.

[AaronGreenhouse]

The logic for checking the end of the connection instance in appendSegment got really tricky. I had to change it to

							boolean finalizeConnectionInstance = true;
							if (toEnd instanceof BusAccess && ((BusAccess) toEnd).getKind() == AccessType.PROVIDES) {
								if (toImpl instanceof ProcessorImplementation || toImpl instanceof MemoryImplementation
										|| toImpl instanceof BusImplementation || toImpl instanceof DeviceImplementation
										|| toImpl instanceof SystemImplementation) {
									finalizeConnectionInstance = false;
								}
							} else if (toEnd instanceof DataAccess
									&& ((DataAccess) toEnd).getKind() == AccessType.PROVIDES) {
								if (toImpl instanceof DataImplementation || toImpl instanceof ThreadImplementation
										|| toImpl instanceof ThreadGroupImplementation
										|| toImpl instanceof ProcessImplementation
										|| toImpl instanceof SystemImplementation) {
									finalizeConnectionInstance = false;
								}
							} else if (toEnd instanceof SubprogramAccess
									&& ((SubprogramAccess) toEnd).getKind() == AccessType.PROVIDES) {
								if (toImpl instanceof DataImplementation
										|| toImpl instanceof SubprogramGroupImplementation
										|| toImpl instanceof ThreadImplementation
										|| toImpl instanceof ThreadGroupImplementation
										|| toImpl instanceof ProcessImplementation
										|| toImpl instanceof ProcessorImplementation
										|| toImpl instanceof VirtualProcessorImplementation
										|| toImpl instanceof DeviceImplementation
										|| toImpl instanceof SystemImplementation) {
									finalizeConnectionInstance = false;
								}
							} else if (toEnd instanceof SubprogramGroupAccess
									&& ((SubprogramGroupAccess) toEnd).getKind() == AccessType.PROVIDES) {
								if (toImpl instanceof DataImplementation
										|| toImpl instanceof SubprogramGroupImplementation
										|| toImpl instanceof ThreadImplementation
										|| toImpl instanceof ThreadGroupImplementation
										|| toImpl instanceof ProcessImplementation
										|| toImpl instanceof ProcessorImplementation
										|| toImpl instanceof VirtualProcessorImplementation
										|| toImpl instanceof DeviceImplementation
										|| toImpl instanceof SystemImplementation) {
									finalizeConnectionInstance = false;
								}
							}
							if (finalizeConnectionInstance) {
								final ConnectionInfo clone = connInfo.cloneInfo();
								clone.complete = true;
								finalizeConnectionInstance(ci, clone, toFi);
							}

This can probably be simplified, but it works this way and the intent seems much clearer like this.

[AaronGreenhouse]

Added unit tests.

[AaronGreenhouse]

Ran all the unit tests. Things are not okay. First test that fails is for Issue 667. The logic for the changes to appendSegment is still incorrect.

[AaronGreenhouse]

I have to revisit the intention of the guard statements identified above in instantiateConnections and appendSegments. Looking at the code as it was before I attempted to fix it, it occurred to me there part of the test, (cat != THREAD && cat != PROCESSOR && cat != DEVICE && cat != VIRTUAL_PROCESSOR), seems to be an optimization to avoid going inside of components that aren't going to have useful internal connections. Whether this check was ever completely true, I'm not sure, but for the current AADL standard it makes no sense. Threads, processors, and devices can all contain subcomponents that have regular ports (never mind the provides access problem), and these ports can be connected to outgoing ports of the thread, processors, or device. The conditional clause above can disrupt this connection.

I can use a similar example as the ones above to demonstrate this. Here we have a thread with an abstract subcomponent, and the connection instances are messed up:

package JustAbstract
public
	abstract A
		features
			p: in out event port;
	end A;

	thread T1
		features
			f1: in out event port;
	end T1;

	thread implementation T1.impl
		subcomponents
			sub: abstract A;
		connections
			c1: port sub.p <-> f1;
	end T1.impl;

	thread T2
		features
			f2: in out event port;
	end T2;

	thread implementation T2.impl
	end T2.impl;

	process p1
	end p1;

	process implementation p1.impl
		subcomponents
			MyT1: thread T1.impl;
			MyT2: thread T2.impl;
		connections
			c2: port MyT1.f1 <-> MyT2.f2;
	end p1.impl;

	system Root
	end Root;

	system implementation Root.impl
		subcomponents
			MyP: process p1.impl;
	end Root.impl;
end JustAbstract;

The connection instances generated are

But, if I take the model above and change the threads and processes to system components,

package JustAbstract_System
public
	abstract A
		features
			p: in out event port;
	end A;

	system T1
		features
			f1: in out event port;
	end T1;

	system implementation T1.impl
		subcomponents
			sub: abstract A;
		connections
			c1: port sub.p <-> f1;
	end T1.impl;

	system T2
		features
			f2: in out event port;
	end T2;

	system implementation T2.impl
	end T2.impl;

	system p1
	end p1;

	system implementation p1.impl
		subcomponents
			MyT1: system T1.impl;
			MyT2: system T2.impl;
		connections
			c2: port MyT1.f1 <-> MyT2.f2;
	end p1.impl;

	system Root
	end Root;

	system implementation Root.impl
		subcomponents
			MyP: system p1.impl;
	end Root.impl;
end JustAbstract_System;

the generated instance model is correct, which two connection instances corresponding to the bidirectionally of the original connections.

[AaronGreenhouse]

I discussed this with Lutz. The guard against threads, processors, and devices is meant to prevent connections ending at subprogram parameters. This is because of timing considerations. Lutz needs to get more clarification from Peter about this.

For now

• Do not go into threads, processors, or devices if the internal connection is to a subprogram parameter, unless that parameter is a requires subprogram or subprogram group access. I think the test here is
• Go into everything else all the time.

Do not go into threads, processors, virtual processors or devices if the internal connection is a parameter connection.

Lots more things to create tests for:

• Obviously test parameter connections inside of threads, processors, virtual processors, and devices.
• Test access connections inside of the above, and inside other items
• test subprograms inside of subprograms, with parameter connections
• Who can have provided subprogram subcomponents: data, subprogram group, thread, thread group, process, processor (but it cannot have subprogram subcomponents??), virtual processor (but it cannot have subprogram subcomponents??), device (but it cannot have subprogram subcomponents??). Check that parameter connections are not allowed in these cases.

[AaronGreenhouse]

Not sure what I need to do on the other end of the connection yet in appendSegment. It's going to have to be a variation of the same test I put in instantiateConnections, because it's just the same issue on the other end of the connection path.

[AaronGreenhouse]

May have gotten carried away in yesterday's discussion. The only things that can have subprogram calls and parameter connections are threads and other subprograms. So I don't see why we need to worry about memory, device, processor, or virtual processor.

Parser/verifier doesn't even allow parameter connections outside of thread or subprogram implementation classifiers.

Also, remember: parameter connections aren't dealt with at all in the instance model.

[AaronGreenhouse]

I tried attacking this problem in a more principle fashion as described above. The test we really care about is the connections inside the component are ParameterConnections.

Again, I have changed things in 3 main locations, plus fiddled with some of the helper methods this time.

1. In instantiateConnections() I replaced

                    if (hasOutgoingFeatureSubcomponents
                            && ((cat != THREAD && cat != PROCESSOR && cat != DEVICE && cat != VIRTUAL_PROCESSOR)
                                    // in case of a provides bus access we want to
                                    // start from the bus.
                                    || ((cat == PROCESSOR || cat == DEVICE || cat == MEMORY)
                                            && feature instanceof BusAccess
                                            && ((BusAccess) feature).getKind() == AccessType.PROVIDES)) {
                        connectedInside = isConnectionEnd(insideSubConns, feature);
                        destinationFromInside = isDestination(insideSubConns, feature);
                    }

with just

					if (hasOutgoingFeatureSubcomponents
					) {
						connectedInside = isConnectionEnd(insideSubConns, feature);
						destinationFromInside = isDestination(insideSubConns, feature);
					}

but, I also changed isConnectionEnd and isDestination to ignore parameter connections:

	public boolean isConnectionEnd(List<Connection> conns, Feature feature) {
		List<Feature> features = feature.getAllFeatureRefinements();

		for (Connection conn : conns) {
			// ignore parameter connections
			if (!(conn instanceof ParameterConnection)) {
				if (features.contains(conn.getAllDestination()) || features.contains(conn.getAllSource())) {
					return true;
				}
				if ((features.contains(conn.getAllDestinationContext())
						|| features.contains(conn.getAllSourceContext()))) {
					return true;
				}
			}
		}
		return false;
	}

	public boolean isDestination(List<Connection> conns, Feature feature) {
		List<Feature> features = feature.getAllFeatureRefinements();

		for (Connection conn : conns) {
			// ignore parameter connections
			if (!(conn instanceof ParameterConnection)) {
				if (features.contains(conn.getAllDestination())
						|| conn.isAllBidirectional() && features.contains(conn.getAllSource())) {
					return true;
				}
				if ((features.contains(conn.getAllDestinationContext())
						|| conn.isAllBidirectional() && features.contains(conn.getAllSourceContext()))) {
					return true;
				}
			}
		}
		return false;
	}

In appendSegment() I changed

                if (toEnd instanceof Parameter
                        || finalComponent && !(toEnd instanceof FeatureGroup)) {

to just

                if (toEnd instanceof Parameter) {

2. I wanted to get rid of the finalComponent flag altogether, but it is used in the following else if having to do with feature group features:

				} else if (finalComponent && toEnd instanceof FeatureGroup) { 

I'm not sure what to do with this yet. I have a feeling this is also wrong and that it will prevent access features inside of feature groups from working correctly. I need a test case for this.

3. I replaced

                            if (((toImpl instanceof ProcessorImplementation || toImpl instanceof DeviceImplementation
                                    || toImpl instanceof MemoryImplementation)
                                    && !(toEnd instanceof BusAccess
                                            && ((BusAccess) toEnd).getKind() == AccessType.PROVIDES))) {
                                final ConnectionInfo clone = connInfo.cloneInfo();
                                clone.complete = true;
                                finalizeConnectionInstance(ci, clone, toFi);
                            }

with

							/*
							 * Issue 2032: If we get here then destination component has internal connections,
							 * not all of which are parameter connections. If there is at least one
							 * parameter connection, as indicated by the hasParameterConnection flag,
							 * we finalize the current connection, but also keep going with processing
							 * the internal connections along the current path.
							 */
							if (hasParameterConnection.get()) {
								final ConnectionInfo clone = connInfo.cloneInfo();
								clone.complete = true;
								finalizeConnectionInstance(ci, clone, toFi);
							}

where, earlier in the method I changed how the list of ingoing connections is computed. I replaced

						List<Connection> conns = AadlUtil.getIngoingConnections(toImpl, toFeature);

with code that gets the connections, but filters out any parameter connections, but also makes note if there were any parameter connections.

						final AtomicBoolean hasParameterConnection = new AtomicBoolean(false);
						List<Connection> conns = AadlUtil.getIngoingConnections(toImpl, toFeature,
								c -> {
									if (c instanceof ParameterConnection) {
										hasParameterConnection.set(true);
										return false;
									} else {
										return true;
									}
								});

where I also changed AadlUtil.getIngoingConnections() to take a predicate, (but also left a compatible original version in place):

	public static EList<Connection> getIngoingConnections(ComponentImplementation cimpl, Feature feature) {
		return getIngoingConnections(cimpl, feature, c -> true);
	}

	/**
	 * Get ingoing connections to subcomponents from a specified feature of the
	 * component impl
	 *
	 * @param feature component impl feature that is the source of a connection
	 * @param context the outer feature (feature group) or null
	 * @param useConnection Predicate that indicates if the connection should be added to the result.  Each ingoing
	 * connection is tested before being added to the result.  This predicate should return <code>true</code> if the
	 * connection should be added to the result.
	 * @return EList connections with feature as source
	 */
	public static EList<Connection> getIngoingConnections(ComponentImplementation cimpl, Feature feature,
			Predicate<Connection> useConnection) {
		EList<Connection> result = new BasicEList<Connection>();
		// The local feature could be a refinement of the feature through which the connection enters the component
		Feature local = (Feature) cimpl.findNamedElement(feature.getName());
		List<Feature> features = local.getAllFeatureRefinements();
		EList<Connection> cimplconns = cimpl.getAllConnections();
		for (Connection conn : cimplconns) {
			if (features.contains(conn.getAllSource()) && !(conn.getAllSourceContext() instanceof Subcomponent)
					|| (conn.isAllBidirectional() && features.contains(conn.getAllDestination())
							&& !(conn.getAllDestinationContext() instanceof Subcomponent))) {
				if (useConnection.test(conn)) {
					result.add(conn);
				}
			}
			if ((features.contains(conn.getAllSourceContext())
					|| (conn.isAllBidirectional() && features.contains(conn.getAllDestinationContext())))) {
				if (useConnection.test(conn)) {
					result.add(conn);
				}
			}
		}
		return result;
	}

My earlier unit tests for this issue pass with these changes. I need to run all the tests now and see what I may have broken. I also want to add additional test of my own:

• Obviously test parameter connections inside of threads, processors, virtual processors, and devices.
• Test access connections inside of the above, and inside other items
• test subprograms inside of subprograms, with parameter connections
• Test access features inside of feature groups (see above)
• test nested abstract components (see the problem above)
• test case where feature has a parameter connection and a connection to an abstract component. ("stop and go" case)

[AaronGreenhouse]

Good news, all the existing unit tests pass under this scenario!

I'm going to add some more tests specific to this issue (see above comment), and hopefully all will be fine.

Need to remember to get rid of any commented out code.

[AaronGreenhouse]

Added tests for

• Virtual bus access connections in system
• Bus access connections in device (instead of system)
• Bus access connections in processor (instead of system)
• Virtual bus access connections in processor (instead of system)
• Virtual bus access connections in virtual processor (instead of system)

The following don't work right now:

• Virtual bus access connections in system
• Virtual bus access connections in processor (instead of system)
• Virtual bus access connections in virtual processor (instead of system)

They all fail the same way, with three connection instances: