End to end flow instantiation creates nonsense flow

[AaronGreenhouse]

In the example below, the end to end flow e2e is broken because the destination of connection c1 doesn't match the starting point of flow fpath. An error about this is generated on the declarative model. But when you instantiate the system top.specific an end to end flow instance for e2e is created, and it contains the erroneous connection. No error is reported on the instance model.

package fred2
public
    thread th
        features
            i: in event port;
            i_wrong: in event port;
            o: out event port;
        flows
            fpath: flow path i -> o;
    end th;
    
    thread implementation th.specific
--        flows
--            fpath: flow path i -> o;
    end th.specific;
        
    thread th2
        features
            o: out event port;
            i: in event port;
        flows
            fsrc: flow source o;
            fsnk: flow sink i;
    end th2;
    
    thread implementation th2.i
        -- trivial
    end th2.i;
    
    process top
    end top;
    
    process implementation top.specific
        subcomponents
            t: thread th.specific;
            q: thread th2.i;
        connections
            c1: port q.o -> t.i_wrong;
            c2: port t.o -> q.i;
        flows
            e2e: end to end flow q.fsrc -> c1 -> t.fpath -> c2 -> q.fsnk;
    end top.specific;
end fred2;

However, if you uncomment the (superfluous) flow path implementation in th.specific, then the end to end flow is not created and the instance model has the error message:

Cannot create end to end flow 'e2e' because the end of the semantic connection 'q.o -> t.i_wrong' does not connect to the start of flow 'fpath'

This is because the error checking of connection endpoints in instantiation only happens when there is a flow implementation. The methods isValidContinuation() are used.

(These methods are currently causing other problems, however, see issue #1984.)

[AaronGreenhouse]

I tried the same case with outgoing connections:

    process implementation top.specific2
        subcomponents
            t: thread th.specific;
            q: thread th2.i;
        connections
            c1: port q.o -> t.i;
            c2: port t.o_wrong -> q.i;
        flows
            e2e: end to end flow q.fsrc -> c1 -> t.fpath -> c2 -> q.fsnk;
    end top.specific;

The instance model correctly ignores this because the collectConnectionInstances returns an empty set when looking for the incoming connections to fsnk in processFlowStep.

[AaronGreenhouse]

I think the fix is to update

				final List<ConnectionInstance> connectionsToUse = new ArrayList<>();
				for (final ConnectionInstance ciToCheck : connis) {
					if ((flowFilter == null || isValidContinuation(etei, flowFilter, ciToCheck))
							&& (nextFlowImpl == null || isValidContinuation(etei, ciToCheck, nextFlowImpl))) {
						connectionsToUse.add(ciToCheck);
					}
				}

so that when nextFlowImpl == null a new version of isValidContinuation() that works with FlowSpecifications is used.

[AaronGreenhouse]

Okay, nothing is easy. This breaks one of the Serializer2Test tests. The problem comes from connection instances burrowing into subcomponents. In this case, the test I added above doesn't work because the end point of the connection instance is a feature of a subcomponent of the subcomponent, where the endpoint of might be a feature of the outer subcomponent. Specifically, we have problems when the flow named in the end to end flow is not fleshed out as a flow implementation in a component implementation.

Need to change things to start looking the the connections that make up the connection instance.

[AaronGreenhouse]

Above problem is distilled in the following example. The end to end flow is not instantiated in top.withImpl_implNoFlow, but is in the other two systems.

package test
public
	system s1
		features
			o: out event port;
		flows
			src: flow source o;
	end s1;
	
	system s2
		features
			i: in event port;
			i_wrong: in event port;
		flows
			snk: flow sink i;
	end s2;

	system s3
		features 
			i2: in event port;
			i2_wrong: in event port;
		flows
			snk2: flow sink i2;
	end s3;

	system implementation s2.implFlow
		subcomponents
			s3: system s3;
		connections
			c2: port i -> s3.i2;
			c2_wrong: port i_wrong -> s3.i2_wrong;
		flows
			snk: flow sink i -> c2 -> s3.snk2;
	end s2.implFlow;

	system implementation s2.implNoFlow
		subcomponents
			s3: system s3;
		connections
			c2: port i -> s3.i2;
			c2_wrong: port i_wrong -> s3.i2_wrong;
	end s2.implNoFlow;

	system top
	end top;
	
	system implementation top.typeOnly
		subcomponents
			s1: system s1;
			s2: system s2;
		connections
			c1: port s1.o -> s2.i;
			c1_wrong: port s1.o -> s2.i_wrong;
		flows
			etef1: end to end flow s1.src -> c1 -> s2.snk; -- should be instantiated
			etef1_wrong: end to end flow s1.src -> c1_wrong -> s2.snk; -- should never be instantiated
	end top.typeOnly;
	
	system implementation top.withImpl_implFlow
		subcomponents
			s1: system s1;
			s2: system s2.implFlow;
		connections
			c1: port s1.o -> s2.i;
			c1_wrong: port s1.o -> s2.i_wrong;
		flows
			etef1: end to end flow s1.src -> c1 -> s2.snk; -- should be instantiated
			etef1_wrong: end to end flow s1.src -> c1_wrong -> s2.snk; -- should never be instantiated
	end top.withImpl_implFlow;
	
	system implementation top.withImpl_implNoFlow
		subcomponents
			s1: system s1;
			s2: system s2.implNoFlow;
		connections
			c1: port s1.o -> s2.i;
			c1_wrong: port s1.o -> s2.i_wrong;
		flows
			etef1: end to end flow s1.src -> c1 -> s2.snk; -- should be instantiated, but currently is not
			etef1_wrong: end to end flow s1.src -> c1_wrong -> s2.snk; -- should never be instantiated
	end top.withImpl_implNoFlow;
end test;

[AaronGreenhouse]

New problems

Failures:
Issue1830Test.testForEndToEndFlow:62 expected:<0> but was:<1>
Issue567Test.issue567:69 In S_top_instance: Expected 1 end to end flow but found 0
Issue575Test.issue575:69 In S_top_instance: Expected 1 end to end flow but found 0

[AaronGreenhouse]

Need to check for refinement along the features.

Fixed that, which fixed 1830, but 567 and 575 are still broken. I haven't looked at them yet.

[AaronGreenhouse]

Problem with 567 and 575 is feature groups. Need to handle those differently.

[AaronGreenhouse]

Updated isValidContinuation() to deal with feature groups:

	boolean isValidContinuation(EndToEndFlowInstance etei, ConnectionInstance conni, FlowSpecification fspec) {
		/*
		 * Issue 2009: Check if the connection instance connects to the flow spec. Here we have a weird
		 * situation. If the subcomponent the flow spec is qualified by is only described by a component type, then
		 * connection end is going to match up with the beginning of the flow. That's fine. If the component
		 * has a classifier implementation AND the flow spec has a flow implementation, then everything will also
		 * fine: either the connection instance is correct and reaches the start of the flow implementation, or it
		 * is incorrect and doesn't. But we can also have the case that the subcomponent is described by a
		 * component implementation but the flow spec does not have a flow implementation. In this case we
		 * still may have the case the connection instance continues into the subcomponent and terminates at a
		 * subsubcomponent. But the flow spec that we have here will be at the edge of the original subcomponent.
		 * so the end connection instance will not match up with the start of the flow spec.
		 *
		 * So what we really need to do is walk backwards along the connections that make up the connection instance
		 * until we find one that connects to the flow because as wee the connection instance may "punch through" the
		 * subcomponent.
		 */
		final Context flowCxt = fspec.getInEnd().getContext();
		final Feature flowIn = fspec.getInEnd().getFeature();
		final List<Feature> flowInRefined = flowIn.getAllFeatureRefinements();
		final EList<ConnectionReference> connRefs = conni.getConnectionReferences();
		int idx = connRefs.size() - 1;
		boolean result = false;
		while (!result && idx >= 0) {
			final Connection conn = connRefs.get(idx).getConnection();
			final ConnectionEnd connEnd = conn.getDestination().getConnectionEnd();
			if (connEnd instanceof Feature) {
				final List<Feature> connEndRefined = ((Feature) connEnd).getAllFeatureRefinements();
				if (flowCxt instanceof FeatureGroup) {
					result = connEndRefined.contains(flowCxt);
				} else {
					result = flowInRefined.contains(connEnd) || connEndRefined.contains(flowIn);
				}
			}
			idx -= 1;
		}
		return result;
	}

This handles the cases where the connection ends at fg and the flow starts at fg or fg.i. It is not possible to have the case where the connection ends at fg.i and the flow starts at fg.i. This is because the actual case were are concerned about the code that gets us here is a connection between to peer components. In this case, the connection must be between to feature groups directly. Feature groups can only be taken apart or put together inside a component, either going down from a parent to child or up from a child to parent, respectively.