Support an $ACTOR shorthand in TDL test cases to easily refer to the SUT actor

[olw-medcom]

Hey @costas80,

I could use some guidance.

I have a test case where I want to simulate a server. The idea is that the SUT should only need to change its endpoint from the real-life endpoint to the mocked/simulated test bed endpoint, and then run as it normally would.

During the test, I expect the SUT to call this endpoint three times. These calls must happen in a specific order, as shown here:

The issue is that I currently have the steps placed in separate threads. This causes some steps to start out of order. For example, the “Send SBDH Ack” step is a SimulatedMessaging step, and it completes automatically. As a result, the diagram becomes confusing, and the intended messaging flow appears out of order.

One option would be to put everything into a single thread, like this pseudocode:

<receive id="1" from="SUT" to="simulatedServer">
  <input name="uri">"same_URI_All_Three_Requests"</input>
</receive>

<verify desc="verify the SUT request 1"/>

<send desc="Simulate the messaging flow" from="SUT" to="SomeOtherActor" handler="SimulatedMessaging"/>

<receive id="2" from="SUT" to="simulatedServer">
  <input name="uri">"same_URI_All_Three_Requests"</input>
</receive>

<send desc="Simulate the messaging flow" from="SUT" to="SomeOtherActor" handler="SimulatedMessaging"/>

<verify desc="verify the SUT request 2"/>

<receive id="3" from="SUT" to="simulatedServer">
  <input name="uri">"same_URI_All_Three_Requests"</input>
</receive>

<verify desc="verify the SUT request 3"/>

However, I am concerned that the SUT and the test engine could get out of sync. For example, while the second receive step is being started, the SUT may already have sent the second request.

What is the best way to handle this?

I was considering building some kind of custom server with a queueing mechanism, which would hold the incoming requests and forward them to the ITB once it is ready to receive them. However, that feels like quite a lot of work and complexity, so I was hoping there might be a simpler approach.

[costas80]

Hi @olw-medcom.

You've touched upon one of the key challenges in test cases that involve multi-message exchanges. The key pitfall of tests involving multiple receive steps is precisely the race condition you describe, where the SUT may automatically send messages before the ITB knows it should be waiting for them. You may have this come up during the test flow, or even with the SUT sending messages before the test session starts (exceptional but still one that comes up).

This is what our complex test development guide deals with in the test case relevant to receiving messages. In a nutshell the approach is as follows:

• Instead of using the HttpMessagingV2 handler in your receive steps use a custom messaging service. This will have its receive operation called when you expect to receive a message.
• Besides the messaging service API (used to interact with the ITB) this will implement a REST (or whatever) API that your SUTs expect to call, as per your message exchange specs. This will effectively replace the listen API/endpoint of the HttpMessagingV2 handler.
• To address the race condition, this service will record two sets of state: (a) ongoing receive steps in test sessions and the conditions to consider them satisfied, and (b) received calls from SUTs and their characteristics that be used to match them against receive step expectations (you weren't far off with your queuing mechanism thought).
• When you receive a receive call from the ITB, check your received SUT messages for an existing match, otherwise record as pending.
• When you receive a call from a SUT, check your pending receive steps for an existing match, otherwise record as buffered.
• When any match is found, build the report you need and notify the ITB. In doing so you mention the session ID (to identify the test session) and call ID (to identify the receive step) to make the match.
• You can also add other cleanup actions depending on what you're going for. For example when a test session is finalised, the service's finalize operation is called giving you a chance to check if there are pending test sessions and, if not, empty your buffered SUT messages.

This approach is followed by all projects I am aware of that need to handle messaging that could cause race conditions. It's obviously more complex that a HttpMessagingV2 handler but the effort pays off. In addition, the design and working sample code are available in the guide (and its related sample repo). Finally, note also (in case useful):

• If you have an existing app with a processing or validation service API, just add the messaging service API to it rather than creating something separate.
• We provide a service template to generate a working messaging service for you. If you have an existing app, you could use this template to create a messaging implementation in a separate project and copy the result to your existing app (i.e., the template will not add code to an existing app).
• As you are using the nightly, you may prefer to implement this extension as a REST service (rather than SOAP) which will be available in the next release. For the time being, see Support for FHIR validation (and use of FHIR-related utilities) in GITB TDL test cases #199 for related documentation on the REST API for test extensions.

Final word on this:

However, that feels like quite a lot of work and complexity, so I was hoping there might be a simpler approach.

I agree with you, but following the approach I mentioned and guide you should find some good help and ready-to-use code for this. Unfortunately managing race conditions like this is too complex and varied to be handled directly by the test engine (at least currently). In any case, if you feel stuck in any step feel free to reach out - we can also have a live call / screen share / whatever to help and go into details.

[olw-medcom]

This was exactly what I was looking for. Thank you @costas80, I will give this a shot in the next few upcoming weeks!

we can also have a live call / screen share / whatever to help and go into details.

Thank you costas, amazing support as always. I think I am able to figure it out, but I appreciate it!

[olw-medcom]

Hey @costas80, if you have the free time I would like to run a more general question by you. This doesn't really concern the ITB, but more general test considerations.

I have successfully made the implementation, but it made me think about how much knowledge testcase developers need to have about the system under test (SUT).

When we mock an actor and have actual messages flowing back and forth, we also need to know quite a lot about the technical transport layer. For example:

• the exact communication protocol, such as SOAP, HTTP, AS4, etc.
• the expected input and output formats
• headers
• authentication
• endpoint configuration
• protocol-specific details

I have not been at MedCom for very long, but my impression so far is that these kinds of technical details would be difficult for many of my colleagues to include directly in their testcases. Historically, MedCom has mainly focused on the data payload itself, rather than the technical details around transporting that data.

So I was wondering whether it would be possible to move some of these transport-related concerns away from the testcase developers and instead let the SUT developers configure them.

For example, could we define actor-level configuration where the user chooses the transport protocol, such as SOAP, HTTP, or AS4? Depending on that choice, the user would then see additional configuration options relevant to that specific protocol.

Then, inside the ITB, we could perhaps use something like an XSLT transformation to convert a simplified MedCom-specific send/receive element into the actual ITB steps needed for the chosen protocol.

Allow me to write some pseudocode as a simplified example. I would adjust the gitb_tdl.xsd to allow a custom element such as this in a testcase.xml.

<testcase id="pseudocode-example-of-transportation"
.../>

<actors>
  <actor role="SUT" name="ParticipantA"/>
  <actor role="SIMULATED" name="ParticipantB"/>
</actors>

<steps>
  <CustomMedComSendElement id="step1Message" to="ParticipantA" from="ParticipantB">
    <input name="dataToSend">"bla bla"</input>
  </CustomMedComSendElement>
</steps>

Then, before uploading the testcase, this could be transformed into regular ITB TDL depending on the actor configuration. For example:

<testcase id="pseudocode-example-of-transportation"
.../>

<actors>
  <actor role="SUT" name="ParticipantA"/>
  <actor role="SIMULATED" name="ParticipantB"/>
</actors>

<steps>
  <if static="true">
    <cond>ParticipantB{step1MessageDataFormat} = 'SOAP'</cond>
    <then>
      <send handler="SoapMessagingV2">
        <input name="uri">ParticipantB{step1MessageURI}</input>
        <input name="action">ParticipantB{step1MessageSoapAction}</input>
        <input name="envelope">"bla bla"</input> <!-- Not entirely sure what an envelope in soap is, so bear with me -->
      </send>
    </then>
  </if>

  <if static="true">
    <cond>ParticipantB{step1MessageDataFormat} = 'HTTP'</cond>
    <then>
      <send handler="HTTPMessagingV2">
        <input name="uri">ParticipantB{step1MessageURI}</input>
        <input name="method">ParticipantB{step1MessageHTTPMethod}</input>
        <input name="body">"bla bla"</input>
      </send>
    </then>
  </if>

  <if static="true">
    <cond>ParticipantB{step1MessageDataFormat} = 'AS4'</cond>
    <then>
      <assign to="step1DomibusHeader">...</assign>
      <send handler="DomibusMessaging">
        <input name="backendAddress">$MedComsDomibusServerAddress</input>
        <input name="method">ParticipantB{step1MessageHTTPMethod}</input>
        <input name="payload">"bla bla"</input>
      </send>
    </then>
  </if>
</steps>

The goal would be to let testcase authors focus on the payload and the business/data validation, while the protocol-specific transport setup is handled through configuration and transformation logic.

Does this idea make sense from an ITB perspective? Or would this become unnecessarily complex and hard to maintain? Is there already something in the ITB that would support this kind of thing?

[costas80]

Hey @olw-medcom. Nice to see you're having these kind of reflections - it shows you're getting to the core of what you want to focus on.

I have successfully made the implementation ...

Great! Building a custom messaging extension and handling things like the race conditions we were discussing, is as complex as it gets. Your ITB kung fu is strong now. Out of curiosity, did you use the SOAP or (the new) REST API to build the extension?

When we mock an actor and have actual messages flowing back and forth, we also need to know quite a lot about the technical transport layer.

Indeed, and this is part of the power of the ITB. It is testing the actual SUT with real exchanges, not just describing what needs to happen. This however requires that you define and manage the details of the underlying exchanges.

.. my impression so far is that these kinds of technical details would be difficult for many of my colleagues to include directly in their testcases.

Fair point! Hiding technical details from developers can be done quite nicely with the ITB and the TDL. Typical approaches followed by most are to either hide such details within scriptlets or to hide them in a custom messaging service. My personal preference where you want to abstract things away is to use scriptlets. You (i.e., the more technical member of the team) implement the scriptlet to do the exchange depending on the configuration of the SUT. This can range from the HttpMessagingV2 handler for simple REST stuff, to the DomibusMessaging for eDelivery, to a completely custom implementation via external messaging service as you are doing now for the race condition thing we discussed. If you need total control and feel its simpler to manage things in code rather than TDL, use a custom messaging service if you prefer.

What you expose to other members of the team (who don't want/need to know the details) is an abstracted API in terms of inputs to the scriptlet: "Send message from A to B with certain characteristics". What happens within and how the sending actually takes place is not exposed. By the way, I like using scriptlets for stuff like this because even if you just wrap a simple send using the HttpMessagingV2 handler, you are hiding this from other test cases and can adapt it in a single location without affecting changes everywhere. It's a classic programming approach really.

Historically, MedCom has mainly focused on the data payload itself, rather than the technical details around transporting that data.

This depends per project. Most projects using the ITB define strict specs for both data and how the data is exchanged. This means that the focus is on both. In your case if you are relaxed in terms of how the data gets moved around then your approach on having this configurable with different sets of options, and hiding the specifics in a scriptlet would work fine.

For example, could we define actor-level configuration where the user chooses the transport protocol, such as SOAP, HTTP, or AS4? Depending on that choice, the user would then see additional configuration options relevant to that specific protocol.

Sure, you can already do this in the ITB. When defining configuration properties, be it at system or actor/statement level, you can define property dependencies. Based on what you describe you would define one required property called "API type" which has a set of preset values "HTTP, SOAP, AS4" as you suggest. Then you define sets of required/optional properties that depend on the specific values of the "API type" property and are enabled accordingly. For example when selecting "HTTP" you would enable properties such as "Endpoint address", whereas for AS4 you would have "Party identifier". From a UX perspective, the user will first select the API type in the configuration, and based on the selection will be presented with further properties to fill in. In fact such dynamic configuration forms is largely why you have the option of previewing the options in the configuration screen.

Then, inside the ITB, we could perhaps use something like an XSLT transformation to convert a simplified MedCom-specific send/receive element into the actual ITB steps needed for the chosen protocol.

Well something like this is not possible, but you can enable/disable blocks of steps depending on the selected configuration values. You can already do this as you well know, and can hide any complexity in doing so within a scriptlet. Your test cases will just see a call to a sendMessage scriptlet without being inundated with configuration and implementation details.

Allow me to write some pseudocode as a simplified example. I would adjust the gitb_tdl.xsd to allow a custom element such as this in a testcase.xml.

I don't see the need for a custom step such as the CustomMedComSendElement. The TDL's send is already an abstraction as it can involve any implementation you like (via handlers and external services). Moreover, if you follow the scriptlet abstraction I explained previously you get to something very similar where other testers would call the send scriptlet rather than add a custom element. The result is the same: implementation details and handling the configuration is not exposed.

The goal would be to let testcase authors focus on the payload and the business/data validation, while the protocol-specific transport setup is handled through configuration and transformation logic.

Great! Follow the scriptlet approach I mentioned and I believe you will have exactly that. You wont have custom transformations but all your conditional logic is hidden within scriptlets.

Does this idea make sense from an ITB perspective? Or would this become unnecessarily complex and hard to maintain? Is there already something in the ITB that would support this kind of thing?

Yes, it makes perfect sense. In fact I see it as a general best practice to use scriptlets to construct your testing vocabulary (even for other stuff like validations). Your scriptlets are your building blocks that you can manage centrally and hide details when not interesting. Most large projects I'm aware of use scriptlets in this way (as a means of encapsulation, centralisation and abstraction). In terms of the "something in the ITB" to support this the things to keep in mind are:

• Use of scriptlets for encapsulation and conditional steps based on your config.
• Use configuration property preset values and dependencies to define the alternate configuration blocks. You can do this both through the UI as well as directly in test suite actor definitions (for actor-level properties).

How does this sound?

[olw-medcom]

Your ITB kung fu is strong now. Out of curiosity, did you use the SOAP or (the new) REST API to build the extension

Thank you sensei 🧙‍♂️😆
I used the same approach as in the sample repo which I am guessing is the old SOAP approach. Question: It's the MessagingServiceImplementation that would change right?

How does this sound?

This sounds almost good, however even with Scriptlets I could imagine it being a big too complex because the testcase author will have to reference the actor configuration naming as that will change from testcase to testcase.
I've actually done something before where I use .xslt transformations before I upload it to the ITB, so I might just continue doing that, but regardless, whether I use .xslt or scriptlets it gives me confidence that you think this feasible! Thank you!

[costas80]

Question: It's the MessagingServiceImplementation that would change right?

Yes, that would become a simple REST endpoint. Plus you don't need any of the extra SOAP stuff (configurations and dependencies). Anyway the result is the same so no worries on this.

I could imagine it being a big too complex because the testcase author will have to reference the actor configuration naming as that will change from testcase to testcase.

Not necessarily. You can look up all configuration in your scriptlet where you hide the complexity in terms of conditional parts and the steps you enable/disable. No need to reference that from the test cases.

Why would the names of configuration values change between test cases? If these are set at e.g. system-level they apply to all test cases and are referenced consistently (e.g., $SYSTEM{apiType}).

I've actually done something before where I use .xslt transformations before I upload it to the ITB.

Several projects I know of do a pre-processing step in which they use the tooling most suitable for them to generate the TDL test cases (FreeMarker, Velocity, ...). This is typically done when projects need to create numerous test cases that are very similar in structure. I'm not sure how such an approach would work in your case as you want to adapt test cases on the fly based on user configuration (not something you know when you're developing your test cases). I think the simplest for you would be the scriptlet approach I described.

I feel your hesitancy is due to "the testcase author will have to reference the actor configuration naming as that will change from testcase to testcase". Care to elaborate why you can't check configs within the scriptlet as I suggested?

[olw-medcom]

Care to elaborate why you can't check configs within the scriptlet as I suggested?

Gladly!
I will do it with an example (pseudocode):

Some testcase:

<testcase>

<actors>
  <!-- Just for the example the actor endpoint configuration is here in the testcase rather than in the testSuite.. -->
  <actor id="simulatedSBDHServer" role="SIMULATED">
        <endpoint name="systemInfo">
          <config name="Send_SBDH_URI"/>
          <config name="Send_SBDH_DataFormat" allowedValues="HTTP,SOAP,AS4"/>
          <config name="Send_SBDH_HTTP_Method" dependsOn="Send_SBDH_DataFormat" dependsOnValue="HTTP"/>
          <config name="Send_SBDH_SOAP_Action" dependsOn="Send_SBDH_DataFormat" dependsOnValue="SOAP"/>
          <config name="Send_SBDH_AS4_PartyIdentifier" dependsOn="Send_SBDH_DataFormat" dependsOnValue="AS4"/>
          <!-- More configs -->
        </endpoint>
  </actor>
  <actor id="SUT" role="SUT"/>
</actors>

<steps>

  <call path="SendMessageScriptlet.xml">
    <input name="from">"simulatedSBDHServer"/>
    <input name="to">"SUT"/>
  </call>

</steps>
<testcase/>

Then if I want to write the ReceiveMessageScriptlet.xml how would I get access to the config names?

<scriptlet>

<params>
  <var name="from" type="string"/>
  <var name="to" type="string"/>
</params>

<if static="true">
    <!-- Here I would want to do $simulatedSBDHServer{Send_SBDH_DataFormat} = 'SOAP' -->
    <cond>From_Actor{Step_Data_Format} = 'SOAP'</cond>
    <then>
      <send handler="SoapMessagingV2">
        ...
      </send >
    </then>
  </if>
<scriptlet/>

So the point I'm trying to make is, since the id of the actor and of the endpoint configs will change from testcase to testcase... How can the scriptlet possibly know how to reference them? How would I write a scriptlet in this case where it would be able to achieve:
<!-- Here I would want to do $simulatedSBDHServer{Send_SBDH_DataFormat} = 'SOAP' -->

[costas80]

Thanks for sharing the example - now I see what your concern is.

The fact that you need to refer to an actor by name is actually some specification baggage we have from years ago. There is a reason for this but in 99.9% of the cases you always have one SUT actor with one configuration endpoint, and the configuration properties you refer to from it are defined and treated as configuration properties from e.g. system or organisation level (it has to do with dynamically generated configuration values from messaging services depending on the actor and endpoint - niche stuff barely used).

To address this concern I would suggest introducing a new predefined map in the test session context named ACTOR which represents the SUT actor and its single config endpoint. Imagine that based on your example, using $ACTOR{Send_SBDH_DataFormat} would be effectively the same as $simulatedSBDHServer{systemInfo}{Send_SBDH_DataFormat} (assuming that simulatedSBDHServer is defined as your test case's SUT actor). This means that you may have different actors, but as long as these have a single endpoint definition and a common configuration property within named e.g. myProperty you will always be able to refer to it as $ACTOR{myProperty}. To be honest, this should have been done ages ago but has been overlooked because most people use system-level properties.

This would resolve your issue quite nicely no?

As an additional comment you should consider whether the API type and connection details are indeed specific to individual conformance statements. If your users are expected to test against multiple specs but should only define once their API preference, then this is best captured as a system-level property (meaning that you could already today do $SYSTEM{apiType} and access related properties as applicable). If it is meaningful to use different API types depending on the conformance statement, then indeed the actor-level properties are the right level of granularity, in which case my suggestion for the $ACTOR alias would work wonders no?

What do you think @olw-medcom?

P.S: If you agree, then with the $ACTOR approach, would you mind creating a separate issue regarding this? I wouldn't want to rename the current one as this was initially related to another topic. Thanks!

[olw-medcom]

Hmm, well that solution would help a bit, but still not solve the issue of the configurations themselves also changing names. Some testcases might have many many send and receive, and each should be configurable by the end-user, so I don't see anyway to always use the same configuration ID's.

In one testcase the endpoint configuration parameter might be called as in the example 'Send_SBDH_DataFormat' while in another it might be called 'Send_Order1_DataFormat'.

[costas80]

In one testcase the endpoint configuration parameter might be called as in the example 'Send_SBDH_DataFormat' while in another it might be called 'Send_Order1_DataFormat'.

Hm, you could then define different scriptlets I guess. How many such different sets of format types do you have? If 2-3 (each with their specific sets of properties and relevant steps) you could use the appropriate scriptlet in each test case.

Note also, in case it helps, that instead of $ACTOR{something} you can also do $ACTOR{$something} where in the latter case the something key is also dynamically determined.

[olw-medcom]

Multiple scriptlets won't work for us, as it's too much work compared to just making an .xslt preprocessing step with out own custom MedCom tags that is then converted to ITB tags before upload to the ITB.
The only way that I see scriptlets working for us would be if it was possible to do something like this:

<scriptlet>

<params>
  <var name="stepName" type="string"/>
  <var name="from" type="string"/>
  <var name="to" type="string"/>
</params>

</scriptlet>

<steps>
  <assign to="Step_DataFormat_Variable_Name">$stepName || "_DataFormat"</assign>
  <if static="true">
    <cond>$ACTOR{$Step_DataFormat_Variable_Name} = 'SOAP'</cond>
    <then>
      <send handler="SoapMessagingV2" from="$from" to="$to">
        ...
      </send >
    </then>
  </if>

</steps>

With the testcase like this:

<testcase>

<actors>
  <!-- Just for the example the actor endpoint configuration is here in the testcase rather than in the testSuite.. -->
  <actor id="simulatedSBDHServer" role="SIMULATED">
        <endpoint name="systemInfo">
          <config name="Send_SBDH_URI"/>
          <config name="Send_SBDH_DataFormat" allowedValues="HTTP,SOAP,AS4"/>
          <config name="Send_SBDH_HTTP_Method" dependsOn="Send_SBDH_DataFormat" dependsOnValue="HTTP"/>
          <config name="Send_SBDH_SOAP_Action" dependsOn="Send_SBDH_DataFormat" dependsOnValue="SOAP"/>
          <config name="Send_SBDH_AS4_PartyIdentifier" dependsOn="Send_SBDH_DataFormat" dependsOnValue="AS4"/>
          <!-- More configs -->
        </endpoint>
  </actor>
  <actor id="SUT" role="SUT"/>
</actors>

<steps>

  <call path="SendMessageScriptlet.xml">
    <input name="stepName">"Send_SBDH"</input>
    <input name="from">"simulatedSBDHServer"/>
    <input name="to">"SUT"/>
  </call>

</steps>
<testcase/>

Is that possible?

[costas80]

The only way that I see scriptlets working for us would be if it was possible to do something like this ...

The issue in the example you mention is the...

  <assign to="Step_DataFormat_Variable_Name">$stepName || "_DataFormat"</assign>

... given that you want to include this variable in a "static" if condition. The problem is that the assign step is a runtime step so you can't use it as a precursor for a load-time check. Moreover, the existing option of using such an expression in a call step's inputs does not achieve what you are going for (test developers would need to define multiple inputs, exposing the details you are trying to hide).

If you are comfortable following the pre-processing approach you mentioned (using XSLT or something else such as FreeMarker or Velocity), this would probably be the fastest way forward for you (and again, I confirm that multiple projects follow such an approach). In the meantime I will note on our end the improvement to be able to refer to SUT actor properties via the $ACTOR shorthand, and will also put some thought into how this "load-time assign" could be supported.

I'll ping you when the $ACTOR shorthand is available in the nightly build (here or in another issue if you want to create one).

P.S: Besides the $ACTOR point, feel free to comment in case you feel something additional should be done here.

[olw-medcom]

Sounds good @costas80, I will go ahead with the XSLT until perhaps some day load-time assignments are possible ^_^

Yes, if you don't mind feel free to use this issue for pinging about the actor shorthand. Should I rename the issue title so it's easier for you to find again?