Is there a way to model "reversible" final states?

[AlCalzone]

I'm quite new to statecharts and this library, so I'm not sure if this is the best way to model what I'm trying to do.

Basically I need to model a situation where I'm waiting for a specific reply from an external source. Before sending this reply, the external source might (but not necessarily need to) request some information from me. I want to be able to respond to these requests, but only while waiting for the reply. And the compound state should not be done while either the reply is missing or we're currently responding. My first thought was using two parallel states:

• Parallel state 1: waitForReply
  • substate 1: waiting, transitions to replyReceived when receiving the reply
  • substate 2: replyReceived (final)
• Parallel state 2: respond
  • substate 1: idle (if nothing happens, this should be final, so the compound state is done as soon as the reply is received. If we need to respond to requests, this should back out of the final state into responding)
  • substate 2: responding, transitions to idle when done.

Is is possible to model this behavior using final states?

[Andarist]

Given your described requirements I've come up with this, quite simple, solution:

processing: {
    invoke: {
        id: 'external_service'
        src: externalSource,
    },
    states: {
        waiting: {
            on: {
                ASKED_FOR_DATA: {
                    actions: respond((ctx) => ({ type: 'DATA_RESPONSE', data: ctx.foo }))
                },
                RESPONSE_RECEIVED: 'done'
            }
        },
        done: {
            type: 'final'
        }
    }
}

[AlCalzone]

Thanks for the quick response, however I'm not 100% sure this solves my problem...
Since I'm breaking things down, I think I missed to mention a relevant detail.

The communication consists of several steps, some of which may be retried in case of failure. I have a separate state machine (let's call it API) for that.
All communication to the external source must be queued and not overlapped, so this machine may only run once at any given point in time. Therefore I need to make sure that the machine invocation is also done before leaving the processing state.

---

Maybe my entire approach to this is suboptimal. I'm currently struggling with this and have several sheets of paper full with ideas that don't quite cut it:
I have a queue of commands that need to be sent. Commands may be added to the queue at any time, but only one command invocation (API Machine) may be active at any given time.
When the send queue is idle, adding a command should immediately start sending it. After one command is done, the next (if there is any) should also be sent. This is easy - except for the hacky assign actions with side effects, but that was the only way I found where resolveCurrentTransaction/rejectCurrentTransaction would be executed before deleteCurrentTransaction:

const resolveCurrentTransaction = assign(
	(ctx, evt) => {
		ctx.currentTransaction.promise.resolve(evt.data.result);
		return ctx;
	},
);

const rejectCurrentTransaction = assign(
	(ctx, evt) => {
		ctx.currentTransaction.promise.reject(
			serialAPICommandErrorToZWaveError(evt.data.reason, evt.data.result),
		);
		return ctx;
	},
);

const setCurrentTransaction = assign((ctx) => ({
	...ctx,
	currentTransaction: ctx.queue.shift(),
}));

const deleteCurrentTransaction = assign((ctx) => ({
	...ctx,
	currentTransaction: undefined,
}));

const createSendThreadMachine = Machine(
		{
			id: "SendThread",
			initial: "idle",
			context: {
				queue: new SortedList(),
				// currentTransaction: undefined,
				...initialContext,
			},
			on: {
				add: {
					actions: [
						assign({
							queue: (ctx, evt) => {
								ctx.queue.add(evt.transaction);
								return ctx.queue;
							},
						}),
						send("trigger"),
					],
				},
			},
			states: {
				idle: {
					// There is no current transaction in idle state, so
					// reset it
					onEntry: deleteCurrentTransaction,
					on: {
						"": { cond: "queueNotEmpty", target: "execute" },
						trigger: "execute",
					},
				},
				execute: {
					// Use the first transaction in the queue as the current one
					onEntry: setCurrentTransaction,
					invoke: {
						id: "execute",
						src: "execute",
						autoForward: true,
						onDone: [
							// On success, resolve the current transaction
							{
								cond: "executeSuccessful",
								actions: resolveCurrentTransaction,
								target: "idle",
							},
							// On failure, reject it with a matching error
							{
								actions: rejectCurrentTransaction,
								target: "idle",
							},
						],
					},
				},
			},
		},
		{
			services: {
				execute: (ctx) =>
					createSerialAPICommandMachine(
						ctx.currentTransaction!.message,
						implementations,
					),
			},
			guards: {
				queueNotEmpty: (ctx) => ctx.queue.length > 0,
				executeSuccessful: (_, evt: any) => evt.data.type === "success",
			},
			delays: {},
		},
);

---

Now the complicated part is that there are two kinds of commands. The simple ones are handled by the machine above.
The second kind (lets call it sendData) adds additional steps:

• After the command was executed, zero or more replies may be received. Only when the final reply was received, the command queue may be resumed.
• It can happen that a sendData command expects a handshake command to be executed beforehand. This handshake is basically a sendData command with different contents, so this also expects a reply.
• If there was a failure or timeout while sending the actual command, a handshake or while waiting for the reply, the sendData command may be retried (including the handshake).
• If we sent the other party a handshake before the actual command, it will send us handshake commands before each reply. We must respond to those - again with a sendData command (but without handshake or waiting for a reply).

[AlCalzone]

I've played around in the visualizer and this does seem to work. However, I'm not sure if this is intended behavior that I can rely on.

  // Available variables:
  // - Machine
  // - interpret
  // - assign
  // - send
  // - sendParent
  // - spawn
  // - raise
  // - actions
  // - XState (all XState exports)
  
  function doStuff() {
    return new Promise(resolve => {
      setTimeout(resolve, 10000)
    })
  }
  
  const fetchMachine = Machine({
    id: 'fetch',
    initial: 'sendData',
    context: {
      retries: 0
    },
    states: {
      finished: {
        
      },
      sendData: {
        type: 'parallel',
        onDone: 'finished',
        states: {
          waiter: {
            initial: 'waiting',
            states: {
              waiting: {
                on: {
                  RESPOND: 'response'
                }
              },
              response: {
                type: 'final'
              },
            }
          },
          responder: {
            initial: 'idle',
            states: {
              idle: {
                type: 'final',
                on: {
                  DO: 'doing'
                }
              },
              doing: {
                invoke: {
                  src: doStuff,
                  onDone: 'idle'
                }
              }
            }
          },
        },
      }
    }
  });

[Andarist]

There is quite a lot to digest here - so I could have easily missed something, but let me answer to some parts of your problem and if it still won't be enough then we'll just continue iterating over this.

1. I would use raise('trigger') over send('trigger') in this case to cause immediate processing - raise puts an event into a higher priority event queue. In this particular example, this probably doesn't matter much, but I would stick to it as a rule of thumb.
2. hacky assign problem - we are aware of this issue and this behavior is not correct, but it was documented and we can't just change it as that would be a breaking change. The problem is that all assign actions take priority over other actions within a single transition. This has already been corrected on the next branch, but it will still take some time before we can release the final v5. You probably could figure out different workarounds for this, but it's probably also not worth it - given that every solution would be just merely a workaround for this.
3. are those 2 different types of commands really different? or is the simple type just a subset of a complex one in a sense that it just happens to have simpler requirements but could be handled by the "complex handler" just fine? Either way - you should be able to just add appropriate on handlers to your execute state to accommodate for bidirectional communication between it and the invoked machine.

[AlCalzone]

If you put it like this, the simple command is just a subset of the more complicated one, yes. I tried to reformulate the requirements and steps, hope its easier to grasp this way. x means always present, (x) means sometimes present, empty cell means never.

Step	Description	simple	sendData
1a	Handshake: Invoke API machine, wait until its done		(x)
1b	Handshake: Wait for specific reply to 1a (exactly 1)		(x)
2	Send command: Invoke API machine, wait until its done	x	x
3a	When requested, respond to handshake requests (zero or more). This means: Invoke API machine, wait until its done		(x)
3b	Wait for specific replies to 2 (zero or more, we can detect when we are done)		x
4	Send data done. This state may be entered when (3a never happened OR the API invocation is done) AND (3b is done)		x

Additionally, I need to retry the entire procedure in the sendData case in the following cases:

• 1a: API machine fails
• 1b: The reply was not received after a certain time
• 2: API machine fails
• 3a/3b: No reply and no handshake request was received after a certain time. Each request/reply should reset the timer

---

These two points currently cause me headaches:

1. As a programmer, I feel the urge to remove some redundancy here, since 1a -> 1b is basically a special case of 2 -> 3b. Does this even make sense in the statechart world?
2. Whats the best way to fulfill the requirement for entering step 4? I could use flags in the context (3a busy, 3b done) and add a guarded transition to both states, but that feels like the classic logic I'm trying to get rid of with statecharts. It seems to work with the machine I posted in Is there a way to model "reversible" final states? #1291 (comment), but I'm not sure if that is intended behavior I can rely on in the future