-
Notifications
You must be signed in to change notification settings - Fork 0
Interpreter control flow explained
Michael Haberler, 4/2014
This document is intended as explanatory background for the 'Interpreter API requirements' document, and should be read before the latter.
Interpreters in LinuxCNC are used to convert some language like RS274NGC into a sequence of machine primitives called canon(ical) commands. This document addresses issues of the current code base, and the current rs274ngc interpreter; however, the problems are not specific to that particular interpreter and relate to integration of any other interpreter or language.
To understand the problems laid out here, it may help to consider first that the initial design of the interpreter into - what was EMC back then - did not support readahead. Readahead is a feature where the interpreter may be ahead of the actual machine in terms of program execution (in fact the interpreter might be finished before the machine starts any movement). Readahead hence requires a queue of commands between the interpreter and the execution logic.
Readahead is needed for motion optimization - a good trajectory planner will look ahead what is next, and might choose to combine or otherwise optimize moves, for instance for higher speed, or to reduce jerk: trajectory planner optimization requires interpreter readahead.
However, readahead also introduces a new problem which is not present in a simpler 'read an instruction, move the machine, go from current machine position to next instruction' type architecture:
-
the semantics of RS274NGC requires that the current machine position is always known when executing a block, because moves are typically computed based on the current position
-
there are RS274NGC words which break position prediction: for instance, the tripping point of a probe run is not known until execution of that code. Such operations are called queue busters; see section 17-17.5 of the Remapping manual for a description of what happens.
-
therefore, such 'queue busters' need to be treated specially: when one is encountered, the interpreter needs to stop, and wait until that particular code is executed. It then can proceed to re-read the machine position, after which the endpoint of the next word can safely be computed.
Interpreters typically have an inner read-eval-print loop: you give the interpreter some input to work on, and the interpreter reads an expression or statement, evaluates it, and computes the output. This is repeated until the end of the input is reached; the process is driven by the inner loop in the interpreter code.
If the interpreter needs a value - for instance the current statement means 'read a value from the terminal into a variable, and continue once the user hit the return key', then the inner loop code would call upon this read function, which blocks until the read is complete.
That is true for any interpreter I know of (Python, Basic, Lisp, Tcl ..) with a single exception:
The RS274NGC interpreter has no inner loop. What it does have is read and execute methods which are expected to be called from using code on a line-by-line basis (this again goes back to paper tape time).
There is a number of interpreter return codes which read and execute may return to the caller:
#define INTERP_OK 0 // statement parsed and executed OK #define INTERP_EXIT 1 // interpreter exited for some reason #define INTERP_EXECUTE_FINISH 2 // interpreter needs to wait, see note #define INTERP_ENDFILE 3 // reached end of file #define INTERP_FILE_NOT_OPEN 4 // file not found #define INTERP_ERROR 5 // error in statement
The curious one is INTERP_EXECUTE_FINISH - it is returned by the execute method if the current block contains a queuebuster operation, i.e. the position of the machine after this block cannot be determined by the interpreter without outside help. This is at least true for the following operations: toolchange, probe, reading an input pin. The way this return code is dealt with is:
-
the using code stops calling read/execute
-
the using code monitors the queue of outstanding commands
-
once this queue is empty, the using code calls the interpreter sync method, which retrieves the current machine state (including position), and synchronizes interpreter state to match it.
-
the using code is expected to call execute() again, to finish the operation.
-
from here on using code continues to call read/execute in turn.
In terms of control flow, this means the driving code for the interpreter must be structured like so:
while (not end of input) {
read();
if (execute() == INTERP_EXECUTE_FINISH) { // a queuebuster was executed
// wait until all outstanding commands have completed
sync(); // synchronize the interpreter state with the current machine state
}
}
Note the interpreter cannot currently do this itself; code using the interpreter must do so.
The way how an interpreter is currently embedded in LinuxCNC is very cumbersome, with dire side effects:
-
the embedding and control flow goes back to punched paper tape time, when execution was single sequence of commands with no readahead of the interpreter, and it made sense back then. It is inadaequate when readahead is desired, and languages with control flow constructs are used (if-then-else, loops, subroutines).
-
it is next to impossible to integrate any other stock interpreter into LinuxCNC (like Python, go, APT..)
-
the severely convoluted control flow has historically been a rich source of bugs.
-
the embedding does not support any other purposes like driving the machine proper; for instance, generating graphics commands for tool path preview, or survey the output of canon commands.
-
the embedding supports a single interpreter instance only, making it very hard to switch context (for instance, pause the machine, switch to a new interpreter context, execute a few commands, then resume the previous context - commonly called 'MDI while paused')
-
the current control flow makes it next to impossible to make the interpreter interact with blocking UI, for instance to ask for parameters; if the interpreter blocks to ask for a parameter, task execution blocks completely. And the machine just stops.
-
the move from the 'paper tape abstraction' to an interpreter supporting control flow structures, and multiple source files has not been backed up by code to correlate source lines and commands under execution. The current identifier (the 'motion id') is just an integer which historically denoted a command or line number, but cannot carry source file information.
I have no explanation other than history, and since I was not around I can only conjecture.
It would be much easier to understand, integrate and debug if the interpreter were a thread, or process of its own, and follow the standard model of a read-eval-print loop. There is no good reason for the interpreter to stop and indicate 'somebody else must do something for me' like waiting for the motion queue to become empty, and 'somebody else please call sync() on my behalf'. Why cannot the interpreter do that iself? if it hits a queuebuster, it just calls a method which waits the queue to empty, and syncs as needed .
It seems to me the interpreter API was designed to be nonblocking because it was intended to run as a subroutine called from the task coordinator program. The design of task single threaded, and assumes all auxiliary code is nonblocking.
The issues above suggest a future revision drop this - in my opinion - design error, and switch to a simpler, API based interface between the task coordinator, and any interpreter. Not only would that make it much easier to integrate arbitrary interpreters, it would also make it easier to use different interpreter contexts as laid out above (e.g. MDI like paused). Also, making the interpreter interact externally, for instance with a UI to ask for parameters, would become possible.
Kent kindly provided the following links which might help understand the history of the LinuxCNC RS274NGC interpreter: