SpidDiscovery

It's becoming clear that life could be a lot easier if we allowed the Solver to know about its spids in advance. Towards this, I'm implementing a process called "spid discovery". This will be done as follows:

• instead of setting its parms to solvable within the first iteration request, MeqSolver sends up a separate solvability/discovery request containing: * Set.State commands for solvable parms; * an Eval.Mode setting of DISCOVER_SPIDS.

• in response to the latter, MeqParms populate a "spid map" record in the rider of their results;

• the results go back up the tree, with intermediate nodes merging the spid maps from their children;

• MeqSolver eventually receive complete information on all the spids it can expect. Pros of this scheme:

• MeqSolver can now trivially conserve memory by preallocating a matrix (since the spids are known in advance), and processing child results one by one.

• Information for TiledSolutions can be passed up as part of the "spid map".

• The "spid map" can contain other useful stuff, such as spid descriptions (making visualization of solver output a lot more informative). Cons:

• An extra request/result going up and down the tree prior to every solution. In terms of node overhead, this essentially amounts to an extra "empty" iteration. Given larger domains (as should be the case for TiledSolutions), this is not significant since we should be compute-dominated.

When does the solver do spid discovery?

The "brute-force" way is to do spid discovery prior to every solution. If the overhead of this proves to be significant, we can go to a somewhat more elaborate scheme:

• The Result returned during spid discovery has a depmask like any other result.
• The depmask will indicate whether spid re-discovery is necessary when going from domain to domain (in case of, e.g., TiledSolutions, it will be, since different domains may be tiled differently).
• When the state of a MeqParm changes in a way that changes its spids (e.g., via the fiddler, or via a sequencer from another branch), the node has to notify its parents somehow. This is pretty much the same as when the caches are cleared "from child to parent", which we already do. I will probably extend this child-to-parent scheme to provide a more generic "notification" mechanism, allowing all sorts of small messages ("claer cache", "clear spid map") to go from child-to-parent.

What exactly is the Spid Map?

The spid map is a field in the Result record. The map itself is also a record, where the spid is the field id (since DMI::Records support numeric field ids efficiently), and the field value is itself a small spid definition record.

The Spid Definition Record

include:SpidDefinitionRecord