Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize transaction state keeping #1832

Open
nkrkv opened this issue Jul 30, 2019 · 0 comments

Comments

@nkrkv
Copy link
Member

commented Jul 30, 2019

XOD 0.30.0 has introduced errors which significantly increased the amount of flash memory consumed. See the forum thread for details.

The problem is the pull algorithm used to propagate errors as opposed to the push algorithm used to propagate dirtiness. We can propagate errors using “push” to avoid much of repetitive code.

If implemented directly, the push algorithm would require an extra 1-byte flag hasUpstreamErrors per node, in every node_NN struct. It would cause massive RAM overhead.

So, we can put the flag inside the dirtyFlags union, as an extra bit. An automatic bonus would be hasUpstreamErrors clean up on transaction end. Storing the flag there should work fine until we get a node with 7 outputs. In this case, we require 9 bits for 7 output dirtienesses, node dirtiness, and upstream error flag. 9 bits is more than DirtyFlags type defined as uint8_t can serve.

But! The 8-bit limitation comes from the era before the loop unroll was done, when we had to choose a single type to represent all nodes. So, we can replace DirtyFlags type in the Node definition with an adaptive type: uint8_t, uint16_t, or uint32_t depending on output count.

This should work, but we can get even further. Most of the nodes have 1-2-3 outputs, thus more than half of the RAM dedicated to the flags is wasted. We can put all the transient state flags in a single super-union struct:

struct {
  bool node_1_isNodeDirty : 1;
  bool node_1_isOutputDirty_OUT : 1;
  bool node_2_isNodeDirty : 1;
  bool node_2_hasUpstreamError : 1;
  ...
  bool node_155_isNodeDirty : 1;
  ...
} transaction;

void runTransaction() {
  ...
  node_56_isNodeDirty |= isTimedOut(node_56);
  node_57_isNodeDirty |= isTimedOut(node_57);
  node_59_isNodeDirty |= isTimedOut(node_59);
  ...
  {
    if (node_155_hasUpstreamError) {
      node_63_hasUpstreamError = true;
      node_67_hasUpstreamError = true;
    } else if (node_155_isNodeDirty) {
      // evaluate
      // propagate dirtiness
      // propagate raised errors (non-recursively)
      ...
    }    
  }
  ...
  memset(&transaction, 0, sizeof(transaction));
  ...
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.