Skip to content

Commit

Permalink
XXX: This is a combination of 58 commits.
Browse files Browse the repository at this point in the history
This is the 1st commit message:

lang: Split FuncValue into FuncValue and SimpleFn

Representing an MCL function value as a golang function from Value to
Value was a mistake, it should be a function from Vertex to Vertex.

Here is why this is a mistake:

    The output of a function like

        $f = fn(x) {
          Shell(Sprintf("seq %d", x))
        }

    varies over time, while a single Value does not. Thus, code like

        Map($f, list(1, 2))

    would first produce the value list("1", "1"), but then it would
    _not_ update to list("1", "2") when "seq 2" produces its second
    line. That's because with the mistaken design, when Map receives a
    new FuncValue or a new input list of N elements, Map calls the
    function from Value to Value N times and produces a single output
    list of N elements.

Here is why the corrected design is better:

    Here's what happens with this new design when Map receives a new
    FuncValue or a new input list of N elements.

    First, Map constructs N item-input nodes, each of which extracts a
    different entry from the list. Then, Map calls the function from
    Vertex to Vertex N times, once for each item-input node, and thus
    obtain N item-output nodes. Finally, Map constructs an
    item-collecting node which constructs a list out of all of its
    inputs, and Map connects the N item-output nodes to the
    item-collecting node. This item-collecting node is the output of
    Map.

    The Vertex to Vertex function constructs and connects its own nodes;
    in this case, it constructs an Sprintf node and connects the
    item-input node to it, and then constructs a Shell node and connects
    the Sprintf node to it, and then returns the Shell node as the
    item-output node.

    The two Shell node in this sub-graph emit a first value "1", which
    propagates to the item-collecting node and causes it to output a
    first value list("1", "1"). Then, the second Shell node emits a
    second value "2", which propagates to the item-collecting node and
    causes it to output a second value list("1", "2"), as desired.

Here is how this commit brings us closer to the above plan:

    Changing FuncValue throughout the codebase is a big change. One of
    the difficulties is that it is not just nodes which are emitting
    FuncValues, there are also many other places in the code where
    FuncValue is used to hold a golang function from Value to Value.
    Some of those places now need to hold a golang function from Vertex
    to Vertex, but other places still need to hold a golang function
    from Vertex to Vertex.

    Thus, as a first step, we need to split FuncValue into two types.
    This commit splits the old FuncValue into two types:

    1. The new FuncValue will hold a function from Vertex to Vertex.
       FuncValue is a Value.
    2. A new type named "SimpleFn" will hold a function from Value to
       Value. SimpleFn is not a Value.

    This commit replaces occurrences of the old FuncValue with one of
    those two new types, as appropriate. This commit does not yet adapt
    the surrounding code to make use of the new representation; that
    will be done in future commits. I have annotated the missing parts
    with the following panic message in order to make it easy to find
    which parts still need to be implemented. The "..." part explains
    what needs to be implemented.

        panic("TODO [SimpleFn]: ...");

Here's where I need help:

    One part of the code which is not clear to me are the parts which
    use reflection. I don't understand the purpose of that code well
    enough to explain what needs to be implemented. I have annotated
    those "known unknown" parts of the remaining work with the following
    panic message in order to make it easy to find which parts still
    need more thinking and planning:

        panic("TODO [SimpleFn] [Reflect]: ...");

This is the commit message #2:

lang: Add the core func graph Txn API

This will eventually let functions change the running graph via a
transaction API.

At the moment the core Lock and Unlock primitives aren't implemented.

This is the commit message #3:

lang: A reversible wrapper around Txn

This is useful for the common case in which we call one FuncValue to
construct a bunch of nodes, and later we switch to a different FuncValue
and so we want to remove all the nodes added by the first FuncValue and
replace them by the nodes added by the second FuncValue.

This is the commit message #4:

lang: move FuncValue to its own package

This is the commit message #5:

lang: combine lang/func/structs and ast

Merging those two packages allows us to avoid import cycles when a
Func needs to add an Expr to the graph.

This is the commit message #6:

lang: CallExpr must generate a subgraph

FuncValues are now manipulating the graph instead of manipulating
values, so the logic for calling a FuncValue must now follow suit.

This is the commit message #7:

lang: ExprFunc does not need to store its value

This is the commit message #8:

spelling

This is the commit message #9:

convert between SimpleFn, FuncValue, and Func

This is the commit message #10:

no need for ExprCall.argVertices

it's the exact same thing as ExprCall.Args

This is the commit message #11:

ExprFunc.Func()'s three cases

This is the commit message #12:

FunctionFunc is no longer used

This is the commit message #13:

ExprBool.MergedGraph()

This is the commit message #14:

ExprIf.MergedGraph()

This is the commit message #15:

ExprFunc.MergedGraph()

This is the commit message #16:

lang: ast, interfaces: Add MergedGraph signature and implementation

This adds a MergedGraph signature to the Expr interface. The txn type is
interface{} until we have that merged.

This is the commit message #17:

lang: ast, interfaces: Add MergedGraph signature and implementation

This adds a MergedGraph signature to the Stmt interface. The txn type is
interface{} until we have that merged.

This is the commit message #18:

Use MergedGraph signature and implementation

This puts it into play, but doesn't initialize the input args at all.

This is the commit message #19:

Restore ExprBool.Graph() and Func()

This reverts commit 3ea3845.

This is the commit message #20:

Restore ExprIf.Graph()

This reverts commit a62889e.

This is the commit message #21:

restore ExprFunc::Graph() and Func()

This is the commit message #22:

fix type errors

This is the commit message #23:

ExprCall.MergedGraph()

This is the commit message #24:

a more precise type for args

This is the commit message #25:

GraphTxn

This is the commit message #26:

ReversibleTxn.AddGraph

This is the commit message #27:

ExprFunc.MergedGraph()

This is the commit message #28:

ExprVar.MergedGraph()

This is the commit message #29:

sub-graph is spelled subgraph in this codebase

This is the commit message #30:

ExprFunc.mkFunc() was unused

This is the commit message #31:

CallFunc.Stream() should add Funcs to the graph, not Exprs

This is the commit message #32:

move ConstFunc to lang.funcs

This is the commit message #33:

move conversion functions to lang.funcs.simple

I wrote some conversion functions from SimpleFn to FuncValue to Func,
and I want to call them from Func.Stream() implementations, so those
conversion functions should be somewhere in lang.funcs.

This is the commit message #34:

it is the responsibility of the function engine to call Init on the nodes

This is the commit message #35:

FuncValue.Call()

This is the commit message #36:

drop MergedGraph's unused txn parameter

This is the commit message #37:

move CallFunc to funcs.simple

This is the commit message #38:

Func from channel

This is the commit message #39:

ChannelBased{Source,Sink}Func

This is the commit message #40:

MapFunc

This is the commit message #41:

remove unused CallFunc field

This is the commit message #42:

[REVERT ME] remove unrelated files to make VS Code happy

This is the commit message #43:

[REVERT ME] remove unrelated files to make VS Code happy

This is the commit message #44:

extend environment with StmtProg's local variables

This is the commit message #45:

delete unused VarFunc

This is the commit message #46:

appease govet

This is the commit message #47:

add imported variables to the environment

This is the commit message #48:

add builtins to the environment

This is the commit message #49:

accumulate all the imported variables

previously, we were only keeping the imported variables from the last
import statement.

This is the commit message #50:

XXX: wip new function engine

This is the commit message #51:

comments

This is the commit message #52:

remove Engine.{Lock,Unlock} placeholders

This is the commit message #53:

f(...) support for ExprCall

This is the commit message #54:

[REVERT ME] remove unrelated files to make VS Code happy

This is the commit message #55:

don't recreate the subgraphInput

This is the commit message #56:

CallFunc now takes a single input, the FuncValue

Also, by using the map_func logic, CallFunc now detects when it can no
longer emits new values downstream.

This is the commit message #57:

GraphTxn must take a pointer in order to modify the graph

This is the commit message #58:

GraphTxn is unused
  • Loading branch information
gelisam authored and purpleidea committed Jun 1, 2023
1 parent 3237f4b commit adef73b
Show file tree
Hide file tree
Showing 58 changed files with 1,478 additions and 3,059 deletions.
107 changes: 0 additions & 107 deletions examples/dhcp_client/dhcp_client.go

This file was deleted.

265 changes: 0 additions & 265 deletions examples/lib/exec-send-recv.go

This file was deleted.

0 comments on commit adef73b

Please sign in to comment.