New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decided to start with tests for FileRes #17
Conversation
@witlessbird Cool... I guess I'll let you keep hacking at this until it's ready, but tests are a great thing! Cheers! Cheers! |
@purpleidea: do you know what "The following file headers are not properly formatted:" build failure for file_test.go is about? |
On Mon, Feb 22, 2016 at 5:27 PM, Dmitri Dolguikh notifications@github.com
Yeah, just put the exact comment boiler plate at the top in your file as is |
Also, updated directory examples to use 'kind' instead of 'res'
@ffrank: fixed formatting |
Hmm, according to Travis, something is still amiss. Also, fixing it in a new commit seems messy to me. Since this is not yet merged, it would be cleaner to |
On Sun, Apr 10, 2016 at 3:06 PM, Felix Frank notifications@github.com
I would work on just getting everything working and passing in however many |
gofmt your code for consistent and idiomatic formatting |
This should make Travis happy. |
I just noticed that this is passing... Here's a quick initial review if anyone is interested:
Ping me when ready and I'll do a more thorough review. |
Iirc we need to rewrite the directory syncing logic in order to shed the dependency. I have that on my plate, but was blocked for a couple of weeks. Hopeful to find some time soon. |
@ffrank: I can do the directory synching/copying logic if you don't have time atm... Let me know! |
I've started this weekend. It's quite a mouthful, but I'll keep at it for a few more days. Pretty sure I'm half way there already. '''Update:''' Just finished writing the first draft last night, and now pushed it to my branch. I will eventually open a PR agains @witlessbird's branch, but first there will be lots of smoke testing, debugging and cleanup. |
The code works now, in principle. However, through our in-person discussions, we found two remaining issues:
I'm working on adding the second. However, I now believe that the first is a non-issue, because
As a result, there is no way for us to hold on to watches that are no longer needed (except those on parent directories, which are not the issue here). I propose we just keep adding watches as needed (recursively in managed directories), and rely on their automatic removal (except for parent directories of course). Cool? |
But does it replace the old watch pointing to inode X, with a new one that now points to inode Y ? Does the fsnotify lib free memory on stale watches that aren't used anymore or do we need to explicitly remove them? Anyways, the idea is that the code looks logical and clean. Let's look at that and go from there. |
Since this has been stale for a while, and a friend was willing to help me through the tunnel vision of the recursive watcher, after I got that first part done, I was "bored" and hacked through an implementation of the CheckApply stuff too. Because of the inactivity here, I figured nobody would mind. I think the code is fairly clean, and while there is still work to do to add more features, I think it's easy to see where they fit in at this point. It is: If there are tests or any other useful code left in this PR, please rebase and send it in as a new PR. Also thank you so much for getting involved in the project. Cheers! PS: Please test and report any issues with the updated file resource. |
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
This is the 1st commit message: lang: Split FuncValue into FuncValue and SimpleFn Representing an MCL function value as a golang function from Value to Value was a mistake, it should be a function from Vertex to Vertex. Here is why this is a mistake: The output of a function like $f = fn(x) { Shell(Sprintf("seq %d", x)) } varies over time, while a single Value does not. Thus, code like Map($f, list(1, 2)) would first produce the value list("1", "1"), but then it would _not_ update to list("1", "2") when "seq 2" produces its second line. That's because with the mistaken design, when Map receives a new FuncValue or a new input list of N elements, Map calls the function from Value to Value N times and produces a single output list of N elements. Here is why the corrected design is better: Here's what happens with this new design when Map receives a new FuncValue or a new input list of N elements. First, Map constructs N item-input nodes, each of which extracts a different entry from the list. Then, Map calls the function from Vertex to Vertex N times, once for each item-input node, and thus obtain N item-output nodes. Finally, Map constructs an item-collecting node which constructs a list out of all of its inputs, and Map connects the N item-output nodes to the item-collecting node. This item-collecting node is the output of Map. The Vertex to Vertex function constructs and connects its own nodes; in this case, it constructs an Sprintf node and connects the item-input node to it, and then constructs a Shell node and connects the Sprintf node to it, and then returns the Shell node as the item-output node. The two Shell node in this sub-graph emit a first value "1", which propagates to the item-collecting node and causes it to output a first value list("1", "1"). Then, the second Shell node emits a second value "2", which propagates to the item-collecting node and causes it to output a second value list("1", "2"), as desired. Here is how this commit brings us closer to the above plan: Changing FuncValue throughout the codebase is a big change. One of the difficulties is that it is not just nodes which are emitting FuncValues, there are also many other places in the code where FuncValue is used to hold a golang function from Value to Value. Some of those places now need to hold a golang function from Vertex to Vertex, but other places still need to hold a golang function from Vertex to Vertex. Thus, as a first step, we need to split FuncValue into two types. This commit splits the old FuncValue into two types: 1. The new FuncValue will hold a function from Vertex to Vertex. FuncValue is a Value. 2. A new type named "SimpleFn" will hold a function from Value to Value. SimpleFn is not a Value. This commit replaces occurrences of the old FuncValue with one of those two new types, as appropriate. This commit does not yet adapt the surrounding code to make use of the new representation; that will be done in future commits. I have annotated the missing parts with the following panic message in order to make it easy to find which parts still need to be implemented. The "..." part explains what needs to be implemented. panic("TODO [SimpleFn]: ..."); Here's where I need help: One part of the code which is not clear to me are the parts which use reflection. I don't understand the purpose of that code well enough to explain what needs to be implemented. I have annotated those "known unknown" parts of the remaining work with the following panic message in order to make it easy to find which parts still need more thinking and planning: panic("TODO [SimpleFn] [Reflect]: ..."); This is the commit message #2: lang: Add the core func graph Txn API This will eventually let functions change the running graph via a transaction API. At the moment the core Lock and Unlock primitives aren't implemented. This is the commit message #3: lang: A reversible wrapper around Txn This is useful for the common case in which we call one FuncValue to construct a bunch of nodes, and later we switch to a different FuncValue and so we want to remove all the nodes added by the first FuncValue and replace them by the nodes added by the second FuncValue. This is the commit message #4: lang: move FuncValue to its own package This is the commit message #5: lang: combine lang/func/structs and ast Merging those two packages allows us to avoid import cycles when a Func needs to add an Expr to the graph. This is the commit message #6: lang: CallExpr must generate a subgraph FuncValues are now manipulating the graph instead of manipulating values, so the logic for calling a FuncValue must now follow suit. This is the commit message #7: lang: ExprFunc does not need to store its value This is the commit message #8: spelling This is the commit message #9: convert between SimpleFn, FuncValue, and Func This is the commit message #10: no need for ExprCall.argVertices it's the exact same thing as ExprCall.Args This is the commit message #11: ExprFunc.Func()'s three cases This is the commit message #12: FunctionFunc is no longer used This is the commit message #13: ExprBool.MergedGraph() This is the commit message #14: ExprIf.MergedGraph() This is the commit message #15: ExprFunc.MergedGraph() This is the commit message #16: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Expr interface. The txn type is interface{} until we have that merged. This is the commit message #17: lang: ast, interfaces: Add MergedGraph signature and implementation This adds a MergedGraph signature to the Stmt interface. The txn type is interface{} until we have that merged. This is the commit message #18: Use MergedGraph signature and implementation This puts it into play, but doesn't initialize the input args at all. This is the commit message #19: Restore ExprBool.Graph() and Func() This reverts commit 3ea3845. This is the commit message #20: Restore ExprIf.Graph() This reverts commit a62889e. This is the commit message #21: restore ExprFunc::Graph() and Func() This is the commit message #22: fix type errors This is the commit message #23: ExprCall.MergedGraph() This is the commit message #24: a more precise type for args This is the commit message #25: GraphTxn This is the commit message #26: ReversibleTxn.AddGraph This is the commit message #27: ExprFunc.MergedGraph() This is the commit message #28: ExprVar.MergedGraph() This is the commit message #29: sub-graph is spelled subgraph in this codebase This is the commit message #30: ExprFunc.mkFunc() was unused This is the commit message #31: CallFunc.Stream() should add Funcs to the graph, not Exprs This is the commit message #32: move ConstFunc to lang.funcs This is the commit message #33: move conversion functions to lang.funcs.simple I wrote some conversion functions from SimpleFn to FuncValue to Func, and I want to call them from Func.Stream() implementations, so those conversion functions should be somewhere in lang.funcs. This is the commit message #34: it is the responsibility of the function engine to call Init on the nodes This is the commit message #35: FuncValue.Call() This is the commit message #36: drop MergedGraph's unused txn parameter This is the commit message #37: move CallFunc to funcs.simple This is the commit message #38: Func from channel This is the commit message #39: ChannelBased{Source,Sink}Func This is the commit message #40: MapFunc This is the commit message #41: remove unused CallFunc field This is the commit message #42: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #43: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #44: extend environment with StmtProg's local variables This is the commit message #45: delete unused VarFunc This is the commit message #46: appease govet This is the commit message #47: add imported variables to the environment This is the commit message #48: add builtins to the environment This is the commit message #49: accumulate all the imported variables previously, we were only keeping the imported variables from the last import statement. This is the commit message #50: XXX: wip new function engine This is the commit message #51: comments This is the commit message #52: remove Engine.{Lock,Unlock} placeholders This is the commit message #53: f(...) support for ExprCall This is the commit message #54: [REVERT ME] remove unrelated files to make VS Code happy This is the commit message #55: don't recreate the subgraphInput This is the commit message #56: CallFunc now takes a single input, the FuncValue Also, by using the map_func logic, CallFunc now detects when it can no longer emits new values downstream. This is the commit message #57: GraphTxn must take a pointer in order to modify the graph This is the commit message #58: GraphTxn is unused
will add more tests as I go.