expose the last evaluated result as `it` in REPL #734

valyagolev · 2022-10-07T06:29:03Z

This depends on #733.

This partially addresses #304.

There's one essential commit with the functionality: 8666e55 . And then there's a small refactoring that I'm proposing, but since I'm unfamiliar with the codebase, I have no idea if it's useful or not.

xsebek

The code and even the refactoring seem reasonable to me.

I just have no idea how the it variable works in the code. I guess you grab the result from the finished REPL computation and set it in the base robot context?

src/Swarm/TUI/Controller.hs

byorgey · 2022-10-07T12:21:40Z

@valyagolev , since this has merge conflicts, and now that you have push access, it's probably easiest to just close this PR and create a new one from a branch in the swarm repo itself rather than from a branch in your fork. (I'll assume you know what that means and how to do it but feel free to ask for clarification/help if not.)

xsebek · 2022-10-07T12:58:42Z

@byorgey in this case the conflict is small enough and should be easy to resolve.

IIRC the only reason for creating branches in the swarm repo was the Restyled Action worked better that way or was there another reason?

valyagolev · 2022-10-07T13:17:38Z

It just needs a rebase :) The branch is the continuation of the one squash-merged. I will do the rebase when I wake up. Thank you for the welcome!

…

On Fri, Oct 7, 2022, 14:58 Ondřej Šebek ***@***.***> wrote: @byorgey <https://github.com/byorgey> in this case the conflict is small enough and should be easy to resolve. IIRC the only reason for creating branches in the swarm repo was the Restyled Action worked better that way or was there another reason? — Reply to this email directly, view it on GitHub <#734 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEVYS5O7ERPRVYNDUOVK3WCANAZANCNFSM6AAAAAAQ7JDAZM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

byorgey · 2022-10-07T13:25:35Z

Yes, I'm sure fixing the merge conflicts will be easy, I just meant if some "administrative" work (such as rebasing) has to happen anyway, it could be an opportunity to reopen a PR from a swarm branch. Not required, just a suggestion.

IIRC the only reason for creating branches in the swarm repo was the Restyled Action worked better that way or was there another reason?

That is one reason, but the other reason is that it makes collaboration easier. For example #669 was a very collaborative effort, and that is harder when PRs are opened from people's private forks where no one else can push. (Not that we are going to immediately start pushing commits to others' PRs! But as trust develops it becomes a possibility.)

valyagolev · 2022-10-08T00:21:10Z

I kept this branch & PR for this one to avoid duplicating the conversation. But next time I'll gladly push a new branch to the main repo directly. Thank you!

The conflict and the comments were resolved.

byorgey

This is looking nice! I just have a few comments about cosmetic issues.

We should probably also talk about the it variable in the tutorial, but we can make a separate issue for that.

src/Swarm/Game/Robot.hs

byorgey · 2022-10-08T01:39:04Z

src/Swarm/Language/Pipeline.hs

@@ -36,6 +37,11 @@ import Swarm.Language.Typecheck
 import Swarm.Language.Types
 import Witch

+-- | A value, or something like a value, along with its
+--   type and requirements
+data Processed val = Processed val Polytype Requirements


What types will end up being substituted for val? I think I saw Value and Maybe Value, will there be others?

I wonder if we can come up with a better name than Processed. To me, the name doesn't really help me guess what the definition is. I guess you named it in parallel with ProcessedTerm, but a ProcessedTerm actually results from processing a Term, whereas that's not the case for a Processed Value. (Though perhaps we should rename ProcessedTerm as well!) Also, seeing the type Processed made me expect that ProcessedTerm is now a type synonym for Processed Term or something like that, but it's not.

Decorated, perhaps? Typeful? Typed? I'm open to suggestions.

What types will end up being substituted for val? I think I saw Value and Maybe Value, will there be others?

I'd also not be too surprised if I saw there: () or Term; [Var] (e.g. as a list of suggestions for a typed hole)... It has a functorial feel to it. Obviously I'm not familiar enough with the codebase to have any idea if it's good or not. I also thought maybe there could be a way to have ProcessedTerm be part of this already, but I'm not sure.

I wonder if we can come up with a better name than Processed.

Completely agree – it's more of a placeholder name, I was hoping for a feedback.

Actually I'm wondering why is it that there's such a hard separation between Type and Requirements. I guess they "mean" different things, but are they actually ever separated from each other anyway? Doesn't their calculation follow similar paths? Wouldn't they be essentially a decoration over a type in other PLs?

So with that in mind, I do like Typed.

Maybe there could be a name for a generalised type like this, e.g... AssemblableType?

then we could have:

data AssemblableType = AssemblableType Requirements Polytype data Assemblable v = Assemblable AssemblableType v

or even:

data Assemblable v = AssemblableType Requirements Polytype v type AssemblableType = Assemblable ()

And yeah, whatever we decide about the meaning of this should determine which file it goes to, as well.

I like Typed, that sounds most natural to me. I might consider using a record though:

data Typed v = Typed { _value :: v, , _typed :: Polytype , _requires :: Requirements }

We use lenses a lot and this will be the result of ix/at so we might as well use lenses for this type too.

Btw. if you want to get feedback on something in PR, it is a good idea to insert some TODO comments so that we don't forget about it. Alternatively, you can make comments on the code here on GitHub or in your editor if you have some plugin for that. 😉

Actually I'm wondering why is it that there's such a hard separation between Type and Requirements. I guess they "mean" different things, but are they actually ever separated from each other anyway? Doesn't their calculation follow similar paths? Wouldn't they be essentially a decoration over a type in other PLs?

Yes, very good observation. Right now requirements are calculated separately, after typechecking, but actually there are several bugs related to the fact that it doesn't work quite right (#540, #394) and I want to redesign things so that requirements are part of types, as they should be (#231).

But let's not get too crazy with refactoring in this PR (what you have so far is fine). I'd rather merge something that works to give us it, and then contemplate further refactoring as a separate PR.

byorgey · 2022-10-08T01:41:07Z

src/Swarm/Game/Robot.hs

@@ -509,6 +513,28 @@ instance FromJSONE EntityMap TRobot where
    mkMachine Nothing = Out VUnit emptyStore []
    mkMachine (Just pt) = initMachine pt mempty emptyStore

+type instance Index RobotContext = Ctx.Var


I like the idea of adding Ixed and At instances for RobotContext, but I would tend to put them right next to the definition of RobotContext. Was there a particular reason for having them down here?

I'm just a bit lost in big files. Also I think the best placement for everything will get clearer if we clarify the specific types proposed here...

I'm also not very sure if I was right not to include the contents of the Store...

There's another way to approach this particular change, namely by changing RobotContext into a product of Ctx (Typed Value) (or whatever we agree on) and the Store. I do think the three Ctxs should go together somehow. Three different maps potentially mean three different sets of Vars: do we want them?

(Such a change will remove the need for those instances)

If you have any suggestions on how to split the Robot source file to be easier to orient yourself, feel free to chime in:

Split large Haskell modules #707 😉

I did not mention this file in the list because it is only over 500 loc, but it is still in our top 10. 😀

There's another way to approach this particular change, namely by changing RobotContext into a product of Ctx (Typed Value) (or whatever we agree on) and the Store.

Yes, we contemplated this previously, in #190 . We abandoned it because it got too complex. But maybe if we try again we can find a simpler way to do it. Though again, I think we should save that for a different PR.

src/Swarm/Game/Robot.hs

byorgey · 2022-10-08T01:46:15Z

src/Swarm/TUI/Controller.hs

+
+        startBaseProgram t@(ProcessedTerm _ (Module ty _) reqs _) =
+          (gameState . replStatus .~ REPLWorking (Processed Nothing ty reqs))
+            . (gameState . robotMap . ix 0 . machine .~ initMachine t (topCtx ^. defVals) (topCtx ^. defStore))


Love it, I knew this code could be simplified somehow.

byorgey · 2022-10-10T15:56:41Z

@valyagolev This is looking good so far. Let us know if you feel like you need more guidance to know how to proceed from here.

valyagolev · 2022-10-10T16:00:29Z

what's the verdict on Processed? I guess I could just rename it to Typed and extract to its own module for now

i will apply other suggested changes soon and merge. thanks!

xsebek · 2022-10-10T17:06:34Z

I think we agreed on Typed, so please do so 🙂

valyagolev · 2022-10-11T09:10:29Z

added 9ca4e46 to fix #304

byorgey

This doesn't seem to work quite right when executing commands. For example, if I type build {move} at the prompt, it says it1 := <r1> : robot, which makes sense. But then when I type it1 at the prompt, it says it has type cmd robot instead of robot. And if I try to actually use it1, it causes a fatal crash. We do have some special case behavior built in for things of cmd type entered at the REPL, and I suspect we're running into that somehow, though I haven't yet figured out what's going on exactly.

We should also reset the replNextValueIndex to 0 every time we start a new game (see the startGameWithSeed function in Swarm.TUI.Model).

src/Swarm/Language/Typed.hs

byorgey · 2022-10-11T10:46:32Z

src/Swarm/Game/State.hs

@@ -361,6 +365,10 @@ robotsInArea o d gs = map (rm IM.!) rids
  rl = gs ^. robotsByLocation
  rids = concatMap IS.elems $ getElemsInArea o d rl

+-- | The base robot, if it exists.
+baseRobot :: Traversal' GameState Robot
+baseRobot = robotMap . ix 0


Eventually I want to stop assuming that the base is always ID 0, and this refactoring is a great first step!

Co-authored-by: Brent Yorgey <byorgey@gmail.com>

valyagolev · 2022-10-11T13:51:57Z

This doesn't seem to work quite right when executing commands. For example, if I type build {move} at the prompt, it says it1 := : robot, which makes sense. But then when I type it1 at the prompt, it says it has type cmd robot instead of robot. And if I try to actually use it1, it causes a fatal crash. We do have some special case behavior built in for things of cmd type entered at the REPL, and I suspect we're running into that somehow, though I haven't yet figured out what's going on exactly.

Oh, this I didn't test. Yeah I will test a few weirder scenarios, maybe we'll need a test for those...

CONTRIBUTING.md

valyagolev · 2022-10-13T16:15:18Z

are we okay with this syntax?

xsebek · 2022-10-13T18:57:56Z

I think it0: int = 1 would look more like the def/let syntax we use currently, but I don't have a strong opinion.

valyagolev · 2022-10-15T09:58:35Z

We do have some special case behavior built in for things of cmd type entered at the REPL, and I suspect we're running into that somehow, though I haven't yet figured out what's going on exactly.

Ok, yes, now I see the problem and it requires making a decision.

Basically if I run a line that returns cmd a, REPL will run the line to get cmd a, then run the command to return a (e.g. a particular robot). So the question is: do we want to store the cmd a or a in it*?

If we store cmd a, we will recreate (e.g.) a robot every time we run it. Because one of the main use-cases is to avoid losing references to the robots, probably it's better to store a. cmd a is available through the repl history anyway, I suppose.

ghci stores a:

λ> print "hello" >> return 3
"hello"
3

λ> it
3

valyagolev · 2022-10-15T10:19:00Z

This looks better

byorgey · 2022-10-15T10:49:18Z

So the question is: do we want to store the cmd a or a in it*?

Yes, absolutely we want to store the returned a value, just like ghci does.

By the way, relatedly, the syntax

it4 : robot <- <r2>

looks strange, because <r2> itself does not have a cmd type, so it's not something you could bind with <-. It should just be displayed with an = sign like everything else. The fact that we executed a command to get that value, instead of just evaluating an expression, should be irrelevant to how we display the resulting value.

valyagolev · 2022-10-15T11:12:05Z

I still think it's good to mark it differently somehow, even if the particular syntax is questionable

byorgey · 2022-10-15T11:19:03Z

I still think it's good to mark it differently somehow, even if the particular syntax is questionable

Why? What difference does it make?

xsebek · 2022-10-15T12:59:05Z

@valyagolev I understand that it could be useful to make it different in order to more easily debug this new feature.

A technically correct syntax would be:

it0 = 0
it1 <- return 1

Honestly the debug features I previously introduced were never useful after merging. But it may just be because that they required recompiling to enable - it would be better if we could e.g. turn them on in the Main Menu.

I am fine with either way, but consider that making the result look different could be more confusing (GHCi does not do it either) and will require more code. 🤷

valyagolev · 2022-10-15T13:13:35Z

No, it's not really about debugging this feature, rather about avoiding loss of information in REPL. If I run `some_func`, I probably also want to see what exactly it returned, value-wise and type-wise ([there's][1] a very useful `:set +t` in ghci). Otherwise there's no visual distinction between two fundamentally different results. I'm sure the output syntax can be tweaked. That said, if the general opinion is still to hide the distinction between `cmd a` and `a` in repl, I'll remove the extra code that shows it. [1]: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#ghci-options

…

On Sat, Oct 15, 2022, 14:59 Ondřej Šebek ***@***.***> wrote: @valyagolev <https://github.com/valyagolev> I understand that it could be useful to make it different in order to more easily debug this new feature. A technically correct syntax would be: it0 = 0 it1 <- return 1 Honestly the debug features I previously introduced were never useful after merging. But it may just be because that they required recompiling to enable - it would be better if we could e.g. turn them on in the Main Menu. I am fine with either way, but consider that making the result look different could be more confusing (GHCi does not do it either) and will require more code. 🤷 — Reply to this email directly, view it on GitHub <#734 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEVYV3XN677UCACJA5NT3WDKTCHANCNFSM6AAAAAAQ7JDAZM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

xsebek · 2022-10-15T14:39:54Z

@valyagolev but there is no distinction between the values that you get:

> return 1
it0: int = 1
> 1
it1: int = 1
> it0 == it1
it3: bool = true

Sure the code at the REPL line has different type, but you can see that in the corner and from the perspective of later lines the results are the same. 🤔

If you have some other difference in mind, please give an example where it would have an impact. I might be missing something. 😅

byorgey · 2022-10-15T15:35:08Z

To offer a bit more perspective: there are essentially two different things you can do at the REPL. You can use it as a "comand center" where you enter instructions (i.e. things with a cmd type) to be executed. Or, you can use it as a "calculator" where you enter expressions to be evaluated. For convenience, both work by entering an expression at the prompt: we just process them differently depending on the type. Mostly, I think players will not even think about the distinction, though it could be useful to have some way to help them understand the difference. However, both modes result in a value, and once you get a value as output there is no longer any difference in how those values may be used. So I think displaying the result values with different syntax is not the right way to go about helping players understand the distinction between executing commands and evaluating expressions. I believe it is more likely to make them think there is some difference in how they can or should use the values later, which is not the case, so it would be more confusing than helpful.

byorgey · 2022-10-15T15:37:17Z

Also, I'm very open to thoughts/suggestions on whether the distinction between execution and evaluation could be confusing to players, and how we might make it less so.

valyagolev · 2022-10-16T08:12:27Z

I pushed a commit to remove the discussed syntax thing

byorgey

This looks great, thanks @valyagolev for your hard work on this!

Now we just need to implement #60 so we can actually see the previous it{n} bindings!

byorgey · 2022-10-16T21:02:19Z

@valyagolev in the future remember you can merge by adding the merge me label, then the Mergify bot will do everything (including making sure that the branch still passes all the tests after merging the current state of the main branch).

valyagolev · 2022-10-17T08:32:06Z

oh right, sorry, forgot about that

xsebek reviewed Oct 7, 2022

View reviewed changes

src/Swarm/TUI/Controller.hs Show resolved Hide resolved

valyagolev and others added 7 commits October 8, 2022 02:07

expose "it" variable to the repl context

dbd41cc

refactor game state to use Processed Value

07fc088

fix new uses of Processed

3b66d41

make Robot indexable by var name

6ee4e0e

simplify adding it

b54ebaa

drop some of the redundant stuff

7f7b1bd

Restyled by fourmolu

5e33d99

valyagolev force-pushed the vg/it-in-repl branch from 44c0aa1 to 5e33d99 Compare October 8, 2022 00:07

docs for topContext

8ab3e16

byorgey reviewed Oct 8, 2022

View reviewed changes

valyagolev added 6 commits October 11, 2022 09:15

Merge remote-tracking branch 'origin/main' into vg/it-in-repl

7a7645b

move the code around in src/Swarm/Game/Robot.hs

2585957

rename Processed to Typed

dd084e3

fixes + fourmolu

81ea2fd

fix build

ddbf7f7

another formatting change

33cff21

valyagolev requested review from byorgey and xsebek October 11, 2022 07:39

valyagolev added 2 commits October 11, 2022 11:07

set it{n} into the base robot context

9ca4e46

reformat

ffa3221

Restyled by fourmolu

e8a3801

byorgey linked an issue Oct 11, 2022 that may be closed by this pull request

Ability to access past REPL results #304

Closed

byorgey reviewed Oct 11, 2022

View reviewed changes

typo

34c87e8

Co-authored-by: Brent Yorgey <byorgey@gmail.com>

xsebek reviewed Oct 11, 2022

View reviewed changes

CONTRIBUTING.md Outdated Show resolved Hide resolved

Merge remote-tracking branch 'origin/main' into vg/it-in-repl

52c1b81

valyagolev added 3 commits October 15, 2022 12:12

handle cmd-types in repl correctly

255b8b6

style

4903449

drop "alternative impl" from topContext

cca77af

restart replNextValueIndex on new scenario

62cf881

silently ignore the cmd context in repl

95ffee4

byorgey approved these changes Oct 16, 2022

View reviewed changes

valyagolev merged commit 977e0ed into swarm-game:main Oct 16, 2022

valyagolev deleted the vg/it-in-repl branch October 16, 2022 20:21

expose the last evaluated result as it in REPL #734

expose the last evaluated result as it in REPL #734

Conversation

valyagolev commented Oct 7, 2022

xsebek left a comment

Choose a reason for hiding this comment

byorgey commented Oct 7, 2022 • edited Loading

xsebek commented Oct 7, 2022

valyagolev commented Oct 7, 2022 via email

byorgey commented Oct 7, 2022

valyagolev commented Oct 8, 2022

byorgey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xsebek Oct 8, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

byorgey Oct 10, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

byorgey commented Oct 10, 2022

valyagolev commented Oct 10, 2022

xsebek commented Oct 10, 2022

valyagolev commented Oct 11, 2022

byorgey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valyagolev commented Oct 11, 2022

valyagolev commented Oct 13, 2022 • edited Loading

xsebek commented Oct 13, 2022

valyagolev commented Oct 15, 2022

valyagolev commented Oct 15, 2022

byorgey commented Oct 15, 2022

valyagolev commented Oct 15, 2022

byorgey commented Oct 15, 2022

xsebek commented Oct 15, 2022

valyagolev commented Oct 15, 2022 via email

xsebek commented Oct 15, 2022

byorgey commented Oct 15, 2022

byorgey commented Oct 15, 2022

valyagolev commented Oct 16, 2022

byorgey left a comment

Choose a reason for hiding this comment

byorgey commented Oct 16, 2022

valyagolev commented Oct 17, 2022

expose the last evaluated result as `it` in REPL #734

expose the last evaluated result as `it` in REPL #734

byorgey commented Oct 7, 2022 •

edited

Loading

xsebek Oct 8, 2022 •

edited

Loading

byorgey Oct 10, 2022 •

edited

Loading

valyagolev commented Oct 13, 2022 •

edited

Loading