New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-node / Cloud FRP #85

Open
stevekrouse opened this Issue Oct 25, 2018 · 4 comments

Comments

Projects
None yet
1 participant
@stevekrouse
Copy link
Owner

stevekrouse commented Oct 25, 2018

This issue is where I'm going to collect work and thoughts on the work for multi-node or cloud FRP. Below are a few "journal entries" that helped inform this work:

@stevekrouse

This comment has been minimized.

Copy link
Owner

stevekrouse commented Oct 26, 2018

Cloud (or multi-node) FRP (originally here)

While my intention was to continue my research in the direction of prototype 4, I have been tantilized by the problem of how to extend the FRP abstraction to "the backend". In other words, apps with data stored somewhere "in the cloud".

To extend the simple counter application, simply make it a multi-computer counter, that aggregates all the counts to a button across all computers. As is often the case, once put in those words, the problem doesn't seem so hard. One approach is to have a "lift" operator that would allow us to transform a button's Event () into a MultiWindow [Event ()], upon which we could do various operations, such as merging all the event streams and counting the occurrences.

Or consider the much more complicated problem of a realtime multiplayer game like agar.io or slither.io. As I've learned the hard way, you cannot simply send the x- and y- positions of each player to each other player and update position accordingly. You must instead anticipate where each player is going to be based on their current position and velocity and project them there until you receive the next word on their position and velocity and then you can subtly nudge that player to where they actually are, and are headed.

How could I model such an arrangement without mentioning low level details such as sockets? I think the concept of perspective is relevant here. From a single player's perspective the x- and y- position, and velocity are all FRP Behaviors, defined at all points in time because their computation happens to quickly. However, those values for other players must be modeled as FRP Events, because we only get glimpses of them at discrete points in time. However we must on the screen display other player's positions as continuous behaviors so we must use the complicated logic to predict and update our perception of the other players to construct a Behavior for them out of our Event of them.

My thinking above is lead by my intuition of FRP, types, and what the abstractions allow. However I feel a desperate need to have a proper medium to help shape my thoughts to only go in proper ways. Potentially Haskell or PureScript could be such a medium. I am considering watching/reading Conal's advice on denotative design again. Maybe I need to come up with mathematical objects to model what I mean by a "multi-widow" or "multi-user" application or event stream. These are the question on my mind when I pick up this stream of thoughts...

@stevekrouse

This comment has been minimized.

Copy link
Owner

stevekrouse commented Oct 26, 2018

Shower denotational design (originally here)

Conal Elliot is the man. My god, I'm so excited to be able to grok a part of what he has to say! Yesterday I re-watched Denotational Design: from meanings to programs and was re-blown away. It's funny: whenever I recommend Conal to friends I have to caveat it with, "I need to watch his stuff multiple times before I get it." This was true of this talk as well.

So here's the denotational methodology as far as I can tell:

  1. Come up with abstract "uses" you want the library to be able to accomplish
  2. Come up with a sketch of an API
  3. Write down the relevant types
  4. Write down the μ (mu, meaning) of those types as mathematical objects (likely, functions)
  5. Write down the μ of each API function
  6. See if any of your types fit existing type classes (monoid, functor, applicative, monad, comonad, etc)

So I tried to follow it in the shower yesterday and I think it went pretty well!

img_20181023_185153

I had a few key insights:

  • There should be a way to turn Events a on one computer to something like Event (Event User a), where the outer event represents this computer's perception of the event happening and the inner event represents when the event actually happened.
  • Another key insight: the default behavior of FRP applications should be to persist data for forever unless programmed otherwise. The resetting of app's data on page reload is a wacky accident!
  • I was having trouble coming up with the semantics for a User... We want a way to authenticate someone and then we need a unique key to identify them. In terms of access for reading and writing, I imagine that logic should live (like everything else) with the relevant pieces of state.
@stevekrouse

This comment has been minimized.

Copy link
Owner

stevekrouse commented Oct 26, 2018

Denotation of users (shower 8/25/18)

  • creating a user is a very imperative notion.
  • much better if we say that a user merely is a pair of keys: user :: (public key, private key)
  • when we want to construct an stream that's tied with a user, we must merely sign :: User -> Stream a -> UserStream a the stream.
  • setting any user details (name, email address, photo) are merely signing data that claims those things for yourself
  • Reading seems less natural, but the idea is that when we want data from another computer, we need to lift a Stream to a Stream (Stream) of some sort (to account for when we get the stream as well as when it happens), and in this lifting we can optionally supply a User parameter. Pieces of state themselves will decide if and how they will allow themselves to be lifted, and to which users.

img_20181025_184131

stevekrouse pushed a commit that referenced this issue Oct 26, 2018

Steve Krouse
## Notes 10/25-26/18
* TOC
{: toc }

### Cleaning .git repo

I spent ~2 hours yesterday morning cleaning up my `.git` repo for this website. It's only fair considering how I abuse this repo so, but it was still quite frustrating. The .`git` repo was over 200MB! Eventually I learned that the issue was all the large `.mp4` and `.pdf` files I added a while ago and since removed but since lurked in the git info. So I removed them, and then accidentally pushed those changes to github. Now I'll have to [eventually fix those broken links](#82).

### The Next 700 PLs

Conal's talk from two days ago pointed me to The Next 700 Programming Languages by Peter Landin. It felt like very modern essay, not one written in 1965!  The beginning didn't speak to me but sections 8 and beyond really did. He coins the term "denotative language" to replace the "functional" vs "imperative" debate: a language with nested subexpressions, where each expression *denotes* something, it's dependent only on its subexpressions listed's values. This is a useful definition!

At the end of this article there's a discussion section where the author and other famous computer scientists discuss these issues, debating what is a declarative language. It's really amazing to see!

### Denotation of users (shower 8/25/18)
I want a place to consolidate all my work on this, so I've started [a github issue for it](#85)

* creating a user is a very imperitive notion. 
* much better if we say that a user merely *is* a pair of keys: `user :: (public key, private key)`
* when we want to construct an stream that's tied with a user, we must merely `sign :: User -> Stream a -> UserStream a` the stream.
* setting any user details (name, email address, photo) are merely signing data that claims those things for yourself
* Reading seems less natural, but the idea is that when we want data from another computer, we need to `lift` a `Stream` to a `Stream (Stream)` of some sort (to account for when we get the stream as well as when it happens), and in this lifting we can optionally supply a `User` parameter. Pieces of state themselves will decide if and how they will allow themselves to be lifted, and to which users.

![img_20181025_184131](https://user-images.githubusercontent.com/2288939/47558190-79bb0480-d90a-11e8-8c8e-3f69851d7354.jpg)


### Todos 10/26/18

* continue research with [prototype 4](https://futureofcoding.org/log#prototype-4) and/or [multi-node FRP](#85) 
* set SPLASH intentions
* Read all of [Kartik](http://akkartik.name/) & redo my /about page
* Organize FoC Thinking & FoC Research lists from Google Inbox --> github issues/project (including these todos)
* Read m fix paper, and understand how the circular monad works... and how I can simulate it in JS

stevekrouse pushed a commit that referenced this issue Nov 1, 2018

Steve Krouse
## Notes 10/31/18-11/1/18
* TOC
{: toc }

### More FRP talk practice!

[Yesterday's practice](https://www.useloom.com/share/d52de5de266b4862afa2777176f652bb) was pretty solid! I'm not sure if I'll have time to review it and do a better one today...

### Stephen Diehl lunch

This morning I poked around on the internet for tutorials related to creating "toy Haskells" particularly interpreted and not compiled. This is related to wanting to create a FP language with a much *livelier* feel, aka [prototype 4](https://futureofcoding.org/log#prototype-4) and sometimes referred to as viv, potluck, stride, etc. I came across Stephen Diehl's many writings on these topics and saw on his Twitter that he went to the London Haskell meetup last night, so I messaged him and we had lunch a few hours later! I love the internet.

He claimed that in order to design a functional language you need to settle on three things:

1. AST
2. Core calculi
3. Evaluation semantics

The code is parsed to the AST, the AST is reduced to something that resembles the core calculi (you can see this in ghci with D dump core), and then the evaluation semantics determine how things are evaluated.

#### Core calculi

* untyped lambda calculus (Scheme)
* System-F (Haskell)
* Martin Lof (Haskell + dependent types)

#### Evaluation semantics

1. Call by value (eval arguments first)
2. Call by need (lazy, like Haskell)
3. Call by push value (he didn't explain)

#### Random notes

* Apparently recursive-do is possible in Stephan's basic interpreter in his book (2k lines of Haskell) because it's just built on top of mfix, which is mutual recursion, and bind...
* He strongly reccomended I read Ben Pierce's book, at least chapters 1-11. One of his coworkers forewarned that it's more of a reference manual than a book to read cover to cover.
* He also recommended Robert Harper's _Practical Foundations_, which is about adding things a la carte to semantics.

### Splash plans

Dinner Thursday night with some folks. Message me if you want to join!

### Interesting cloud data projects

I found [edsu.org](https://edsu.org/) via [this HN post](https://news.ycombinator.com/item?id=18316116). Seems cool. Similar to a lot of other projects floating around the ether these days. The comments are full of them. Just wanted to jot it down here.

### Todos 11/1/18

* continue research with [prototype 4](https://futureofcoding.org/log#prototype-4) (including learning about the stuff from Stephan Diehl above) and/or [multi-node FRP](#85) 
* Read all of [Kartik](http://akkartik.name/) & chat with Antranig and Philip & redo my /about page
* Organize FoC Thinking & FoC Research lists from Google Inbox --> github issues/project (including these todos)
@stevekrouse

This comment has been minimized.

Copy link
Owner

stevekrouse commented Nov 14, 2018

  • At SPLASH, I was really excited to find ScalaLoci which was about placement types of reactive programming in Scala. Very related to this work. I need to peruse their Related Work section for more details.
  • I found a way to make sense of reading (which I was having trouble with above). All data is "public" but by default anything you do is encrypted so only you can read it. You can also encrypt it with others' private keys if you want them to have access. We'd probably want to make who can have access be a Behavior so that we can change this if we want to revoke access.
  • Eventually, the next step here will be implementing this, probably in Haskell

stevekrouse pushed a commit that referenced this issue Nov 14, 2018

Steve Krouse
## Mid Nov 2018 Many Thoughts
* TOC
{:toc}

### SPLASH 2018 recap

I normally don't like to travel but this trip was possibly the best thing I've had to travel for in my life. I'm very excited to do more things like this, maybe a couple times per year.

#### Internet friends IRL

I've been referring to this period of my life as my "Twitter Friends Phase" because I am making so many friends and then I also get to see them in person. Just in last week in Boston alone, here are all the amazing people I got to spend time with:

Will Chriton, Joel, Jonathan Edwards, Ravi, Brian, Cyrus, Glen, Sean McDirmid, Paul Chuisano, Geoffrey Litt, John Maloney, Charles Roberts, Chris Granger, Eyal Lotem and Yair Chuchem from Lamdu, Roben Kleene, Daniel Moon, the ReScala team from Technische Universität Darmstadt (Ragnar, Pascal), Josh Horowitz, Roly Perera, Caleb Helbling, Evan Czaplicki

Sadly I only have one photo from the event. I should do better next time!

![image](https://user-images.githubusercontent.com/2288939/48514489-683f8b00-e82c-11e8-91aa-6b54dd936273.png)

#### LIVE 2018

I have trouble with lectures. I'd much prefer sitting at home in my sweatpants, listening at 2x speed and bouncing if it's not for me right now. However, LIVE 2018 was one of the best days ever, not in spite but *because* it was a day full of mind-blowing demos one after another.

I recorded them on my phone and uploaded online to a lot of thank-you's. The Bootleg page had a fun run, including Jeremy Asheknas's transcript which landed on the front page of HN for the day (not when I posted it, but when he posted it a few hours later).

### Next Research

I really have two very interesting directions to go in: making FRP experience better, or expanding the FRP universe to multi-node. We choose FRP experience for now, but I'm thinking about the other thing on the side. 

#### p4 11/14/18

(Tracked at #86)

The goal is to submit this work to \<Programming\> in Feb and a good title for the PX workshop (says JE) is "FRP eXperience".

There are a few different levels here: 

* the abstraction / code
* the streams, including data and how they fit together
* the output UI

##### Infinite canvas drawings

![image](https://user-images.githubusercontent.com/2288939/48514496-6d9cd580-e82c-11e8-8bf4-1b4af9aef054.png)

##### Other tools for inspiration

* WYSIWYG hybrids such as Sketch-n-Sketch
* Aprt.us, which would be a more visual language. Key question is how to do higher-order & cyclic, and other abstractions
* Facebook Origami
* Lamdu/Luna/Dark and all the projectional editors
* principle - design tool with timeline editor
* framer

##### Worst parts of Reflex/Haskell

But then I thought: let's focus on the really key issue here. What's the highest-leverage improvement to be made? What's the *worst* part of using Reflex/Haskell now? In the shower note on the right below, it says:

1. feedback loop: knowing if the syntax is right, if the types make sense, what the code I'm looking at *is* from both a type, data, and UI perspective
2. knowing what's possible (syntax, API, types)

##### p4 prioritized features

1. With this framing, a quick compile time (or intepreter time) could be key. Reflex seems to take 4+ seconds. Can I do sub 1? Or faster? How about hot reloading?
2. Great docs, autocomplete, type annotations everywhere
3. Automatic plumbing:  can we semi-automatically lift transforms inside monadic contexts? Ditto for automatic type coersion plumbing like int to string, etc
4. Stream visualization, including how streams make other streams, the shape of streams, and also the data for each stream
5. (Bi-directional) Direct manipulation of UI

![image](https://user-images.githubusercontent.com/2288939/48357110-d0904000-e665-11e8-9154-3768ad3ce414.png)

##### Implementation

JE send me an email: 

> Can you implement your FRP eXperience using FRP, and apply it to itself? That might be challenging, but would also be very impressive. It will be seen as a limitation if it can't apply to itself, although that might be unavoidable in the first phase. But I think that at the very least you need to implement it in some FRP framework to avoid the charge of hypocrisy.

My response: The only implementation of the FRP I argue for in my paper is Reflex/ghcjs. It doesn't exist in PureScript, Elm, F#, etc. Turns out laziness makes it a lot easier to implement. My next experiment is to try to do it in JS with CycleJS. I've tried this in the past. It's not easy -- it's not built for this use case -- but it seems possible, even cycles. More to follow...

* I did a very naive implementation of streams stuff in JS here: https://codesandbox.io/s/81225lrop2
* It'd probably be better to build off cyclejs. Over a year ago I tried to do something similar so maybe I can start there? https://codesandbox.io/s/xlrynkqoqp Inspecting the types of streams is key. Maybe I can use the cyclejs devtools and/or augment them somehow. Could typescript or something similar help me encode the types of streams of streams of streams? Could monaco help with autocomplete suggestions?

Some other ideas for building this:

* racket turnstule dsl for building typed langauges
* L. Kats and E. Visser. The spoofax language workbench. 
* C. Omar. Reasonably Programmable Syntax. PhD thesis, Carnegie Mellon University, 2017.
* Build a JS interpreter with laziness based on [Stephen Diehl's work](https://futureofcoding.org/log#stephen-diehl-lunch)

#### Cloud FRP 11/14/18

(Tracked at #85)

* At SPLASH, I was really excited to find [ScalaLoci](https://scala-loci.github.io/) which was about placement types of reactive programming in Scala. Very related to this work. I need to peruse their Related Work section for more details.
* I found a way to make sense of reading (which I was having trouble with above). All data is "public" but by default anything you do is encrypted so only you can read it. You can also encrypt it with others' private keys if you want them to have access. We'd probably want to make who can have access be a `Behavior` so that we can change this if we want to revoke access.
* Eventually, the next step here will be implementing this, probably in Haskell

#### Name ideas 11/14/18

Now that I'm clear I am building a system, and not just producing research, I am thinking on a name. Here are some themes it could embody:

* malleability
* conviviality
* casual
* customizability
* cooperation
* denoational

And here are my favorite names of the moment:

1. Potluck
2. viv
3. Stride (probably can't do because there's already a similar-ish thing)

And here's what Twitter thinks:

<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Preferences on potential new programming language name..?</p>&mdash; Steven Krouse (@stevekrouse) <a href="https://twitter.com/stevekrouse/status/1061441377890942976?ref_src=twsrc%5Etfw">November 11, 2018</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

#### 2019 Conference deadlines

-   \<Programming\>: Salon de Refuge (Jan 7, April 1) Genoa, Italy
-   \<Programming\>: PX/19 (Feb 1, April 1) Genoa, Italy
-   PPIG (June-ish, Aug) UK
-   SPLASH: Onward! (mid-April, end-Oct) Athens, Greece
-   SPLASH: LIVE (June/July, end-Oct) Athens, Greece

### Possible Dec 2018 Re-group Projects

I got this project started a year ago as fast as possible, and it seems like now-ish is a good time to regroup a bit and work on a few upgrades... Maybe I'll set aside Dec in do some of these things...

#### Meetups more a thing

It seems like there's interest in hosting/attending these in lots of cities, and with a bit of effort (a website with guidelines) we can have them in a bunch of cities. I can do London and maybe New York, Caleb Helbing has been talking about starting for Boston, Amjad Masad (of Repl.it) says he can take SF...

#### Branding

* Amjad suggested that a cool logo could go a long way. I actually had a dream a night or two ago about making the logo a interesting color blue...
* Relatedly, the webiste styling could use an upgrade, a color scheme, top nav, footers, make it clear I accept pull requests on github, comments on pages
* The podcast could really use intro and outtro music
* Change the Slack URL to futureofcoding.slack.com, and make a doc explaining channels (maybe fix this up a bit) with descriptions and threading

#### Re-organize systems

* This log system is a bit janky (too much git integration, adding date to headers, hashtag based system could be nice for collecting for topics, todo system could be better than constant appending)
* Jekyll seems simple but is actually quite complex and I've found it to be a nightmare to get things the way I want. Maybe I switch to a different platform. Maybe I slowly roll my own static site generator. Maybe I do it on now.sh...?
* Especially now that Google Inbox is dying, I need to get my FoC research + FoC Thinking lists out and into another system
* Relatedly, Github Issues could use some love. Potentially I could move everything here, and then come up with a better input mechanism to github issues...

#### Sustainability

* I have had a number of people strongly encourage me to set up a Patreon so they can support my work. This is terribly exciting! Top priority.
* Here's a probably silly idea: a future of coding job board. A place where people in the community can connect with companies in the community. Maybe if it gets big enough, companies can sponsor it to get listed higher?

### Todos 11/14/18

* Dec regroup projects (above)
* p4 storyboards (maybe 2-3 totally different takes, shallow, brainstorming mode)
* schedule JE podcast in late Nov
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment