Skip to content

Latest commit

 

History

History
524 lines (407 loc) · 14.6 KB

TODO.org

File metadata and controls

524 lines (407 loc) · 14.6 KB
Raw

TODO.org

roadmap

Current: Hypermedia Client and VRSJMP Command Bar Client

Hypermedia Client - vrsjmp

  • [X] Runtime Pubsub
  • [X] Hypermedia Client Design
  • [X] VRSJMP Command Bar Client

Runtime Core v0.2

Goal: Build Core Runtime Features

  • [ ] Init File
  • [ ] File IO for Simple Storage
  • [ ] Command IO
  • [ ] Process Links and Supervisor
  • [ ] Rust Macros for Code Compression
  • [ ] Lyric Macros
  • [X] Parallel Development / Release Instances
  • [ ] Daemon Installation Hook

VRS Cloud - Replace Things / Are.na

Goal: Deploy VRS to the Cloud as a Rust Backend Service This is in service of desktop + web + mobile replacement for Things / Org Mode

See Also: Integration with paradigm service

Lispy DX

Goal: Replicate Emacs-like Development Workflow to build VRS software E.g. Emacs Eval, SLIME-like workflows for molding Software

runtime

Conn-less Process Spawning

Goal: Non-connection-backed spawn from kernel.rs

Purpose:

  • Spawn from Process / REPL
  • Spawn from Init File

Service Discovery / Mgmt

Goal: Replace existing pid 0 hardcoding w/ service mgmt Skip interface / imports map for this one - see also “Runtime Linking” Convenience APIs to see running services and some level of introspection

  • [X] register
  • [X] ls-srv
  • [X] find-srv

IPC - srv and bind_srv

Goal: Provide better wrappers over call

callee serves unique named-interface w/ a set of callable symbol IDs:

  • Runtime linker records: The PID of callee, and exported symbols

caller calls bind, which:

  • Requests service information from runtime linker
  • Runtime linker returns a set of SymbolId and Val to add into Env
  • Newly added bindings use send pid recv pid id to send + wait for result

Want svc for service and bind for callers See also - pattern matching / destructuring

Maybe also cover imports / exported interface and discoverability

# *** service-side
(srv :launcher
     :export '(get_favs, add_favs))

# *** client side (gets imports!)
(bind_srv :launcher)

PubSub API

  • Support pub/sub channels
  • Proof-of-Concept: Pub/Sub listening to “counter” variable on new task execution model
  • Reactivity of PubSub over PubSub
  • Method to access pubsub topic via vrsctl

Hypermedia Client

Goal: Build CLI Hypermedia Client over VRS See also - “Revisit Client Connection”

APIs to manipulate lyric S-Exp more effectively See also s-explib https://github.com/janestreet/sexp

See Also: CLI Client Shell

  • [ ] DOM API - Set Content of “Page”
  • [ ] DOM API - Get Element with ID

Supervisors and Links

Supervision Trees

Goal: Spawn Link to be notified of errors in spawn-ed processes

Purpose:

  • Service Mgmt on Startup (?)
  • Restarting on error

Dotfiles / Init Scripts

File IO for S-Expressions

Read / Write S-Expressions to File Goal: Replace hard-coded data in launcher.ll

Command IO - Better exec

Goal: Better interface over Command to access:

  • Process output
  • Exit code

Revisit Runtime / Client Boundary

Goal: Revisit protocol b/t/ Runtime / Client w/ New Learnings

Idea - Few standardized interactions?

  • REPL - text only (?)
  • Command-Bar / Browser (List, Search, Select, Navigate)

Learnings:

  • Instead of always sending Form, what client wants depends:
    • REPL always wants text - it’d rather have error message v.s. “cannot serialize” error
  • How to reconcile with Hypermedia API?

Controlling Terminal Mechanics for Connection

How to handle nesting? e.g. calling a yielding call in peval while it is already yielding Nested Yielding + Input routing for connection

Allows:

  • Currently running program as “url”
  • Interactions on that program are routed
  • “Interrupt” to exit to prompt, like Ctrl-c

Similar to read and input() working in shell + python respectively

Preemptive Multitasking

  • Start a process that infinite loops, (loop 5)
  • Show that other screen is still responsive

Universal Entities, Search, Indexing

Goal: Unified Interface to persistent storage Goal: Expose entities in runtime programs via core primitive

Plumber / Embark Mechanics for Entities

Define actions on entities to act on shape of data I.e. the hypermedia is generated based rules on entities it matches

“What can I do with this data” - like Emacs Embark Thought - Running embark-like mechanics is itself a cmd - i.e. “show me embark meno on this item”

General User IO

Give VRs capability to see (web pages, screen), Hear (user, text) and prompt for input (text, voice, selection)

Sqlite IO for S-Expressions

lyric lang

Add cond

Add cond macro to avoid pyramid of if.

Destructuring def

Pattern Matching / Destructuring

Goal: Provide way to extract params from messages w/o indexing

Add match

Add pattern matching conditional branching Test: Make sure that (a b a) only matches (1 2 1) but not (1 2 3)

[#A] BUG: Lambdas can be used to break process isolation

A process can send a lambda (which captures its environment) to another process to break process isolation

Repro Steps

  • Define variable in process A, count, and a function inc updating that variable
  • Send the inc lambda to another process B via send
  • Call inc from process B

Issue: Above can be used to update process A from process B without message passing

Not an issue in erlang / elixir due to pure functional types. No values mutating via references captured.

Thought: Allow but discourage passing lambdas

Better Escape Characters

Currently, escaping backslash is hacked together - see lexer for strings

Add proper support for:

  • \n
  • '
  • "

And appropriate printing of those values

quasiquote, unquote, unquote-splice

Macros

Pipeline Macros / Threading Operator

(Streaming?) https://clojure.org/guides/threading_macros

  • Thread first, thread last, etc

Colocated Docstring for defn

Conditional Operators not, and, and or

and EXPR... - left-to-right, and stop if one expr evaluates to #f. Otherwise #t or - left-to-right, stop if one expr evaluates to NOT #f

Function Calling Arguments - Lambda Lists

Support richer lambda list

Alternate Idea: Only support rest - but build convenience for:

  • Supplying value for optional arg from rest list
  • Extracting keyword argument from rest list
  • [ ] optional
  • [ ] rest
  • [ ] keyword arguments

Math Operators -

Pinning Symbols in Pattern Matches

Using the value of symbol instead of symbol

E.g. (match '(+ a b) would not work, since + would be bound to Val::Symbol instead of Val::NativeFn

Fiber API within lyric

Allow creating and running fibers within lyric? E.g. Use Yielding Function as an Iterator

Local Variables in Stack Frame

CallFrames have base pointer to stack Local variable access is relative to base pointer Removes need for Env - Env is implicit (?)

Wait… how do closures work?

Pretty Printing Code and S-Expressions

live on

Init File

Goal: Replace /scripts/serve.sh hack w/ proper init rcfile load path

Log File

Goal: Introspect running runtime via logs

Better Errors - Process Tombstones?

Better Error Reporting when Process Crashes Lots of noise for fiber execution.

// E.g. below fails in =call= BUT the real error is at ping_pong :interface '(echo) - echo is not defined for this program
let echo_prog = r#" (begin 
    (spawn (lambda () (begin
        (defn ping (msg) (list "pong" msg))
        (defn pong (msg) (list "ping" msg))
        (srv :ping_pong :interface '(echo)))))
    (list
        (call (find-srv :ping_pong) '(:ping "hi"))
        (call (find-srv :ping_pong) '(:pong "bye")))
)"#;

Hot Reload / Auto Restart

Goal: When runtime crashes, auto restart

Demo: Timeline + Capture + Todo + Notes

Quickly capture notes, todos, thoughts, tasks Recall + Search them

Demo: Command Bar: Use Query String in Hypermedia Interaction

Use query string! e.g.:

  • Open URL typed, instead of selecting item
  • Pass query string to command
  • Run CLI command from launchbar

Demo: Process Manager over ps

Exercises Embark Mechanics + Consuming Program Output Generate hypermedia based on ps output

Notification Service

Journal / Brain

A replacement for “self-texting”

A chat interface for:

  • Storing information (notes, messages, URLs, etc)
  • Recalling information
  • Timeline Record
  • Brainstorming
  • Task Management
  • “Assistant”

Chat Service over Cogni

rnd

LLM Blocks

Macro expansions, powered by LLM to generate UI Use bret-blocks to interactively tweak

# Key idea: It's generated, and prompt is inline - but evaluation is NOT at runtime. Develop time expansion
(prompt "An user interface for X"
    ...)

Durable Execution

Goal: Recovering from process or host failure

Program Migration

Goal: Moving snapshot of running program from machine to machine

Agent Programs

debt

Native Bindings that should be Lambda

E.g. built-in lambda bindings - e.g. call_fn

/// Binding for call
pub(crate) fn call_fn() -> Lambda {
    Lambda {
        params: vec![SymbolId::from("pid"), SymbolId::from("msg")],
        code: compile(
            &parse(
                r#"
            (begin
                (def r (ref))
                (send pid (list r (self) msg))
                (get (recv (list r 'any)) 1))
        "#,
            )
            .unwrap()
            .into(),
        )
        .unwrap(),
        parent: None,
    }
}

Native Bindings w/ hand-rolled macos

E.g. srv definition - srv_fn

tooling

Inferior VRS Mode in Emacs

Extend existing emacs mode with inferior mode comint shell, like run-python

Emacs Code-Signing Workflow

Add ability to sign a script from editor - e.g. (sign "Ey.....") and workflow with Emacs

Debuggers and Breakpoints

Editorialize History / Interaction

Seamless flow from REPL / Past Interaction, into durable program

Interactive Inspection in Emacs

E.g. See values E.g. See value of watch-ed topic

Test Runner

Expect Testing

See Jane Street

Observer

See:

  • Processes / Services
  • Messages

Structured Editor

Victor Blocks

https://recurse.zulipchat.com/#narrow/stream/102312-consciousness/topic/Leo.20Shimonaka/near/396280622

Inspiration - Bret Victor Ladders of Abstraction

Quickly do interactive “prop testing”

  • Go from a function
  • Parameterize over range of values
  • Immediately see output results

Do this… in source?

Extend to visual elements? UI:

  • Visually drag and adjust parameters, which get reflected in source
(defn my-fn (a b)
    (+ a b))

(defn my-ui (title subtitle)
    (list :title title
          :subtitle subtitle))

(bret-block
    (my-fn (a :from 0 :to 10)
           (b :from 0 :to 10))
    ; => See matrix of results

    (render (my-ui (title :in '("ONE" "TWO" ...))
                   (subtitle in '("one" "two" ...))))
)

learn - Charles - Fern

shell

CLI Client Shell

  • [ ] CLI to subscribe to a topic that is the “interface”
  • [ ] Hypermedia Interface shows the “Actions”
  • [ ] “Search Query” can be provided via CLI
  • [ ] Actions can be invoked via CLI - Opening Apps, URLs, “Pushing” secondary menus

Voice Client Shell

integration

Selected Text

Clipboard

Browser Page

Image

perf

Use FxHash

String Interning

Compact Bytecode

Tail-Call Optimization

bugs

bug? - Scope bug for Functions used to spawn child process

(defn child (parent_pid)
    (def results '())
    (subscribe :my_topic)
    (send parent :child_ready)     # ** BUG ** - parent was not defined here!
    (loop (match (recv '(topic_updated _ _))
        ((_ t :done) (send parent (list :child_results results)))
        ((_ t val) (set results (push results (list t val))))
        (_ (error "Unexpected result"))
    )))

(def parent (self))
(def child (spawn (lambda () (child parent))))

(recv :child_ready)

(publish :my_topic :one)
(publish :my_topic :two)
(publish :my_topic :three)
(publish :my_topic :done)

(recv '(:child_results _))

Busy Looper sometimes Crashes Runtime

(loop 0) blocks new connection only via REPL, but not =vrsctl -c “(loop 0)”=

Using (map (ps) (lambda (p) (kill p))) results in runtime crash sometimes

Disconnecting from REPL after running loop 0 does not quit as expected.

Sending client request for undefined method hangs indefinitely

Test case to repro

//! Tests for [Client] Pubsub API

use lyric::parse;
use vrs::{Client, Connection, Form, KeywordId, Program, Runtime};

/// Test Client::subscribe between client and runtime service process
#[tokio::test]
async fn client_pubsub_process_and_service() {
    let rt = Runtime::new();

    // counter service
    let prog = Program::from_expr(
        r#"(begin
        (def count 0)
        (defn increment (n)
            (set count (+ count n))
            (publish :count count))
        (srv :counter :interface '(increment)))
        "#,
    )
    .unwrap();
    rt.run(prog).await.unwrap();

    let (local, remote) = Connection::pair().unwrap();
    let client = Client::new(local);
    rt.handle_conn(remote).await.unwrap();

    // subscribe then increment
    let mut sub = client.subscribe(KeywordId::from("count")).await.unwrap();
    client
        .request(parse("(increment 1)").unwrap())
        .await
        .unwrap();
    client
        .request(parse("(increment 10)").unwrap())
        .await
        .unwrap();
    client
        .request(parse("(increment 31)").unwrap())
        .await
        .unwrap();

    assert_eq!(
        sub.recv().await.unwrap(),
        Form::Int(1),
        "should publish 1 after (increment 1)"
    );
}

// TODO: Pubsub from another client

Processes that run srv does not exit after vrsctl exits

I.e. killing connection does NOT terminate proc.

bug - (publish :my_topic) terminates vrsctl w/o helpful error

Errors in REPL are sub-par quality

wishlist

Record of “I want to be able to” thoughts

Open a visual structured editor for building interfaces from source, with changes reflected in source

(interactive)-like mechanics for dealing with common user-input arguments

E.g. (completing-read) and (interactive "f") in Emacs