Two-way communication (aka real-time)

[brillout]

The idea is to create a new library called Televar that works independently of Telefunc.

See new design here.

See also: #135.

[brillout]

CC @louwers @nitedani

[brillout]

This is low-priority on my side, but I'd be up to co-create this, in case someone is up for it.

[nitedani]

I don't see the advantage of variables over functions, but I think that is because I don't know what you have in mind. How would reactive code look like using variables? How would I know when the value of a variable has changed in the browser? Would the variable itself be converted to an observable when imported in the browser? Would we export utility functions to convert a variable to react state or similar? Somehow I would still need to tell the server to start sending me something(in this case messages), and for that I would still need an api call/telefunction.

For example, using functions it would look like this:

// server
export const getMessages = (cb) => {
    setInterval(()=>{
        cb('hello')
    }, 1000);
}


// browser
export const Messages = () => {
    const [messages, setMessages] = useState([]);
    useEffect(() => {
        getMessages((msg) => {
            setMessages([...messages, msg])
        });
    }, []);
    ...
}

Or like this:

// server
export const getMessages = () => {
    return new Observable(subscriber =>{
        setInterval(()=>{
            subscriber.next('hello')
        }, 1000);
    })
}


// browser
export const Messages = () => {
    const [messages, setMessages] = useState([]);
    useEffect(() => {
        getMessages().subscribe((message) => {
            setMessages([...messages, message]);
        });
    }, []);
    ...
} 

Using variables, it could look like this:
messagesVar could be stored on server,browser, or both. For example if it's stored only on the server side, then every access in the browser would need to be proxied to the server variable. If it's stored only in the browser, then every access from the server side would need to be proxied to the browser variable. If it's stored both sides, then only mutations need to be sent to the other side, and access can be local.

export const messagesVar = defineVar([])
export const getMessages = () => {
    setInterval(()=>{
        messagesVar.push('hello')
    }, 1000);
}

[redbar0n]

Remember https://en.m.wikipedia.org/wiki/Fallacies_of_distributed_computing

[brillout]

using functions it would look like this:

I also thought about this, but I eventually found many problems about that approach.

Actually, thinking more about Televar, I found major limitations around designing everything around "remote variables".

So far, I can't find a design that fits every need.

Real-time needs differ highly depending on the use case. E.g. a chatroom like discord has different requirements than a game with a single user-shared global state.

I think we should make Telefunc focused on RPC for now and postpone real-time support.

That said, I'd be more happy to have a look at a prototypes, if someone is up for designing and implementing a prototype.

[brillout]

Edit: See latest design at #36 (comment).

[brillout]

We found a design.

createChannel

// Chat.telefunc.ts
// Environment: server

import { createChannel } from 'telefunc'
import db from 'my-favorite-db'

export function onChatInit(chatId: number) {
  // **********
  // *** DB ***
  // **********
  const {
    // Initial list of messages
    data: messagesInit,
    closeDatabaseConnection
  } = db.queryAndWatch(`SELECT * FROM messages WHERE chatId = ${chatId}`, {
    // Or whatever ORM API
    callback(messages: string[]) {
      // Send new messages to client
      channel.send(messages)
    },
    // Or whatever ORM API
    onError(err) {
      console.error(err)
      // Send the error message shown to the user on the client-side
      channel.send({ error: 'Something went wrong. Try again.' })
      // Close the channel
      channel.close()
    }
  })

  // ****************
  // *** Telefunc ***
  // ****************
  const channel = createChannel<string[] | { error: string }>()
  // Initial messages
  channel.send(messagesInit)
  // User switches chat room, or closes the website
  channel.onClose = closeDatabaseConnection

  return channel
}

// Chat.tsx
// Environment: client

import { onChatInit } from './Chat.telefunc'

const channel = await onChatInit(42)
channel.listen(messages => {
  console.log([
   'Received messages:',
   ...messages
  ].join('\n'))
})

channel.close()
console.log('Connection closed')

On the client-side, you can manually stich everything together but it's tedious. The following high-level hooks provide first-class DX.

Click me to see tedious glue code.

// Chat.jsx
// Environment: client

import { onChatInit } from './Chat.telefunc'
import { useEffect, useState } from 'react'

export default function Chat({ chatId }) {
  const [messages, setMessages] = useState([])
  const [error, setError] = useState()

  useEffect(() => {
    let channel
    let isPrematureUnmount
    const setup = async () => {
      const channel = await onChatInit(chatId)
      if (isPrematureUnmount) {
        channel.close()
        return
      }
      channel.listen((messages) => {
        if (messages.err) {
          setError(messages.err)
          channel.close()
        } else {
          setMessages((prev) => [...prev, ...messages])
        }
      })
    }
    setup()
    return () => {
      if (channel) {
        channel.close()
      } else {
        isPrematureUnmount = true
      }
    }
  }, [chatId])

  if (error) return <div>{error.message}</div>
  return <ul>{ messages.map((msg) => <li>{msg}</li>) }</ul>
}

useLiveArray

Instead of stiching things manually, the user can use the component hook useLiveArray() which takes care of all the glue code:

// Chat.tsx
// Environment: client

import { onChatInit } from './Chat.telefunc'
import { useLiveArray } from 'telefunc/react'

function Chat({ chatId }: { chatId: string }) {
  // Note that `useLiveArray()` returns `string[] & { error?: string }`
  const messages = useLiveArray(() => onChatInit(chatId))
  // That's it, everything is taken care off...
  if (messages.error) return <div>{error.message}</div>
  return <ul>{ messages.map((msg) => <li>{msg}</li>) }</ul>
}

useSyncArray

// Chat.tsx
// Environment: client

import { useRef } from 'react'
import { onChatInit } from './Chat.telefunc'
import { useSyncArray } from 'telefunc/react'

function Chat({ chatId }: { chatId: string }) {
  // `messages` is an array proxy (like MobX)
  const messages = useSyncArray(() => onChatInit(chatId))
  const inputRef = useRef<HTMLInputElement>(null)

  if (messages.error) return <div>{messages.error}</div>

  function handleSend() {
    messages.push(value) // `value` sent to sever via `channel`
    inputRef.current.value = ''
  }

  return (
    <>
      <ul>{ messages.map((msg) => <li>{msg}</li>) }</ul>
      <input ref={inputRef}/>
      <button onClick={handleSend}>Send</button>
    </>
  )
}

With minimal diff needed on the server-side to make useSyncArray() work.

Click me to see diff.

  // Chat.telefunc.ts
  // Environment: server

  import { createChannel } from 'telefunc'
  import db from 'my-favorite-db'

  export function onChatInit(chatId: number) {
    // **********
    // *** DB ***
    // **********
    const {
      // Initial list of messages
      data: messagesInit,
      closeDatabaseConnection
    } = db.queryAndWatch(`SELECT * FROM messages WHERE chatId = ${chatId}`, {
      // Or whatever ORM API
      callback(messages: string[]) {
        // Send new messages to client
        channel.send(messages)
      },
      // Or whatever ORM API
      onError(err) {
        console.error(err)
        // Send the error message shown to the user on the client-side
        channel.send({ error: 'Something went wrong. Try again.' })
        // Close the channel
        channel.close()
      }
    })

    // ****************
    // *** Telefunc ***
    // ****************
    const channel = createChannel<string[] | { error: string }, string>()
    // Initial messages
    channel.send(messagesInit)
    // User switches chat room, or closes the website
    channel.onClose = closeDatabaseConnection

+   channel.listen((msg: string) => {
+     db.query(`INSERT INTO messages (chatId, content) VALUES (?, ?)`, [chatId, msg])
+   })

    return channel
  }

useLiveObject() & useSyncObject()

Same idea but for objects.

useLiveQuery

Client

import { useLiveQuery } from 'telefunc/react-query'

// Snapshot-only mode (invalidate works; push/remove without getId are ignored)
const { data: settings } = useLiveQuery({
  queryKey: ['settings'],
  queryFn: onGetSettings,
})

// Array-aware mode (push + remove apply smart in-place patching)
const { data: posts } = useLiveQuery({
  queryKey: ['posts'],
  queryFn: onGetPosts,
  getId: (post) => post.id,
})

Server

import { invalidate, push, remove } from 'telefunc/react-query'

export async function onAddPost(text: string) {
  const post = await db.post.create({ data: { text } })
  push(['posts'], post)       // all subscribers get { kind: 'upsert', item: post }
}

export async function onDeletePost(id: string) {
  await db.post.delete({ where: { id } })
  remove(['posts'], { id })   // all subscribers get { kind: 'remove', item: { id } }
}

export async function onRenameTag(id: string, name: string) {
  await db.tag.update({ where: { id }, data: { name } })
  invalidate(['tags'])        // all subscribers re-run telefunction, get { kind: 'snapshot' }
}