AceBase + Svelte = end to end fully reactive system! Thoughts on design

[ahdat81]

im just starting to get into the realm of reactive / spa web coding. been playing with svelte the past couple week. ive been thinking about acebase for a while after learning about its features, and finally decided to give acebase a whirl because i had this nagging feeling svelte + acebase will do exactly what ive been thinking 'reactive' coding should be. After banging on it the past couple days, i got a test system that's fully reactive end to end. make a change to the db on the server, and it'll auto proprogate to all the clients and the respective binded values and guis. make a change to a field on the client, and it'll proprogate out to the server, without writing any db code. Insanely cool!

I'm expecting to run into limitations with anything beyond simple queries/objects, but still a lot to learn with acebase, so we'll see. But before i keep going down this path, thought i'd ask if there's any expected potential issues with my design/implementation. Basically i have a server and browser client (+ cache). I'm letting that handle all the data syncing so it doesn't disrupt the user and keeps my side very very simple. With svelte, I have a store that's storing a proxy for every single record that needs syncing. I would imagine a power user can have up to 2000 or so various objects. then im tracking any mutation events for every proxy. Some basic tests of 1000 objects seems to work fine so far, but im kinda worried there might be an issue needing to have that many proxies open at a time. also if its deployed to al live server and may run into unexpected bandwidth usage /latency (not sure how the proxy and syncing works under the hood) thoughts?

[appy-one]

Sounds cool! I don't know about your particular use case, but do you really need thousands of proxies? If you want to monitor and sync an entire collection of thousands of records, you could just use 1 single proxy on a parent collection? The only hard limit for live data proxies I can think of is memory - if you have too much data in the proxy you'll run out of it. Regarding bandwidth used, that depends on your configuration. If you are using a local cache db and the server has transaction logging enabled, a live data proxy will download all data from the server once, cache it, and then only transfer object mutations back and forth ever after. (Unless last sync was too long ago, then it will fetch fresh data again after sending any local mutations done offline). Without transaction logging enabled on the server, the proxy will sync local (offline) changes and retrieve fresh data from the server every time it is created. You won't have to worry about latency, updates are only as big as the actual mutations (previous / new value pairs for changed properties). More proxies won't result in much higher latency.

[ahdat81]

ahh yes that was me not understanding how things worked in acebase yet, heh. When I realized we can proxy an entire collection, simply tying a svelte store to an proxied array was a very clean solution. Unfortunately any type of sorting will modify the entire array so that was inefficient. Svelte stores like arrays much more than a collection of json objects, and acebase works better with collections. So i tried a few things and was able to proxy a collection once, then drop in each object in the svelte store and manipulate it from there. that seemed to work great! was able to manipulate the array without triggering any mutations, but any changes to indivdual object will still propogate out. Next step is to try more complex queries and joins so we'll see how that goes.

es thanks for the tip about enabling transaction logging, you just saved me a few hours. That was EXACTLY how i envisioned it to work! Acebase has been pretty awesome, great work man. There's definitely a baas business here. I imagine whats capable here to be the next evolution. Real end-to-end reactivity and being able to completely abstract away the database is pretty wicked.

[ahdat81]

nothing clean enough to release. maybe i can put something together in a week or two. are you running into any specific issues?

[ahdat81]

here's a very quick and dirty startup

on your main template file, add this to the top to load the external js lib

<svelte:head>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/cuid/1.2.5/browser-cuid.min.js" on:load={()=> {console.log('cuid loaded')}}></script>
	<script src="https://cdn.jsdelivr.net/npm/acebase@latest/dist/browser.min.js" on:load={db.onAceBaseLoaded}></script>
    <script src="https://cdn.jsdelivr.net/npm/acebase-client@latest/dist/browser.min.js" on:load={db.onAceBaseClientLoaded}></script>
</svelte:head>

<script>
    import * as db from '$lib/libs/db'
    import { onMount } from "svelte"

   let loaded
   onMount(async()=>{
        await db.init() // wait for db to initalize before load page
        loaded = true
   })
</script>

here's the db.js file

// set from env variables
const host = import.meta.env.VITE_DB_HOST
const port = import.meta.env.VITE_DB_PORT
const dbname = import.meta.env.VITE_DB_NAME
const https = import.meta.env.VITE_DB_HTTPS == 'true' ? true : false

export var db
var localCacheDb, acebaseLoaded, acebaseClientLoaded

async function sleep(millis) {
    return new Promise(resolve => setTimeout(resolve, millis));
}

export const onAceBaseLoaded = () => {
    acebaseLoaded = true
}

export const onAceBaseClientLoaded = () => {
    acebaseClientLoaded = true
}

// wait for db to initialize before any db calls
export async function init(){
    while(!acebaseLoaded || !acebaseClientLoaded) // wait for external js libs to load
        await sleep(100)

    const options = { transactions: { log: true, maxAge: 30, noWait: false } } // optional transaction logging
    localCacheDb = await AceBase.WithIndexedDB('offlinecache', options) //optional for local cache
        
    db = await new AceBaseClient({ 
        transactions: { log: true, maxAge: 30, noWait: false }, // again optional for transaciton logging
        cache: { db: localCacheDb }, // optional, enables offline access
        host: host, port: port, dbname: dbname, https: https
    })

    await db.ready()
    return db
}

from there just import db to whatever page you need and access the db obj. I ended up creating a class to manage each acebase node. but the key is this

        import {db} from '$lib/libs/db'
        var proxy = await db.ref(nodepath).proxy(); // create acebase proxy to the node you want
        export var store = writable(proxy.value) // svelte store to db node
        proxy.onMutation((mutationSnapshot, isRemoteChange) => {
            console.log(`Value of path "${mutationSnapshot.ref.path}" was mutated by ${isRemoteChange ? 'somebody else' : 'us' }`, 
 mutationSnapshot);
                console.log('refreshing store...')
                this.store.update((cur) => {
                    cur = cur
                    return cur;
                })
        })

inside the class is a svelte store obj (to the acebase node) that you use throughout your svelte project. on any mutation, trigger an update to the svelte store. this code isn't tested, just pulling snippets out of my project. assuming you're decent with svelte, shouldn't be hard to put together. hope that helps

[donl]

Thanks! There is a lot of good information there.

The async initialization was one of the pain points I was having.

Does your proxy.value follow the form of:

{ <cuid> : { a: 1, b: 2, ... }, ... }

If so, I'm wondering the best way to traverse the cuid to get to the actual data. Currently, I'm converting the object to an array using Object.entries() and iterating -- it seems clunky.

[ahdat81]

Do you mean traversing it in html? you can set it to a store obj locally, then use lodash to manipulate the collection. if you're not familiar, its a library that gives you alot of the built in array functions onto a collection. you can check out their page for all the methods they support. but something like this. (side note, i add an id property to every object, same as the cuid. this comes in handy in many scenarios)

import {filter} from 'lodash'
$:store = writable(proxy.value)
..

{#each Object.entries($store) as key,obj} // simple
{#each Object.entries(filter($store, a => { return ... filter ....})) as key,obj (obj.id)} // fancy. noticed i have the id property here.   svelte uses this for various things to better update the list

[ahdat81]

actually now that im thinking, you shouldnt even have to create a store if you're directly accessing the proxy

{#each Object.entries(proxy.value) as key,obj}

if you need to access a particular cuid, its just a json collection

let obj = proxy.value[cuid]