How difficult is to create an offline-first app?

[frederikhors]

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report  
[ ] Performance issue
[ ] Feature request
[x] Documentation issue or request
[ ] Support request
[ ] Other... Please describe:

I only discovered this project today.

Congratulations for this great job!

In your opinion, how difficult is it to create an offline-first Web/Desktop/Native application with this project?

Anything that respects these rules?

[NetanelBasal]

This is a general question. I can't answer this without knowing what you need to do. Sorry, but I don't have time to dive into the article.

[frederikhors]

We are evaluating your project because we want to build a CRM-like that must have all the data and a 100% offline CRUD with synchronization once online.

I cannot understand if we can achieve this with your project.

Some of the questions in the article:

1. Are conflicts handled? How?

2. How much data does the client need to store to fully replicate (full offline data access & editing)?

3. Does it require all data to live in memory, or can it work with mostly-persisted data? (such that large datasets are usable)

4. Does it correctly handle working on multiple tabs in the same browser session?

[NetanelBasal]

Akita is a state management solution. You can persist the data to web storage. You can keep browser tabs in sync using localStorage.

[NetanelBasal]

Anything else you'll have to manage yourself

[frederikhors]

Thanks.

[frederikhors]

It would be amazing to sync state from browser to server just to handle offline situations.

And I'm not talking about conflicts for now (I don't care at this stage, I can use "last writer wins" or handle the error in the server and communicate it to the single client).

What do you suggest as an effective sync system to any backend?

[NetanelBasal]

I'm not sure what you mean by "an effective sync system to any backend"? Can you elaborate?

[frenetic]

Hey @frederikhors, have you checked Pouchdb? It basically is a solution for your problem. However, you will need to have a backend running CouchDB our that implements CouchDB API.

Akita is a state management library.
It basically tracks the entities that exist and what happened to get them to a specific state.
Using localStorage or other browser storages, you can save the "final" state of your app so, when opening it again, you dont have to fetch all that data you previously had.

With that being said, and complement what @NetanelBasal said, you can use Akita to BUILD your own solution.
The simplest way to do it is to have a modified boolean field on your model.
So, you can have a service that queries Akita for entities marked as modified and push them to the backend. And get the backend to perform anything you need...

For sure you can do something more advanced, something like a CQRS + event system - but that will need you to work on the backend.

Why Im telling you all this:
Im using Akita on a project and using it offline, doing the "simplest" approach. The biggest downside is the overhead. You never know what you need to get back from the backend, what was deleted, what the user lost permissions... So, we need to download everything again and again and again...
For the sake of quickly delivering an offline first solution, it works. But then you will want to move to something with less overhead.
Which means building something that forces the app to do validations on timestamps or versioning, as well as the backend, so the app can send and request only the required data (thus the CQRS + event system pattern).
It is not easy, but doable.

[frederikhors]

I'm not sure what you mean by "an effective sync system to any backend"? Can you elaborate?

Something to synchronize the operations Akita saves in the browser once the connection with the server is restored.

[NetanelBasal]

We don't have this functionality. Can you give me an example of how you imagine a JSON representation of this?

[frenetic]

Something to synchronize the operations Akita saves in the browser once the connection with the server is restored.

May be you can use a service-worker for this: https://developers.google.com/web/ilt/pwa/introduction-to-service-worker

When your user performs an action, add the result of the action to Akita and send the HTTP request through the service-worker. The service worker, on the other hand, would listen to successful requests and push the results back to Akita.

[mattbodman]

I am implementing something similar using Service Workers, IndexedDB and Akita ngEntityStore

[frenetic]

@mattbodman I know this is not the scope for a/this github issue, but Id love to read about how you guys are implementing it, challenges and victories.

[kragoth]

@frederikhors just want to give a bit of insite into this question as it may be of use to you or anyone else looking to implement an "offline first" application.

My team have delivered a fully functioning "offline first" application using Akita as the data store. But, remember, Akita is just there to store your data and give you a nice way to access it.

To "syncronise" with the server we have the following

We have a isDirty property on every entity that is set using preAdd/preUpdate events.
We listen for a connection back to the server (there's at least 3 different checks we do here) and once we have a connection all data where isDirty == true is sent back to the server. If the server acknowledges the post, we clear the isDirty flag.
Now, there's a gotcha in clearing the dirty flag. The data may have been changed in the time you just spent sending it to the server.... so, we have a updatedDate that is compared with what is in the store and what was sent to the server. If the updatedDate in the store is greater than what we just sent to the server, we do not clear the dirty flag.

So far all of that is by far the easiest part of an "offline first" application.

Dealing with multipe devices updating the same record, devices coming back online and syncing data that has been updated by other people/devies or system processes, handling application upgrades that change your API/schema and dealing with old versions of data. Keeping the application up to date when they are online.... all of these things presented a much larger challenge.

Our app is quite complicated

• a spatial job tracking application where users can see on a map where their jobs are
• 23 different entity stores (Basically following https://netbasal.com/working-with-normalized-data-in-akita-e626d4c67ca4)
• many spatial (geoJSON) properties that we display on maps
• users are able to draw their own shapes and save them
• users can collect client details name/phone etc
• users can collect spatial waypoints
• users can scan barcodes

The list goes on but, suffice to say quite complex. Any given user has roughly 20,000 records being synced to the Akita store.

So, can Akita be used for what you are trying to do.... yes. But, you'll need to do the sync management work yourself.

@NetanelBasal I'd like to thank you for your work on Akita as I have definitly enjoyed using it in our app.