New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discussion - using Relay offline #676

Closed
zuhair-naqvi opened this Issue Dec 11, 2015 · 42 comments

Comments

Projects
None yet
@zuhair-naqvi

zuhair-naqvi commented Dec 11, 2015

We're planning to use Relay for a messaging application we're building with React Native. As you'd assume the 2 things that are key to achieving this are subscriptions and offline support both of which seem to be in the works.

We'd love to contribute to these efforts but need some validation if the below proposal could work for offline support or if it's plain stupid!

Inject a custom network layer that does the following before handing queries and mutations over to Relay's default network layer:

Queries:

  1. When online, log all query responses to a persistent local store.
  2. When offline, switch out the default network layer for something like https://github.com/relay-tools/relay-local-schema to query the persistent local store instead of the GraphQL endpoint.

Mutations:

  1. Create an immutable persistent log producer for React Native. (perhaps use Kafka?)
  2. Provide optimistic updates as well as log all mutations on device when offline
  3. When the device comes online, connect to a log consumer service and compact mutation logs into GraphQL requests, then somehow trigger fatQueries for the logged mutations so any data not covered by the optimistic updates is applied to state via standard relay.

How do you Facebook folk currently handle offline for native relay apps? such as the Ads Manager? And other GraphQL clients like Facebook for mobile?

@zuhair-naqvi zuhair-naqvi changed the title from Offline support with a custom network layer + apache kafka to Offline support using a custom network layer + apache kafka Dec 11, 2015

@zuhair-naqvi zuhair-naqvi changed the title from Offline support using a custom network layer + apache kafka to Offline support using a custom network layer + immutable log stream Dec 11, 2015

@eyston

This comment has been minimized.

eyston commented Dec 11, 2015

The RelayStore has the idea of a CacheManager which sets as a layer in the store hierarchy:

  • queued : place optimistic mutations go
  • records : normal cache where query responses go
  • cache : your thingy goes here

When you issue a query to the store it takes the first result it hits -- so your cache manager layer would sit at the bottom and be able to handle data requests before anything goes to the network layer.

I haven't done anything with this, so could be 100% wrong, but @steveluscher mentioned it in his meetup talk. I believe it would allow you to save this to local storage or any other place that would be longer lasting than the normal store layer.

@zuhair-naqvi

This comment has been minimized.

zuhair-naqvi commented Dec 11, 2015

@eyston do you mind linking me to this meet-up talk? Is there currently a way of hydrating RelayStore and bootstrapping the Cache from this hydrated Store at a later point?

@eyston

This comment has been minimized.

eyston commented Dec 11, 2015

It was an aside to the talk -- maybe an answer to a question -- so not on slides or anything (and I'm not sure if there are even slides).

The part in code: RelayStoreData#injectCacheManager : https://github.com/facebook/relay/blob/master/src/store/RelayStoreData.js#L158

I don't think RelayStoreData is exposed publicly (just RelayStore). RelayStoreData#getDefaultInstance would get it privately tho~.

Again, haven't done anything with this myself, but here is the interface: https://github.com/facebook/relay/blob/master/src/tools/RelayTypes.js#L154

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Dec 11, 2015

Ordinarily this would be an ideal question to post on Stack Overflow (we're trying to keep GitHub issues focused on bugs and enhancements), but since the discussion has already started let's keep it here.

There are two main things to handle for offline:

Queries

One option is to inject a custom network layer and cache queries/responses, and play them back when offline. This might work depending on your use case.

As an alternative, Relay supports injecting a cache manager (with the normal network layer). Whenever data is queried while online, responses will be logged to the cache manager so that it can persist data. When a query is executed while offline, Relay will attempt to fulfill the query by reading from the cache manager.

You can inject the manager with RelayStoreData.getDefaultInstance().injectCacheManager(). The interface is documented in code.

Mutations

Mutations are a combination of the mutation class and props. One option here would be to maintain a queue of mutation calls, caching them when offline and playing them back when online.

@zuhair-naqvi

This comment has been minimized.

zuhair-naqvi commented Dec 12, 2015

@josephsavona I'll post any how to's on StackOverflow moving forward but offline support is an enhancement right?

Thanks 💯 for your answers

Ultimately we want to contribute the work on offline sync back to upstream so wondering if anyone at Facebook's working on this / planning to work on this in near term? If so, might make sense to join efforts rather than potentially go in different directions.

@josephsavona josephsavona changed the title from Offline support using a custom network layer + immutable log stream to Offline support discussion Dec 12, 2015

@josephsavona josephsavona changed the title from Offline support discussion to Discussion - using Relay offline Dec 12, 2015

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Dec 12, 2015

As mentioned in the Roadmap, offline support is something we're actively exploring. In practice, this means that:

  • The cache manager interface is reasonably stable and you're welcome to try it out. Just keep in mind that the interface is experimental and may change going forward.
  • We're not ready to open-source an implementation of the cache manager interface, and we don't have the bandwidth to provide support for a default implementation in react-relay. Instead, we'd encourage you to publish your own implementation as a separate repository. We'll do our best to answer questions if you cc us on PRs.
@zuhair-naqvi

This comment has been minimized.

zuhair-naqvi commented Dec 19, 2015

@josephsavona @eyston

Let me know if the below makes sense:

We've got relay working with React Native, the next challenge is to offer an offline first experience. To do this one approach I've been thinking of is to bundle the schema with the app using https://github.com/relay-tools/relay-local-schema and back it up with https://github.com/facebook/dataloader persisting the cache to disk. The trade off is you'll still be making multiple requests from the client however majority of the requests will be resolved locally through data loader and the few that do go over the network will be asynchronous as far as the app is concerned as Relay abstracts these for us. So this drawback may not be so much of an issue depending on your use case.

There will also be a pub/sub mechanism on top of the dataloader that can selectively invalidate the cache when the data changes on the server.

The business logic and communication with the database will be kept outside of the resolvers and loaders (in a persistence API on the server) which is good practice anyway.

This way, with minimal change you might be able to build offline native apps using Relay - unless I'm missing something obvious, keen to hear your thoughts!

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Dec 19, 2015

That sounds like a solid approach. Data loader will help to reduce the number of network requests, and the injected relay-local-schema is just a network later as far as Relay knows.

Let us know how this works for you!

@zuhair-naqvi

This comment has been minimized.

zuhair-naqvi commented Dec 22, 2015

We were really excited to give this a shot but we can't seem to get graphql-relay-js to import into the React Native app as the https://github.com/graphql/graphql-relay-js is still based on babel 5.x and our fork of RN 0.16 (which is working with Relay) is using babel 6. We tried upgrading graphql-relay-js to babel 6 but then it wouldn't find babel-runtime helpers even though we're using babel-plugin-transform-runtime.

Then we tried manually importing babel-helpers (extends, createClass etc.) but babel-helpers/typeof would just not work.

Possibly related to facebook/react-native#2000 ?

Any suggestions?

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Dec 22, 2015

good question. cc @sebmck @amasad @DmitrySoshnikov

@vjeux

This comment has been minimized.

Contributor

vjeux commented Dec 22, 2015

@taion

This comment has been minimized.

Contributor

taion commented Dec 22, 2015

@josephsavona

Thanks for linking me here. That makes a lot of sense, and it's a really neat implementation. I'll update the README on relay-local-schema appropriately.

In a perfect world, I'd like to be able to entirely avoid the waterfalls and send un-cached GraphQL queries wholesale to the master (especially in a mobile context), but just using DataLoader sounds like a really neat implementation.

@skevy

This comment has been minimized.

Contributor

skevy commented Dec 22, 2015

@zuhair-naqvi this is a known issue with how the RN packager deals with babelrc.

The way around it is to define your own transformer, that either does something similar or extends https://github.com/facebook/react-native/blob/master/packager/transformer.js. You can put in the babelRelayPlugin there. This is how its done:

https://gist.github.com/skevy/1a814befb036b98b30d2

You would then call the packager with "--transformer=pwd/transformer.js"

@zuhair-naqvi

This comment has been minimized.

zuhair-naqvi commented Dec 23, 2015

@skevy thanks for the suggestion. @josephsavona thanks for your help so far, you've been great!

At this stage, we've decided to use Redux for our project given the timelines we're working with but I'd be keenly watching how the Relay and React Native communities toy with this idea as it could potentially kill three very hairy birds (i.e. Offline, Real-time and Local state) with one teeny-tiny stone (if it works).

@amasad

This comment has been minimized.

Contributor

amasad commented Dec 23, 2015

cc @hzoo @thejameskyle @loganfsmyth any insights on the babel-runtime issue mentioned above?

@skevy

This comment has been minimized.

Contributor

skevy commented Dec 23, 2015

@amasad I'm almost positive all the issues @zuhair-naqvi described above have to do with facebook/react-native#4062

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Dec 23, 2015

@zuhair-naqvi thanks for starting the discussion, bringing up the Babel issue, and for the follow up. Good luck with the project and let us know how it goes!

@jamiebuilds

This comment has been minimized.

jamiebuilds commented Dec 23, 2015

I thought the typeof issue was resolved by @loganfsmyth as of babel-runtime@6.3.19. https://phabricator.babeljs.io/T6644#68981

@skevy

This comment has been minimized.

Contributor

skevy commented Dec 23, 2015

@thejameskyle you're probably right. I'm 90% positive this a RN packager issue that's being described, not a babel one.

@shahankit

This comment has been minimized.

shahankit commented Jul 6, 2016

@josephsavona I'm trying to make relay offline using relam db and relay-local-schema. I want to update relay store manually where I have a query and payload from my custom network layer. I have used Relay.Store.getStoreData().handleQueryPayload(query, payload). where query is created using Relay.createQuery(relayQueryQL, variables) where relayQueryQL is create using Relay.QL ${queryString}``

When I try to provide queryString like this:

   query UserRoute($id_0: ID!) {
      user(id:$id_0) {
        id,
        ...F0
      }
    }
    fragment F0 on User {
      id,
      name,
      email
    }

similar to one that goes over network it throws error. Can you suggest how to use handleQueryPayload using such type of query strings. I tried looking in RelayRenderer.js to check how query and data are written RelayStore.

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Jul 7, 2016

@shahankit this is a good question for stack overflow: tag your question with #relayjs and post a link to it here and we or the community can answer. I'd recommend including a full code snippet so we can help diagnose.

@shahankit

This comment has been minimized.

shahankit commented Jul 7, 2016

@josephsavona I have created a stackoverflow thread here: http://stackoverflow.com/questions/38236178/updating-relay-store-for-queries-with-multiple-definitions. I would be very nice if you could answer it there or here if possible.

@helielson

This comment has been minimized.

Contributor

helielson commented Mar 5, 2017

@josephsavona I also created a SO question: http://stackoverflow.com/questions/42604115/use-relay-cache-data-on-react-native-app-while-fresh-data-is-being-fetched
I would appreciate if you share some thoughts there.
Thanks in advance.

@sibelius

This comment has been minimized.

Collaborator

sibelius commented Mar 28, 2017

@shahankit did you get Relay working with Realm? do you have some repo or gist showing the code?

@josephsavona are we gonna see more support to offline on Relay modern?

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Mar 28, 2017

For current Relay, check out relay-cache-manager, which implements the CacheManager interface and allows cached data to be persisted and restored for offline functionality.

@zuhair-naqvi

This comment has been minimized.

zuhair-naqvi commented Mar 30, 2017

@josephsavona how does this strategy (relay-cache-manager based offline first) change with Relay modern?

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Mar 30, 2017

@zuhair-naqvi We're still iterating on how offline mode would work in Relay Modern. I touched on this very briefly in the architecture doc about RecordSource.

@tslater

This comment has been minimized.

tslater commented Aug 8, 2017

I have a really simple example of one approach to using relay offline: https://github.com/tslater/reactnative-relay-offline.

Note that this is a totally offline solution, but could be altered to be hybrid if you use multiple environments or add some logic to your environment to decide whether or not to resolve things locally. In my case, we already have a really good code for syncing our cloud to sqlite locally, which is a huge topic in itself. Consequently, this idea presupposes having a sync with sqlite. It's a very simple idea I had over a year ago, but one I haven't seen done yet. Basically, you resolve GraphQL queries locally to sqlite.

One benefit to doing this is that you can do your initial sync in the background, but not force the user to wait for that download/sync to finish. The user can use Relay with the cloud while the database syncs, then they can just switch to the local query resolver when the sync is finished. In this way, you could always have an option of resolving things locally, or in cloud. In our case, some features, like searching or graphing data in the distant past, we could choose to treat as "always online", while treating other data as "always local" once the sync is completed.

@jhalborg

This comment has been minimized.

jhalborg commented Aug 27, 2017

Tempted to move to Apollo at the moment, with it's Redux integrations - offline support is super important for mobile. A simple 'save-current-relay-store-to-storage-and-rehydrate-on-startup' strategy would be fine as an intermediate, but I'm not sure how this could be achieved at the moment.

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Aug 27, 2017

A simple 'save-current-relay-store-to-storage-and-rehydrate-on-startup' strategy would be fine

@jhalborg This is trivial:

Save the store with serialized = JSON.stringify(environment.getStore().getSource()). On startup to rehydrate, do new Environment(new Store(new RecordSource(JSON.parse(serialized))).

@taion

This comment has been minimized.

Contributor

taion commented Aug 27, 2017

@josephsavona That seems not quite enough, though? Per #1881 – we'd need to use lookup query renderers, but those would have their own problems.

@josephsavona

This comment has been minimized.

Contributor

josephsavona commented Aug 27, 2017

@taion True, you'd have to use a QueryRenderer that did a lookup(), which is also straightforward.

@patrick-samy

This comment has been minimized.

patrick-samy commented Nov 2, 2017

@taion any luck with persisting the store for offline support?

@taion

This comment has been minimized.

Contributor

taion commented Nov 2, 2017

I haven't done any further work.

The lookup() approach above isn't quite good – you want to actually go through the network layer to e.g. set up a subscription, so it's not correct as a general solution.

@patrick-samy

This comment has been minimized.

patrick-samy commented Nov 2, 2017

I find it strange that no one outside FB seems to have documented this or added support for it in boilerplates. Anyways, I don't plan on using server-side rendering for my application.

Sounds like the following approach should work, I'll give it a go:

Save the store with serialized = JSON.stringify(environment.getStore().getSource()). On startup to rehydrate, do new Environment(new Store(new RecordSource(JSON.parse(serialized))).

@taion

This comment has been minimized.

Contributor

taion commented Nov 2, 2017

If you want SSR, follow the example from Found Relay. You’ll have a nicer time doing things at the request level. It’s perfect, but it’s good enough and easy enough. Just serializing the store won’t help with the stock query renderer.

@taion

This comment has been minimized.

Contributor

taion commented Nov 2, 2017

The request hydration stuff there is separate from Found and does not require its use.

@patrick-samy

This comment has been minimized.

patrick-samy commented Nov 3, 2017

In this case, does it even make sense to rehydrate the store? Does QueryRenderer rely on the store at all before performing the fetch?

Maybe only caching query responses would achieve a PWA offline-mode? Like these implementations:
http://taiki-t.hatenablog.com/entry/2017/09/05/181931
https://github.com/yusinto/relay-modern-ssr/blob/master/src/universal/relayEnvironment.js

@taion

This comment has been minimized.

Contributor

taion commented Nov 3, 2017

In a standard SSR implementation, I wouldn't rehydrate the store at all. I'd just use the request cache and handle everything at the request level.

You don't need anything additional to accomplish SSR – just using the request cache is sufficient.

@taion

This comment has been minimized.

Contributor

taion commented Nov 3, 2017

There wouldn't be any benefit to populating the store initially anyway.

@ghost

This comment has been minimized.

ghost commented Nov 20, 2017

@jhalborg, have you moved to Apollo? How do you setup the offline mode? redux-offline (https://github.com/redux-offline-team/redux-offline)?

@jhalborg

This comment has been minimized.

jhalborg commented Nov 22, 2017

I haven't yet @johnunclesam . And as I understand it, redux-offline is a more opinionated version of redux-persist, you might want to look there first instead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment