Some theory and examples for async client-side fetching

[apapirovski]

I've seen a lot of questions about this problem and none of the solutions I've seen out there quite address it in a flexible enough manner for my purposes, so I decided to do this little write-up. If nothing else, hopefully it generates some useful discussion.

Disclaimer: This is an approach that worked for me, I don't claim it's perfect, correct or that it addresses your particular needs.

The problem
You're building an app that needs to fetch some data before it renders the routes. This is simple if you don't care about transitioning down the path before this data is fetched. In such case, you can just use componentDidMount (or use Rx to react to something like redux-router state change) to run whatever methods are needed. The server-side aspects of this have been covered endlessly and the match solution works well.

Let's say, however, that you do care about transitions. I'm currently working on a universal React-based e-commerce website where the data has to be fetched first before the previous components unmount. The site should also be capable of displaying a global loading indicator if any data dependencies are being fetched.

A more relatable example: GitHub project view. The page transitions happen after page data is loaded — while displaying a global loading indicator — not before.

The theory and the potential solution
The react-router render tree always starts with one component, this means that we can nest our actual routes within any number of components that don't have a path on them and that render absolutely no markup (but rather simply this.props.children).

By creating a top component that sits below the router but above our app, we can use componentWillReceiveProps and shouldComponentUpdate to isolate our app from router changes while letting it go about its business undisturbed.

Furthermore, this also means you're free to pass down new properties that will, for example, show a global loading indicator (or progress bar). Because the code relies on componentWillReceiveProps, this means that it also won't run on the back-end. It could probably be expanded to support server-side but I already had a match based solution that worked for me.

Now, for some sample code:

import React, { Component, Children } from 'react';

import _map from 'lodash/collection/map';
import _isEqual from 'lodash/lang/isequal';

export default class TransitionManager extends Component {
  static propTypes = {
    children: React.PropTypes.element.isRequired
  }

  constructor(props) {
    super(props);

    this.state = {
      isAppHydrating: false
    };
  }

  // this basic version only re-renders if we're done hydrating
  shouldComponentUpdate(nextProps, nextState) {
    return !nextState.isAppHydrating;
  }

  // 1. Check that the new location is actually different and requires our intervention
  // 2. Get all dependencies and run them, passing in whatever properties you need.
  //    Keep in mind, these are static functions that don't have any awareness beyond
  //    what you pass to them.
  componentWillReceiveProps(nextProps) {
    if (_isEqual(this.props.location, nextProps.location)) {
      return;
    }

    const promises = _map(
      getAllDataDependencies(nextProps.children),
      fetchData => fetchData(nextProps, this.props)
    );

    if (promises.length > 0) {
      this.setState({
        isAppHydrating: true
      });

      Promise.all(promises).then(this.willTransition.bind(this, nextProps));
    } else if (this.state.isAppHydrating) {
      this.setState({
        isAppHydrating: false
      });
    }
  }

  // For cases where the user presses another link before we're done,
  // we need to check that our location data is the most recent.
  // This could be avoided by using promises that support cancellation.
  willTransition(transitionProps) {
    if (!_isEqual(this.props.location, transitionProps.location)) {
      return;
    }

    this.setState({
      isAppHydrating: false
    });
  }

  render() {
    return this.props.children;
  }
}

// Iteration here is more complex due to these being actual elements rather classes.
// If you're using redux-router, you'll also need to test for WrappedComponent.
function getAllDataDependencies(children) {
  return iterate(children, []);
}

function iterate(children, accumulator) {
  Children.map(children, child => {
    if (child.type.fetchData) {
      accumulator.push(child.type.fetchData);
    }

    if (child.props && child.props.children) {
      iterate(child.props.children, accumulator);
    }
  });

  return accumulator;
}

Hopefully the comments made that clear enough to parse. From this base state, you could further extend the class by storing a copy of old this.props.children in the app state and using cloneElement to generate a version that passes new props down to the component — these could contain things like data loading progress status or simply information about the transition about to happen.

Note that if you're not doing server-side rendering then you will need to make sure to update this code to actually run on componentWillMount, otherwise you will not get any initial data. You'll probably also want to make other tweaks to enable loading indicators, etc.

Further Disclaimer: It's possible this breaks some internals of react-router. I haven't found any issues yet but I haven't used the full api to be completely certain.

[benbonnet]

"You're building an app that needs to fetch some data before it renders the routes. "
So what is it people call "isomorphic", and what's the point of doing server-side rendering, if there's no data in it ?

[vojtatranta]

Well this problem probably goes beyond react-router. To accomplish this whole thing you need some state management and something that tells you what needs to be updated.
This something might be Relay, but I haven't tried it yet.
By the time I simply listen for route change and if route changes that I request server for new page but instead getting HTML I ask server for JSON only.
You may take a look if you like.
Server side, notice routes /:date and / and sendState function.
Listening for location change on client and most important is fetching new state from server on location change

[benbonnet]

that depends on architecture yep but the concern seem quite recent;
so far 've never heard that much about it, googling around a lot about react
hope it'll get more attention

[vojtatranta]

@bbnnt wait mate, seems you don't understand.
Once you send http request to an URL data are being fetched on the server and with this data your default app state is being rendered.
So there are data in the app, that's purpose of server side rendering. We wipe out need for ajax call for data once is page firstly rendered.
Just like Gmail has this progress bar when you hit refresh, with react, there's no need for this anymore. With data got from database or whatever, you feed react that renders html for your app and app state in JSON you send along with HTML to the browser.
Look at project I referenced above. You see everything there.

[benbonnet]

Nope I don't fully understand what's going on :/
I'll sure look into your project; but let me ask you :

• "We wipe out need for ajax call for data once is page firstly rendered." : you'll never fetch the server again ?
• "Just like Gmail has this progress bar when you hit refresh, with react, there's no need for this anymore. " : new mails will come up your app automatically ?

Right now I'm confused when you point "fetching new state from server on location change", what's happening at this specific moment ?

[vojtatranta]

• you do fetch server again, when you need it of course (e. g. you filter over emails, you go to different page). On the contrary, with angular you mostly make ajax call once javascript loads and in that ajax you receive state (data - emails list, filters - list, user object...) of the app.
• if Gmail used server-side rendering than you would received rendered HTML of the app and somewhere in HTML you'd get <script>window.initial = "//state object here with emails and everything"</script> which you may use further in your app.

[benbonnet]

My concern is about his quote "The site should also be capable of displaying a global loading indicator if any data dependencies are being fetched."

[vojtatranta]

Practically yes.
He wants to display progress bar when there's some Ajax fetching, but is has nothing to do with refreshing of whole page.

But in Gmail case. It has to fetch data using Ajax on every page refresh, with server-side rendering, you don't have to fetch anything to render app, as stated above.

[apapirovski]

@bbnnt A hypothetical lifecycle of an isomorphic app would go something like this:

1. Server receives a request for a path
2. We use match to get the correct render tree
3. Get data based on the render tree, where each route might have its own dependencies
4. Send fully rendered HTML together with the initial state to the client
5. <Link> is pressed by the user and triggers a new route
6. react-router does its things and passes new props to its render tree
7. TransitionManager component, as described above, intercepts and delays rendering new components until all data dependencies have been retrieved.

@vojtatranta I believe your solution displays a loading indicator but it also renders the new state of the app, without the new data. Correct me if I'm wrong on that.

[vojtatranta]

@apapirovski nope, I mean exactly same this as you.
I used Gmail as example of app that is not rendered on the server.

[apapirovski]

@vojtatranta I understand that you're rendering on the server but in your app, when history listener is triggered, it initiates data fetch. That's all fine but it doesn't stop the new routes from being rendered while the data is fetched.

Imagine your app had another route (say About) which rendered a completely different component (StaticPage) from a different state (Static). Now the Gallery component would transition out and the new component would transition in, but it would have no data to show and so would just show a loading indicator.

The point of the middleware component is to avoid that blank render because in some apps it's undesirable. That's why I gave the example of GitHub — when you switch between views, it fetches the data while displaying a loading indicator over the current page.

[vojtatranta]

@apapirovski Ahaaa,
I see. Yup, that is true.
In this case I rely on Redux, because since the state is not changed (router objects are just mutated and redux ensures implementation of shouldComponentUpdate), the whole app does not rerender.
But otherwise, you are right.
I haven't yet used TransitionManager, looks bit too complicated for simple task like this.
I would also recommend you to use some kind of state managing library (redux, some flux) which would solve this just like in my case.
You would also got rid off setState calls, which would make everything more clear

[apapirovski]

@vojtatranta I do use Redux. The solution above is generalized for people who don't. Redux or the redux-router don't solve the problem.

Think about it like so:

• You have a route component App with path /
• You have 3 sub-routes each rendering a separate component with a separate store, those sub-routes might nest further with further dependencies
• You're fetching data from more than one service (but even if it was one, doesn't matter)

Here are the issues:

• You need a Promise that tells you whether ALL of your dependencies down the new render tree have been fetched. It's not enough that the app state has changed (something else might've changed, not the exact stores we need), it has to be fully hydrated.
• You need to be able to pass new properties such as isFetching down the render tree, which means you need to be able to clone the old render tree, so you avoid rendering the new components that react-router wants to render.
• And finally, we want to allow our user to continue interacting with our app. Blocking shouldComponentUpdate in our App component means that we might not let through their interactions while the data is loading.

Which is why we abstract the solution above the App — it gives us more flexibility and keeps our App pure. TransitionManager can be pulled out of the routes at any time without impacting the rest of the app.

[vojtatranta]

To those issues:

• I solve this by creating action type STATE_CHANGED which I implemented on stores where I knew that they were affected by url change, this works nice but causes many app rerenders (which I have in TODO)

Otherwise you are right I am curious about any full-featured example with TransitionManager, would be great if you shared one. Thanks.

[vojtatranta]

Actually this turns out to be very interesting problem. I would like to know some other opinions @knowbody, and maybe someone from redux community...

[apapirovski]

I don't have a full redux implementation that I'm happy with but in my head it goes something like:

• dependecy decorator to define actions that need to happen before render
• TransitionManager to crete a wall between the router and the app (and only allow router to impact the app when it's ready for it)
• Store that tracks the lifecycle of the transition, that way we can avoid stupid stuff like cloneElement and just listen to the transition where necessary.

Something like that. To make it fully featured, I think the dependency probably needs to specify whether it's blocking or not, that is whether the app can render without it.

[benbonnet]

@apapirovski a route change that would behave like a promise ?
Feels like a component tends to behave differently, depending on the fact that it is reached right away (render after data-fetch) or that you get to it from a route (data-fetch/wait then render)

[taion]

BTW, I have a separate implementation of async data fetching in a routing context for react-router-relay: https://github.com/relay-tools/react-router-relay/blob/v0.6.2/src/Container.js#L59-L64 (sorry, this code snippet isn't going to make as much sense as it might if you haven't seen the code for Relay.RootContainer).

It's quite possible that whatever I have in the Relay context will have to stay separate, just because Relay has its own semantics around loading states, and it'd make more sense for react-router-relay to stick to those than to general React Router async data handling, but I thought it was pertinent to this discussion.

[taion]

Oops hit tab once too many times.

[imouto1994]

@apapirovski Any problems with your solution so far? I am very excited about your solution and plan to use it for my project :)

[kennethaasan]

I'll ask the same question as @imouto1994 to @apapirovski. Becuase it's difficult to do the same thing that was so easy in the earlier version of react-router, and this seems to be the best solution I can think off...

[apapirovski]

No problems with it although I'm using redux-router and have switched over to a middleware approach.

[imouto1994]

Are you following the way of react-redux-universal-hot-example ?

[apapirovski]

I have't looked at their implementation but I assume it's similar. I don't think there's much flexibility in terms of how you do it if you're using middleware.

Edit: Yeah, it's pretty similar. In mine, I actually created a higher-order component that defines a getDependencies static method that fetches the data. That way I don't have to traverse react-redux's WrappedComponent like they do. I just have to make sure my higher-order component is always the top most wrapper.

I also have a separate transitions reducer that stores the current transition state to allow rendering of things like global loading bars / indicators, etc.

[imouto1994]

@apapirovski Do you mind sharing about how you tackle the problem that fetched data before navigation complete might crash the current route if it also uses same data with the next route? I find it as one of the main problem for this approach. I am thinking about applying shouldComponentUpdate in each smart components to check whether the router is currently navigating or not.

[apapirovski]

Assuming you're using Flux or redux, you could just structure your stores in such a way that data is tied to a key, which could be a param or a slug or something else unique to each path. Then the component would only update when both conditions are met.

shouldComponentUpdate would also work but would have the side-effect of not allowing any re-renders on that component, even if they're not related to the fetching.

[kennethaasan]

This one worked for me:

import React from 'react';
import Actions from '../../actions/Actions';
import isEqual from 'lodash/lang/isEqual';

const ComponentProvider = React.createClass({
    propTypes: {
        Component: React.PropTypes.func.isRequired,
        initialData: React.PropTypes.object
    },

    getInitialState() {
        return {
            isAppHydrating: false,
            data: undefined
        };
    },

    // re-render if we're done hydrating
    shouldComponentUpdate(nextProps, nextState) {
        return !nextState.isAppHydrating;
    },

    componentWillReceiveProps(nextProps) {
        if (isEqual(this.props.location, nextProps.location)) {
            return false;
        }

        if (nextProps.Component.fetchData) {
            this.setState({ isAppHydrating: true });
            this._addLoading();
            this._getDependencies(nextProps);
        }
        else if (this.state.isAppHydrating) {
            this.setState({ isAppHydrating: false });
        }
    },

    _getDependencies(transitionProps) {
        transitionProps.Component.fetchData(transitionProps.params)
            .then(data => {
                this._saveNewData(transitionProps, data);
            })
            .catch(nodata => {
                this._saveNewData(transitionProps, nodata);
            });
    },

    _saveNewData(transitionProps, data) {
        if (!isEqual(this.props.location, transitionProps.location)) {
            return false;
        }
        this._removeLoading();
        this.setState({
            isAppHydrating: false,
            data: {
                [transitionProps.Component.displayName]: data
            }
        });
    },

    _addLoading() {
        Actions.toggleLoading(true);
    },

    _removeLoading() {
        Actions.toggleLoading(false);
    },

    render() {
        const { Component } = this.props;
        return 
             <Component 
                 {...this.props}
                 data={this.state.data || this.props.initialData} 
             />;
    }
});

export default ComponentProvider;

If server rendering or initial render, I'm passing in initialData.

[taion]

https://github.com/rackt/async-props

[kellyrmilligan]

@apapirovski I know this issue is old, but did you get an example working where the old route hangs out until the promises are resolved?

EDIT: this is from shouldComponentUpdate, my bad

[kellyrmilligan]

@apapirovski i'm also seeing the app scroll to the top when the transition starts, any ideas on that?

[kellyrmilligan]

apapirovski did you get a redux version of this working? I have had success with just setState, but integrating it with redux has proved to be more challenging than I thought.

[kellyrmilligan]

for anyone landing on this from google, I have built a module for redux usage that fits the bill!

https://www.npmjs.com/package/react-redux-transition-manager