Proposal: API for explicit side effects

[jlongster]

Note: this PR should not be merged, it's just here for discussion and will require improvements

I've been porting some of the Firefox devtools code to use a redux-like architecture (not actually Redux yet because we need to clean stuff up first), and it's been interesting to try to implement complex use cases. Porting my blog to redux has also helped gain insight on where we can do better.

I used to think that it was really nice and clean for all async work to only happen in action creators, using something like the thunk middleware. It is neat to consolidate complex workflows there, and reducers stay pure and synchronous. But there are a few downsides:

• You frequently want to do things that heavily depend on the shape of the state, like caching. See here in my blog: https://github.com/jlongster/blog/blob/e9224e6113399ca6f26f4a0eb1360b4df37e26fa/src/actions/blog.js#L26. The logic for caching it split apart in two places: one where it sets the cache and the other where it checks it. That's not too bad though.
• You also want to do more complex things like control the asynchronous flow: batching requests, forcing specific order of network calls, etc. You actually can't do this at all with async action creators, because this inherently requires the ability to read the flow of actions being dispatched and perform side effects.

I've seen redux-remotes which aims to solve this as well. It allows you do read from the stream of actions and perform async calls based on them, and you have access to dispatch and getState like in an action creator. But I don't like that it introduces another primitive, when we already have action creators and reducers.

There's also #307 and other issues, and I'm sure there will be lots of opinions about this.

@gaearon has been tweeting about Elm recently, and he's right: Elm has been leading the pack here, and it'd be good to follow it's steps. In Elm, you do something that up until now we've been preaching against: reducers can have "side effects", which essentially are just async work. At this point I'm ready to just give up and embrace side effects in reducers.

This PR adds the ability for reducers to return side effects, which are just functions that have access to dispatch and getState (like async action creators).

Example:

const initialState = {
  itemsById: {}
};

function items(state = initialState, action) {
  switch(action.type) {
  case constants.FETCH_ITEM:
    return withSideEffect(state, dispatch => {
      dispatch({
        type: constants.FETCHING_ITEM,
        status: 'begin'
      });

      setTimeout(() => {
        dispatch({
          type: constants.FETCHING_ITEM,
          status: 'success',
        });
      }, 1000);
    })
  case constants.FETCHING_ITEM:
    if(action.status === 'success') {
      const item = action.value;
      return {
        itemsById: Object.assign({}, state.itemsById, { [item.id]: item })
      };
    }
  default:
    return state;
  }
}

This works by using withSideEffect to return a state object and a side effect. You are free to run this side effect or not. In the normal scenario, side effects are run after the reducers are run in dispatch. But if you were replaying actions, you would simply ignore the side effects.

We don't have a type system like Elm, but we can take advantage of our dynamic typing by returning a class instance so that we can internally check it. I added an internal StateAndEffect class, and check if that is returned, and if true I know side effects are returned. It doesn't matter that this is a class instance because we never care about serializing side effects.

It's basically a monadic operation but I decided against the terms "lift" and "unlift".

Higher Order Reducers

We could apply the higher-order concept here too, if you want to separate pure reducers and reducers with side effects:

// items.js
const initialState = {
  itemsById: {}
};

function items(state = initialState, action) {
  switch(action.type) {
  case constants.FETCHING_ITEM:
    if(action.status === 'success') {
      const item = action.value;
      return {
        itemsById: Object.assign({}, state.itemsById, { [item.id]: item })
      };
    }
  default:
    return state;
  }
}

// itemFetcher.js
const items = require('./items');

function itemFetcher(reducer) {
  return (state, action) => {
    switch(action.type) {
    case constants.FETCH_ITEM:
      // We still call the lower reducer, but we don't really have to
      // in this scenario
      return withSideEffect(reducer(state, action), dispatch => {
        dispatch({
          type: constants.FETCHING_ITEM,
          status: 'begin'
        });

        setTimeout(() => {
          dispatch({
            type: constants.FETCHING_ITEM,
            status: 'success',
          });
        }, 1000);
      });
    default:
      // On the init action, this *should* pass undefined as state so
      // it still gets the correct initialState, right?
      return reducer(state, action);
    }
  }
}

// One problem here though is that this will be attached to the state as `state.itemFetcher`,
// not `state.items` but I'm sure that's solvable
module.exports = itemFetcher(items);

withSideEffect is monadic: if passed another StateAndEffect type is will correctly compose the state and effect into a new StateAndEffect type. When the side effects run, both the effects from this reducer and any lower reducer will run.

Advanced Example: Ordering Requests

Here's an example of something you can't do with async action creators. Say you want to do an HTTP request because an action comes through. That's fine. But if the user wants to initiate multiple of these requests, you want to make sure that only one happens at a time. Additionally, for simplicity, if multiple come through before the first finishes, we just do an additional single request after the first one finishes (we don't run 5 in order if 5 comes through at once, we run 1 to completion and then run an addition 1 request).

Demo here: http://jlongster.github.io/redux-experiments/side-effects/
Code here: https://github.com/jlongster/redux-experiments/tree/master/side-effects

The neat part is that we can use the sequence of actions to control what actual side effects happen, and I can extract this functionality out into a single library. Let's call it ensureCompleted:

const { withSideEffect } = require('redux');

function ensureCompleted(initialState, fetchActionType, statusActionType, reducer) {
  // This could be more complex, of course, just keeping it simple for
  // a demo
  let queued = false;
  let fetching = false;

  return (state = initialState, action) => {
    switch(action.type) {
    case fetchActionType:
      if(fetching) {
        queued = true;
        return state;
      }
      fetching = true;
      return reducer(state, action);

    case statusActionType:
      if(action.status === 'success') {
        if(queued) {
          queued = false;
          fetching = false;
          return withSideEffect(reducer(state, action), dispatch => {
            dispatch({ type: fetchActionType });
          });
        }
        else {
          fetching = false;
        }
      }
    }

    return reducer(state, action);
  }
}

module.exports = ensureCompleted;

This is a higher-order reducer, and wraps the item fetcher reducer here: https://github.com/jlongster/redux-experiments/blob/master/side-effects/reducers/items.js#L50. The item fetcher reducer simply does the async call blindly, unknowing that it's actually being batched/reordered/whatever.

Questions

• Does this mess up any higher-order stores or middleware? Only anything dealing with what comes back from a reducer needs to know about this, and I don't think this will break any libs.
• I'm still not sure about composition of effects. Higher-order reducers are neat, but they mess up how we currently combine reducers. The name of the higher-order reducer will be used as the name of the piece of state, when you probably want it to be something else (the "dumb" reducer). Elm takes quite a different approach to composing state it seems. "Reducers" are just simple models that can be composed: https://github.com/evancz/elm-architecture-tutorial/#example-6-pair-of-random-gif-viewers.

For example, let's say I have a reducer that fetches blog posts and stores them. I fetch a blog post with an id of foo. But the blog post also has a "readnext" property which is the id of the next blog post to read, and I need to fetch that after I get foo. In normal async world I'd do something like this:

let post = yield getPost('foo');
let readnext = null;
if(post.readnext) {
  readnext = yield getPost(post.readnext);
}

I'd love to figure out how to fit all of this into something that doesn't obfuscate the async workflow too much.

• Currently I run side effects on the next turn of the event loop (after calling the reducer) because it could immediately dispatch, and you don't want cascading dispatches. This feels a little weird, but I don't know if that's an indication that this is the wrong path or there's a better fix for that.

This is a bit of a braindump, I hope it isn't too confusing. I didn't spend much time gently introducing these concepts, so if you are confused, check out this Elm tutorial first: https://github.com/evancz/elm-architecture-tutorial/. Let me know what you think. Or if I'm horribly, horribly wrong.

[staltz]

👍 for inspiration from Elm
I was fortunate to meet Dan in person yesterday and we talked about Elm's Effects. To have its equivalent in Redux or whatnot (Cycle.js? 😄) is to essentially build an I/O Monad. I have not inspected too much of your PR and the long message, but it seems like that's indeed the direction: a data structure (StateAndEffect I guess) to hold the instructions to perform the effect, without yet carrying out that effect, delegating its execution.

I don't understand much of the reducers architecture (particularly middleware, higher-order reducers, etc), but I believe soon we are simply recreating Elm's equivalent in JavaScript, without the awesomely strong benefits of Elm: no runtime errors, and guaranteed immutability.

I'm this close > < to just migrating to Elm.

UPDATE:

Currently I run side effects on the next turn of the event loop (after calling the reducer) because it could immediately dispatch, and you don't want cascading dispatches. This feels a little weird, but I don't know if that's an indication that this is the wrong path or there's a better fix for that.

That's a consequence of not having a designated output of the program. In Elm, that's main. Also Cycle.js has a main. This is how you get a gate from the pure world (FP in the program) and the effectful/external world.

[acdlite]

@jlongster This is really cool (still digesting all of it) but I think this PR/experiment would be more useful in the form of a userland extension. It looks like the code you've written would work perfectly as a store enhancer, a la applyMiddleware(), devtools(), persistState(). (I believe @gaearon has decided to leave store enhancers out of the docs for now, and push people toward middleware instead, but they're essentially like decorators or higher-order components.) That way we can play with the ideas without needing to depend on a specific Redux branch.

[staltz]

I'm still not sure about composition of effects.

This should be done with monadic bind. Monads are there for compositionality, and AFAIK the I/O Type (Monad) in Haskell is malleable before it is outputted by the main.

[jlongster]

I don't understand much of the reducers architecture (particularly middleware, higher-order reducers, etc), but I believe soon we are simply recreating Elm's equivalent in JavaScript, without the awesomely strong benefits of Elm: no runtime errors, and enforced mutability.

Haha, yes. There certainly seems to be a strong convergence of similar ideas on the UI front. Relay is a whole other story, too, which relegates the need for a lot of this interestingly, and personally I would use ClojureScript everywhere if I could.

However, there are large projects in JS and I feel somewhat empathetic to those that are stuck with it. I have a vested interest improving my project, the firefox devtools, and we can still benefit from the ideas.

[jlongster]

@acdlite definitely, I considered going that approach but I figured I try this first, since it seemed like Dan was tweeting about wanting effects in here. (edit: but it certainly makes sense to go userland and then merge if found useful)

[jlongster]

I'm still not sure about composition of effects.

This should be done with monadic bind. Monads are there for compositionality, and AFAIK the I/O Type (Monad) in Haskell is malleable before it is outputted by the main.

If you look at StateAndEffect, I think it has that property. If you pass another StateAndEffect into withSideEffect, it will compose the effects (when the side effect is run, it will run the first one first and then the second).

However, the composition that I'm not sure about is reducers in general. Right now we pretty much assume that we have a top-level object with a bunch of reducers as the values (basically a flat list of reducers). In Elm, it's very common to compose models so it forms much more of a tree shape. Look at my example about posts and readnext at the end, I'm not sure yet how to express that cleanly with side-effect-producing reducers.

[acdlite]

@jlongster I agree this something that should be supported by the core, since it affects the entire community — e.g. the Redux Devtools would need to be updated to ignore side-effects when re-scanning a series of actions. However, even if/when we do move this into core, I imagine we'd want to implement it in the form of enhancers and higher-order functions, anyway. So I think it'd be useful to experiment using those primitives as well.

[staltz]

However, there are large projects in JS and I feel somewhat empathetic to those that are stuck with it.

They are most likely not written in Redux (yet). The problem of going functional but not fully functional is you lose the strongest benefits of FP which are its hard solid guarantees, read more here http://queue.acm.org/detail.cfm?id=2611829. E.g.: it is fairly "easy to reason about" if all data is immutable. You can replicate that in JS by using discipline, not a compiler, but in a large JS project, there will be a chance of some mutability sneaking in, and that'll make all that "easy to reason about" speech crumble. Same argument applies to reasoning about runtime errors. Elm guarantees an absolute zero amount of runtime errors.

[jlongster]

They are most likely not written in Redux (yet). The problem of going functional but not fully functional is you lose the strongest benefits of FP which are its hard solid guarantees, read more here http://queue.acm.org/detail.cfm?id=2611829. E.g.: it is fairly "easy to reason about" if all data is immutable. You can replicate that in JS by using discipline, not a compiler, but in a large JS project, there will be a chance of some mutability sneaking in, and that'll make all that "easy to reason about" speech crumble. Same argument applies to reasoning about runtime errors. Elm guarantees an absolute zero amount of runtime errors.

I'm literally just now starting to lead an effort to migrate Firefox devtools to React and something like redux. Moving to Elm is a no-go. Our entire team is made up of JS experts, and that would require a full rewrite, whereas moving to Redux is far less work. If we require updates to the screen to go through an immutable data structure (the screen literally won't show updates if mutated), that's good enough for us.

(I get your point tho)

[acdlite]

Right now we pretty much assume that we have a top-level object with a bunch of reducers as the values (basically a flat list of reducers)

Only if you use combineReducers(), which you must invoke explicitly. If it's a good idea, it shouldn't be hard to convince people to move to a different state structure. After seeing the response to Redux these past few months, I'm not as concerned anymore about people being reticent to embrace strange new ideas. :)

[staltz]

I'm literally just now starting to lead an effort to migrate Firefox devtools to React and something like redux. Moving to Elm is a no-go. Our entire team is made up of JS experts, and that would require a full rewrite, whereas moving to Redux is far less work. If we require updates to the screen to go through an immutable data structure (the screen literally won't show updates if mutated), that's good enough for us.

The only actual cost of migration to Elm is learning a new syntax (which thank God isn't as offensive as Haskell). But I honestly understand your situation and I won't insist in this vector in this discussion. Carry on

[acdlite]

Does each effect need to run in a separate task or could we queue all these up and loop through them at the end of the current dispatch cycle?

[jlongster]

This will run all the effects produced in a single dispatch cycle on the next tick (the effects are combined into a single effect by combineReducers). I think that should be ok. They can't really rely on each other, so they aren't racey. They will only just fire off async stuff.

Not sure how I feel about event delaying this a tick though, because async calls will be started a tick later, but I guess async code isn't that sensitive to perf. I don't see how we could get around delaying this because dispatch could be called immediately (is that a bad thing?)

[staltz]

Note: tutorial on how to gradually transition from React/Flux (Redux also) to Elm: http://noredinktech.tumblr.com/post/126978281075/walkthrough-introducing-elm-to-a-js-web-app

[gaearon]

@jlongster ❤️

[bhauman]

I have worked with these ideas for a while and I just want to make sure that one thing is understood.

An event occurs and we want a pure transaction. After trying many ways to frame this I have decided that adding a bunch of ceremony and process here is really unneeded.

The event handler itself can be thought of as a package of asynchronous ops that have various requirements and orderings. And the pure transaction or transactions are part of that package. That's all that's needed nothing more. Everything else is just us trying to comfort ourselves with the idea that we are adding shape to this state of affairs.

The event handler can side effect all it wants including the pure state transactions. The handling of this asynchronous stuff can take any form. You are still going to have a list of transactions that you can rollback replay whatever.

Be careful about adding ceremony where it really doesn't add anything.

[glenjamin]

This seems similar-ish in API to the action-interceptor concept in https://github.com/glenjamin/fluctuations#action-interceptors - although my reason for introducing it is very different (I'm fairly sure interceptors could be implemented for redux as middleware).

As I understand it the primary advantage is that it lets you co-locate async and sync logic when the two are related?

The other benefits seem like they can be achieved with existing tools, albeit in slightly different ways.

Here's an approach which I think is similar to the "Ordering Requests" example? using a sort-of higher-order actionCreator.

function oneAtATime(actionCreator, startAction, endAction) {
  var running = false;
  return (dispatch, getState) => {
    if (running) return;
    actionCreator(action => {
      if (action.type == startAction) {
        running = true;
      }
      if (action.type == endAction) {
        running = false;
      }
      dispatch(action);
    }, getState);
  }
}

exports.fetchItems = oneAtATime((dispatch, getState) => {
  dispatch({type: "START"});
  setTimeout(() => {
    dispatch({ type: "END", some: "data" });
  }, 1000);
}, "START", "END");

That said, I think that being able to co-locate async & sync stuff is a benefit that is not to be underestimated.

In the first home-grown flux framework I implemented, we didn't have anything like actionCreators, and simply did async stuff in stores when we felt like it, emitting change events whenever necessary. Over a year later, the app containing that framework is still going fine, with no major issues in that regard - certainly caching data in the store is something that fits very nicely when doing things this way.

[jlongster]

@bhauman

The event handler itself can be thought of as a package of asynchronous ops that have various requirements and orderings. And the pure transaction or transactions are part of that package. That's all that's needed nothing more. Everything else is just us trying to comfort ourselves with the idea that we are adding shape to this state of affairs.

I relish in simplicity; I'm not known to add unneeded complexity. You have to explore though to discover things that truly are needed, things that really do simplify. So far the redux style of state has proven itself, and we are exploring how to fit async into that.

What you describe is very vague. I'd love to see other options, I'm trying to figure out the simplest way possible to do it.

The event handler can side effect all it wants including the pure state transactions. The handling of this asynchronous stuff can take any form. You are still going to have a list of transactions that you can rollback replay whatever.

Show me some code, I'd love to see what you have in mind.

[jlongster]

@glenjamin I meant to tweet you this issue, glad you found it. There's one crucial difference in your oneAtATime code: it doesn't fire subsequent requests. If you run my demo, you'll see that if a 2nd request comes through before the 1st one finishes, it queues up another request to send. Then, when the 1st request finished, it fires the 2nd request.

Action creators cannot read from the stream of actions going through the system.

Action Interceptors do look similar, yeah. As long as you record actions after they have run. One of the benefits of the Elm-style is you can simply choose whether or not to run side effects, so you can replay actions and just ignore side effects. Or you can clearly log which actions have side effects.

However, I'm unsure if this is really the right level to be implementing network batching/ordering/etc. anyway. Promise, observables, channels, etc provide a lot of infrastructure for doing complex async workflow.

The thunk middleware returns the result, so you can actually get back a promise/channel/whatever when dispatching an action, as seen in my blog code for loading a post: https://github.com/jlongster/blog/blob/e9224e6113399ca6f26f4a0eb1360b4df37e26fa/src/components/post.js#L215. This makes composing async action creators very nice. Instead of listening for actions, you could use the actual async abstraction (like waiting on a promise). But there are some limitations to this, I gotta run maybe I can flesh that out later.

[staltz]

@bhauman

The event handler itself can be thought of as a package of asynchronous ops that have various requirements and orderings.

I'll assume you meant using something similar to handleOnChange here:

var Timer = React.createClass({
  getInitialState: function() {
    return {toggled: false};
  },
  handleOnChange: function () {
    this.setState({toggled: !this.state.toggled});
    jQuery.getJSON(requestUrl, function(responseData) {
      // ...
    });
  },
  render: function() {
    return (
      <div>
        <input type="checkbox" onChange={this.handleOnChange}/> Toggle me
        <h4>{this.state.toggled ? 'ON' : 'off'}</h4>
      </div>
    );
  }
});

React.render(<Timer />, document.querySelector('#mountable'));

This is not simplicity, this is complexity. ("Simple" as in "untangled")
Suddenly render() is not anymore about rendering only, and the component is not "just the View" anymore, it's a complete program. This is just conflating everything in the component.

Also, there are other use cases where you need to do HTTP requests unrelated to some user event, and in those cases it makes no sense to put HTTP requests or other side effects in any component event handler.

I guess in the absence of a main() as the designated gate between the app and the external world, we try to look for something close enough to that, and event handlers are what React has. But still, bad choice.

[glenjamin]

@jlongster mm, I think I could expand my example to do that, assuming the async thing is always triggered via the same actionCreator - but this does not hold true in general.

To-date, I've mostly done stuff like this outside the normal react flow, in the "API utils" on the traditional flux diagram.

The action-interceptor stuff acts a bit like there are two-dispatchers. Handling an action there stops it being sent to stores, unless explicitly dispatched again. Dispatches from interceptors only get sent to stores, not other interceptors. There is a separate API for sending something "back to the top". This is all stuff I made up based on initial production usage, but has been working well so far (~4months).

As I understand it now, the current proposal is roughly equivalent to letting reducers call dispatch, but with a way for the logging stuff to identify the side-effecting transitions and treat them differently?

On 18 Aug 2015, at 23:53, James Long notifications@github.com wrote:

@glenjamin I meant to tweet you this issue, glad you found it. There's one crucial difference in your oneAtATime code: it doesn't fire subsequent requests. If you run my demo, you'll see that if a 2nd request comes through before the 1st one finishes, it queues up another request to send. Then, when the 1st request finished, it fires the 2nd request.

Action creators cannot read from the stream of actions going through the system.

Action Interceptors do look similar, yeah. As long as you record actions after they have run. One of the benefits of the Elm-style is you can simply choose whether or not to run side effects, so you can replay actions and just ignore side effects. Or you can clearly log which actions have side effects.

However, I'm unsure if this is really the right level to be implementing network batching/ordering/etc. anyway. Promise, observables, channels, etc provide a lot of infrastructure for doing complex async workflow.

The thunk middleware returns the result, so you can actually get back a promise/channel/whatever when dispatching an action, as seen in my blog code for loading a post: https://github.com/jlongster/blog/blob/e9224e6113399ca6f26f4a0eb1360b4df37e26fa/src/components/post.js#L215. This makes composing async action creators very nice. Instead of listening for actions, you could use the actual async abstraction (like waiting on a promise). But there are some limitations to this, I gotta run maybe I can flesh that out later.

—
Reply to this email directly or view it on GitHub.

[jonathan]

Just an interjection. It sounds like you guys are talking about changing reducers into something like clojure's transducers. Your new reducer (transducer) would take a reducing function (server fetch and logging) and return new state.

Actually, it sounds like the combineReducer() function is doing a lot of what a transducer would do. Maybe the transducer api is what you guys are looking for.

[bhauman]

@jlongster @staltz I was on an iPhone when I replied before.

I felt a sense of urgency because I have been down this very road with a library called frontier and I feel like this road was followed in Pedestal as well. Both resulted in a tremendous amount of complexity when it comes to feeding side-effectful pure transactions back into the pure transaction queue.

It could be possible that you guys aren't following the same line of thought. But what happened in frontier is that: when I made pure transactions primary and had them return thunks of effects, I kept running in to exceptional situations that couldn't be expressed which lead to further "innovations" in my model.

My main realization was that when I made functional pure transactions primary and tried to encapsulate effects within them I was creating an expressive constraint that in the end added no value and was effectively equivalent, more complex and adorned, compared to code that made the impure side-effecting async code primary which called a transact! method (which operated on state) with an pure operation when it needed to. This relationship works and can express anything.

Here is a code example in pseudo CLJS (because it's terse):

(defn view-accounts [user-id]
   (go
       (let [account (<! (fetch-account user-id)]
            ;; pure state transition
            (transact! [:update-account-data {:user-id user-id :account-data acount}])
           (if-let [checking-balances (<! (fetch-checking-balances user-id)]
               (do 
                  (transact! [:update-checking-balances {:user-id user-id :balances checking-balances}])
                  (transact! [:focus-accounts {:user-id user-id}]))
               (transact! [:warning {:message (diagnose-environmental-problems)}]])))

(defn account-control [user-id]
  ;; react element
   [:a {:onClick (show-acounts user-id} "View Accounts"])

Now wether you use monads or csp or whatever in view-accounts, by making the side-effectful stuff primary I have full expressivity. And I'm not forced to do jump through some system that is thought to be better but is just a shape.

This road it will be tempting to create side-effectful pure transactions that requeue new side-effectful pure transactions and from there things just get kinda nutsy. When you get there you need to ask why you need this in the first place.

I say this in the full humility and knowledge that my experience is very very limited and in full respect for what you guys are doing. Heck I'm a crazy minimalist.

[ashaffer]

@jlongster I'm curious to know what you think of my proposal: #544 I think the advantage it has over what you're suggesting is that side-effects are transparent (whereas closures are opaque) to middleware, which makes caching and other more powerful features trivial to implement in a fully general way in middleware. And it also allows you to implement features like caching fully orthogonally to the implementation of the actual requests.

EDIT: Also @staltz, I think my proposal is precisely what you are suggesting. Essentially treating the redux middleware stack as the IO monad.

[matystl]

+1 on this idea.

I never liked idea of splitting business logic between action creators and reducers based on their sync/async nature. Actually I liked that in original flux you could make ajax request inside stores. What i don't like in original flux was that you can change state based on result of ajax without going through dispatcher. In this design you can in reducer make async calls(return them as side effect) but result of them have to come back in form of action. So this is nice

What is missing here is some form of middlewares for effects. Like in #544 is proposal for declarative api for action creators i would like to have same power for effects returned from reducer. Maybe best way would be to pass things back through actions middlewhare so if you return function it would be picked up by thunk middlewhare and executed. If you return description of ajax call than it would be picked up by some ajax-middlewhare. Also this would give you ability to just return another action from reducer(not sure if this is necessary or good idea but I would leave it to user land and later document best practices).

Also question to discussion is if you return sync effects(like just action to dispach, or ajax that immediatly dispach that it started) whether they should run in same loop as dispached action or view should be updated in between and actions are run on next js event loop.

And for copying elm i realy like architecture but for lot of people it's hard to get different syntax and architecture at same time. So it would be nice if we copy ideas from elm to js, people will get use to different architecture(like mondic ajax calls) and then maybe will switch to another syntax(elm) and get benefits for more safety and guaranties(from compiler).

[jlongster]

@staltz

I guess in the absence of a main() as the designated gate between the app and the external world, we try to look for something close enough to that, and event handlers are what React has. But still, bad choice.

This issue is covering some really broad topics, so there is a lot we could talk about! I don't want to go too far down this discussion here (otherwise this PR will get huge), but I think there is interesting value in "colocation", where React components do contain relevant information. However, obviously there are bad ways to do it, as the need for Flux shows (and I agree with your example). But once you have something like Relay, which provides a query language and opaquely takes care of the network requests for you, it's actually really nice to colocate data queries right with the component.

Anyway, I don't think that goes against what you're saying, but I think Relay is such an interesting contrast to all the needs we are describing in this PR. Really, once you have Relay, a lot of this just goes away. David Nolen talks about this with Om Next (the next version of Om), where they copy Relay and everything like cursors just goes away: https://www.youtube.com/watch?v=ByNs9TG30E8

[jlongster]

@glenjamin yes, that's basically it. It's a pretty simple implementation, because side effects only exist as a function that can be called. I don't try to describe them in any more detail, they are opaque, which personally I think is the way to go because you can do anything inside of them (more and this in replies I'm about to write to others)

[jlongster]

Alright, I've given a lot of thought to this. I tried some more use cases with it, studied Elm a bit more, and really just been trying to get to the bottom of what we really need.

My conclusion: this is the wrong path. I'm fine if some form of effects make it's way into redux core, but given how aggressively simple redux core is, and the push for external libs to augment it, I'm not sure it belongs here.

My Use Case Was Bad

I put forth a strange use case: needing to block and reorder remote requests, which requires keeping state between the requests to do so. My "advanced example" is not a good reason for a side effect API. As @glenjamin hinted, this kind of stuff is far more appropriate at a lower-level networking layer, and an action creator simply uses it (the instance of the networking engine being exposed to the action creator via middleware, probably).

This networking engine (whatever it is) can totally keep state around internally. We don't care about tracking that state. Right now we can't track any async state: if you are using promises, for example, if you snapshot your state and load it up later you have no idea if there were pending promises in memory.

I was thinking that I need to track that state in my redux state, but that's misguided.

Unnecessary Boilerplate

My simple example involved calling a fetchItem action creator to send a FETCH_ITEM action to trigger the network request as a side effect, which in turn dispatches FETCHING_ITEM actions. I don't really see what this buys us. What do we gain over the fetchItem action creator immediately performing the network request (or doing it whatever wants to express the request, instead of dispatching a dumb action), which then dispatches FETCHING_ITEM actions?

It's been bothering me that in dispatch I had to run side effects with a setTimeout of 0, and I think it indicates this is a bad path.

JavaScript is not Elm

Elm is very cool, but very different. It's like Haskell in that you cannot immediately perform an IO. Any side effect must be passed all the way back up to the runtime which will perform it outside of the "pure" Elm function run by main.

This has a lot of advantages, but pretty much forces the API the Elm currently has for side effects. We are not Elm, and most JS folks are just going to be confused by this.

It doesn't get us much, either. Elm has the ability to snapshot and replay the entire application because it's so pure. We don't get that in JavaScript, we can only snapshot and replay React+Redux apps. So doing IO immediately in action creators is fine.

Colocation is good

After all of this, I figured out the main thing I really wanted: I just want to declare action creators in the same file as my reducers! Most of the time these two go hand-in-hand together, and you could still have a globalActions.js file that defines actions applicable to everything.

Here's an example: https://github.com/jlongster/redux-experiments/blob/master/colocated-async-actions/reducers/items.js. I export an update function and a set of actions. This just requires a different combineReducers function which checks to see if an update method exists on the passed object instead of assuming it's a function.

The only annoying thing is sometimes I wish the thunk middleware could somehow just give me my current reducer's state when I called getState, not my entire app state. But I'm sure that's solvable.

[jlongster]

Going ahead and closing this issue. If someone else wants to pick this up (or @gaearon still finds this an interesting idea and wants to keep it open), please reopen. My code can be found on this branch: https://github.com/jlongster/redux/tree/withSideEffects

[utanapishtim]

@jlongster fyi, thanks for opening the issue. This spawned some great discussion and got me thinking quite a bit about my side-effecting code and just side-effecting code in general. I , for one, learned and thought a lot from this. Thanks!

[jlongster]

(@jonathan I still plan to think about reducers in terms of transducers, probably just need to look at it differently, thanks! I think there was an issue about how they apply as well.)

@utanapishtim great!

[bhauman]

What an amazing discussion. I'm really impressed.

[sjmueller]

I've read the various discussions happening around async actions, and collectively there's a lot of brainpower around how to make this pattern more elegant. Suffice to say, this is one challenging area of redux that is still maturing.

While I'm not well-versed enough in redux to offer a solution, I do know what kind of code I (and maybe other folks) want to write. It would look something like this:

async function login(username, password) {
  const credentials = { username, password };
  dispatch(credentials);
  try {
    const user = await fetch('/login', credentials);
    dispatch(actionType.success, user);
  } catch (e) {
    dispatch(actionType.error, e);
  }
}

Clean, simple, and expressive. Note that other flux frameworks like alt allow you to express async in a similar style. Of course, this example is getting away from pure functions (e.g. where does dispatch come from? what ), but I don't think there's a javascript developer out there who wouldn't immediately understand what's going on in this code.

Just something to keep in mind as this pattern evolves.

[ashaffer]

@sjmueller Since this thread is closed, there is a similar discussion happening over in #544

[matystl]

@jlongster I realy liked you idea so i was sad that it get closed so easy. So my contribution is in this repo:

redux-effect-reducers

Main differencies from original solution

1. It's in userland not in core
2. Effects are not speactial entity it's anything pure that some existing middlewares can parse. (so with thunk-middleware you can have effects like original proposal)
3. Handling of effects from dispach is in same event loop as original dispach. Also listeners for store changes is notified only after all effects has runned.

Disadvantages

1. It's working with devtools(and replay) but it's hacky solution.
2. combineReducersWithEffects is almost same as in redux core and i couldn't reuse it. Maybe some injection points would be nice? This can become important if multiple implementation for combining store will come up to that time it's probably ok to have separate implementations.
3. you can dispach sync action from reducer

Longer description is in repo so i will not copy it here. If there will be some interest i can publish it on npm.

@gaearon Would love if you look at it.

[tomkis]

I would still say keeping side-effects inside reducers is not that bad idea. We are using this approach for few months in production application and it works really great. There is a also a post advocating that approach http://blog.javascripting.com/2015/08/12/reduce-your-side-effects/ but the question is, do this really need to be a part of redux core? I wouldn't say so.

[jedwards1211]

I haven't read all of this discussion yet, but my primary beef with async action creators was wanting to be able to swap out the entire control logic easily, so that in one version a network call is made as a result of an action, and in another version no network call is made. If async calls are formed in action creators, turning async calls on/off or modifying them is awkward (use global variables? use environment variables? use action creator creators?)

My solution right now is to make all the async calls in my own middleware, so that in the bootstrappers for the different versions of the app I can just apply different middleware. But already I'm wishing I could manage the async calls and immediate state updates in one place.

There are various reasons for wanting to swap control logic. In my current project there could be multiple variations of the app running in different network environments. In another project back in my Java/Swing days, the business wanted to shift certain work from the client to the server, but I knew doing it on the server would be a waste of time, money, and resources for something that would perform poorly and be error-prone, so I made a version of the controller that fetched data computed by the server and a version that did the computation on the client, and structured things so that I could pick one with basically one line of code. That meant I was well prepared when, sure enough, they realized doing all the work on the server wasn't going to perform well. So from then on, I thought being able to change the control logic completely while keeping the view and model the same would usually be a wise thing to do, and shouldn't be too difficult to enable.

The also has the interesting consequence that I want to be able to hot reload my custom middleware, which was pretty simple to solve using a wrapper.

One interesting thought though: it would be neat to be able to fire some actions as sync only, meaning they wouldn't be able to trigger any async actions. Doing so would make it a bit less likely that other developers who aren't very familiar with the entire system could create event loops of some kind. Of course in my own custom middleware I could ignore actions with a noasync flag attached. But it might be a useful standard concept for building async systems.

[ashaffer]

@jedwards1211 I think you might like redux-effects :)

[gregwebs]

I didn't like the exact approach of redux-effect-reducers (the disadvantages section listed here kind of explains it), so I created redux-side-effect. The README explains the approach and compares it to alternatives.

[jedwards1211]

@ashaffer Thanks, I am using Meteor for the backend instead of a traditional REST app, but I may look into it at some point.

[jedwards1211]

@ashaffer one of the things that came to mind when reading redux-effects code was why terminate the middleware chain and return a promise instead of just dispatching an EFFECT action that contains the promise to be used by the redux-effects middleware? I see no major advantages or disadvantages but since all actions would go all the way through the middleware and reducer, there could theoretically be more flexibility.

[ashaffer]

@jedwards1211 That is an interesting idea, I hadn't thought of that. It would be somewhat more flexible I think you're right. I'll have to think that through a bit but it might be worth making that change.

One downside is while it eliminates the coupling on the frontend, it causes the effect middleware to have direct knowledge of their composition strategy. That might be ok though.

[jedwards1211]

So I just decided to go with this extremely minimalist solution in my own project:
sideEffectMiddleware.js:

/**
 * Enables reducers to perform side effects by adding a `sideEffect`
 * function to the `action`.  In your reducer you just call
 * `action.sideEffect(({dispatch, getState}) => {...});`
 */
export default store => next => action => {
  let sideEffects = [];
  action.sideEffect = callback => sideEffects.push(callback);
  let result = next(action);
  sideEffects.forEach(sideEffect => sideEffect(store));
  return result;
};

It's a little bit of a hack to tack something onto the action of course, but personally I find it way less invasive than the other libs and pull requests I've seen around here.

[gregwebs]

@jedwards1211 I published your code as actionSideEffectMiddleware in version 2.1.0 of the package.

[vladap]

I'm still new to FP so feel free to correct me wherever I'm not accurate enough, but take into account that I wanted to stay illustrative in explanations rather than precise. For more precision follow included links, I could misinterpret something - still learning.

@staltz, @gaearon

I was fortunate to meet Dan in person yesterday and we talked about Elm's Effects. To have its equivalent in Redux or whatnot (Cycle.js? ) is to essentially build an I/O Monad. I have not inspected too much of your PR and the long message, but it seems like that's indeed the direction: a data structure (StateAndEffect I guess) to hold the instructions to perform the effect, without yet carrying out that effect, delegating its execution.

Rather Free Monad or Free Applicative Functor should be used to achieve this. The purpose of Free Monads is to build Abstract Syntax Tree (AST) of instructions, they don't execute anything. This AST is then executed by interpreters and one can write different interpreters for different purposes. This AST can as well go through optimization or meta-programming step which rewrites the AST. Interpreters have to execute somewhere => already mentioned "main" which puts it together.

In type strict languages types are used to encode instruction types and pattern matching is used to implement interpreters (f. e. in Scala). Pattern matching can be used for optimization/meta-programming to search for patterns in AST and rewrite its branches as seen fit.

Unlike Free Monads, Monads are quite limited. They are sequential control structure, when one step fails the remaining steps are not executed, they can't inspect program before its execution and possibly rewrite it or skip steps. And Monads of different types doesn't compose.

If you read through this you can say that you already know these aspects - Javascript Promise is (quite proprietary and purist would say incorrect) implementation of a monad. The "then" method is the bind method, constructor is "unit". Promise/Future is well known monad which models a value which will be available later, Option monad models value which maybe won't be available at all, List is a monad which models computation which can return more then one result, etc - monads model computations, sequential transformation of an input into output, step by step.

If you want to compose monads of different types you have to use either Monad Transformers or Kleisli Arrow. Kleisli is the better option.

Applicative Functors unlike Monads model parallel execution and can inspect the composition (I know it from some clever book but I can't use this aspect yet, and they abstract on amount of input arguments (arity)). If we would squint a bit we can see that Javscript Promise has Applicative aspect in its Promise.all(array_of_promises) - which doesn't abstract arity though, still one input argument, hence the array as an input.

I believe that IO monad works alike Free Monad in Haskell. Everything written in Haskell is pure, its monads don't execute any side-effect, they just build the composition. This composed program is then passed into compiler which creates AST from it, then it goes into optimizer which does a lot of magic. Then the optimized AST is sent to runtime for execution. I would say that both IO instructions and IO interpreter are build-in in Haskell. Haskell has Free Monads as well, they are still needed to build custom interpreters. I don't use Haskell and I might be simplifying.

In languages which are not purely functional and doesn't have such an advanced runtime for IO like Haskell - f. e. Scala, IO monads usually refer to an implementation where function wraps the side-effect in the same way like shown in OP withSideEffect. IO monad itself then execute these side-effects. It is very limited implementation and useful only for simple programs for the reasons already mentioned describing monad limitations.

Because we don't have Haskell like runtime in Javascript Free Monads with a custom interpreter would have to be used to build it to get something flexible. Or the approach without Free Monads and craft AST like data structure (f. e. in plain js object) manually together with using some spec. It is already suggested there, and I believe the same approach is used in Cerebral. I find it inferior to Free Monads though.

There are no types in Javascript to describe instructions. So again either manually build AST with spec or the second option is to use some kind of runtime reflection for it - using function or class names to describe instructions then inspect them and interpret, or similar approach with objects like they describe Actions.

But pure reducers have to be close to side-effects. In FP it is said that pure function runs in some CONTEXT (sometimes called EFFECT). These contexes model side-effects. Then these contexes are composed whenever needed. Basic types of contexes are invented, the basic ones I know about are - Functor, Applicative Functor, Monad, Free Monad, Free Applicative Functors, Streams. If you want to compose them into larger good - Kleisli. Trying to side-step FP abstractions when one wants to achieve goals which FP is doing for decades is wrong. These abstraction were crafted by many clever minds to arrive at a minimal set of methods required to model particular behavior and then build more complex programs in terms of these minimal interfaces.

And they are not that hard really. Many just well define what I was already using I just haven't seen it. The problem is their explanation is usually very convoluted and turns most away. But if you know Builder pattern (method chaining) and Decorator pattern you are almost there to understand Monad. Especially if you already know how Promise works without understanding what Monad is. And if you replace the single value wrapped by a Promise with an endless sequence of values you are almost there to understand Streams. And you probably know that when two things have different type/shape you need something to convert/transform/mediate between them hence you are almost there to understand Monad Transformers and Kleisli Arrows.

Streams are the latest advance so the question for me is - should I just chose one of its implementation? Probably.

Don't be confused by the strange names/terms in FP which has no relation to what the thing is providing. It is because these patterns are so generalized that trying choosing some more specific name wouldn't make a sense in some other context were this abstraction can be used. I heard that OOP programmers like to give different names to the same thing, mathematicians and FP programmers like to give the same name to different things. This alone is confusing for OOP till one starts to see different things as the same (or partially same). It is what FP is learning me I already see how it changes how I think about software.

This helped me to understand monads - they are explained in terms of Builder and Decorator pattern:
Douglas Crockford: Monads and Gonads: https://www.youtube.com/watch?v=dkZFtimgAcM

"In addition to it begin useful, it is also cursed and the curse of the monad is that once you get the epiphany, once you understand - "oh that's what it is" - you lose the ability to explain it to anybody."

This is the best walkthrough about monads in Javascript I know of:
Translation from Haskell to JavaScript of selected portions of the best introduction to monads
https://blog.jcoglan.com/2011/03/05/translation-from-haskell-to-javascript-of-selected-portions-of-the-best-introduction-to-monads-ive-ever-read/

Free Monads - it is for Scala but the explanation still holds for anybody regardless
Monadic IO: Laziness Makes You Free
http://underscore.io/blog/posts/2015/04/28/monadic-io-laziness-makes-you-free.html

Great book if you want to get deeper
Functional Programming in Scala
http://www.amazon.com/Functional-Programming-Scala-Paul-Chiusano/dp/1617290653
It includes the following topics:
PART 4 EFFECTS AND I/O
* External effects and I/O
* Local effects and mutable state
* Stream processing and incremental I/O

Another good one, for Scala though, but even you wouldn't read the code it can learn quite a bit
Functional and Reactive Domain Modeling
https://www.manning.com/books/functional-and-reactive-domain-modeling

Encoding algebraic data type in JavaScript (I found this in my bookmarks, so why not this FP overview by mentioning Algebraic Data Types, even when it might be a bit OT)
http://kwangyulseo.com/2015/06/23/encoding-algebraic-data-type-in-javascript/

If you would like to better understand Scala code because there are some good resources about FP, this is probably the best starting book (it has no advanced concepts though)
Learning Scala: Practical Functional Programming for the JVM
http://www.amazon.com/Learning-Scala-Practical-Functional-Programming/dp/1449367933

[ashaffer]

@vladap You may want to check out redux-effects. It is an implementation of the free monad pattern exactly as you describe it :).

[tomkis]

I built simple Store enhancer which allows to yield side effects within reducers https://github.com/salsita/redux-side-effects - another approach for solving the problem.

[gaearon]

Of all recents attempts to solve side effects in a nice way I think Redux Saga is the most elegant. Check it out if you haven't already, maybe try to give it a go in a real app and let us know how it goes!

#1139