New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling lists in Cycle.js #312

Closed
staltz opened this Issue May 16, 2016 · 58 comments

Comments

@staltz
Copy link
Member

staltz commented May 16, 2016

Handling lists in Cycle is hard. It's the only sad thing about Cycle in my opinion so far, specially because lists are so common. Otherwise, Cycle is beautiful. Because it keeps on popping up so often, let's take it a bit more seriously. This issue is to track progress being done to update docs (cycle.js.org), Nick's library Cycle Collections, and other initiatives that may help people feel comfortable building lists in Cycle.js.

There are 4 solutions, here are their pros & cons:

  1. Manually build the dataflow, which will contain circular dependencies. This is what we do e.g. in TodoMVC.
    • Pros: explicit, no magic; follows Cycle.js principles of dataflow and purity; is being improved if people use xstream, it's easier to do that in xstream than it is with RxJS.
    • Cons: hard to understand and be comfortable with; probably a bit verbose.
  2. Wrap the Cycle.js subapps as Web Components https://github.com/staltz/cycle-custom-elementify
    • Pros: easier for most web developers to understand; explicit, no magic; enables Cycle.js subapps to be used outside Cycle.js;
    • Cons: subapp cannot have effects besides DOM-related; Diversity-support is annoying; Web Components spec is still in flux or not official yet.
  3. Global state at the top-most component (main).
    • Pros: a bit simpler and familiar mental model (like Redux); still aligned with Cycle.js principles of dataflow and purity.
    • Cons: still some circular dependency machinery to set up; not fractal, but if you know the tradeoffs, this may not be a problem at all.
  4. Stanga and similars.
    • Pros: Like the above, but circular dep machinery is hidden and automatically managed; is fractal; lenses API as a plus.
    • Cons: not aligned with Cycle.js principles of dataflow and purity with regard to drivers, harder to reason about what should be in a driver and what should not be.
  5. Cycle Collections, the library Nick is working on.
    • Pros: aligned with Cycle.js principles, easier to understand the API and reuse, not so much boilerplate/verbosity.
    • Cons: "magic", you give it X and Y and "things just work".
  6. Cycle Onionify, https://github.com/staltz/cycle-onionify
    • Pros: fractal with "lenses/cursors"-like API, only one state tree, hides away both fold and imitate, removes props vs state dichotomy.
    • Cons: currently only xstream support. No props vs state may lead to undesireable concurrent state modifications between parent and children.

CC @Widdershin @laszlokorte @milankinen @TylorS

@staltz

This comment has been minimized.

Copy link
Member

staltz commented May 16, 2016

Here's an example of how xstream makes the task of doing solution (1) a bit easier:

  • Eliminate some connect() calls cyclejs/examples@88ee425?diff=unified#diff-ac697d5a649cf8a49dd205b246a4c01cL29 because of xstream's multicast-only, sync start, async stop.
  • Use imitate() on proxy streams. We can build circular dependency logic directly in imitate(), such as not propagating errors forward (to avoid infinite synchronous cycle of error throwing, and stack overflow) and other issues.
@whitecolor

This comment has been minimized.

Copy link
Contributor

whitecolor commented May 20, 2016

https://github.com/whitecolor/cycle-circular for those who are stuck with rx/most

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented May 21, 2016

Just for demonstration, I converted the advanced-list example to use cycle collections.

Have a look: Widdershin/cycle-examples@2c3208d?diff=split

I think it's actually quite a nice improvement.

Things the user no longer has to think about:

  • Isolation
  • Ids, or dispatching actions to the correct place
  • Passing the parent sources and component around in order to be able to add a new item
@staltz

This comment has been minimized.

Copy link
Member

staltz commented May 21, 2016

Thanks @Widdershin, I put some comments on the commit, just in order to try to understand it better.

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented May 23, 2016

For anyone else interested, @staltz and I have been discussing/iterating on the above commit in this PR. cyclejs/examples#25

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented May 25, 2016

I have a thought, why is it we wan't to isolate a list item in a component, from it's parent list, why dont we just declare for the list, and the list item, in the same component. Mutable lists will break the cycle one way or the other, so maybe the cycle component abstraction should stop here?

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented May 25, 2016

My number one reason is to save the trouble of having to match up events from the DOM with the item they apply to. Isolation makes that really elegant.

@staltz

This comment has been minimized.

Copy link
Member

staltz commented May 25, 2016

Aron, that is a viable alternative and I did that in some examples in the past. But Nick pointed out one problem. The other problem is you might want to build bottom-up most of the time. Meaning: I have a component, I want to make a list of many of them, and the parent is isolated from each of them.

@staltz staltz added this to the Aug/Sep major projects milestone Jul 22, 2016

@staltz staltz added the size L label Jul 25, 2016

@staltz

This comment has been minimized.

Copy link
Member

staltz commented Sep 5, 2016

Added one more to the original list:

Wrap the Cycle.js subapps as Web Components https://github.com/staltz/cycle-custom-elementify

  • Pros: easier for most web developers to understand; explicit, no magic; enables Cycle.js subapps to be used outside Cycle.js;
  • Cons: subapp cannot have effects besides DOM-related; Diversity-support is annoying; Web Components spec is still in flux or not official yet.
@hermanbanken

This comment has been minimized.

Copy link

hermanbanken commented Sep 30, 2016

When using a list of isolated components with events I came across this problem myself and tried to solve it. My data is like this:

in
[ { id: 1, data }, { id; 2, data }, { id: 3, data } ]
[ { id: 1, data }, { id; 2, data } ]
[ { id: 1, data }, { id; 2, data }, { id: 4, data } ]

intermediate when applied function "Component"
[ { id: 1, DOM$, clicks$ }, { id: 2, DOM$, clicks$ }, { id: 3, DOM$, clicks$ } ]
[ { id: 1, DOM$, clicks$ }, { id: 2, DOM$, clicks$ } ]
[ { id: 1, DOM$, clicks$ }, { id: 2, DOM$, clicks$ }, { id: 4, DOM$, clicks$ } ]

out DOM$
[ vtree 1, vtree 2, vtree 3 ]
[ vtree 1, vtree 2 ]
[ vtree 1, vtree 2, vtree 4 ]

out clicks$
[ clicks 1, clicks 2, clicks 3 ]
[ clicks 1, clicks 2 ]
[ clicks 1, clicks 2, clicks 4 ]
or
all clicks$ merged in single stream as they track their target anyway

I now created a operator - which is a bit dirty since it is mutating (although local):

  observableProto.combineOpenGroups = function(selector, groupTransform){
    if(typeof selector == 'undefined' || selector == null) {
      selector = (v) => v;
    }
    if(typeof groupTransform == 'undefined' || groupTransform == null) {
      groupTransform = (v) => v;
    }

    return this
      .flatMap(group => {
        return groupTransform(group)
          .map(item => [group.key, selector(item)])
          .concat(Rx.Observable.just([group.key, false]))
      })
      .scan({}, (memo, next) => {
        var copy = Object.keys(memo)
          .reduce((o, k) => { o[k] = memo[k]; return o }, {});
        if(next[1] === false) {
          delete copy[next[0]];
        } else {
          copy[next[0]] = next[1];
        }
        return copy;
      })
      .map(obj => Object.keys(obj).map(k => obj[k]))
  }

which can then be used

input
  .flatMap(n => n)
  .groupByUntil(
    d => d.id,
    v => v,
    d => input
      .map(list => !list.some(ad => ad.id === d.key))
      .filter(b => b)
  )
  .combineOpenGroups(null, Component)

However, this does not provide the clicks$ in any way. That is the problem cyclejs/collection solves right?

@hermanbanken

This comment has been minimized.

Copy link

hermanbanken commented Oct 3, 2016

I solved it using custom operator, which probably does just what the Cycle Collection plugin does, but it is very very short for what it does. Might be an interesting solution: https://gist.github.com/hermanbanken/a00898288a223cadb787cef9f9bc3d81

Which can now be used as:

var components = [{
  DOM: Rx.Observable.of("A1", "A2"),
  event$: Rx.Observable.empty()
},{
  DOM: Rx.Observable.of("B1", "B2", "B3"),
  event$: Rx.Observable.just(1).delay(2, scheduler)
},{
  DOM: Rx.Observable.of("C1"),
  event$: Rx.Observable.just(4).delay(5, scheduler)
}];

const collection = Rx.Observable
  .fromArray(components)
  .collection({
    latest: {
      DOM: (item) => item.DOM,
    },
    merge: {
      click$: (item) => item.event$.map(item)
    }
  })

and then the resulting collection is a object, with the DOM and click$ fields.

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented Oct 3, 2016

Cool operator. How would you handle removing components? E.g. removing a todo in a todo list.

@hermanbanken

This comment has been minimized.

Copy link

hermanbanken commented Oct 3, 2016

Through completion of component streams, or as a separate (feedback)
stream. I have used and reworked this operator today a lot, so I made some
changes. Will update the gist later tonight.

@hermanbanken

This comment has been minimized.

Copy link

hermanbanken commented Oct 4, 2016

Updated the gist. Adding the completion is not completely trivial: the flatMap/publish in there wraps the completion, but also creates a persisted multicast. Since the subscribe to the next list (output of scan) happens béfore the unsubscribe of the previous list, the published Observable is always connected, causing it not to restart (when cold) or replay (when hot).

Maybe a custom combineLatest which has the signature

Observable<Observable<T>>.combineLatest(): Observable<[T]>

which combines all open/not-completed observables and creates a single subscription to each inner observable can remedy the issue...

@staltz staltz removed this from the Aug/Sep major projects milestone Oct 11, 2016

@staltz

This comment has been minimized.

Copy link
Member

staltz commented Oct 11, 2016

I added a 6th option to the list in the first comment. I'll also add me as an assignee since those 6 options are efforts from multiple people, not just Nick. In my opinion, this issue can hang around until we as a community settle down on one of those 6 as the default choice, while of course not excluding the possibility to use the others.

In my personal experience, I really like (6) but that doesn't mean I'll push down the other options, regardless of who built it. For instance, I also built option (2) and I'm still interested in experimenting with that one. Also curious to see how people use and experience (5).

@staltz staltz assigned staltz and Widdershin and unassigned Widdershin Oct 11, 2016

@maiermic

This comment has been minimized.

Copy link

maiermic commented Oct 22, 2016

@staltz Great topic, thanks for your overview of solutions. I'd like to join in the discussion, but I'm missing a description of the tackled problem.

Handling lists in Cycle is hard.

  • What kind of list are we talking about?
    • dynamic vs. static
    • what kind/type of elements? components?
    • ...
  • What is hard?
    • state management?
    • communication between parent and child components?
    • connecting/combining sinks of parent and child components?
    • ...
  • Why is it hard?
  • How do the mentioned solutions tackle those issues?
  • Are those issues list specific or is it a common issue of collections?
@staltz

This comment has been minimized.

Copy link
Member

staltz commented Oct 24, 2016

What kind of list are we talking about?

dynamic

What is hard?

If each item in the list is a component (many source streams to many sink streams), then a dynamic list is a stream of an array of sinks, and that's a rather cumbersome entity to work with. It's a stream of array of object with streams. Add in the middle parent/child communication too.

@jvanbruegge

This comment has been minimized.

Copy link
Member

jvanbruegge commented Apr 18, 2017

It is not closed...

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented Apr 18, 2017

I am the creator of @cycle/collection. I don't believe we should present any library as the one true way of managing lists of dynamic components in Cycle.js. Collection and Onionify are both viable approaches to this problem, with their own set of pros and cons.

Try different approaches. Figure out what works best for your application. We can always find better abstractions and mental models, but we have to keep innovating.

@cluelessjoe

This comment has been minimized.

Copy link

cluelessjoe commented Apr 18, 2017

Thanks @jvanbruegge, it's issue #504 which is closed and shown as such in the comment above. My bad.

@Widdershin: thx for the reply. Due to its naming I was wondering if @cycle/collection kind of "won", but it doesn't look like it anymore.

Currently we build the dataflow manually, but we're looking if/how to do it differently.

thanks again you all for cycle and your availability :)

@staltz

This comment has been minimized.

Copy link
Member

staltz commented May 1, 2017

Check out this onionify proposal where we are experimenting with an API that looks like both Cycle Collection and Cycle Onionify: staltz/cycle-onionify#28

@staltz

This comment has been minimized.

Copy link
Member

staltz commented May 18, 2017

Onionify+Collection is going well as an experiment. People can try it out installing cycle-onionify@4.0.0-rc.3. I wrote some examples here too https://github.com/cyclejs/cyclejs/tree/onionify-collection

@Widdershin what do you think? The API looks like const items$ = collection(Item, sources) and assumes under the hood that sources.onion.state$ is an array$. Everything else is reducer-controlled, so there isn't the add$ arguments.

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented May 18, 2017

👍 I think this addresses the biggest weakness onionify has.

Haven't had a chance to try it out yet or look at the code, but I'll try to soon.

Have you considered making collection part of the returned onion source?

Ie: sources.onion.collection(Item, otherSources). Not sure exactly what benefit there would be but it might be interesting to consider as an API.

@staltz

This comment has been minimized.

Copy link
Member

staltz commented May 19, 2017

Have you considered making collection part of the returned onion source?

I haven't considered, because collection() basically needs have also pickCombine and pickMerge, so sources.onion.collection, sources.onion.pickCombine, sources.onion.pickMerge start to look ugly. In general I try to keep sources.something for data and query of data, while helper functions can be imported separately.

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented Jul 26, 2018

Here is my take on state management.
I call it cycle-spoke.
https://gist.github.com/aronallen/f327f6571b4af425542393dcf7cbc867

@staltz

This comment has been minimized.

Copy link
Member

staltz commented Jul 26, 2018

@aronallen Interesting! It flips around onionify: source is a stream of reducers, sink is a stream of state. What is your reasoning behind this? I'm curious about the pros and cons.

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented Jul 26, 2018

@staltz so internally it has the same direction as onionify, source is a stream of state, sink is a stream of reducers, but a spoked component reverses this contract externally. This allows a parent component to subscribe to the derived state of a child component, and a parent may inject a reducer as a source to the child component.

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented Jul 26, 2018

@staltz the idea of spoke is that you have many spokes in your application, like you may have many isolates. It doesn't solve the problem of a global state tree, but you can still build a global state tree if that is what you want.

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented Jul 26, 2018

So after writing spoke, I see it doesn’t really solve the list problem very well. But I think it is a bit more approachable than onions.

The main issue facing me is how the child will let its parent know that it wants to be removed. Since intentions of state modification are emitted as reducers to the sink. And all reducers must be if the same type, how could we emit ‘DELETE’ to a sink? Then my thoughts went a bit wild, what if the convention is that when the reducer sink completes, the parent continues that reducer sink with one additional reducer that removes the row.

I am toying with these thoughts, how to make a counterpart for spoke that works well for lists. My working title is a function called hub, because spokes sit on a hub.

@shfrmn

This comment has been minimized.

Copy link

shfrmn commented Aug 1, 2018

Hi guys!

Just to add my two cents to the conversation. I've been mostly satisfied with @cycle/collection as a solution, however lately I became really frustrated with the level of typescript support in both @cycle/collection and onionify.

Out of this frustration I just created a similar library called cycle-lot. It's 100% percent in typescript and uses proxies to make API simpler. I'd really appreciate it if you tried the library and let me know your thoughts on it.

P.S.
I also posted in this issue and I hope it won't come across as spamming.

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented Aug 1, 2018

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented Aug 2, 2018

@shfrmn I really like the idea with proxies.

I guess the AND and OR operation of Observables map to Combine and Merge, thus providing these two by default is a great idea.

Could there be a need for a third proxy that is a function that receives a custom operator?

type Operator<T> = (...Stream<T>[]) => Stream<T>;
// later in the code
lot.compose(merge).DOM
@shfrmn

This comment has been minimized.

Copy link

shfrmn commented Aug 2, 2018

@aronallen Nice idea, can be added with almost no efforts. Can you suggest any candidates for the method name other than .compose?

@Widdershin

This comment has been minimized.

Copy link
Member

Widdershin commented Aug 2, 2018

.pipe(...operators)?

I think a big plus of .compose is consistency with xstream.

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented Aug 6, 2018

@shfrmn thanks for picking up my suggestion.

Looking through your example, it seems there is no way for the parent to decide to remove items from the lot, as all new items are just appended to the lot.

What if the parent could control this through how it consumes the lot.
By providing a second argument to the compose method, we could define how we consume the returning sinks.

lot.compose(combine, join).DOM // would combine all DOM sinks from all time.
lot.compose(combine, switch).DOM // would combine all DOM sinks from the latest sources$ event.
@shfrmn

This comment has been minimized.

Copy link

shfrmn commented Aug 8, 2018

@aronallen I’m always happy to get new ideas from you. However this time I’m not sure I completely understand your proposal and the motivation behind it.

Right now cycle-lot supports two ways of removing components:

  1. Component can remove itself by emitting a value in the designated sink stream.
  2. The whole lot can be reset from the outside. Reset stream comes as a parameter to the constructor function. (This feature is not yet documented other than in its typescript signature).

A parent component can always remove items from the lot because it has full control over the arguments it passes to the lot constructor.

My aim with this library is to create a practical solution to a very specific set of problems and I’m not sure which specific problem or an inconvenience you are pointing to in your comment.

I don’t mind discussing things on a conceptual level (as in how everything should be), however we better do that some place else since my thoughts on it go way beyond the scope of this issue or even cycle framework.

@shfrmn

This comment has been minimized.

Copy link

shfrmn commented Aug 8, 2018

@aronallen One conceptual problem here is that we don’t actually know what it means for a component to be removed. What we probably want to achieve is being able to completely disconnect a component. However then we have a dilemma again: should components accept a stream of state or a single value. Component accepting a stream of values is more flexible and persistent by design. Component accepting a single value is by design disposable and should be treated similar to how we treat immutable data. In the first case we only want to disconnect a component when, say, a number of values in array became smaller than the number of components we currently have on our hands. In the second case we’d like to remove a component on each change of state. (However this does not make such component a simple view because it might still map one plain value to a set of streams that it sinks).

You suggested that parent components could choose how they “consume” sinks from lots. I can’t really perceive this idea out of the context I tried to describe above. Hence my somewhat grumpy response.

In the end, I have my doubts whether “component” is the right and necessary abstraction at all.

@aronallen

This comment has been minimized.

Copy link
Contributor

aronallen commented Aug 24, 2018

@shfrmn

I see, I was not aware of the reset feature.
My thinking was more along that if the source state consisted of Observable<Array<T>> with T being the item. Then one could have a system that maps Observable<Array<T>> to Observable<Array<Instance<T>>> Instance being the result of invoking a Component.
with Observable<Array<Instance<T>>> we could then decide to scan over the Observable and provide a custom reducer that folds into a subset of Observable<Array<Instance<T>>> of all time.
With that we can then fold down on the specific channels with combine, merge, or a custom combination. It is quite different than lot, maybe I should try to code the example up.

@staltz

This comment has been minimized.

Copy link
Member

staltz commented Oct 29, 2018

With @ cycle/state released, this issue is done

@staltz staltz closed this Oct 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment