Resubscribing identical subscriptions

[hpx7]

If I do

Meteor.subscribe('foo');
Meteor.subscribe('foo');

Meteor will subscribe twice, instead of ignoring the second subscription.

This is especially an issue with the new template subscriptions -- it's pretty common that templates will overlap in their data requirements, and currently we get a lot of unnecessary loading while we wait for the server to tell us that we already have the data.

Technically fixing this would actually change existing behavior (e.g. consider the case where the foo publication returns a random cursor), but I hope no one is relying on that kind of behavior. I could always write a package that patches Meteor.subscribe, but I really feel like this is closer to a bug and should be fixed in core.

[stubailo]

The problem right now is that subscriptions are too flexibly to actually know what the effect will be of calling it a second time (as you mentioned). The solution might be to have a special subset of subscriptions, for example those that simply return a database cursor, behave in a more intelligent way.

[hpx7]

I think this is an important optimization to make if we really want to encourage modular component design using template level subscriptions.

Can you think of any legitimate use cases where "don't resubscribe to existing publications" would break functionality?

[avital]

Don't we dedup subscriptions on the server anyways? So subscribing twice would be a NOOP, but still send a subscribe message to the server? /cc @glasser

[rclai]

In the Meteor docs in the paragraphs under Meteor.subscribe it says:

However, if the next iteration of your run function subscribes to the same record set (same name and parameters), Meteor is smart enough to skip a wasteful unsubscribe/resubscribe.

I am under the impression that if you do:

// server/publish.js
Meteor.publish('todos', function () {
  console.log('publishing todos');
  return Todos.find();
});

// client/todos.js
Meteor.subscribe('todos');
Meteor.subscribe('todos');

I should only see the publishing todos message once, however, I see two.

I would think that the second subscribe call wouldn't even hit the server, but apparently it does.

[glasser]

@rclai That doc note is specific to reactive contexts.

[ghost]

@glasser I am having the exact same problem within a reactive context. Namely, I call Meteor.subscribe multiple times from different Templates within their autorun's. An autorun is a reactive context.

Also, considering the concept of publish & subscribe, why would you ever publish something a second time with the exact same arguments? It should return the exact same result (unless of course the relying data sets have changed, but it would probably have auto-updated anyway). If you would rely on this republishing logic, I would actually interpret it as abusing the system.

As Meteor wants to go into the modular, reusable template way, why would you want to promote users to have to do their own centralized bookkeeping? I completely agree with @hpx7 and hope this bug is fixed as soon as possible. Whenever a subscribe with the exact same arguments happen, you should only publish the results once to the client.

[stubailo]

While it is convenient to subscribe to things in lots of different places, we now officially recommend limiting the number of places you do subscriptions, by having a small subset of your components be "smart components" which do the data loading, and having them pass the data down to where it is needed. http://guide.meteor.com/ui-ux.html#smart-components

[ghost]

@stubailo Thanks for your response! If I understand it right, any templates in need of the data, needs to be a child of the "smart" component. However, creating such a component, makes it only fit to be a parent of a single specific set of child components which decreases reusability (the template is nearly static). Also, the parent component can only pass data in one specific form making it suitable for a small amount of child components. Not even to mention the difficulty of implementing dependencies on multiple parents. What if you have a component A which relies on X and Y but also B, which relies on X but you want to place it somewhere else not near Y? You would indeed have to create 2 instances of X and have two outstanding subscriptions anyway. How is this the better solution? Or am I misinterpreting the way smart components work?

The alternative is to increase the complexity and bookkeeping in a specific part of the framework. If multiple duplicate subscriptions are detected, they should all be gathered and sync on the same outgoing subscription. If the data is already there, the onReady can be popped immediately. This results in the less or same amount of outgoing subscriptions and reduces the complexity of the app significantly. The merging of similar subscriptions is already implemented; you would only need to centralize the bookkeeping for exact duplicate subscriptions.

I am curious to what I am missing. ^^

[stubailo]

Basically, you should try to build as many "reusable" components as possible - so if you have some kind of widget, or list of items, or whatnot, there are huge advantages to building it in a such a way that it doesn't use any subscriptions or collections. This will let you (1) use that component in other places in our app and (2) test that component with a variety of inputs.

Once you have a bunch of these reusable components, you now need to craft a custom top-level "page" smart component that renders all of them with the appropriate data. I think in many cases every route in your app will have a different smart component that loads and massages data into the right format for the components it wants to render.

Also, the parent component can only pass data in one specific form making it suitable for a small amount of child components.

I think this is what I would disagree with here - you should first build the child components, and then make a totally custom parent that passes exactly the data that those children need. I wouldn't suggest having some sort of generic parent component that works with a wide variety of children.

[ghost]

Ah ok, so it seems I am interpreting the main concept of a smart component correctly. But why would creating custom parent components be a better solution than introducing the bookkeeping complexity in the framework? I understand the parent-child component structure works with the current version, but wouldn't the optimization on subscriptions fall in the categories of "radically less code" and "fully integrated and flexible"; both of which Meteor is trying to achieve? Right now you expect users to write custom parent components for different placement, children and shared data structures which would be optional if the optimization is introduced. The only downside (I can think of) of using the optimization over the parent-child structure, is a few more minimongo queries to be executed.

[stubailo]

But why would creating custom parent components be a better solution than introducing the bookkeeping complexity in the framework?

One big problem with Meteor today is that it does a lot of things under the hood which sometimes work in surprising ways. For example, you might make two subscriptions that you think do one thing, but actually do something else because of an optimization under the hood.

"Introducing bookkeeping complexity" is one of the things we want to do only when absolutely necessary from this point forward. Also, you can already use something like subs-manager: https://github.com/kadirahq/subs-manager

[hwillson]

To help provide a more clear separation between feature requests and bugs, and to help clean up the feature request backlog, Meteor feature requests are now being managed under the https://github.com/meteor/meteor-feature-requests repository.

This feature request will be closed here, but anyone interested in migrating this feature request to the new repository (to make sure it stays active), can click here to start the feature request migration process. This manual migration process is intended to help identify which of the older feature requests are still considered to be of value to the community. Thanks!