Refreshing partially loaded records

[rvega]

Here's the scenario:

You load a list of records with findQuery but the server only returns a few fields for each record. Afterwards, when the user navigates to a "detail" screen, you need to fetch the rest of the fields for the record from the server so you call App.store.find(type, id) and the store returns the already fetched record, there's no way to tell the store to fetch ask the adapter for the record again.

Also, any suggestions for a work around are appreciated.

[tchak]

Agree we need refresh method on Models and ModelArrays

[joewest]

Why not have a separate model for your extended data/attributes (or a model to represent just the summary information)?

[cloke]

The issue I find with creating a summary model is that in my situation the user has the ability to modify what I am summarizing so I end up right back at the same situation. The main model is modified, then I have to sync the summary, and now the adapter wants to commit my summary (which I could just ignore on the server), but seems like even more work arounds.

[rvega]

@cloke yeah, similar situation here.

[wycats]

I have updated the title to properly reflect the request.

You could "sideload" the summary model in your adapter from any create or update request. It is now possible to load records into the store that are not dirty, so you can simply treat the "summary" model as read only and return an update to it from every request.

[chanks]

I also really need a refresh method, but my use case is somewhat different, and the workaround @wycats suggested won't work for me. I'm writing a test-taking app - when the user decides to take a test, the Ember app creates a client-side test record and informs the server, which inserts it in the DB. The server starts up a background task to build the test (pick questions for it). I need my Ember app to keep reloading the test record every x seconds until it receives the full, built test.

Does anyone have an idea for a simple workaround?

[tchak]

@chanks this is probably not something that will go in the official package but this is my current take on the problem
tchak@130d308

The main problem with reloading is to be able to "merge" underlying data.
I believe having a strong infrastructure to merge underlying data is a must have for ember-data

[chanks]

Thanks @tchak - I ran into one problem, where findQuery would try to call a nonexistent "call" function on the model I was refreshing (see here: tchak@130d308#L2R436). I defined an empty function for it to call, though, and things seem to be working alright now.

[tchak]

sorry my bad, I fixed it in my local branch but have not pushed. Will push as soon as possible

[tchak]

@chanks Hey, I juste updated my fork, it should be ok now and it is rebased on latest master.
Have fun

[chanks]

@tchak Great, thanks!

[abloom]

I was running into a similar issue, wanting to load partial records up front and later load more detail based on what the user wants to see.

The above referenced commits no longer exist so I came up with the following:

App.Store = Ember.Store.extend
  reload: (obj) ->
    @adapter.find(this, obj.constructor, obj.get('id'))

It seems to simple, can anyone see any problem with this approach?

[ppcano]

@abloom, i have extended your solution checking the record state.

    reload: function(record, rollback) { 

        if ( record.get('isDirty') ) {

            if ( rollback === true ) {

                // this will rollback the record to the last loaded values in the
                // store
                record.send('rollback');

            } else {

                // skip rollback values

                var manager = record.get('stateManager'),
                        data = record.get('data');

                record.withTransaction(function (t) {

                    var dirtyType = manager.getPath('currentState.dirtyType');
                    t.recordBecameClean(dirtyType, record);

                });

                data.adapterDidUpdate();
                manager.goToState('loaded');

            }

        }

        this.get('adapter').find(this, record.constructor, record.get('id') );

    },

@wycats, @tchak, is it there any limitations to avoid reloading dirty records? I guess, i am hacking too much to get this functionality working.

[wagenet]

This issue doesn't seem to be progressing. At the moment we don't explicitly support partial loads. This is something we've considered and may add at some point in the future.

[brennanmceachran]

I know this is a closed and old thread, but I'm curious if anyone has found a work around. I'm in a similar situation as @rvega.

I can't load an the entire model in a list senario due to huge load times. I can shave off properties from the return w/ get parameters which makes the calls to the lists more performant.

model.reload() helps because I can call model.reload() when I transition to a route which will display the full model. However if the list gets called again somewhere else in my app, the full model pops back to a partial model.

Are there any ways around this?

[bobbus]

@brennanmceachran , I would suggest you to read this answer on SO. I think it's a good solution for this problem.

[brennanmceachran]

@bobbus it's a potential way around a problem, but I don't think it's really a solution. You're modifying your backend/api because your frontend can't differentiate between fully and partially loaded models, and adding a second model on the frontend.

Would that not mean all benefits of passing an object via {{#linkTo "singleroute"}} are gone? As you now have to load up your second model / you can't bank off of the data already loaded.

I would have assumed that a call to a "full" model would just add the properties that were missing. :(

[bobbus]

You're right, I would not mean this will be the final solution, I just can't find a better workaround for now. I read partial objects loading is in the roadmap for ember-data but I guess it's not planned for now.

[coladarci]

I'd love to know if in the past 6 months, anyone's found a better solution. While @bobbus is right that there's a decent work around, it results in having to keep two separate models manually in sync (if you update/delete the full record, the teaser record doesn't know to be updated/deleted).

[brennanmceachran]

We still haven't found one... but i think the jj-abrams branch is attempting to solve some of these issues...

[chopper]

Any updates on this? I've been trying a few hacks, but haven't found a satisfactory solution :-/

[elsurudo]

Having the same issue. This seems like a really, really common use-case, no? Should definitely be something baked into the framework, IMO

[sly7-7]

@chhopper @elsurudo I'm not sure, but I think the store support this feature at some point. I can see the push method taking a 'private' third optional argument which indicates the incoming data should be merge instead of replacing the current one. Perhaps even the update method is aimed to be public. You can use it, but the data you be normalized.

@bradleypriest Do you know what's the status of this ?

[bradleypriest]

@chhopper @elsurudo @sly7-7 yes and no, depending on your specific usecase.
No there isn't specific partial model support in ED.
However since this ticket was first opened, both reload and push methods have been added which solve most of the issues mentioned in this ticket.

[sly7-7]

@bradleypriest thanks for the answer. So perhaps using the store.update() I previously mentioned is the way to go in their use case. Is this something we want to go further with ? I was thinking something like calling the serializer.normalize and then push the normalized partial data.

[bradleypriest]

If someone gives me a use-case I'd be happy to work through it, all the examples are pre-jj-abrams ED so I don't know what people are struggling with

[coladarci]

@bradleypriest , a use case that is more complete but based on the one that started this thread is you have a blog with articles:

• homepage as a list of blog teasers - you only want the title of the article, so you only return the title from the server. (the reason being your full objects are monsters)
• you click on one of them and are taken to the blog post page, at which point you'll want to "refresh" the record so that you get the rest of the data that wasn't return for the teaser representation. I've gotten this far by using the hooks provided in the model.
• The problem comes when you are now on that blog detail page and you need to refresh the teasers (a stretch for this example, but let's say the teasers always need to be present - like in a drop down or a side column) . When that store updates, the teaser version of the model that is returned from the server wipes out the full version of the model you just pulled for the detail page so your detail template breaks.

FWIW, a hacky way around this problem is to create a separate Model for the teaser (PostTeaser) versus the full (Post). Other than having to monkey with your backend to accommodate the api, the only problem I ran into is that this doesn't play nice with link_to ("link_to 'post' teaserobjectinstance" needs to point to the post route, not the teaser instance model route). I think if link_to would accept an id, instead of an object, this would work (i.e PostTeaser.id == Post.id, so if I could just pass the id to link_to, all might be well).

[sly7-7]

@coladarci I was thinking about the same use case, it seems to be fair. In our app we have this kind of scenario.
As a side note, I was thinking that passing an id to the link-to helper should work. Maybe I'm just confused.
Seems like it should really work, it's documented here: https://github.com/emberjs/ember.js/blob/227f5cf6847b68941d51434b6963787a4fe27870/packages/ember-routing/lib/helpers/link_to.js#L646

[bradleypriest]

For the teasers and the full posts don't you already have two different endpoints then so they're two different models anyway?

[coladarci]

@bradleypriest , the way I set it up, no - I have different logic in my index api function versus my show api function. One asks for teasers and the other asks for the full.

[coladarci]

@sly7-7 - glad you linked to that - I gave up on this months ago before 1.0 was released. I'm thinking that was a later add. So I actually think between @bradleypriest point about having a different endpoint for teasers (which sounds reasonable) and the fact that link_to can take an ID, we are best off having two models and link_to with the id! For those who are confused about why this helps with the "refresh" issue - the moment we are switching models on the detail route, it'll be forced to pull a fresh model by id since there is nothing already loaded..

[bradleypriest]

Yeah, I'm not even sure what the library would be able to provide here that would solve your problem in a simpler way. It doesn't seem like it's too much code on top of the basics whichever way you slice it

[coladarci]

@bradleypriest - I agreed completely - the link_to no longer working was a deal-breaker as I had to start manually forming complicated links with unbound variables and then those links' active states weren't working correctly as routes changed.

The library would have to support a nondestructive load where existing models aren't wiped out if less data is provided from the server. That said, I can see why this would be problematic to say the least.

[bradleypriest]

@coladarci that support exists via the update function. The original usecase was for websockets and friends but it wouldn't be too hard to wrap it to provide partial_find_and_update

[coladarci]

Interesting - it sounds like more work than to just maintain the teasers with a separate ObjectModel all together - is there an example of update being used in this way that you could point us to?

[chopper]

I had the same use case as many here. The way I solved it was to have a separate models (a long App.Payment and short App.PaymentSearchResult with some additional information about the search, e.g. distance to the search location). App.PaymentSearchResult stores the id of the corresponding App.Payment and I use linkto with the payment_id (note that just using payment via a DS.belongsTo will result in many unnecessary requests to your backend). Thanks to @sly7-7 for the hint!

This is an ok solution for me, but since this is a very common use case it could make sense to have support for this baked in. For example, what if the backend passed a _partial attribute with the serialized data, which would allow the store to distinguish between full and partial data? I'm sure there's implications and trade-offs, but something along those lines could be useful to many, rather than having this re-implemented over and over.

Also this approach required me to monkey-patch my backend (since requests now go to /payment_search_results). This isn't necessarily a problem but it would be nice to be able to specify a URL on a model-by-model basis (that way I could, for example route requests to /payments/search). This functionality used to exist in ED, but I believe it was removed. Does anyone have updates on this?

[watsonbox]

I've written a blog post outlining a partial model loading solution which works well for me. You can also find an alternative store-level approach to avoiding overwriting complete models with partial models, and more discussion on this subject, here.

[ZenCocoon]

There's now a Ember CLI addon to solve this at https://github.com/BookingSync/ember-data-partial-model

[toobulkeh]

Are there downsides to including that addon into ember-data core?

[tchak]

@toobulkeh including properly this kind of functionality in core require to make all attributes accessors async (aka return a promise). This is something that is discussed but have a lot of implications. It could be not too much of a problem once async/await are more wildly used.