New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Ember Data url templates #4

Open
wants to merge 4 commits into
base: master
from

Conversation

Projects
None yet
@amiel

amiel commented Aug 20, 2014

This is my third draft after feedback from @igorT.

There are still some details that need to get ironed out, but I thought it was about time to get community discussion on this.

Thanks!

(RFC content pasted in here by @machty)

  • Start Date: 2014-08-20
  • RFC PR:
  • Ember Issue:

Summary

url-templates improves the extesibility of API endpoint urls in RESTAdapter.

Motivation

I think that Ember Data has a reputation for being hard to configure. I've often
heard it recommended to design the server API around what Ember Data expects.
Considering a lot of thought has gone in to the default RESTAdapter API, this
is sound advice. However, this is a false and damaging reputation. The adapter
and serializer pattern that Ember Data uses makes it incredibly extensible. The
barrier of entry is high though, and it's not obvious how to get the url you need
unless it's a namespace
or something pathForType
can handle. Otherwise it's "override buildURL". RESTSerializer was recently
improved to make handling various JSON structures easier; it's time for url
configuration to be easy too.

Detailed Design

buildURL and associated methods and properties will be moved to a mixin design
to handle url generation only. buildURL will use templates to generate a URL
instead of manually assembling parts. Simple usage example:

export default DS.RESTAdapter.extend({
  namespace: 'api/v1',
  urlTemplate: '{host}{/namespace}/posts{/id}'
});

Resolving template segments

Each dynamic path segment will be resolved on a singleton object based on a urlSegments
object provided in the adapter. This urlSegments object will feel similar to defining
actions on routes and controllers.

// adapter
export default DS.RESTAdapter.extend({
  namespace: 'api/v1',
  pathTemplate: '{/:namespace}/posts{/:post_id}{/:category_name}{/:id}',

  pathSegments: {
    category_name: function(type, id, snapshot, requestType) {
      return _pathForCategory(snapshot.get('category'));
    }

    post_id: function(type, id, snapshot, requestType) {
      return snapshot.get('post.id');
    };
  }
});

Psuedo implementation

export default Adapter.extend({
  buildURL: function(type, id, record, requestType) {
    var template = this.compileTemplate(this.get('urlTemplate'));
    var templateResolver = this.templateResolverFor(type);
    var adapter = this;

    return template.fill(function(name) {
      var result = templateResolver.get(name);

      if (Ember.typeOf(result) === 'function') {
        return result.call(adapter, type, id, record, requestType);
      } else {
        return result;
      }
    });
  },
});

Different URL templates per action

Configuring different urls for different actions becomes fairly trivial with this
system in place:

Usage

// adapter
export default DS.RESTAdapter.extend({
  urlTemplate: '/posts{/:post_id}{/:id}',
  createUrlTemplate: '/comments',
  hasManyUrlTemplate: '/posts/{:post_id}/comments',

  pathSegments: {
    post_id: function(snapshot) {
      return snapshot.get('post.id');
    };
  }
});

Drawbacks

  • Building URLs in this way is likely to be less performant. If this proposal is
    generally accepted, I will run benchmarks.

Alternatives

The main alternative that comes to mind, that would make it easier to configure
urls in the adapter, would be to generally simplify buildURL and create more
hooks.

Unresolved Questions

  • How do we handle generating urls for actions that do not have a single
    record? This includes findAll and findQuery, which have no record, and
    findMany and findHasMany which have a collection of records.

@amiel amiel force-pushed the amiel:path-templates branch from 9206dc0 to 03a5aa2 Aug 20, 2014

@amiel amiel changed the title from Initial path templates proposal to RFC: Ember Data path templates Aug 20, 2014

@amiel amiel force-pushed the amiel:path-templates branch from 03a5aa2 to d7a0b11 Aug 20, 2014

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Aug 20, 2014

How would you see configuring for multiple fetch types, e.g. findHasMany, etc.., using the pathSegments hash? If that was needed.

@amiel

This comment has been minimized.

amiel commented Aug 20, 2014

@knownasilya Unless I am misunderstanding your question, there is a section on this. Basically, buildURL would get passed an action type and would look up a template specific to that action (hasManyPathTemplate, createPathTemplate, etc).

@amiel amiel force-pushed the amiel:path-templates branch from d7a0b11 to 0eb6f63 Aug 20, 2014

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Aug 20, 2014

@amiel ah, ok so it would reuse the existing segments in the pathSegments hash.

@amiel

This comment has been minimized.

amiel commented Aug 20, 2014

@knownasilya Correct, and there might be segments in pathSegments that only apply to some of the templates.

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Aug 21, 2014

I really like this proposal, and can see it being one of the things that makes ED very usable in many situations.

@trek trek added the ember-data label Aug 22, 2014

@jayphelps

This comment has been minimized.

jayphelps commented Sep 30, 2014

Can someone add some usage examples, from the store-side?

e.g. given:

// comment.js
export default DS.RESTAdapter.extend({
  pathTemplate: '/:namespace/posts/:post_id/comments/:id',

  pathSegments: {
    post_id: function(record) {
      return record.get('post.id');
    }
  }
});

How do I find one of these comments via the store methods?
Something like this?

var comment = this.store.find('comment', 1, { post_id: 15 });
@amiel

This comment has been minimized.

amiel commented Oct 1, 2014

@jayphelps Good question.

I like your idea of

var comment = this.store.find('comment', 1, { post_id: 15 });

This is kind of related to this unanswered question.

@blimmer

This comment has been minimized.

blimmer commented Oct 1, 2014

I really like this proposal, and would love to see something like this in Ember Data. I've done a lot of reading on this topic today, and have worked around the issue as follows.

For certain actions, we namespace the URL to a particular customer. For instance,

http://endpoint.com/customer/123/settings.json

for populating the settings model.

Because we don't have something as described here in Ember Data, we're doing this hack in pathForType in our adapter:

  pathForType: (type) ->
    switch type
      when 'settings' then "customer/#{@get('session.customer.id')}/settings"

Where session is injected into our adapter and we require the user to be logged in to make this request.

It would be preferable if we could do this the way @amiel has suggested (expect with a findAll instead of a find:

@store.findAll('settings', { customer: @get('session.customer.id') }

So, anyway, I'm really a big fan of this idea and would love to see this implemented 👍

@garrettlancaster

This comment has been minimized.

garrettlancaster commented Nov 21, 2014

👍

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Nov 21, 2014

Will it also be possible to add query params to the paths? e.g.

pathTemplate: ':namespace/:type?owner=:owner'

Or maybe even something like:

this.store.find('comment', 1, { queryParams: [['owner', 4]] });
pathTemplate: ':namespace/:type/:id'
// api/comments/1?owner=4

And they are added if specified.

@amiel

This comment has been minimized.

amiel commented Nov 21, 2014

Yes, it will be more like:

pathTemplate: ':namespace/:type/:id'

this.store.find('comment', 1, { owner: 4 });
// api/comments/1?owner=4
@MiguelMadero

This comment has been minimized.

MiguelMadero commented Nov 23, 2014

👍

@nikz

This comment has been minimized.

Contributor

nikz commented Nov 27, 2014

👍 for this - let me know if there's anything I can do to help with development!

@stefanpenner

This comment has been minimized.

Member

stefanpenner commented Dec 5, 2014

Doesn't the jsonapi adapter have some support for this? Is this something that must be backed into ember data, or does everything exist for it to be an adapter concern?

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Dec 5, 2014

@stefanpenner at a minimum it should be in the RESTAdapter by default.

@davidmaitland

This comment has been minimized.

davidmaitland commented Jan 8, 2015

Any update on this? This would potentially stop us moving away from ember-data to a custom adapter.

@amiel

This comment has been minimized.

amiel commented Jan 9, 2015

It was my intention to implement this. Since I wrote this RFC I have changed companies and am now just getting approval for some open source time. I'll see if I can get started next week. If anyone is interested in pairing on this, let me know. I'll also be at Emberconf, and would love to chat about it there :)

In the mean time, I would love some input on the Unresolved Questions.

@wycats

This comment has been minimized.

Member

wycats commented Feb 6, 2015

I really like the idea of using URL templates for this problem. However, I think it would be better to use the URL template spec for this purpose. For example, the URL template spec allows you to expand arrays or maps into various structures (like comma-separated lists or query params), which is much more expressive than the :segment syntax.

We might consider supporting the :segment syntax as a shorthand for people familiar with Ember/Rails/etc. routing syntax, but I really like the expressiveness of the template syntax. This is also why we use URL templates in JSON API rather than a simpler ad-hoc solution.

Can people investigate the smallest, most embeddable URL template expander written in JavaScript? If there is none that is particularly small, it shouldn't be too hard to make a new microlibrary, but I bet one already exists.

amiel added a commit to amiel/data that referenced this pull request Feb 21, 2015

Move buildURL and related methods to a mixin
This is my first step towards emberjs/rfcs#4.

The intent here is to separate url building from the RESTAdapter so that
it can eventually be included in other adapters and hold the logic for the
path templating described in emberjs/rfcs#4.

amiel added a commit to amiel/data that referenced this pull request Feb 21, 2015

Move buildURL and related methods to a mixin
This is my first step towards emberjs/rfcs#4.

The intent here is to separate url building from the RESTAdapter so that
it can eventually be included in other adapters and hold the logic for the
path templating described in emberjs/rfcs#4.
@knownasilya

This comment has been minimized.

Contributor

knownasilya commented May 6, 2015

@jamesarosen see #32 and davewasmer/ember-data-actions#1

Would love your feedback on that last issue.

@amiel

This comment has been minimized.

amiel commented May 7, 2015

@jamesarosen overriding verbs seems handy, but I think it could be handled more simply than with url-templates. I'll suggest an implementation in emberjs/data#3047.

@amiel

This comment has been minimized.

amiel commented May 7, 2015

@knownasilya @davidmaitland Let me know if you have any issues with ember-data-url-templates. I only have it set up in one project, so it may not be as generic as I think.

Also, I've tried to document installation and usage, but I'm probably forgetting something. I'd be happy to pair on helping you get set up :)

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented May 7, 2015

@amiel might be worthwhile sending a PR to @davewasmer's https://github.com/davewasmer/ember-data-actions repo to use this addon.

@amiel

This comment has been minimized.

amiel commented May 7, 2015

@knownasilya I'll look in to it.

@mgenev

This comment has been minimized.

mgenev commented Jun 24, 2015

It's been a year, is there no solution for this yet?

@igorT

This comment has been minimized.

Member

igorT commented Jun 24, 2015

@mgenev the addon has been up and running for a long time now

@amiel

This comment has been minimized.

amiel commented Jun 24, 2015

@mgenev We're looking for input with the addon. Please let me know if you get a chance to try it.

https://www.npmjs.com/package/ember-data-url-templates

@mgenev

This comment has been minimized.

mgenev commented Jun 24, 2015

@amiel thank you, i'm on it, i'll report back

@billpull

This comment has been minimized.

billpull commented Jun 24, 2015

@amiel add-on working for me thank you.

Anyone on ember-data team have an indication of if this path will be integrated into ember or if this will become deprecated and another nested url solution available?

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Jun 24, 2015

Would love to see an example of this add-on working with the new adapter options..

@amiel

This comment has been minimized.

amiel commented Jun 24, 2015

@knownasilya what new adapter options are you talking about?

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Jun 25, 2015

@amiel

This comment has been minimized.

amiel commented Jun 25, 2015

@knownasilya Oh sweet! I'll have to take a more in depth look. Maybe I missed it; are these options passed to buildURL?

@igorT

This comment has been minimized.

Member

igorT commented Jun 27, 2015

@billpull the idea is for people to use this addon, and then judge based on the feedback whether it's worth the complexity of brining it to ED itself. As addons now work pretty well, seems sometimes better to opt out of making core bigger and bigger

@bcardarella

This comment has been minimized.

Contributor

bcardarella commented Nov 24, 2015

I'm curious to know what the status of the RFC is. We've used the addon: https://github.com/amiel/ember-data-url-templates and it works great.

@amiel

This comment has been minimized.

amiel commented Jan 4, 2016

FYI: There's a bunch of new documentation up at https://github.com/amiel/ember-data-url-templates/wiki

I'm not sure how the ember-data core team feels about bringing this in to core yet. Here are my thoughts:

First of all, while ember-data-url-templates still feels new (there haven't really been many changes since the first release), it is getting used by a lot of projects (according to ember observer it is in the top 10% of downloads). Also, the amount of code in the actual plugin is very small (< 100 lines).

If that were the end of it, I'd vote strongly for including it in ember-data core immediately.

However, the current templating library (geraintluff/uri-templates), is 4.5k minified and I think that is too much to include in ember-data by default.

geraintluff/uri-templates supports level 4 of RFC6570, which is great for the addon, but maybe not necessary for the basic use-case.

I think ideally ember-data would include a micro-library for supporting RFC6570 level 1 or 2 and that it would be easy to plugin in support for level 4 templates if needed.

@oligriffiths

This comment has been minimized.

oligriffiths commented Jan 6, 2016

@amiel Hey, this is interesting, just came across this. We've been working on a prototype system to interface with a fairly unstandardised API, with very little convention. Some routes use the correct HTTP verbs, some don't, some require body data, some don't.

I developed a very basic URL scheme mapper, that would resolve a URL based on a provided object (model), a list of changed attributes, and an "action" that is being performed "fetchRecord" etc.

It looks like this could solve some of our problems, but our API is so non-standard some things become very hard. For example, on a post object we have a liked attribute that says whether the current user has liked the post previously. In order to like a post, we set the liked property to true, and save the post. When this happens, we need to send a POST request to a different URL than the one the post usually saves to.

What are your thoughts on this...

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Jan 6, 2016

@oligriffiths you are probably better of using something like https://github.com/mike-north/ember-api-actions, and do model.like() where that likes and saves the model to the like endpoint.

@oligriffiths

This comment has been minimized.

oligriffiths commented Jan 6, 2016

@knownasilya Thanks ill check that out.

However, we still have an issue that some routes don't map to HTTP verbs, e.g. to delete a post, we send a POST request to /post/delete with data of: id=X.

@knownasilya

This comment has been minimized.

Contributor

knownasilya commented Jan 6, 2016

@oligriffiths that would be solved by @amiel's template addon

@amiel

This comment has been minimized.

amiel commented Jan 6, 2016

@oligriffiths The concept behind this RFC is implemented in https://github.com/amiel/ember-data-url-templates. Based on what you said, I think https://github.com/mike-north/ember-api-actions will be useful to you. You can use ember-data-url-templates along with ember-api-actions.

@oligriffiths

This comment has been minimized.

oligriffiths commented Jan 7, 2016

@amiel Thanks, they both look excellent, however I'm not sure they cover the following cases:

Custom HTTP verbs for specific actions, e.g. POST to do a DELETE action
Custom request bodies for specific actions, e.g. POST data of id=X for a DELETE action

@amiel

This comment has been minimized.

amiel commented Jan 7, 2016

@oligriffiths that makes sense. I think you'll want to override deleteRecord then.

@oligriffiths

This comment has been minimized.

oligriffiths commented Jan 7, 2016

Ok, I'll try working on an extension to handle this case then, along with using those recommended addons

@erichonkanen

This comment has been minimized.

erichonkanen commented Jan 9, 2017

What is the status of this? It's been a year since last comment and I've found this while researching how to handle sub-resources with ember-data. It seems like there isn't a clear way to do what we're looking for which is to have a model e.g. User with a relation field User.friends: hasMany('user') where that relation field GET/PUT's it's data from a sub-resource /users/:id/friends

Is that what this rfc/related package aims to handle?

@rmharrison

This comment has been minimized.

rmharrison commented Jan 9, 2017

Yes, as far as I understand it, that's exactly what this RFC is intended to do. Back in Oct, I gave a talk [slides, video, sample repo] outlining my understanding of the problem, and the current workarounds available with @amiel's ember-data-url-templates addon.

To my understanding, there are two main hang-ups.

  1. As discussed, one of the hang-ups is the large file size of the uri-templates lib.
  2. Even with path resolution, the question of how to pass data between the route and the model adapter is still open. There isn't an attribute, in for example snapshot, present in all of the dozen or so fetchRecord et al. methods.
@amiel

This comment has been minimized.

amiel commented Jan 9, 2017

@rmharrison is correct. ember-data-url-templates is the current implementation of this RFC.

@erichonkanen Please let me know if you have any questions or feedback :)

pzuraq added a commit to pzuraq/emberjs-rfcs that referenced this pull request Nov 27, 2018

Merge pull request emberjs#4 from pzuraq/update-design-and-motivation
updates the design and motivation sections, along with some others
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment