Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upModule Unification #143
Conversation
This comment has been minimized.
This comment has been minimized.
|
@dgeb - Thank you very much for iterating on this! I am very much in favor of this proposal, I have only one or two relatively small concerns that I will try to comment on later tonight. |
This comment has been minimized.
This comment has been minimized.
|
@rwjblue - Thanks as well to you for all your help - your migration tool has proven invaluable. Looking forward to hearing any remaining concerns. |
This comment has been minimized.
This comment has been minimized.
|
I don't think I could be any happier about how this RFC has turned out. Our iterations have paid us back in dividends! |
This comment has been minimized.
This comment has been minimized.
willmanduffy
commented
May 9, 2016
|
Overall I think this is super cool and I can tell from just my first look into this that there was a lot of thought put into it. So for starters, thanks for your hard work! Couple things I wanted to ask about:
You list this as a design constraint, but does the RFC address this? For example I cloned down https://github.com/rwjblue/--ghost-modules-sample/tree/grouped-collections/src and opened up 10 files (a common use case for me) and suddenly... Secondly I have a mild concern about naming the folder Not a deal breaker at all, just something I wanted to raise. |
This comment has been minimized.
This comment has been minimized.
Yes. Take the following example:
In these cases, there are only a single file and the file can be named after its type. In today's pods you would have:
Extrapolating that out, would result in many many files named Now, as you have noticed it is not guaranteed to solve the full title bar issue (because there will likely still be many files named |
btecu
reviewed
May 9, 2016
| template.hbs | ||
| paginator-control | ||
| component.js | ||
| template.js |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
willmanduffy
commented
May 9, 2016
•
|
No argument here that this definitely improves a lot over the current pods structure, but alternately doesn't it make it worse for people migrating from a non-pods app (like me)? Suddenly I'm going from.
to
I can definitely see this making the titlebar problem worse for some. I should note that I probably have a pair of templates / js for maybe 70% of my components but this is anecdotal data tied to my coding style obviously. No info on how others work. |
This comment has been minimized.
This comment has been minimized.
landongn
commented
May 9, 2016
|
The only thing that seemed unclear to me was the designation of Otherwise this looks awesome! I'm a big fan of pods, but this just making the pod structure more organized, which is a growth pain that many medium-to-large projects will face when using pods. |
This comment has been minimized.
This comment has been minimized.
I'm concerned if this might create backward incompatibilities if we don't allow overrides / aliases. For example, if |
This comment has been minimized.
This comment has been minimized.
The modules on disk will not be changing. The |
dgeb
force-pushed the
dgeb:module-unification
branch
from
18df372
to
220516f
May 9, 2016
dgeb
force-pushed the
dgeb:module-unification
branch
from
220516f
to
fca9184
May 9, 2016
This comment has been minimized.
This comment has been minimized.
@landongn |
This comment has been minimized.
This comment has been minimized.
@willmanduffy I've tried to explain this concept in the teaching section for collections and types. Please let me know if this clarifies the waters at all or if I still have work to do. |
This comment has been minimized.
This comment has been minimized.
Agreed, but many other parts of this structure are quite a bit more ergonomic. For example, when working deeply inside a specific route structure in Also, even in classic structure the "titlebar problem" can happen. You may have I personally think that I would be fine with removing that from the list of constraints. I agree with you that we haven't fully addressed it, but I believe that the consensus on the core team is that we are satisfied with that aspect of the proposal. |
This comment has been minimized.
This comment has been minimized.
|
Editor plugins like https://atom.io/packages/ember-tabs go a long way to making the titlebar problem moot. |
This comment has been minimized.
This comment has been minimized.
|
@willmanduffy @rwjblue I agree that it seems like the RFC in its current form doesn't address this specific problem. I'm beginning to think that perhaps the best path forward is to identify the top N popular editors in the community, and recommend to users that they adjust their configuration accordingly to better display a relevant filename. Additionally, many editors allow you to drop a configuration file in the root of a project directory—we should explore whether there are better out-of-the-box configurations Ember CLI could generate for new projects. |
This comment has been minimized.
This comment has been minimized.
landongn
commented
May 9, 2016
This makes sense, though I'm struggling slightly with the name of 'collection'. The purpose of the A few of the terms took multiple parsing before I understood exactly what was happening. Am I parsing too heavily on the semantics side for this to be relevant? Probably, though it's worth considering if the muddying of the waters with regards to separation of concerns is worth discussing here for this RFC. |
This comment has been minimized.
This comment has been minimized.
I completely agree, that would be odd indeed. This proposal limits the locations for |
This comment has been minimized.
This comment has been minimized.
landongn
commented
May 9, 2016
•
|
@rwjblue my mistake, that was not clear to me. Thanks for clarifying! edit err...
Is edit edit: is there anything preventing me from importing a component from |
landongn
reviewed
May 9, 2016
|
|
||
| Every resolvable module must have both a `name` and a `type`. The `type` | ||
| typically corresponds to the base class of the module's default export (e.g. | ||
| `route`, `template`, etc.). |
This comment has been minimized.
This comment has been minimized.
landongn
May 9, 2016
How does this work if I'd like to export a singleton that's not a service or a discrete ember subclass? This phrasing implies that there is a requirement for a module to be a specific and explicit type, though I don't think that's the intention.
This comment has been minimized.
This comment has been minimized.
dgeb
May 9, 2016
Author
Member
@landongn The utils collection is intended for general ES modules. Its contents are completely ignored by the ember resolver. The name and type requirement is specific to resolvable modules.
This comment has been minimized.
This comment has been minimized.
dfreeman
May 9, 2016
•
That distinction threw me a bit in the listing of what private collections are allowed where.
Since the contents of utils aren't intended to be resolvable anyway, is it necessary to declare that utils is an allowed private collection everywhere?
This comment has been minimized.
This comment has been minimized.
dgeb
May 10, 2016
Author
Member
@dfreeman utils is allowed as a private collection to enable co-location of lib files wherever they are needed. This makes relative ES module imports cleaner, and makes for easier movement of whole "branches" of functionality.
The - prefix signals to the resolver that a directory is a private collection and has its own rules distinct from the collection that contains it. If you did not place local lib files in -utils, the resolver would treat them as members of the parent collection and try to interpret their name, type, etc.
landongn
reviewed
May 9, 2016
| Modules can be grouped together with other modules of related types in | ||
| "collections". Collections are directories with type-aware resolution rules | ||
| which allow related modules to share a namespace. For example, the `models` | ||
| collection contains models, adapters, and serializers. |
This comment has been minimized.
This comment has been minimized.
landongn
May 9, 2016
If I am not utilizing ember-data, but utilize the data folder collection, is there restrictions on naming things based on potential collisions with ember-data types, such as Adapter and Serializer?
This comment has been minimized.
This comment has been minimized.
dgeb
May 9, 2016
Author
Member
@landongn The ember-data addon will define the data group, the models and transforms collections, as well as associated types for any ember project in which it is installed. If ember-data is not installed, another addon could declare these same things or variations thereof.
Without any such declarations, ember's resolver would have no understanding of these concepts, and would not be able to resolve types like adapters and serializers. I suppose you could still use these same directories to store ES modules (like a lib dir), but the ember resolver would not resolve them.
This comment has been minimized.
This comment has been minimized.
landongn
reviewed
May 9, 2016
| // Rule 3 | ||
| src/data/models/author (with `export default DS.Model.extend()`) |
This comment has been minimized.
This comment has been minimized.
landongn
May 9, 2016
Same question here in regards to namespace collision. I don't understand enough about the rules for resolution here ( as well as why they're doing what they're doing ) but I don't use ember-data and have concepts of models that somewhat align with this use case, but export Ember.Object rather than DS.Model. Is this a problem?
This comment has been minimized.
This comment has been minimized.
rwjblue
May 9, 2016
Member
No not at all. But if you intend to use the resolver for finding your models/adapters/serializers then you will need to register the collection and allowed types yourself (this will be a thing that ember-data does for you if you had it).
This comment has been minimized.
This comment has been minimized.
landongn
May 9, 2016
Got it. Thanks. Has this been documented in the guides anywhere (self-registering modules)?
This comment has been minimized.
This comment has been minimized.
rwjblue
May 9, 2016
Member
As part of the work to implement this RFC, the guides and API docs will need to be updated. Nothing exists today, since this is the first time we have discussed it.
This comment has been minimized.
This comment has been minimized.
landongn
reviewed
May 9, 2016
|
|
||
| The main module also declares other properties that help the Ember resolver | ||
| understand relationships between projects. For instance, the main module can | ||
| declare which modules in an addon are available to a consuming app's resolver. |
This comment has been minimized.
This comment has been minimized.
landongn
May 9, 2016
In the use case of a custom resolver, given the folder structure:
- ui
- components
- filter-select
- pointer
- component.js
- template.hbs
- controller
- component.js
- template.hbs
- touch
- component.js
- template.hbs
would additional work need to happen to tweak that custom resolver to utilize the new module patterns? (e.g; the part under components where a specific component, as an example, a filter-select that needs to be loaded based on some app state (e.g; we're in a 'mode' for the app that corresponds to the subdirectories under src/ui/components/filter-select/[pointer|touch|controller]/component.js -- can we still abstract that as part of the resolvers duties, or would this require elimination of the resolver or some alternative approach?
This comment has been minimized.
This comment has been minimized.
rwjblue
May 9, 2016
Member
It will absolutely be possible to create a custom resolver that extends from the resolver changes proposed here. I would assume that your current custom resolver would need to be rewritten to take advantage of the new system, but it should be straightforward.
dfreeman
reviewed
May 9, 2016
| * `ui` - components, partials, routes | ||
|
|
||
| The collection and type system is designed to be extensible, so that addons can | ||
| contribute their own collections and types. The `data` collection and its |
This comment has been minimized.
This comment has been minimized.
dfreeman
May 9, 2016
•
Can addons define additional allowed types for an existing collection? For instance, I could imagine ember-data-model-fragments wanting to create a fragment type that's allowed in the data collection.
This comment has been minimized.
This comment has been minimized.
rwjblue
May 9, 2016
•
Member
Yes, assuming fragment was not already reserved by something else, addons will be able to add types to existing collections. The specific node-land API to do this is still TBD, but it will exist. There is nothing "magical" about the collections discussed in this RFC other than that they apply to almost all Ember app. These collections and types will all be declared via the same node API that other addons use.
willmanduffy
reviewed
May 9, 2016
|
|
||
| The following collections and allowed types (rules 1 & 2) are proposed: | ||
|
|
||
| * `components` - COMPONENT, HELPER, template |
This comment has been minimized.
This comment has been minimized.
willmanduffy
May 9, 2016
Feels pretty confusing to me to bundle helpers in with components like this. Is there a reasoning on why they can't be a collection of their own? I totally see the advantage if the helper is coupled tightly with a specific component, but I have a generic helper like format-money that feels weird when grouped under components with
src/ui/components/format-money/helper.js
It's like we're burying the lede that it's a helper and not a component.
This comment has been minimized.
This comment has been minimized.
willmanduffy
May 9, 2016
Also (sorry for the double email notification) the distinction occurs to me that helpers do not require a - in their names, unlike components. Will that cause any friction if we are treating them as the same here?
This comment has been minimized.
This comment has been minimized.
rwjblue
May 9, 2016
Member
The RFC discusses this a bit in the "Components" Collection section. Dan writes it much better than I can here.
In general, the idea is that there is a single group of things that are invokable in a template. It makes a ton of sense that this group share a directory structure. If nothing more than to make it very clear that they are sharing a competing namespace.
For example, if you had both app/helpers/foo-bar.js and app/components/foo-bar.js in a "classic" structured app what do you expect to happen when you invoke {{foo-bar}}? I know what does happen, but it is definitely not obvious what is the "least surprising" thing.
Sharing a single directory for both types allows us to remove this manual constraint management that everyone currently has to do when creating a helper or component...
This comment has been minimized.
This comment has been minimized.
mattmcmanus
May 10, 2016
I feel similarly here. When I was picking up ember, reasoning about why I should write something as a component or a helper wasn't very clear. I defaulted to making everything a component and then spent some time porting several to helpers after running into glimmers too-many-components rendering issues. This change feels like it may make the distinction even less clear.
This comment has been minimized.
This comment has been minimized.
sandstrom
May 21, 2016
•
I feel similar. However, I totally buy the namespace argument, it's a good one! And I don't have a major objection against components including both components and helpers. Still, perhaps there is another name that could be used?
pieces, bits or aid are a few words that could work.
This comment has been minimized.
This comment has been minimized.
@landongn Perhaps the key point here is that the Ember resolver will use the conventions defined in this RFC to understand where to find things. If you don't care if your base component is accessible from the resolver, you could place it anywhere and import it into your subclass. That is all happening at the ES module dependency level. However, I wouldn't recommend doing this because chances are that you will also want to to co-locate tests for your base component, and the test runner will also use a resolver to find tests in collections. Incentives like this should be strong encouragement for developers to follow conventions. |
This comment has been minimized.
This comment has been minimized.
mitchlloyd
commented
May 10, 2016
|
I'm trying to understand the need for As mentioned earlier, utils are not relevant because Ember doesn't resolve utils. Now that there is no way to invoke nested component (e.g. Is the intent here to prevent naming collisions in case someone wants a route named |
This comment has been minimized.
This comment has been minimized.
webark
commented
Oct 9, 2016
|
i think in order to fufill all of the design constraints, things like having styles and tests alongside thier respective components, having components that are specifically scooped to other components, relocatabilty, local lookups, amongst others, a directory tree approach is needed. these trees would now be representing routes including sub routes, as well as nested components. So you can't think of it as a single level. I've been using pods as they are, and for me, even though there are definatly downsides which this rfc addresses, are far superior then haveing 10's to sometimes 100's of components all in one directory. like for even the date picker component.. that is not the only component you would have.. using this pods, or this rfc, you could organize it to have a month component, that inside it, and scooped to it, you could have a week, then inside that a day. In my experience this has led to more manageable, powerful and elegant solutions due to having a component at each level, and everything that comes with them. to do this in a flat pattern can be coveluted,and lacks the nessicary contexts. Plus with the ability to have your styles, templates, and eventually tests all at that same level greatly aids in portability and manageablity. However it seems that having all the specific types of files named exactly the same thing exasperates a decent and vocal chunck of the community, seems unwise to ignore. and though i feel directories are a must, my ... TL;DR |
This comment has been minimized.
This comment has been minimized.
TameBadger
commented
Oct 10, 2016
•
@Gaurav0 |
This comment has been minimized.
This comment has been minimized.
Ember CLI's addon system allows for a great deal of flexibility. Some people are experimenting with ESLint while I and most of the Ember ecosystem continue to use the jshint based linting currently included by default in the stable version of ember-cli. When it is determined by consensus what the best ESLint rules are for Ember, we will hopefully all coalesce around a single shared configuration, just as we are in the process of doing here for file structure. This will best allow us to all use the best practices as discovered by the community, while continuing to have all the advantages of having us all do the same thing. One such advantage is, as mentioned in this RFC, being able to write an addon that upgrades source code for us to the new directory structure we've decided on. No matter what file structure is decided on in this RFC, you will be able to change it for your project by writing an addon. But I hope you will realize that sharing the solution agreed upon by others is a better choice. |
This comment has been minimized.
This comment has been minimized.
TameBadger
commented
Oct 10, 2016
|
That sounds awesome! I agree a 100 percent on:
I'll add to this that, I don't think it's worth trying to convince everyone there is one solution that will suit everyone. It's a better approach (IMO) to say, we have primitives available to allow people to create solutions that are shareable, with a main solution as the "Ember" solution. And that's basically what you are also saying with:
So I'm happy if that's the case. |
This comment has been minimized.
This comment has been minimized.
and
We actually do have this today and retain it with the approach being discussed in this RFC. Under discussion is the default behavior of the resolver. But anyone can choose to supply their own custom resolver that augments this default behavior or, if you wish, entirely replace it with a resolver that finds modules dynamically based on your preferences. In that sense @rtm's desire already exists:
We wouldn't need to revisit the concept of the resolver because this a point of the resolver. That said: many specific but arbitrary project structures places a slight burden on developers moving from project to project. And, as @Gaurav0 so eloquently notes, this places a high burden on addon authors. So, if you opt into a entirely custom resolver it's unfair to expect addons to work bug-free in your custom app or to expect addon authors to make their addon work with your app. A world where an application developer needs to expend effort to make addons correctly interact with their application is the world everyone using Ember is explicitly opting out of. |
This comment has been minimized.
This comment has been minimized.
grapho
commented
Oct 11, 2016
|
I will just put in my two cents: I have been at Ember for a while now.. and have seen many changes.. and many breaking changes as well. Though I sympathize with concerns (especially those of beginners and outsiders) that pushing people into a new organizational structure will case some upset. At the end of the day, Ember is a framework and well, sometimes you need to change the frame to allow bigger and better things. All I personally care about is if I use the framework, will my life be easier, or will it be harder? If I just follow the new rules of the framework.. will I be able to make Apps better, faster, stronger? I believe that is going to be a vast improvement and I cannot wait for this to land.. just the module structure.. for now, I don't even know yet how excited I am about the things that will follow Module Unification.. all I know is that the classic app structure is outdated and inefficient for a large app.. and Pods does not solve my issues. Anyway, good work, that is my final comment :) |
This comment has been minimized.
This comment has been minimized.
TameBadger
commented
Oct 12, 2016
|
I'm with you @grapho ! Really excited for this, it's already going to be massive improvement and is indeed necessary. @trek, on the following:
I'm just wondering if there is a possibility for resolvers to all output some structure that addon authors can count on..like a resolved-map structure that others can create regardless of file and directory naming and structure. This is probably over simplifying it, and more investigation necessary. Any recommended code to look at will be appreciated, I will start with https://github.com/ember-cli/ember-resolver. |
This comment has been minimized.
This comment has been minimized.
|
I'd like to unpack the philosophy behind the organizational structure more because I think it will go a long way towards alleviating concerns about "too much magic" through conventions and give a glimpse into some options for ember modules in the future. Let's start by saying that every default type in a collection (e.g. components, models, routes, etc.) may have a default representation plus named aspects that represent different facets of it. For example, a "component" can have a default representation that is a This concept of a default representation plus named aspects aligns well with ES6 modules, which can have both default and named exports. This RFC carries this parallel further into the file layout, which allows two forms of naming in collections:
Let's call the first form "compact" and the second "exploded". The compact form bundles together all the related aspects of a "thing" (i.e. component, model, etc.) into a single file with named and/or a default exports. The exploded form spreads the named aspects into separate files in a subdirectory. Let's take a look at a simple date-picker component in both forms. Here's the exploded form, spread across three files: // src/ui/components/date-picker/component.js
import { Component } from 'ember';
export default Component.extend({
});
// src/ui/components/date-picker/template.hbs
<div class="date-picker"></div>
// src/ui/components/date-picker/style.css
.date-picker { color: red }This is equivalent to the compact form, all in one file: // src/ui/components/date-picker.js
import { Component } from 'ember';
export default Component.extend({
});
export const template = hbs`
<div class="date-picker"></div>
`;
export const style = `
.date-picker { color: red }
`;Note that we're not actually recommending the compact form now because of the need for:
Expect future RFCs for both. The purpose of this example is to point out how the compact and exploded forms are compatible and complementary. You could define your default component export in // src/ui/components/date-picker.js
import { Component } from 'ember';
export { default as template } from './date-picker/template.hbs';
export default Component.extend({
});
// src/ui/components/date-picker/template.hbs
<div class="date-picker"></div>All the conventions are doing is eliminating the need for the above re-export. Because this RFC's layout and ES6 modules themselves are all statically analyzable, we have the opportunity to do this kind of merging of modules at build time to minimize the number of modules actually used and thus improve run-time performance. We are not actually taking away the right or ability to do this kind of wiring on your own. We are simply eliminating the need to do so when you use the default resolver, which will be a big plus for developer ergonomics. One other thing to note is that modules in their compact form may go a long way to reducing heartburn over file-name repetition. A component can be exported from I hope that this expansion of the philosophy behind this RFC relieves some concerns. I had initial concerns about over-complicating an already long RFC by unpacking this, but the subsequent discussion has made me realize it's important to share. |
This comment has been minimized.
This comment has been minimized.
|
My thanks to everyone who has contributed to this RFC thread. A proposal that touches one of the most fundamental aspects of an application—how its file structure is laid out—is one that is important to get right. We can't do that without feedback from as wide an audience as possible. That this discussion was so spirited, respectful, detail-oriented and productive (given the internet's tendency to be otherwise) makes me incredibly proud to be part of this community, and I think shows the value of our RFC system. After two Final Comment Periods (FCPs), the core team feels confident that any glaring problems have been identified and we can now begin working towards implementation of the RFC. The biggest tradeoff of the current proposal is editor ambiguity caused by identically-named files. We've heard those concerns loud and clear. The goal of an RFC is not to achieve perfection out of the gate. In that case, RFCs would take years, if they ever shipped at all. Rather, RFCs should provide a clear, incremental step forward, while still leaving us the opportunity to address in the future any issues that were identified. In this case, we feel confident that the editor confusion issue can be addressed in one or more of the following ways:
I'd like to thank @dgeb for his hard work and diligence over the more than six months he's been working on this. This proposal unlocks many different performance optimizations that will be critical for improving the load time of mobile apps, as well as providing a more consistent experience for developers that's in line with Ember's principle of Convention over Configuration. I'm excited to use the new file structure in my own apps. |
tomdale
merged commit e26f13d
into
emberjs:master
Oct 18, 2016
tomdale
removed
the
Final Comment Period
label
Oct 18, 2016
This comment has been minimized.
This comment has been minimized.
webark
commented
Oct 20, 2016
|
I know this is closed, but just a question on where mixins will got with this? Or are you planning on phasing those out. |
This comment has been minimized.
This comment has been minimized.
|
@webark mixins are not part of the resolver story, to use them you have to |
This comment has been minimized.
This comment has been minimized.
webark
commented
Oct 21, 2016
|
@locks ... |
kellyselden
referenced this pull request
Oct 28, 2016
Closed
[Fastboot support] Integrate building & serving fastboot in ember-cli #6350
cibernox
referenced this pull request
Nov 20, 2016
Open
Enhance contextual-components & helpers #179
This comment has been minimized.
This comment has been minimized.
paulyoder
commented
Dec 6, 2016
|
FYI: here's the issue in Ember CLI's repo that's tracking the progress of this feature. |
Turbo87
referenced this pull request
Jan 10, 2017
Merged
Resolve absolute imports from the ember module prefix #103
This was referenced Mar 2, 2017
This comment has been minimized.
This comment has been minimized.
bartocc
commented
Mar 29, 2017
|
Here's another issue in the emberjs repo tracking this |


dgeb commentedMay 9, 2016
Rendered