Resource/context folder structure instead of structure based on file type #642
Replies: 27 comments
-
I really like this idea! I like how it's similar to a lot of the React applications I've worked on where the files are placed based on hierarchy. That being said, what are your thoughts on this kind of file structure with less emphasis on the
|
Beta Was this translation helpful? Give feedback.
-
@BlakeWilliams Thanks for the feedback! Could you explain a bit more what you mean by common/shared not being special? I didn't think it was but maybe I'm misunderstanding |
Beta Was this translation helpful? Give feedback.
-
Sure! I think it's my assumption/misunderstanding here. I assumed that since Does that help clarify my thought process on it? |
Beta Was this translation helpful? Give feedback.
-
Oh yes, I see what you mean. I agree with that. My only concern here is that that may require using require "src/common/**" # Load common/shared stuff first
require "src/apps/**" # Load everything else after That way any new folder you add to Alternatively, if you had a folder structure like this:
You'd have to manually require the feature folders. Maybe that is worth it though? require "src/common/**" # Load common/shared stuff first
require "src/users/**"
require "src/store/**" Thoughts? |
Beta Was this translation helpful? Give feedback.
-
Or maybe it could be required like this... require "something_that_needs_to_be_loaded_first.cr"
require "src/common/**"
require "src/**" # Load everything else after Maybe that would be nice. The structure would look like this
|
Beta Was this translation helpful? Give feedback.
-
I've seen some frameworks that do weird structures like this as well, and I'm not a fan. Looking at a folder structure for an app shouldn't be where you look to see what the app does. It would be like looking under neath a car to see how fast it goes. This breaks convention because every app is going to function a bit different. Right now our largest Lucky app doesn't have users, or authentication, and our models have been abstracted in to a separate shard allowing us to use the same models across different apps. By segmenting all these directories out, the app becomes prone to a lot more duplication just from confusion of "this thing could go in this place and that place".
I'm not sure how. If you're looking for a service object, and you look in the services folder, it should be pretty easy to find, no? Plus, worst case is you bust out your grep fu 😛 I think a structure like you're proposing might be ok for brand new apps from the ground up, but porting over existing ones (like my company is doing) makes things a lot more difficult since it means re-architecting. |
Beta Was this translation helpful? Give feedback.
-
Ah, I didn't even think about autoloading everything. For better or worse I was in a React mindset where you have to I do like the idea of loading everything in common first, and then loading app code afterwards. I think your last suggestion, or your first suggestion are great (with the first suggestion meaning separate directories). I'm not super familiar with the specifics and pitfalls of requiring dependencies in Crystal, but I could see having a file where you explicitly require your folders be nice in cases where load order is causing issues. |
Beta Was this translation helpful? Give feedback.
-
@jwoertink Yeah I guess it isn't hard to find, but more that it is hard to find what is already made for an app. Part of me also just likes seeing things organized this way because it makes more sense to me.
I think in this case it'd go in src/common. A lot of frameworks that have done this don't have a nice place or structure for shared stuff (IMO). Luckily even if this structure changed it'd be super easy to use the old way because you just need to change a few lines in I'm not 100% sold on this approach so I'd love to hear other opinions too. Maybe this could be optional? I don't love the idea of have two ways to structure things though 🤔 |
Beta Was this translation helpful? Give feedback.
-
One thing to note about doing those mass glob requires is that there has been talk about removing that feature since it causes a lot of confusion (though, I really like having it). But the main reason was because in some cases you have to make sure you files are required in a specific order. As for a question about this proposed structure, I have an app that has a user model. The user model has a flag to denote if the record is an admin. When logging in, if the user is admin, they are directed to an admin interface, and if they're a regular user, they go to a whole different members dashboard section. How would that structure look? Would there still be a |
Beta Was this translation helpful? Give feedback.
-
@jwoertink Yeah I really hope they don't remove that, especially because there is no way to reference files relative to project root. If they do though you'd need to do the same require dance with either structure :( I'd structure it like this probably:
So pretty much the folders would match whatever the resource is. But I dunno, would that make sense in your case? |
Beta Was this translation helpful? Give feedback.
-
That would probably solve my case, but now it feels like each app is having to change its folder structure. So if I'm building apps for clients, each app I open I'm having to re-learn the structure of where things are in order to best suit the needs of that app. It feels very javascript/php to me. Maybe we could look at some popular apps, and think about how they would be structured like Twitter, Youtube, Simple Blog, e-comm store, and something multi-tenant like Shopify (single engine, multiple domains). I think seeing something like that might help to clarify the concept like "If you wanted to build an app like A, then the structure would look like B". |
Beta Was this translation helpful? Give feedback.
-
I am concerned about the same thing ("it feels like each app is having to change its folder structure"). But I also don't love putting things in folders by type because I think it can be hard to find what forms or service objects are intended to be used for which parts of the app. I also like the idea of having fewer deeply nested folders, e.g.
I'm not totally sold on this point. I think the convention is:
What would you have to re-learn? I really am curious. Since I thought about this structure a lot it is hard to view it with fresh eyes! I am still concerned about making it harder to developer because people have to overthink where to put things. I had that issue with Phoenix contexts. But I think the default for Lucky would be: folder structure should match the resource name. Extract to something else once you feel comfortable with a better name. But maybe that still is too much overhead? I love the idea of what some sample apps would look like. Here's one I can think of that I've been working with:
Or for GitHub
So the basic structure here matches the route namespace by default. If you want you could group things differently maybe Does that makes sense? What do you think about that? |
Beta Was this translation helpful? Give feedback.
-
I also updated the original description to reflect this new class name -> folder name convention, along with examples. I also added another example of a recent problem I had on a project where I spent a lot of time tracking down which files and classes were used for a pretty complex part of the business |
Beta Was this translation helpful? Give feedback.
-
I'm sorry I can't read through the whole thread right now, but wanted to pitch my 2 cents real quick. Hope there isn't too much repetition. Another advantage I see in the On the other hand, I'm not sure if this convention also adds a little value to Rails, where it is relatively easy to write engines and plugins that can be modified by overriding models, views and controllers. I can't think of a way the |
Beta Was this translation helpful? Give feedback.
-
Ok, so actions and pages would fall under a resource, as opposed to a group? Where would static pages like Also, would a brand new empty lucky app just generate a Assets would be tied to a resource/context too, right? So you'd have your customers related js in |
Beta Was this translation helpful? Give feedback.
-
These are excellent questions that I hadn't thought of at all.
If I understand the question correctly, either one, but by default it would match the class name, which usually is named after the resource.
This is a tricky one for either file structure I think. In the current structure I guess I'd still make a group called
I'm not sure I understand. Could you give an example of what you'd do in the current structure?
This is more confusing in the current structure I think because what I see most people do (myself included) is put the page in either one of the two folders, even though it belongs to both. Either that or I'd put it in
It would generate the config files and
Even without auth you'd have a home folder for the default page:
The generators would also use this new structure. For example There would also be the new
I'm not sure. I'd have to see how/if that would work with web pack, but I think that would be really nice to have JS files there. Of course once something is shared I'd move it to What do you think? Does that make sense? |
Beta Was this translation helpful? Give feedback.
-
Thanks for your thoughts @perezperret
I hadn't thought of that! Great point.
Hmm I'm not sure. I'll keep that in mind though! |
Beta Was this translation helpful? Give feedback.
-
Ok, I'll say that I'm still not 100% sold, but I think it's a lot more clear to me now, and I can see how a different structure like that could be pretty nice. I think I'd have to actually build an app in that style to see where I run in to issues, and feel it out. I feel like it wouldn't be too bad, but obviously there's so many variables when it comes to building an app, that we could be missing some major hurdles. Thanks for explaining! |
Beta Was this translation helpful? Give feedback.
-
When I moved from building apps mostly with Rails to building apps mostly with Phoenix, contexts were one of the things I appreciated the most. That said, for me the biggest win wasn't the folder structure itself, it was the idea of having each context be an API surface. This felt like it massively simplified testing in particular, especially when dealing with a operations that involve multiple models changing or interacting in the same operation. Regardless of the complexity of what happened, there was always just one function that needed to be tested to see if that operation was working. Huge relief. I don't know how hard or easy or canonical this would be in an object oriented language, but if this change in Lucky also made it easy to, for example, create a module for each context that served as an API surface for all the contained business logic, I think that would probably make a big positive difference for me. |
Beta Was this translation helpful? Give feedback.
-
@jwoertink I'm glad that helped a bit, and I'm glad you brought all that up because it helped clarify things for me too. I'm not 100% sold on it yet either. Maybe I'll make the change and we can experiment with it. I'm totally fine rolling it back if we (the community) are finding it is more painful than helpful. @neurodynamic Thank you for reaching out. I wasn't totally in love with contexts, but that may be just me. I do like the idea of having a simple interface that is easy to test. You can definitely do this in Lucky/Crystal using a module or a class. Forms were also made to encapsulate a lot of what you'd do in these operations. They are fairly simple to test, but I still need to write a guide on how to do it :) But you'd do something like this once the test helpers are done: params = Lucky::Params.new(email: "paul@thoughtbot.com", password: "")
SignUpForm.create(params) do |form, user|
form.password.errors.should eq ["is required"]
WelcomeEmail.new(email: "paul@thoughtbot.com").should_not be_delivered
end You could so something similar with a service object like |
Beta Was this translation helpful? Give feedback.
-
@paulcsmith That seems similar, yes. I should say that I'm speaking from a position of relative ignorance about Lucky itself, so it's possible I'm just missing the ways that what I want is already possible. I've been following the development interestedly, but I am still working primarily in Phoenix right now. I also agree with you that contexts do complicate things in some ways as well in terms of decisions about which context a particular piece of functionality that might be ambiguous or shared should go in. Overall I have still preferred them to the Rails way of doing things, mostly because of how much more I enjoy testing with Phoenix, but I think the pros and cons you are citing are real. |
Beta Was this translation helpful? Give feedback.
-
I am a big fan of the Principle of Proximity so that any objects that are closely related should be placed together and those that are not related should be placed apart. In software development this leads to a structure where objects related to a particular feature are placed in a folder for that feature and unrelated objects are somewhere else. Software is developed in layers. A lower layer provides services to upper layers, i.e. shared objects provide services to feature objects. A User object is (usually) shared among features so it is a lower level object and would be placed in a common/core/shared folder hierarchy. Everything distinctly related to the xyz feature (models, actions, views, business logic, javascript, stylesheets) can be placed in the xyz folder. In practical terms in my (Amber) application my folder structure is spec/
All pretty standard so far (from an Amber/Rails perspective) but these folders only contain lower level objects that are used across the application or are not directly related to a feature
So if I am working on the abc feature, be it the api or the views or the javascript it is all located within the src/modules/abc folder. It is a toss up whether having javascript shared between one or more features should be in src/lib/js or src/modules/js - it doesn't matter a great deal as long as I am comfortable with the choice. I prefer src/modules/js because that is the upper level of features abc and xyz and is more conformant with the principle of proximity. I find there is no need for assets/javascripts in the pathname when the abbreviation 'js' is clearly well known and understood. In contrast I recently worked on a Ruby on Rails contract that had a combined 2383 files in the codebase. To work on a feature I had to navigate around the source tree to find the place where a file was located that I wanted to view/edit and this involved looking in: confile/locales/nl-BE Not to mention the 545 migrations in db. Suffice to say there was a great deal of scrolling and hunting involved. This project had started small - like so many Rails projects but had grown over the years into a huge monolith, and this was just the api and CMS. The customer facing frontend application was another 1181 files. In my structure I would reduce 14 folders to 1 folder. It is my view that once your application grows and you have invested so much time in a flat codebase structure it is more difficult to change to a more hierarchical and more easily managed structure. Therefore it is better to start with the right structure from the beginning as this proposal suggests. The only question for me is whether the feature folders are better in a src/(features|modules) folder or up a level as src/feature_name so that they align more closely with the URL. Having the the codebase organised in levels allows me import the levels in a specific order, e.g
If I was going to place feature folders directly in src rather than in a modules folder then I would probably create a features file in src and require that in my application file and require each module explicitly in the features file. Also when generating a new feature with the scaffold generator I would want the generator to add a require statement for the feature to the features file. With regard to test suite objects I hope to one day get time to sort out a process so that when I modify a file in a feature folder an appropriate test process is started that tests that feature only and when a common/shared file is modified the whole test suite is run. I hope that this will also allow me to put the test suite objects for a feature in the same feature folder alongside the artifacts that implement the feature so that all of my application is modularized. BTW we solved this modular application problem in Amber with support for recipes and creating a modular recipe. You might want to consider that route as it would also add a lot more to the Lucky framework while leaving the 'by object type' code organisation option for those that prefer the old school way :) |
Beta Was this translation helpful? Give feedback.
-
One thing about learning a new language after living so long in another ecosystem (Ruby and Ruby on Rails in particular), one gets to take a step back and think about these issues from a fresh perspective. One thing I loved about Ruby on Rails is convention over configuration. One thing I hated about Ruby on Rails was convention over configuration. :-) Convention is great because:
For these points, I love convention. What I hate about it:
Not to get too far off topic, one thing I was looking at today as I built my first set of working pages in Lucky was the file structure we have. One thing I'm liking is that it's breaking out controllers, actions, and rendering in a way that encourages (Single Responsibility Principle)[https://en.wikipedia.org/wiki/Single_responsibility_principle] and (Separation of Concern)[https://en.wikipedia.org/wiki/Separation_of_concerns]. What I'm not liking is all the extra typing (and clicking on directory folders) to bring up the files to edit. When I'm working on a feature, I'm typically highly focused on the full CRUD of the data being updated, so I'm thinking collectively about my controller, actions, and views all at once. One idea came to mind (and I'm not suggesting it's adopted here), which might help with this discussion: What about grouping by higher level concerns? For example, finding all the parts related to Users during authentication and profile management was quite a hurdle in this first Lucky project I'm working on and it made me immediately wish I had something like this:
That is, for everything I'm going to hang off Basically, the folder structure mimics the routes I'm matching them to. Anything shared can live in appropriate folders off the Another wording up above that caught my eye was "interfaces" The "public" (unauthenticated) interface, the Admin interface, the "private" (authenticated/logged in) interface. That immediately suggested to me another level to the basic folder structure that is basically /<concern_route>/[concern files] If it's shared, it lives off an immediate subfolder of "./src/..." or "./src/shared/..." (a.k.a. "./src/common"). Otherwise, it's organized by feature/concern closely following the intended service route URL. |
Beta Was this translation helpful? Give feedback.
-
+1 For some default grouping by concern/route instead of file content. May it make sense to also name the files with a "concern-prefix", by convention?, to also allow dropping all files in a single folder, or folders sorted however desired? For example, I'd like omitting the /shared or /common subdir for just placing the common files in the parent folder. Supporting "container" folders with files "shared" by subfolders could also be nice if there are many folders. So just reading in all files from the entire tree, plus mapping e.g. all |
Beta Was this translation helpful? Give feedback.
-
Hey @testbird! Thanks for chiming in. I'm having a hard time visualizing the suggestion. Could you share an example of what that file structure would look like, with some examples actions, models, etc? |
Beta Was this translation helpful? Give feedback.
-
Hi, hello everybody! New to this, I can't say very much concerning a specific optimal default structure, only that I do (currently?) prefer folders that group files that "belong together" instead of by "type". Concerning specific things to consider, I can only, and really like to recommend two examples I still remember years later, as having been extremely well thought out. [Minutes spend might make things easier later (modules, content level files,...?) ] The ideas I had, were independent from whichever structure may be best for a default or for particular (example) apps. Actually they are rather allowing for adaptions by the users, however it may fit them (the possibility to override everything). Basics
Make folder structure modifiable for different purposes, by keeping it optional.
Does it make some sense? Should pretty much allow to put and move the files in any custom structure, not withstanding a default structure.. |
Beta Was this translation helpful? Give feedback.
-
Example: |
Beta Was this translation helpful? Give feedback.
-
Right now we have a structure that many frameworks have. This structure is based heavily on the type of file:
This can sometimes make it hard to find related files (query objects, service objects, etc.). It also makes it hard to tell at a glance what the application does and how things relate to one another (see "Other Advantages" below).
I've seen some applications handle this with "contexts" or "applications" that encapsulate different domains.
I've had a few issues with this approach though:
Proposed solution
I experimented with a few approaches and I think this would resolve the above issues.
Admin::Users::Index
goes insrc/admin/users/index.cr
.Api::Users::Show
goes insrc/api/users/show.cr
, etc.src/common
by default because they are often shared across boundaries. You can have more specific forms or queries in folder if you want.src/store
in the example below), but default (class name -> folder structure) is the way to start so you don't have to overthink things too earlySo default convention is:
apps/features/contexts
folderapps/features/context/whatver it is called
folder is named after the resource by default. Makes it easy to get started without premature extractioncommon
Other advantages:
I recently had an issue with a project where we were trying to figure out what was going on after records were imported. This took a day of digging to find all the workers and related code that was scattered across multiple folders. Here's what I would have loved to have done afterward:
This makes it a lot easier to find related code without tracing method calls between your apps and stitching it all together in your head or writing a markdown doc of everything you've found
What do you think?
Thoughts/concerns about this approach? Any ideas for naming things?
Beta Was this translation helpful? Give feedback.
All reactions