Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

this is not a standard Go project layout #117

Open
rsc opened this issue Apr 9, 2021 · 140 comments
Open

this is not a standard Go project layout #117

rsc opened this issue Apr 9, 2021 · 140 comments

Comments

@rsc
Copy link

@rsc rsc commented Apr 9, 2021

The README makes clear that this is not official, but even the claim "it is a set of common historical and emerging project layout patterns in the Go ecosystem" is not accurate.

For example, the vast majority of packages in the Go ecosystem do not put the importable packages in a pkg subdirectory. More generally what is described here is just very complex, and Go repos tend to be much simpler.

It is unfortunate that this is being put forth as "golang-standards" when it really is not. I'm commenting here because I am starting to see people say things like "you are not using the standard Go project layout" and linking to this repo.

@iarosb
Copy link

@iarosb iarosb commented Apr 13, 2021

@rsc could you please share a few links to some trustworthy (from your point of view) resources to learn more about the best practices for writing "idiomatic " Go code? Thank you in advance.

@henvic
Copy link

@henvic henvic commented Apr 15, 2021

@iarosb, I always recommend anyone starting with Go to read

https://golang.org/doc/effective_go
https://github.com/golang/go/wiki/CodeReviewComments
https://github.com/golang/go/wiki
https://golang.org/ref/spec

I feel relieved to see Russ Cox opening this issue!
I always hated how this repository sells the idea of being something official and how inaccurate it is.

Quite a few times, I had a hard time because of its "official-ish" appearance – it makes developers, especially people starting with Go, blindly follow whatever recommendation is here. Awful!

@iarosb
Copy link

@iarosb iarosb commented Apr 15, 2021

@henvic the reason I ended up here asking for links is that I've seen many developers referring to this repo as if it has some official status, but it seems to be very questionable in a way it relates to the core ideas behind Go's creation. I'll go through those resources and your help is much appreciated.

@rakyll
Copy link

@rakyll rakyll commented Apr 17, 2021

What @rsc describes here is a problem I observed many times. I saw many projects trying to "fix" their layout based on what's provided as a reference here. It creates confusion and conflict among contributors. The project is doing a good job having a disclaimer in the README, but it doesn't seem to be effective because many people don't read the text and assume this is an officially endorsed project.

leoschet added a commit to leoschet/gaivota that referenced this issue Apr 18, 2021
@theckman
Copy link

@theckman theckman commented Apr 23, 2021

I am 100% aligned with both @rsc and @rakyll on their comments. We experience the same misconception about the status of this repository in the Slack workspace, especially for those who are new to the language and are trying to figure out how to do things "right". I am not sure that I get a sense, at least in our space, that it's getting any worse, but it's frequent enough that I would be comfortable classifying it as a problem.

Considering the feedback being given in this issue, @kcq how would you feel about moving this repository out of the current Organization and onto a different one (or your username) to benefit the larger Go community? I'd actually be happy to provide your repo as a resource for Gophers if it wasn't asserting itself as a standard.

Edit: I had forgotten I filed this awhile back, regarding the pkg/ directory: #10

@flowchartsman
Copy link

@flowchartsman flowchartsman commented Apr 24, 2021

I usually point people to https://eli.thegreenplace.net/2019/simple-go-project-layout-with-modules/, and have done so in several issues in this very repo.

You've had several long-time gophers and a couple of go maintainers chime in now. It would be really great if you could archive this repo or, at the very least, move it to a less-misleading org. I don't like saying it, but overall community impact is negative at this point. It's hurting more than helping.

@MrTravisB
Copy link

@MrTravisB MrTravisB commented Apr 27, 2021

The issue is the different requirements between a simple system package (which I tend to think the Go team over optimizes for) and more complex apps and services. Every company I've worked at that had larger apps written in Go has a repo that looks very similar to this. Even the pkgsite repo uses some of the structure that this repo recommends. Obviously I would not use this structure for a small system library that isn't meant to be used in a standalone way but in my experience a lot of what this repo recommends is very valid and preferred for large apps.

@flowchartsman
Copy link

@flowchartsman flowchartsman commented Apr 27, 2021

I think the primary issue is that the org is titled golang-standards which, combined with its age, lends it a certain credibility (not to mention SEO) that is unwarranted and has long been a source of confusion for new developers.

Differing requirements is certainly a thing to be aware of, but they're impossible to capture in a single directory hierarchy, and that's part of the problem. This repo is complex and, despite periodically stating that features are optional in documentation, has a monolithic structure which comes across as prescriptive. This is at odds with the language, IMO. Go is a relatively simple language to get started with, and this is not a good jumping off point. I've personally helped several developers rekindle their waning interest by showing them a simpler way to lay out their code. I'm sure many other folks in community forums have had similar experiences.

For those that do persist, as mentioned above, they often cause further confusion by directing others to follow these guidelines in the form of PRs, issues, or just well-intentioned advice the recipient doesn't know any better about.

It would be really great if at the very least the org name could change. For the reasons discussed above, this is not a very good standard, much less the standard.

@SteelPhase
Copy link

@SteelPhase SteelPhase commented Apr 27, 2021

I am in support of the idea this project represents, but these conversations bring up one thing to me. Why doesn't something like this get published by the Go team. It doesn't have to be complex, but I feel like there is a gap in the documentation that this repo partially fills. I stumbled on this when I was in search of this information, and it was extremely helpful. I'm well past this point in my learning of Go, but I have to assume others could benefit from an official version of sorts of this type of information.

@ivarec
Copy link

@ivarec ivarec commented Apr 27, 2021

I am in support of the idea this project represents, but these conversations bring up one thing to me. Why doesn't something like this get published by the Go team. It doesn't have to be complex, but I feel like there is a gap in the documentation that this repo partially fills. I stumbled on this when I was in search of this information, and it was extremely helpful. I'm well past this point in my learning of Go, but I have to assume others could benefit from an official version of sorts of this type of information.

Because the Go team cannot know all uses of Go and what project layout each type of project would benefit the most. In general, just tossing a few .go files in the same dir will take you very far.

@SteelPhase
Copy link

@SteelPhase SteelPhase commented Apr 27, 2021

I think I missed something in my original comment. This was at a time when I was moving from small one off projects, to something significantly larger at work while I was still learning an uncertain about what I was doing. I don't know the best solution, but it seems like there could be at least a bit more in that regard. Honestly even if it was a wiki page with some "hey this could be useful" comments regarding laying things out when the time comes.

@flowchartsman
Copy link

@flowchartsman flowchartsman commented Apr 28, 2021

This blog post has always been a great place to start, and it's worth noting that @eliben, whose article I posted above is in the Golang organization. There's also this post by @rakyll (former Go Team member) and this one by @peterbourgon. These articles may not be quite as specific or official as some people might want, but like our earlier poster mentioned, everyone's needs are different, and the official guidelines have had a correspondingly broader scope.

Should is a tough concept in software engineering, and once you say things should be done a particular way, it's tougher to evolve and to develop new best practices. The more specific and proscriptive you are, the tougher it becomes, from my experience. Far better to have clear, philosophical guidelines describing the guiding principles of the language and how those can shape the reusable components you write. And I do think we have that.

Put another way: "it depends", and giving it a proper treatment involves, ideally, multiple case studies on "well structured" packages that take design, intent and API footprint into consideration, along with how those decisions were arrived at, why, and how they may have evolved through the life of the project. Perhaps theres some room for this in the Go Blog, but even then the answer is still "it depends", and what works for one package might not work well for another, even if the guiding principles are the same.

And, of course, when I google "structure golang programs", this repo is still on top, which is the crux of #117

@cep21
Copy link

@cep21 cep21 commented Apr 28, 2021

22k stars shows there is a community need for something like this. I understand the "it depends" camp for complex projects, but there are guidelines that most people can agree with. For example, the vast majority of simple go projects, when they are starting out, have a go.mod at the root path with a package that probably matches the github repository. There's probably a LICENSE file and README file. They probably should check in their go.sum file. There should probably be a package_test.go file.

There are other best practices where their existence is important, even if the details are not. For example, almost all have CI, but we don't have to make a decision on which CI is best.

With 22k stars, the community is signaling a strong need for guidance on how to structure go projects. I believe it would be high leverage for an official voice, more recent than 2012, with access to golang.org, to guide people here.

@theckman
Copy link

@theckman theckman commented Apr 28, 2021

I am not sure we can assume the user's intent when they star a project. I know for many it signals an interest to understand or explore a project later. It does not mean that they have a need for it. I star some projects I never use, nor ever have a need for.

@masakatsu-corp
Copy link

@masakatsu-corp masakatsu-corp commented Apr 28, 2021

@rsc Does the Go team have an official project layout pattern?

@xhit
Copy link

@xhit xhit commented Apr 28, 2021

Go doesn't have an official project layout.

I always use the layout like here: https://github.com/golang/go/tree/master/src and itś simple: a folder with that package and all related.

Go is simple, I don't know why developers want to use patterns of others languages here.

@theckman
Copy link

@theckman theckman commented Apr 28, 2021

@xhit considering we have go fmt to consistently style source code, I could see some then wanting or even expecting to find a style to use consistently to structure their project. I think there are some patterns we've discovered over the years that can help projects be more easily maintained, but I think that's much different that some sort of one-size-fits-all "standard" layout.

@SteelPhase
Copy link

@SteelPhase SteelPhase commented Apr 28, 2021

I think an excellent start to anything even remotely like this could just be to point out the resources you've all been dumping as alternatives. There's all this great info spread all over. Should be enough to at least have a wiki page that you can point people to, when they've been mislead. It's also a good option incase this org sticks around longer.

@clausecker
Copy link

@clausecker clausecker commented Apr 28, 2021

I think discussions about a standardised project layout miss the point. Go encourages you to derive package names from the last path component. When importing multiple packages, you want the names of the packages you import to be unique to avoid naming collisions. A standard layout would mean that it's very likely that many of the packages have the same, standard names and hence collide. This is to be avoided. So in order to make for a good programmer experience, the layout must not be standard but rather custom and different for each project.

I believe that people overthink this way too much. Just let your projects grow organically. Except for a few hard coded directory and file names (like vendor, testdata, and go.mod) you are free to chose whatever directory names you like. You should
make full use of this freedom and put your stuff into whatever directory name seems most appropriate for the problem at hand. And if this choice turns out to be a poor one, you can always refactor later.

@krak3n
Copy link

@krak3n krak3n commented Apr 28, 2021

I think that this stems from new people coming to the language looking for guidance on best practices for project layout out. This is especially true if you come from a very well structured world like Django, Flask or Rails and so on where project structure is defined for you, this is where you put your handlers and your models go here etc etc.

Coming to go where that safety net does not exist can be a little daunting for new comers, for myself coming to go from the Django world I struggled with how to layout projects, "I was always asking myself am I doing it right?" and while the correct answer is that there is no right way, only the way makes sense for what you are building, the golang-standards/project-layout gives a guide to how you might start and grow from there.

I've long stopped laying out my projects specifically like this because I've grown in experience and naturally figure out project layout on the fly and it's naturally evolving as the codebase grows and evolves, things will move around a bit and thats ok.

Perhaps the issue is that this project aims to be THE way to layout a go project when really it's more like a here is a place to start to inspire you. It's certainly not the standard way though.

@LyonNee
Copy link

@LyonNee LyonNee commented Apr 28, 2021

Not long ago, I also encountered such a problem. I used to be engaged in the development of other languages ​​and have certain development experience. When I want to use golang to develop a project, and for this to establish a good code directory structure, I am confused.
Later, after looking at this project and understanding the compilation rules of golang, I developed a "standard layout" of my own project.

@higker
Copy link

@higker higker commented Apr 28, 2021

As a young gopher, I prefer simplicity,i like the layout list :

@batara666
Copy link

@batara666 batara666 commented Apr 28, 2021

Yeahh, let's delete this missguided repo

@bitfield
Copy link

@bitfield bitfield commented Apr 28, 2021

I have some sympathy with @rsc about this (OK, a lot of sympathy). I don't think the layout shown in this repo represents either the best way to structure Go projects, or even the most common way (those are quite different things, as you'll appreciate).

That said, though, I guess the author of this repo can publish whatever he wants, even describing it as a "standard", and no one can say boo about it—not even Russ. That's the joy and the hurt of open source. No one is obliged to pay any attention to the maintainer's ideas about Go project structure, and the flipside of that is that he's not obliged to pay any attention to us. If someone thinks they can do better (and the Go team would seem to be an obvious candidate for this), let them go ahead and do so.

I read about a dozen blog posts a day about Go which are (in my opinion) at best misguided, and at worst positively harmful. It would be a full-time job for me to try to get the authors to change their minds, and posts, and it's none of my business anyway. Instead, I publish my own opinions, and let Gophers decide whose ideas they like best. That seems to me a sensible way of going about things.

@batara666
Copy link

@batara666 batara666 commented Apr 28, 2021

@bitfield The org should be renamed. It's that simple.

It's misleading, it harms devs new to Go.

It's ok that Go doesn't have a standard project structure. Any given project's structure is a function of its size, complexity and type.

Go is versatile, so it makes sense that there would be a bunch of different structure choices depending on what exactly your project is.

It also fits that new devs would be looking to better organise their code once the single main.go starts to get a bit out of hand, especially when coming from a file per class language. They search Google, they find this repo. Because of its name not its readme.

It's very easy to miss that comment in the readme. It's by no means a sufficient disclaimer. That comment is a massive copout from the owners because they know they'll get no further traction if they renamed the org. Maybe they should instead focus on demonstrating how different potential project layouts could fit different use cases, and make it actually useful to new Gophers with some signposting of that nature. "Start here with a basic layout", "here's one with a bit more organisation", etc.

@clausecker
Copy link

@clausecker clausecker commented Apr 28, 2021

@m2q The standard opiniated directory structure for Go is no directory structure. Name every package according to its function. Chose unique names, not names everybody else uses. No fun programming if you import 10 packages named util. And do not add boilerplate path components like src, util, or pkg. Nobody needs that.

Perhaps the only thing resembling a convention is having cmd for main packages as the second-to-last path component. And of course the few hard-coded directory and file names.

@colin-p-hill
Copy link

@colin-p-hill colin-p-hill commented Apr 28, 2021

It's fine to put out a recommended layout, even one that may be imperfect. @bitfield is right to point out that there are plenty of blog posts with questionable suggestions out there. I'm not sure this can be put into that bucket, as blogs published under the names of individuals or organizations will be generally understood as reflecting only the author's perspective. Who would have the audacity to call their organization "golang-standards" if they have neither the authority to set standards nor consensus from the community that what they are promoting is a best practice? It's not uncommon for new gophers to make the fairly sensible assumption that the answer to this question is "nobody", and rather than pursuing further perspectives, they take this repo to be the final answer that its name implies it to be. (No, a disclaimer in the readme does not adequately mitigate that. How often do you read a readme from top to bottom?) It's a fine project, but an irresponsible name, and it would be tremendously helpful to the community if the name were changed to convey the humility which I'm sure the author intended to bring to this project.

@quenbyako
Copy link

@quenbyako quenbyako commented Apr 28, 2021

Lol, i have a great idea:

If this layout is not official (yet, maybe), instead of renaming, why not to work on it? 🤔 I mean, go still doesn't have best practices of directory layout (unlike rust, dart, etc. note that they have recommendations, not restrictions), why not create a recommended only standard of layout (only as recommendation, like effective go, gofmt, etc.) that suits most of gophers tasks including language core developers?

People say here right thing (like @flowchartsman): go is so simple that you can write the code as you feel it and it looks good. Honestly, this repository is not needed at all in current status 😂 but despite this, some kind of agreement is still needed where long-time gophers where to find specific logic with unknown project to them (for example, Kubernetes and Prometheus are rather complex sources, but at the same time coode looks slightly similar, I do not want to be tied to this particular projects or this repository at all, so with such recommendations it is easier to understand where someone else's code stores and how it relates to each other.)

The question is not so much in standardizing the layout, but in explaining to newbies how long-time gophers are storing their code by functionality

guys how dislike it: instead of being toxic, explain your opinion. I look at your thumbs down and not fully understanding whats wrong with my comment, why not to explain your position?

@meblum
Copy link

@meblum meblum commented Apr 30, 2021

The claim here seems to be that

  1. Newbies find this repo through google search and think it's official.
  2. They go through the readme for advice on how to structure their repo, and get misleading advice on project structure.

What I don't understand is, if someone reads the README, they also see the disclaimers right at the beginning that it's not official and not a one-size-fits-all. If they don't read the README, how are they being misguided by just starring it?

@ivarec
Copy link

@ivarec ivarec commented Apr 30, 2021

@nicerobot respectfully disagree. The beginners that should be using a few .go files for their starter projects are the ones being harmed by the overhead imposed by the "standard". Projects with higher ambitions will have their own structure regardless.

@DamareYoh
Copy link

@DamareYoh DamareYoh commented Apr 30, 2021

@nicerobot

Then how about an added disclaimer in the project with exactly that point?

There is a disclaimer there already, and that you haven't noticed this is exactly the point. People (particularly new folks) come to this repo thinking this is the official recommendation for how repositories should be because the org name is "golang-standards" and then they don't read the README because why should they? They are immediately presented with what they were looking for - a directory structure that they think is the "standard" because of the org name. Then these users go on to create these directory structures in their project without understanding the pain points they will later encounter, or they go around trying to evangelize it on other repos that don't need it.

Nobody is upset that there's a proposal in this repo. The problem is that it's sitting in an org called "golang-standards" and it's not anything close to the standard for the language.

E- Also before anybody rehashes the argument about shifting ownership of the org, nobody is asking for that either

E2 - @meblum, when re-reading over this, it looked like I was responding to you directly. Apologies for the confusing format. I was intending this post to be a counterargument to your point that the readme disclaimer is sufficient.

@covrom
Copy link

@covrom covrom commented Apr 30, 2021

I've been waiting for this discussion for a long time. It seems to me that this repository is the tricks of Russian hackers 😱 or a diversion from Java developers 🐸🐸🐸

@bconway
Copy link

@bconway bconway commented Apr 30, 2021

If the repo included a README.md justifying each design choice with examples, I think it would really cut down on the sniping, here and elsewhere. For example, the apparently-maligned pkg/ pattern could have a README.md that shows dozens of repos using this pattern, like Google, Kubernetes, etc, such that we'd know why it was included.

@theckman
Copy link

@theckman theckman commented Apr 30, 2021

I would strongly encourage anyone in the community to avoid using K8s as inspiration for project structure. 😅

@therealplato
Copy link

@therealplato therealplato commented Apr 30, 2021

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.
John Gall, Systemantics: How Systems Really Work and Especially How They Fail 1975

@DamareYoh
Copy link

@DamareYoh DamareYoh commented Apr 30, 2021

@nicerobot

Then couldn't this issue have simply been closed almost immediately by filing a PR for this issue to fix the READMEs? Did this entire thread just go wild because it was in the newsletter?
How about more like #121 ?

Have you actually read the readme?

It already contains the disclaimer you're asking for and it is obviously not working. The PR you linked is just updating a translation.

@coip
Copy link

@coip coip commented Apr 30, 2021

im a fan of the orgs other template example and think, as-it-is (#aa3ea0f), others might agree it's aligned with the spirit herein :-)

rm .gitignore might provide even higher signal:noise, but not terribly picky.

https://github.com/golang-standards/project-template

@DamareYoh
Copy link

@DamareYoh DamareYoh commented Apr 30, 2021

@nicerobot

My point is, what's your proposal? Just to complain?

Why should I reiterate solutions to you when you could read the recommendations already presented in the thread?

If I'm complaining about anything, it's about you posting without understanding what you're talking about.

@TwinProduction
Copy link

@TwinProduction TwinProduction commented Apr 30, 2021

I'm glad that somebody's pointing out the fact that while this project has garnered a decent amount of stars, it isn't necessarily representative of the vast majority of Go projects.

Beyond agreeing with @rsc's statement on the usage of golang-standards being a bit high handed regardless of whether the repository's README explicitly mentions it being unofficial, I do agree that some extra effort could be made to make it a bit more obvious, such as by prefixing the repository description with Unofficial.

I understand that the internet will never cease hosting disagreements about everything and nothing, but just like you wouldn't argue that every single projects in these hundreds of List of awesome projects repositories to actually be measurably "awesome", I think you should take any repositories outside of the official organization hosting the actual maintainers with a grain of salt.

That being said, please, there's no need to be disrespectful or threaten people, the maintainer of this repository was obviously just trying to do some good. Instead of trying to start a flame war, come up with helpful suggestions and provide valid reasons why you think X action needs to be taken.

@francoposa
Copy link

@francoposa francoposa commented Apr 30, 2021

I have found this pattern incredibly straightforward and easy to adapt to Go: https://docs.microsoft.com/en-us/dotnet/architecture/microservices/microservice-ddd-cqrs-patterns/ddd-oriented-microservice

@sirkon
Copy link

@sirkon sirkon commented Apr 30, 2021

This layout only makes sense in one kind of cases:

  1. There's one or more executable packages in a project
  2. There're directories what are not Go packages: rpm folder, static data, etc.
  3. The project provides public API

In this case it is useful indeed. But things get ugly very quickly: it is a big question if the layout is useful when just one point out of these 3 is out:

  • Not command line utilities and only a few folders with non-Go directories: it is better to put packages right in the root and combine static data into special directory.
  • No packages and only a few folders of static data: same solution.
  • No static data: just putting packages into the root seems like a preferable choice.

And this is plain simply when there are only packages or only executables: just put everything into the root.

@sirkon
Copy link

@sirkon sirkon commented Apr 30, 2021

@nicerobot respectfully disagree. The beginners that should be using a few .go files for their starter projects are the ones being harmed by the overhead imposed by the "standard". Projects with higher ambitions will have their own structure regardless.

Can't agree more. Had an imbecile at my job: once wasted several hours to choose a good name for a repository to hold a parser of some kind, just a lib with a single package. In the end we got something like

repo/
    pkg/
        someparser
@mantzas
Copy link

@mantzas mantzas commented May 1, 2021

The lack of any other standard makes it the only known solution and given the fact that it also has over 23K stars means most probably that there is a huge need for it. I would suggest that this could be a nice opportunity for the Go team to take a look at it? /cc @rsc

@m2q
Copy link

@m2q m2q commented May 1, 2021

I would strongly encourage anyone in the community to avoid using K8s as inspiration for project structure. 😅

@theckman this remark really irks me. How is a developer supposed to know that this big project is not a good reference? People say the same about docker.

I think the fact that Go doesn't have an official reference for project structure is orders of magnitudes worse damage than whatever this repository does.

Any developer worth their salt reads through disclaimers like this. Anything else suggests they are copy-pasters who can't think independently. Instead of blaming this repo, blame beginners who are impatient and irresponsible.

How about opening an issue on Go repo instead, because that's where the real problem lies. If Go would provide devs with proper answers to these questions, everything would be easier. Blaming this repo here is just a band-aid fix.

@theckman
Copy link

@theckman theckman commented May 1, 2021

@m2q how is any developer supposed to know whether any arbitrary block of code they look at is well written or not? They don't. The same applies with project structure. Much of these aspect of things are more art than science, and I'd rather not put process around creativity.

I am not aligned with your assessment of the damage being done by a lack of guidance. Can you show me some concrete evidence of that? I and many other developers have had zero guidance, some for nearly 10 years, and there hasn't been a case I can recall where that's caused a problem. Knowing that I've written Go since 2013, that seems like a long enough time to run into a real problem with the lack of explicit guidance.

Edit: the one problem I have continually run into has been talking people off the ledge of assuming this is actually a standard, when they are about to blindly follow advice that doesn't benefit them and may cause more work or toil. Not the lack of guidance.

If you care about this and think it's a problem, then you can choose to make it your responsibility to file a Go proposal to catalyze a change. Everyone in the community has the power to raise proposals and champion their acceptance, if you feel there is value in making a change.

So far with the context provided to me in this thread, I do not perceive this to be a problem that the Go team should spend cycles addressing. You can file the proposal and let the Go team communicate that to you directly (if they feel the same way).

But also, what's wrong with the issue that someone already opened?

@DamareYoh
Copy link

@DamareYoh DamareYoh commented May 1, 2021

@mantzas

I would suggest that this could be a nice opportunity for the Go team to take a look at it?

@m2q

How about opening an issue on Go repo instead, because that's where the real problem lies.

Did you both miss that someone has posted a proposal over on the go github already?

image
golang/go#45861

@m2q
Copy link

@m2q m2q commented May 1, 2021

@theckman Without guidance, learning is slower & less efficient. More trial and error, more time spent experimenting on dead-end approaches. Less exposure to well-crafted ideas. How are you supposed to know that there's a better way? Unless you speak to other developers, get feedback and outside opinions, you won't. Of course since developing is a very social activity, you're bound to be exposed to external influence at some point.

So naturally this isn't a big deal in the long run, but it definitely is in the short term. There are so many languages and technologies devs are exposed to, but we can't afford endless studying. Time to productivity is an important factor. We're trying to adapt into new ecosystems as rapidly as possible. In this context, I want to spend as much time as possible learning from successful and experienced people/projects/approaches, not derive my own solution for every trivial matter.

A good example for the damage a lack of guidance does in the short term: refactoring. Bringing a product or service to market, and having to do refactors is a real PITA. So what do many devs do? Delay it. And then we have debt. Seen it at startups that are growing rapidly, and their codebase is not built to accommodate a growing number of developers. So much downtime.

I mean project organization is just one out of many potential problems that can choke a dev at scale. How is a dev new to Go able to avoid the majority of these traps? If the answer for trivial topics like this is "years of experience" then that's frustrating.

I feel like an idiot writing this, because folder choice is one of the more harmless "mistakes" someone can make. But I believe the point is still valid

@theckman
Copy link

@theckman theckman commented May 1, 2021

@m2q multiple folks in this thread have experienced exactly what you are describing, and in their cases it was because someone thought this was a standard (the true way) and either argued about it, or implemented an inferior project structure that had to be refactored. These are actual user stories talking about real impact this had, and so far nobody has offered similar stories about the lack of explicit guidance.

Learning well isn't always done fast. Becoming even just proficient in something takes a substantial amount of time. There is no one true project structure, and maybe not even 5. So what you're advocating for is to teach these newbies that there is N proper ways, and anything outside of that is wrong. You don't teach them to explore their space and to make informed decisions based on their own explorations, instead you teach them to believe dogma exists around their project structure regardless of the problem they are solving. That's just factually untrue, and they should let their project dictate the structure.

Refactoring is in the same bucket. How or what you refactor depends on variables we cannot possibly enumerate or communicate, including client specifics, product behaviors, and user expectations. It's impossible to be prescriptive about those efforts.

So far I feel like you've provided some vague / high level problems, but nothing concrete or with explicit examples.

@theckman
Copy link

@theckman theckman commented May 1, 2021

@m2q Oops, forgot this: because changing folder structure is a breaking API change, it's not harmless.

@m2q
Copy link

@m2q m2q commented May 1, 2021

@theckman
You're correct that learning is done slowly. Also correct that teaching someone the way you described is not ideal.

But the issue here is that a huge chunk of beginners are taking the repo at face value, and aren't critically reading the disclaimers. There's no dogma contained here. The second I read that this is not official, I took everything with a grain of salt. The author also mentioned the words "intentionally generic" which signals to me it's bloated with tons of folders that I am very likely gonna be ignoring.

I'll give you some specifics. This repo told me about both the /internal feature and the /vendor feature - with the former being in the 1.14 release notes. How would I ever find that out by myself? It's not something you just randomly stumble upon, and since most go projects on github are libraries, you won't see that folder there often. Now, the /vendor exposed me to a way to include dependencies inside the module directory... how could I have found that out? Go through this beast https://golang.org/ref/mod or try to come up with the right keywords (which wasn't easy, considering vendor is not what I'd google). Why is vendoring not contained here https://golang.org/doc/modules/managing-dependencies? Same applies to cmd.

Reading this repo, it took me 20 seconds to understand 2 very helpful features of Go. They're not complicated. But they're both related to organization, handily contained in the same README. And both containing relevant links to inform myself.

For me this repo can be a really good starting point that gives you a bit of context in very plain language. Gives you recommendations, some very useful features, a collection of invaluable links and a good sprinkle of misinformation. It's like a friend explaining stuff to me. Of course I know it isn't the holy word of god. But it's a massive time saver because it consolidates relevant information. A net benefit in my book

I hope it's now a bit more clear what I meant

@peterbourgon
Copy link

@peterbourgon peterbourgon commented May 2, 2021

@m2q

this remark really irks me. How is a developer supposed to know that this big project is not a good reference? People say the same about docker.

In fact you probably shouldn't look to any of the "flagship" large projects for inspiration. As projects grow they develop their own gravity, and once a project gets to the "household name" size it's likely to have created its own bespoke conventions, idioms, structures, processes, etc. that overrule those of the implementing language. This makes total sense! It's not a flaw in those projects in any way. But it does mean you're not going to extract useful information by studying them. You're much better served by looking smaller: for example, instead of Kubernetes, consider HashiCorp's Nomad, which is still really really big, but tractable in a way that Kubernetes isn't. And really I'd encourage you to look even smaller: the good stuff, as far as I've seen, is generally around ~10k SLoC.

@goodforever
Copy link

@goodforever goodforever commented May 3, 2021

@lesismal
Copy link

@lesismal lesismal commented May 3, 2021

golang is not java. please keep go simple and don't mislead people in the go community with java thinking.

compared with scripting languages, golang is largely a system-level language. golang developers should have project planning capabilities and should plan project structures based on different project scales and business types instead of using unified templates.

don’t agree with @avelino. newcomers can get started from a simple project. for simple projects, this template is obviously complicated that increases the complexity of learning for newcomers.

@ramonmacias
Copy link

@ramonmacias ramonmacias commented May 3, 2021

I am in support of the idea this project represents, but these conversations bring up one thing to me. Why doesn't something like this get published by the Go team. It doesn't have to be complex, but I feel like there is a gap in the documentation that this repo partially fills. I stumbled on this when I was in search of this information, and it was extremely helpful. I'm well past this point in my learning of Go, but I have to assume others could benefit from an official version of sorts of this type of information.

I will say also, that it depends on the arch you are trying to apply to your project. Each arch will affect for sure the package and hierarchy of your packages and files. Some of the most populars archs rely on OOP languages, and I saw many times to try to fit some particular Java implementation into Go doing a kind of copy&paste structure.

@deltamualpha
Copy link

@deltamualpha deltamualpha commented May 3, 2021

Again, people are getting caught in the weeds of if there’s a standard and what it should be, but this issue is about the fact that the name of this organization, as “golang-standards”, is misleading about the authority of this repository. That’s all. It’s easy to bikeshed about what the proper way to structure a project is! But this is not the place to do that.

@kcq if you’re not going to change the organization name, just say you’re not going to, close this issue, and let the go team decide what to do next. Anything else just leaves everyone here to argue minutiae.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet