Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples for using Standard as a toolkit for multi-project webdev monorepos? #165

Closed
montchr opened this issue Oct 18, 2022 · 6 comments
Closed

Comments

@montchr
Copy link
Contributor

montchr commented Oct 18, 2022

Standard positions itself a tool for managing monorepos (I think?). Most of my encounters with monorepos have been within the JavaScript ecosystem. Usually these tend to use Yarn, PNPM, Lerna, or some other newer tools (Vercel’s Turborepo also comes to mind, and there's also the very-confusingly-named "Nx"!).

I’ve been looking through some the examples and tutorials but am struggling a bit to find a middle ground between the simple tutorial examples and the complexity of the Bitte/Cardano examples. JS monorepos are more complex than the single-binary Rust example in the template but generally don't need to concern themselves with the complexities of managing hosting / distributed apps infrastructure.

Some notable real-world examples:

And some template examples I've come across:

So, all that is to say, does Standard fit the bill for these types of mega-projects containing smaller projects containing their own packages, components, utility function libraries, etc.? Or am I attempting to plug a square into a circle?

If Standard does fit such applications, in theory, how would Thee Standardisers map the concepts/structures of cells / cell blocks / runnables / data / etc. to one of those existing projects?

Another even more abstract question: how would Standard be applied as a tool for managing a web agency's client+project structures, or the many desks and newsrooms of a large media publication? If we zoom out and spin around, a larger variety of domain-boundaries appear: cells within organisms within hives within colonies within ... planets...


On the Broader Subject of Standard as Monorepo Tool

Perhaps most directly, let's refer to https://monorepo.tools/ (made by Nrwl, the company responsible for that confusingly-named "Nx" tool):

https://monorepo.tools/#monorepo-features

What monorepo tools should provide

  • Local computation caching -> fast
  • Distributed computation caching -> fast
  • Distributed task execution -> fast
  • Transparent remote execution -> fast
  • Detecting affected projects/packages -> fast
  • Local task orchestration -> fast
  • Workspace analysis -> understandable
  • Dependency graph visualisation -> understandable
  • Code sharing -> manageable
  • Consistent tooling -> manageable
  • Code generation -> manageable
  • Project constraints and visibility -> manageable

They then list several monorepo tools and compare them using these criteria. I do think Standard+Nix accomplish at least some of these requirements. Does Standard belong on that list too? Should it?

I am a little skeptical of the self-promotional aspect of that website, so here is another reference point with quite a lot less gloss:

https://www.robinwieruch.de/javascript-monorepos/

@montchr montchr changed the title Examples for using Standard as a toolkit for multi-project JavaScript monorepos? Examples for using Standard as a toolkit for multi-project webdev monorepos? Oct 18, 2022
@blaggacao
Copy link
Collaborator

blaggacao commented Oct 19, 2022

@rhyslbw I'm not sure if you actually have or want to have an opinion on this. But I think you might be the only person I know that sits on the JS-side (or in fact somwhere in between) of the intersection.

I remember our discussion how the nix cache can potentially disintermediate quite some of those novel domain-specific shared cache techniques in JS, such as in nx (as if not only the name was a shameless rip 😉) and potentially solve some of the (re-)build duration issues, that developers reportedly face (or faced).

All that, while modulo "usability", currently not much is left on the nix side worth exploring in practical terms in a productivity setting.

@blaggacao
Copy link
Collaborator

blaggacao commented Oct 19, 2022

Another even more abstract question: how would Standard be applied as a tool for managing a web agency's client+project structures, or the many desks and newsrooms of a large media publication? If we zoom out and spin around, a larger variety of domain-boundaries appear: cells within organisms within hives within colonies within ... planets...

Beyond the repo, there is the interconnecting nix tissue (this project would have been almost named tissue). Standard has a very friendly interface for downstream users (modulo issues with exponential lock file growth that is upon nix upstream to solve).

The fundamentals of this power is undoubtedly nix' fetcher class of functions.

For an agency, I'd probably use one repo per client (due to separation of accounts, while the cross-repo/cross-account integration penalty on the nix 'tissue' is low).

For News Agencies, I'd also try to find repo boundaries around teams amd spans of control (whatever that actually means when looking at it from the technical angle).

std, the CLI aims to a friendly reach-out and hand-holder for the average tech savvy consumer and can reach, with some very limited training on the command line, into non-technical users short of a web frontend for the stuff that DevOps puts out there.

It goes into the direction of a nix-based self service strategy. Yes, you heard that right! Modulo windows/mac support (which @jmgilman has done an excellent job in tackling via recent devcontainer support).

@dzmitry-lahoda
Copy link

dzmitry-lahoda commented Oct 26, 2022

General guidance on monorepos could be awesome.

I think to have any possible subfolder by any possible CODEOWNERS team to have is own cells to grow.

So root flake.nix is registered with many subfolder which are than flat out into packages/shells/apps/nixops/home/managers/arions/oci-images/.. according some pattern.

@blaggacao
Copy link
Collaborator

blaggacao commented Oct 30, 2022

I want to say that arion (aka docker-compose on nix) support just landed in #173 (closed by #175). It's one piece in the puzzle of making Standard increasingly work for the average web developer. Not directly related to the mono-repo aspect, though.

@blaggacao
Copy link
Collaborator

@montchr in the JS world, I'd point you to https://github.com/canva-public/js2nix

With a little bit of flake-wiring and softening of hard-coded input contracts to the CLI, that looks like an excellent integration candidate for std + js.

@montchr
Copy link
Contributor Author

montchr commented Jul 1, 2023

I'm going to close this since it's too broad, and I don't really know how one would pick anything actionable from my initial post 🙃

@montchr montchr closed this as not planned Won't fix, can't repro, duplicate, stale Jul 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants