RFC: A builder library to power koch

[alaviss]

koch is imperative, and this creates huge friction in understanding what every step does, composing them, perform parallel builds, etc.

So I'm proposing to rebuild koch with a declarative approach in mind where we have build targets, dependencies declared ahead of time, akin to a build system like CMake or Meson. This should improve code quality by depending less on the immediate state of the system like with the current system, and maybe the library can then grow into a standardized nimskull build system in the future.

I don't have much to sell about this idea yet, I'm still doing research on it. Hopefully I will have something to show in a month or so.

This work, if it turns out right, should replace koch and niminst as they are right now.

[haxscramper]

I'm all in favor of having a proper build system - it would be beneficial to all projects with at least some level of complexity. Nimble's task section is absolutely insufficient, and writing custom tool for each library quickly becomes really annoying.

[CyberTailor]

It's better to add Nimskull support for Meson, writing a brand new build system is not an optimal solution. Let's start this after rebranding.

[CyberTailor]

CMake is more powerful, so the compiler can be bootstrapped with it. Meson's Compiler support is for end users

[saem]

I will not under any circumstances allow cmake, meson, etc anywhere in the development cycle of the compiler. I barely tolerate python koch and I look forward to the day we can ditch that too.

[disruptek]

I will not under any circumstances allow cmake, meson, etc anywhere in the development cycle of the compiler. I barely tolerate python koch and I look forward to the day we can ditch that too.

That's really not the diplomatic, cooperative, constructive, helpful language I'd expect to see here.

[canelhasmateus]

What is the problem with the aforementioned build systems?
Asking as someone who is not familliar with these tools, or their strength and shortcomings.

[saem]

Reply below covers much of my issue with these big systems. #123 (reply in thread)

I want less stuff not more in the dev cycle.

• fewer dependency issues that come with these
  • often less big balls of python
• a big thing to learn, especially for debugging setup/version distro issues
• inevitably lots of stuff ends up in these systems and yet more action at a distance

Right now it's annoying but much of it is the language all the way down. The easier the language is to debug the further someone can go with a smaller set of skills.

[disruptek]

You've convinced me to share an alternative perspective.

If you have plenty of contributors and you know what you want the software to do, it's quite easy to replace or improve software. I'm far from convinced that you know what you need, let alone what you want, and introducing a novel piece of Nim/skull to do this job is just creating a new maintenance burden that doesn't need to exist to achieve superior functionality with less work. There's plenty of documentation, track record, and maintainers for other build tools; why exactly do you favor a new piece of code with none of that? Some people who might contribute to Nim already understand how to wield these tools, and do so powerfully. This is not the case for an imaginary piece of software that is so new that it hasn't even been designed. Do you want to develop yet another build tool or do you want to develop a compiler? Et cetera.

If this is already a decided issue, with all these (and surely more) arguments to the contrary, I really don't see how this project can claim to hold any of its ethos as written.

[alaviss]

This issue is nowhere near set, as there are no implementations yet. The possibility for Meson, CMake, etc. is still on the table.

There are a few reasons why we might want this new piece, other than just for koch:

• We need a replacement for extccomp and its caching system.

• testament rework wants a better test runner for parallelization and to order test dependencies.

Both of the above need a solution, and if one were to be created it could be used for koch too, eliminating a dependency.

Admittedly, while I've been working on and off on a solution, I have not yet produced an implementation that I deem of adequate quality for upstreaming. So there are plenty of chances for Meson, CMake, etc. to come in as a replacement.

Though, a few of our developers have their grievance on those build systems, so addressing that would be a requirement for other proposals of the same nature.

[disruptek]

Though, a few of our developers have their grievance on those build systems, so addressing that would be a requirement for other proposals of the same nature.

Maybe those developers can chime in with an enumeration of problems with other systems which prohibit their adoption for Nimskull, even as an interim solution.

[saem]

My fundamental issue with these tools is that they're very large systems solving very large problems. When one has those problems they'd be remiss not using those tools.

I'm much more in favor in having the compiler ditch more and more stuff to simplify it so we don't have as many of these big problems as possible.

So before shoving meson/cmake/etc into the compiler build, I'd much rather eliminate the complexity wherever possible.

Now if another project needs it because nimskull is playing a small part inside a grander picture then our artifacts, cli, etc should not be hostile to that use case so as to sabotage it.

[disruptek]

Why not do both: simplify the requirement set of Nimskull and then choose the least imposing solution which meets that set of requirements. But, first, simplify the requirement set, right. We agree that skipping that part doesn't make sense, yes?

[saem]

Yes, simplifying the requirements is a must. That first step so we have an initial compiler exe is it, where said step just works across all supported compiler build host platforms, we don't even need the full bootstrap cycle.

After that, well we can handoff to a nimskull program that can cover testing that initial compiler, finishing the rest of the bootstrap, having fancy orchestration for testament, whatever...