Why encourage more builders?

[joshgoebel]

Why do we encourage a proliferation of builders when the spec itself is so precise? Wouldn't a single canonical builder in a popular language be easier to maintain and push the spec forward with new features, etc? It seems like a lot of duplicated effort and destined to lead to "dead" builders... (which has already happened over the intervening years)

It seems easy to show how the community benefits from additional templates and themes, but much less so from having many fragmented builder implementations, each implementing a different version of the spec, and in slightly variant ways.

Recently the idea was floated of perhaps not letting older builders [old spec] into the new organization... that has it's own problems (IMHO) but only solves the problem today, not tomorrow when the maintainer of builder-xyz is hit by a bus and no one else really cares about xyz because abc/def/ghi builders work so well...

Obviously if someone really wants to build a builder it's a free world - but should we encourage this vs one official builder that works on all popular platforms? (OS X, Windows, Linux, etc)

[joshgoebel]

I should be clear I can sorta maybe see the desire for per language library bindings (not builders) to work with scheme files... though this is largely (though not exhaustively) covered by a language having YAML bindings....

But even then what would you do with these bindings other than building builders?

And if the goal was to "support base16 themes" ... "as an application"... isn't that why we have templates? IE, wouldn't it be better to instead create a template for your app... and then that template turns the native Base16 theme into whatever format your language/app would most prefer to consume? Insulting it (largely) from future changes to the builders/spec.

The more libraries/builders we have the more fractured the ecosystem becomes when a new version of the spec is announced... you end up with:

• Application B (Python) only supports Base16 v12
• Application A (Node.js) only supports Base16 v9
• Application C (Go) supports Base16 v15

I'm wondering if this is avoidable. Right now once ANY single builder is updated all applications immediately support the latest spec.

[belak]

Thanks for bringing this up - you bring up a number of good points.

I think you're right - supporting a single builder which works on all platforms would probably be ideal, but providing libraries (and perhaps other integrations) for other languages would also be helpful.

One issue is that I'm not sure which builder we'd want to focus on - I'd obviously prefer Go (partially because I wrote it, but also because it's easy to build/deploy in a cross-platform manner, though they're binaries so you can't download the same thing everywhere), but the Node builder is up there too, especially because Github Actions runs a nodejs environment by default (Docker is also supported by Actions, though not quite as comprehensively).

There are a few places I could see builders being used, which might help us decide:

1. template maintainers - used when locally building and experimenting with templates
2. scheme creators - used when experimenting with a scheme, probably with certain templates
3. CI - used to automate building all schemes, possibly through Github Actions, possibly another way
4. the "managers" concept - this has never been formalized, but providing a way to build/install multiple schemes

[actionless]

i think having multiple builders can't be avoided - for example:

1. if we'll agree on rust builder - for people would be VERY hard to use it inside browser or on some lofi hardware without rust support, or generally some people would be not inspired downloading and building some exotic for them toolchain for just generating the color palette for terminal let's say

2. if we'll agree to use javascript builder - people who would like to use it from c or python lib wouldn't be excited having js machine as a requirement for adding colorpalettes support to their lib

so having one reference implementation is alright - but having that one reference implementation instead of spec for other implementations - would really close a lot of current usecases of base16 ecosystem

especially given what base16 is based only on 2 very popular libraries - mustache and yaml, which available for endless amount of target platforms

[joshgoebel]

would really close a lot of current usecases of base16 ecosystem

Which use cases do you feel that having a reference implementation (itself serving as a rough spec) would close off?

[actionless]

@joshgoebel

1. any other use case than running CLI manually/from the script, ex. using it as a library, not as CLI tool;
2. running CLI manually/from the script on platforms for which having rust buildchain/js runtime/etc is crazy overhead.

[joshgoebel]

using it as a library, not as CLI tool;

I'm not sure how having a reference builder implementation stops anyone from writing Base16 libraries or tools in any language... the scheme and template specs are what matter for library creators, not builders.

rust buildchain/js runtime/etc is crazy overhead.

No one is forcing you to use a reference implementation (whatever it is) if you don't like it - you can always write your own in C or Nim or whatever makes you happy...

[actionless]

@joshgoebel i having the impression what before replying you reading only 10% of the messages

please reply again after reading them full (as the point which you're so actively arguing about was never written by anyone in this thread)

or for example try quoting full sentences, not just 2 words out of context

[actionless]

instead of removing alternative builders i think it should be just defined the retention policy for the builders inside org - if builder getting unsupported for too many versions or too many months/years in a row - it getting removed from org

btw such retention policy also should be defined for templates - as they could break as soon as major version of the app for which they written for will be released

[joshgoebel]

I assure you I'm reading your messages. I fully read your last post before I responded. I still don't understand your two concerns about a reference implementation. Perhaps we're hitting a language barrier? I can repeat my thoughts in different words:

1. How does a reference builder implementation HURT library authors?
2. A reference implementation need not be the only implementation.
   • (nor does it imply that IMHO).

[actionless]

but i wrote it clearly what i'm not against having reference implementation - i'm against discouraging other alternative builders being part of the org (i'm not going to submit my builder to the org, so i don't have any personal benefit in this question)

even if choosing between js builder vs rust builder as in example i gave - it's not possible to choose one of them to be the one and only builder as both have pros and cons - whilst retention policy would just let the ecosystem naturally form

[belak]

Here's how I see the future of builders:

1. One or maybe two "official" builders aiming to always support the latest released version of the spec. Any extra builders could be put in @base16-project-contrib, or simply not be included. If we keep more than 1, we should have a reason. This currently looks like Go and Node, which could be used for different purposes (Maybe Go for CLI because it's easy to build/release binaries, Node for CI because it's supported directly by GH Actions - we're currently using Go via Docker for CI which would be another option). As a note, if we decide to drop one of these, it may be hard to choose which one.
2. Provide libraries in a number of languages for building templates from schemes rather than full builders.
3. Drop non-official builders from the README, add references to the libraries.

[joshgoebel]

which could be used for different purposes

I think "different purposes" may be a bit wide (seems to allow lots of room for new builders). Though personally I'm not in any hurry to lose the node builder as it's a measurable part of my motivation to stay involved - it allows me to easily (and quickly) prototype things like my semantic color proposal or the recent stats I posted on accessibility... being a maintainer of a builder means it took only a few minutes to generate those stats just by writing a few lines of JS.

I think the "committed and invested maintainer" shouldn't be wholly discounted as a reason for a build tool to exist... though having contrib as a fallback is interesting... I'm not sure how I would feel if my builder got "kicked out" and into contrib. :-) Obviously this is more of a problem for the two existing official builders... it's easy enough for us to just say "no more official builders"... though I wonder if we didn't already "drop" builders during the recent transition itself?

Provide libraries in a number of languages for building templates from schemes rather than full builders.

I still don't fully grasp this. The best real life test that those "builder libraries" worked would be to write the remaining looping code to create a full builder - and then build. We could even have a script to "build world" (using the simple CLI proposed elsewhere) and then SHA256 the resulting output to certify that all builders were producing the same exactly output for all schemes/templates.

I mean I personally don't think we need 100 builders but I also don't see how 100 build libraries that are just 1% short of being a builder makes a lot of sense. At that point you might as well go all the way and be a library + unofficial builder.

[belak]

I think "different purposes" may be a bit wide (seems to allow lots of room for new builders).

Fair point.

Though personally I'm not in any hurry to lose the node builder as it's a measurable part of my motivation to stay involved - it allows me to easily (and quickly) prototype things like my semantic color proposal or the recent stats I posted on accessibility... being a maintainer of a builder means it took only a few minutes to generate those stats just by writing a few lines of JS.

This is one of the reasons I stay involved as well - I like maintaining a builder and I like lots of the low level tooling.

I think the "committed and invested maintainer" shouldn't be wholly discounted as a reason for a build tool to exist...

This makes sense to me. Lets make this the requirement for now - similar to templates, builders would need to be "sponsored" by a primary maintainer, but in this case we can also say that maintainer needs to be already involved in other parts of base16. I'm not sure of a way to phrase that succinctly though.

though having contrib as a fallback is interesting...

I wanted contrib to be a place where less-maintained schemes (or schemes without an active primary maintainer) get moved. Maybe this will be something to revisit in the future.

though I wonder if we didn't already "drop" builders during the recent transition itself?

Yes, I dropped a number of them a long time ago - Chris used to maintain the PHP builder, but hasn't for a number of years... and I'm not a huge fan of PHP as a CLI scripting language anyway.

I still don't fully grasp this. The best real life test that those "builder libraries" worked would be to write the remaining looping code to create a full builder - and then build.

I mean I personally don't think we need 100 builders but I also don't see how 100 build libraries that are just 1% short of being a builder makes a lot of sense. At that point you might as well go all the way and be a library + unofficial builder.

Yes, that's a fair point.

Maybe we can specify that official builders will export a library? I'm trying to think of an alternate method of making "managers" possible to build without requiring a very specific CLI interface.

[joshgoebel]

@Misterio77 I understand flavours is a builder now as well - I didn't know that before. Could I ask what led you to writing your own builder vs wrapping one? One thing we're trying to figure out is how to best support and grow the broader ecosystem.

Imagine if every theming tool was their own builder... any hard work we put into the style systems and reference builders won't benefit those tools... This seems like a problem from an ecosystem building and duplication of effort perspective. Instead of every tool potentially automatically gaining support for Base17, Base24, Ansi16, etc as soon as we release - instead nothing would happen. And all that work would have to be repeated over and over...

Do you have any thoughts or ideas around this?

[Misterio77]

Yeah, I agree with the thought!

Simply put, my main issue was that existing builders seemingly weren't really designed with tool development in mind. Each builder does stuff differently and don't really treat their interfaces as something other developers would use.

Before writing flavours, I wrote a shell script that used the Python base16 builder. The python builder had good features for end users, but wasn't very scriptable or particularly fast, so building a tool with the UX I wanted (mostly centered about defining templates in a config file and using the tool to change between schemes) would be impossible.

The rust builder at the time didn't (and probably still don't) exports any of its features as a library, and calling other (slow) builders through shell commands would totally defeat the point of building a tool designed to be fast.

That's pretty much why I think we should have a clear definition of builders, that are actually designed to be small, scriptable, and that can possibly be used as libraries.

Flavours is pretty much in maintainance mode as I don't use it anymore. I switched over to NixOS and now use it to fill in colors in my apps (using nix-colors). I do plan on writing a decent general purpose rust builder and overhauling flavours at some point (managing scheme and template files is specially bad, so I plan on doing something similar to what plugin managers do)

[Misterio77]

My main gripe is that builders try (and mostly fail) to include porcelain features for end users, and in the process cripple the scriptability and minimalism plumbing tools should have.

[joshgoebel]

Each builder does stuff differently and don't really treat their interfaces as something other developers would use.

But you only need to use one - I'm not sure why slightly different interfaces would be a huge problem... you don't need to use all 20 builders, just the one you pick.

calling other (slow) builders through shell commands

How fast does it need to be? Node builder builds all 233 schemes (1 template) in only 0.7 seconds on my system... I've made no effort to optimize it, but I don't find that particularly slow. I imagine building 1 scheme to multiple templates would be a more useful benchmark but I don't have an easy way to test that quickly - and I'm convinced it should be equally fast.

The last time I "built world" (all schemes * all templates) I recall it only taking several seconds...

Honestly though, shouldn't the builds be cached after generation? Or mass pre-cached upfront? I'm not sure why you would prefer live building unless you're generating dynamic/random schemes or something.

and that can possibly be used as libraries.

I'm pretty sure this isn't the right answer, but I know some of us want to see tons of libraries... but to me it seems the builder problem all over again... as the specs gets more complex we're duplicating work and effort 10x, 20x... with each library having it's own unique flaws, bugs, differences, etc...

My main gripe is that builders try (and mostly fail) to include porcelain features for end users, and in the process cripple the scriptability and minimalism plumbing tools should have.

• What is a porcelain feature?
• What would your dream builder scriptable tool (not library) look like?

[belak]

Alright, I've got a few things here -

1. The spec is currently very simple, which makes it easy to create a builder rather than using an existing one (as an example see https://github.com/afq984/base16-xfce4-terminal which provides a minimal build.go to work with that repo, but they also encourage using other builders).
2. Providing lots of builders is mostly possible because the spec only requires you to read in color values, and export variables based on them - there is not currently any resolving variables, aliases, or other things. If we add these features it probably makes sense to focus on the builders we directly support rather than making it easy to create new builders.

@Misterio77 I'd be happy to add a porcelain interface to base16-builder-go, as an option rather than the only interface - I'm refactoring that builder at the moment so it may be hard to make a bunch of changes yourself, but if you're willing to work with me on a proposal, I'll build it.

[belak]

At the moment here's where we are:

• I have been keeping the base16-builder-go updated
• All spec updates I make have a corresponding PR to base16-builder-go before being merged
• We're going to focus on builders which have a "sponsor" willing to maintain them. At the moment, that is primarily the Go builder, but we semi-officially maintain a Node builder and I believe Jamy is also working on a Rust builder.
• I have no preference as to which builder we end up keeping long-term, but I agree that when we start adding more complicated features, maintaining a bunch of different builders will not be ideal.

I've made my opinions known and I don't believe this ticket is currently actionable, so I'm closing it. If you think that's a mistake, please let me know or open a new ticket.