To fork, or not to fork, that is (not?) the question

[LebedevRI]

(please move to discussions, intentionally created as an issue to have wider notification reach)

(There's TLDR at the end of this comment.)

Dear maintainers.

As it is apparent, i'm relatively new in this repository.
While i have knew of Halide's existence since, perhaps, at least ~2016,
it wasn't until recent that i have become sufficiently unhappy with the
intricacies and horrors of the implementation detail, that,
if you hold the tools right, lead to somewhat reasonable performance.

So i've wanted things to change. While there are many possibilities,
in the end, Halide did seem like a reasonable fit to improve QoL.
Of course, it was expected that there will be hard edges that will
need some tuning, there always are some.

On the surface level of immersion into the codebase, things are nice!
However, the mode i dug, be it either during attempts at addressing
the rough edges that i've run into, or just trying to get a grasp of
the codebase, the more confused i become. You see, the problem was,
unfortunately, i had a (mis?) fortune of hacking at LLVM/clang before,
so i had a rough expectation of what'd expect to see.

Long story short, here's some of the things, in no particular order,
that rubbed me the wrong way:

• Unreadable codebase - i don't have infinite-width monitor.
• An extremely weird choice regarding integer wrapping-vs-2's complement arithmetic - why is it that specifically 32-bit ints have their overflow as undefined? That should be type's property.
• Everything is a public API
• Basically no UB checking
• Lack of documentation, specifications
• Lack of serialization
  • Lack of translation verification tools (not surprising!)
  • Lack of reduction tools (have fun with reproducers!)
• Poor fuzzer coverage
• Lack of integration of fuzzers into oss-fuzz
• Not distinguishing between fallible input checking, and invariant checking (all asserts are always on)
• A particular preference of recursive algorithms in codebase
• Separation from the greater LLVM ecosystem
• Everything is an end-to-end test, starting from C++ frontend
• A particular lack of unit tests
• Endianness pickinness

Now, let's take a step back. Why am writing this? Because i care,
and i want to have something that is better and faster than plain C/etc,
but without having to have a really bad time when one has to dig
in the internals of the underlying codebase. And while the first part,
we could say, is mostly satisfied, i think we all can all agree
that the second part isn't.

I understand that the project is not exactly obscure, and is actually used
in production, and it is also a research project. But what that means is,
it's not widely used either, and the, far fewer, people that also end up
developing it, are more preoccupied in keeping it afloat than addressing
the more fundamental problems.

So to continue the original thought, the issues i have listed are addressable,
but as i have said, i'm relatively new in this repository, and don't have
any existing stake in it, or any existing halide-based code that i have to
maintain, so i'm somewhat wary of sunk cost fallacy, or specifically,
sinking it further instead of making some other choice.

To summarize, i would like to address those issues, i really would,
but i feel like we need to take a step back here, reassess the modern
state of the world, and, having learned the lessons that we have learned
getting here, start anew with a more defined architecture,
on a modern base, with a wider audience?

TLDR:
I would like to propose to discuss starting a new upstream LLVM sub-project,
which would effectively be a modern-day reboot of the existing Halide,
with almost identical front-end and results, but not so identical internals,
and wider audience/reach.

Would any of the existing Halide contributors be interested to contribute with that?
Would any of the existing Halide consumers be interested in the improvements mentioned, and in migrating over?

Now, this stand-up comedy is over. Please feel free to drive me out of the stage with your laughter. Thank you.

Roman.

[pranavb-ca]

@LebedevRI - Thanks for sharing your thoughts above. I don't know if this discussion is where we wish to dive into taking action on some of the specifics mentioned above but @aankit-ca and I are interested in solving the "Lack of serialization" problem that you talk of above. It'll be an important step towards creating a bugpoint-like tool for Halide.

@abadams @steven-johnson - are you guys aware of anyone who has already started working on this and would need help? If not, we'll start from scratch.

[steven-johnson]

@abadams @steven-johnson - are you guys aware of anyone who has already started working on this and would need help? If not, we'll start from scratch.

@zvookin has done some thinking about serialization; I don't know to what extent he has written them up.

[zvookin]

I have a flatbuffer definition which is getting moderately close to complete as a first draft. I have not hooked it up to anything yet. I'll share it later today or tomorrow. Flatbuffer is a good choice but it is certainly not the only possibility.

[pranavb-ca]

@zvookin - Thanks, looking forward to it. Any specific reason you chose flatbuffers over protobuf?

[zvookin]

Apologies for the delay. Theory is flatbuffers will result in both less code size and smaller serialized data. Mostly I consider it a starting point. Start on the schema is here: https://github.com/halide/Halide/blob/halide_ir_flatbuffer/apps/flatbuffer_ir/halide_ir.fbs . Next steps are to finish the toplevel nodes (Pipeline, Module) and to implement conversion to/from Halide's internal IR graphs. It would be great to replace our IR graphs entirely but I don't think that is possible with anything like flatbuffer/protobuf/Cap'N Proto, etc. Please let me know your thoughts and if you want to start working on it, happy to coordinate with you. I figured having a concrete starting point would help move the work forward.

[pranavb-ca]

Thank you @zvookin - @aankit-ca is out of the office today. He'll be back tomorrow and will take a look at this.

[LebedevRI]

Now posted https://discourse.llvm.org/t/interest-in-a-perhaps-simplistic-image-processing-front-end/65431.