Question regarding inlining

[chenglou]

@mrakgr thanks for this project! I've been observing for a while now. Being able to have a nice type system + inlining guarantees could unlock some nice use-cases.

I'm still new to this subject, but I'm wondering what you think of C++'s constexpr or Jai's inlining features. But really, any language with some inlining guarantees, like a type-safe macro system. I'd love to try it out myself, but I'm on macOS.

Thanks for your work and feel free to close this issue =).

[mrakgr]

My view on how type safe macros should be done has not changed ever since I first grasped the direction and began work on Spiral. They should be functions. They should be pervasive and rather than being add-ons to a language, they should be its most dominant feature. That is the Spiral's way. To me it feels common sense and the only surprise is that it took somebody like me to come along and do it as it should be done way out on the timeline in 2017. It is a pity nobody created a language similar to Spiral 20 years as it would have saved me a lot of time making it. Had it been done, I bet it would have been the dominant language for GPU programming for example.

It is a pity, but languages follow an evolutionary trajectory and as a result there is a risk of branches getting missed.

There is of course a lot of research work done on metaprogramming - Lisp and Scheme are dominant in dynamic languages. In static languages that work falls under the umbrella of partial evaluation and I'd say that Scala has the most impressive work done here. Scala in particular certainly needs it. All highly abstract, powerful languages do.

However pretty much all the languages with the ambition of having efficient high level abstractions that I've surveyed before I began work on Spiral are trying to add partial evaluation features on foundation that might not support them.

C++ is probably the exemplar of such a language. Extrapolating the current pace of its development, I think that in 20 years it will have about half of what Spiral has today, except with a vast amount of baggage that would make the kind of style used in Spiral exceedingly tedious.

Jai to me seems like a C derivative. Its creator boasts 500k LOC/s compilation times. Spiral is realistically 1/100th of that and for it to be usable for large projects somebody (me) would need to develop significant capabilities for incremental compilation.

I would really love for Spiral to be 10s and 100s of times faster than it is now during compilation, but I do not think that will be happening.

---

There is a kind of pervasive meme going on right now in some circles that GC is to be avoided at all costs in systems programming. Languages like Rust are especially driven by such a consideration. If Spiral were a fully fledged project tasked with having its own runtime, it would be worth implementing tagless GC.

GC is always a point of conflict in systems languages. The motivation of avoiding it is the driver behind some languages like Rust, but I think it should be mentioned in these discussions that languages like Java and C# are slow not necessarily because of GC - it is because they are too dynamic. Their runtimes and GC were built on research done on Lisp-like languages and it might be possible to gain significant improvements by setting Spiral as the starting point instead and developing a lightweight runtime especially suited for it.

The important thing in my view is not so much avoiding GC as is having the capability to not need heap allocation for abstractions. Spiral's inlining capabilities actually make it significantly easier to avoid having to do heap allocation compared to C derivatives.

A goal for GC in a systems language that would be worth pursuing is that the overhead of GC is to be proportional to heap usage. Meaning the programs which do all their work on the stack would have no overhead from GC even if the language comes with it. And for the rest, it might be possible to make GC so it is minimally invasive and even faster than manual allocation.

I am just speculating that this last part might be possible - I have no plans to actually pursue this line of work as I am busy with machine learning, but Spiral definitely has research potential in this direction.

[chenglou]

Thanks for the reply! Btw, OCaml's GC is exactly that: it's a function of how much is retained. Might be an interesting target for Spiral one day? Easier to use on linux/macOS/web too (ocaml can compile to JS), though I understand this isn't a priority

[mrakgr]

Hmmm, does .NET not run on Mac? My search does turn up results for .net mac. I think you should be able to get Spiral to compile on Mac and then try it out if you are really interested. It should not be hard, Spiral is pretty small and does not have much in the way of dependencies.

My goal with Spiral is having a language that I could use to compile efficient GPU code for the sake of doing ML and nothing else at the moment. I'll eventually use it for other things like games and devices better suited for ML than GPUs, but now I just want to thoroughly master the current stage before moving on. And by moving on, it probably won't be to other OSs, but rather what I just mentioned.

A few years back when I first got interested in ML there weren't much in the way of good ML libraries - not on Windows anyway. As an example, for a long time Tensorflow and (now defunct) Theano were Linux and MacOS only. I think that Tensorflow and PyTorch supporting Windows only happened quite recently, around 1-1.5 years ago.

That might have been one of the triggers - had I been starting ML now, I probably would not go to the extreme of making my own language just for ML. What Spiral represents to me now is a way of marrying my programming ability with my desire to do ML. If other users find a need for Spiral, I'd be happy to have them, but otherwise have no intention of doing anything to attract them to the language.

[mrakgr]

I'll close the issue since the thread seems to be done, but if you have any more questions please feel free to ask.