Status update & summer break

[perlun]

Hi,

Just like last year, I thought I'd share a bit of status update from my end now before the project goes into "summer break" again for a while.

The 0.3.0 release was made a few months ago. See the release notes for more details. Just like last time the Markdown source is the place to go to in case you find some spelling errors or other mistakes in the document.

Musings in the world of native memory (de)allocation

Since the 0.3.0 release, I have started looking into some new interesting features, most notably:

• (many) Implement AsciiString for ASCII-only strings #377
• (many) Implement Utf8String for strings with non-ASCII content #385

I also started trying to use these new strings a bit in practice (#389), but really, this will not be complete until we take it all the way and use Perlang strings everywhere (#390).

And if we've already done that... why not go even further and make running Perlang programs completely independent of the .NET platform altogether?

Yes, it's a significant effort, but if we're ever to do it, the best time could be now(). I touch the subject a bit in #376:

This aligns well with the overarching goal of not making Perlang be overly dependent on the .NET platform itself; ideally, I don't want to close any doors right now as for the future of "compiled" Perlang. It may compile to MSIL, it may also compile to ELF/Win32 PE binaries. Both of these should remain possible. The recent steps in the first #370 implementation PR (#377) where we move towards using unmanaged byte* pointers for the string data seems well-aligned with this goal.

LLVM to the rescue

To this end, I've started experimenting a bit with LLVM, more specifically trying out the Kaleidoscope tutorial (https://github.com/perlun/llvm-tutorial - I got to chapter 8). The feeling I get is that it's actually quite easy to use this tooling for generating "real" AOT-compiled images, i.e. .o files on Linux. (Thanks to JetBrains for throwing me a zero-cost copy of CLion to use when playing around with this btw, you guys are the best. ❤️)

There's only one catch though: the LLVM Core libraries are C++-based. C++ cannot be directly be used from C# (with P/Invoke), because of name mangling.

LLVMSharp to the rescue... kind of.

After testing LLVM from C++ (which was easy; just follow the tutorial and copy the example code into your project), I also tried to get it working with LLVMSharp. Getting this running on LLVMSharp is a bit of a different beast unfortunately. The project is pretty much a one-man show with no activity the last few months, with little documentation. It's probably useful if you know a lot about how to use it and know P/Invoke nitty-gritty details by heart, but at least for me the learning curve has proven to be a bit too challenging.

What would be a real help: C++/CLI

The "easiest" way to deal with this could instead be to interoperate with the LLVM libraries using C++/CLI. C++/CLI provides a convenient "bridge" between the managed world of C# and C++. Using C++/CLI, we could keep the majority of our code (for parsing Perlang code, doing type inference/other checks, etc) and just write a small C++ "glue" layer to fit the C# parts together with the LLVM C++ API. Unfortunately though, C++/CLI is not supported on Linux, and might never be: dotnet/core#2508

Rich Lander from the .NET Team explains why in this comment: dotnet/core#2508 (comment)

We don't plan to enable C++/CLI on Linux (or any other OS other than Windows). It would require us to do one of two things:

• Enable the MSVC compiler on Linux. That would be a very large project since MSVC has only ever targeted and run on Windows.
• Enable the C++/CLI feature in Clang/LLVM or GCC. That, too, would be a very large project.

We haven't identified any reasonable cost version of this project. It's a very large and complicated feature set that is sensitive to the C++ compiler, OS, and architecture. I cannot think of another feature like that or that is even in the same ballpark.

The future is not set

So, how to proceed from here? I honestly don't know.

One way would be to start writing a new C++ implementation of Perlang from scratch. This has many drawbacks:

• I'm not a big fan of C++. I've honestly hardly used in since the early 2000s, where I wrote a C++-based Qt application at work. I find the C++ language quite complex, and the whole concept of "header files" that have to be kept in sync with the implementation is just so archaic when you have worked with .NET and Java for a while.

• It's by necessity a "big-bang" style of change. If we can't interop between C# and C++, we can't reuse really anything of our existing code. Particularly losing all the unit/integration tests and having to reimplement all of this in C++ would be a significant effort.

• It will consume all the project's resources for a huge amount of time. Perlang is not a big corporate project with many developers. It's just me. 🙂 And to be honest, because of family-related issues and the pressure life brings, neither my motivation nor the time I've been able to put into the project hasn't been that great lately.

The summer is (almost) here

All of this gives us a natural transition to the forthcoming summer break. 😅 It'll be great to try and take a break from this for a while and keep my mind on some other things. Who knows, perhaps I'll come up with some great ideas when I take some time away from the project?

Best regards,
Per

[perlun]

Some months have passed since I wrote the previous message, and I'm happy to inform you that #409 has now been merged. Take a look at #406 for the overall status of the "compiled mode" effort.