What language is rakudo written in?

[AlexDaniel]

I've seen this question many times. I think that we need an answer in our FAQ.

[safinaskar]

I want to see full path of building rakudo. I. e. to build rakudo you need such and such compilers and interpreters. And to build them you need such and such etc, etc. Then we will eventually get to perl 5 and what next? What language perl 5 written in?

Do we need in any point in this path to run some untrusted binary or some unreadable generated text file? Is this possible to build rakudo using only human-written non-generated text files and C and C++ compilers and nothing more?

[safinaskar]

For example, we have C and C++ compilers, human-written perl 5 and perl 6 sources without any untrusted generated files (both binary and text). Is this possible to build trusted rakudo binary from this? Not vulnerable to trusting trust ( http://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf ) attack in perl? (Of course, perl will still be vulnerable to trusting trust attack coming from C and C++ compilers, but is it vulnerable to perl trusting trust attack?)

[gfldex]

The details of the compiler called Rakudo are an implementation detail and should be explained at the compilers homepage. The docs document the language Perl 6 and will cover implementations only briefly. It's reference point is roast not Rakudo.

[stmuk]

I don't think exact compile time requirements belong in either the faq or
this doc repo.

Most users are expected to compile using Rakudo Star and this information
belongs there (and is already mostly there) in the top level README. Note
the compile requirements are different for Rakudo and Rakudo Star.

Maybe some top level details about rakudo implementation could be a new
"Implementations" faq section (eg. "rakudo is written in perl6 and NQP").

There is a binary blob used to bootstrap at one of the levels (either NQP
or MoarVM I forget) and it might also be interesting to explain this in the
new faq section -- without scaring people too much about the trust issues
which are common to most such systems and not just perl 6.

On 16 June 2016 at 04:05, Wenzel P. P. Peppmeyer notifications@github.com
wrote:

The details of the compiler called Rakudo are an implementation detail and
should be explained at the compilers homepage. The docs document the
language Perl 6 and will cover implementations only briefly. It's reference
point is roast not Rakudo.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#599 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AEUuY4FLxDwJ2jHICjnS2Rib-spUOMUbks5qML2AgaJpZM4I27NB
.

4096R/EA75174B Steve Mynott steve.mynott@gmail.com

[jnthn]

There is a binary blob used to bootstrap at one of the levels (either NQP
or MoarVM I forget) and it might also be interesting to explain this in the
new faq section -- without scaring people too much about the trust issues
which are common to most such systems and not just perl 6.

Also, the git history can, I believe, be used to trace all the source that led to those blobs (and reproduce them), should anyone want to do that.

[AlexDaniel]

In any case, I think we should have an answer for that question in FAQ (because it is a frequently asked question). It's fine if the answer is just a link to rakudo.org or whatever.

[safinaskar]

Also, the git history can, I believe, be used to trace all the source that led to those blobs (and reproduce them)

"I believe"? Eh? Did anyone really tried to reproduce this? Some reproducing instructions should be published. So that anyone can perform security audit of human-readable sources and then build them

[jnthn]

If I knew somebody had, I wouldn't have had to qualify it with "I believe". :-) I'd be happy to point anyone who wants to work on reproducing it and documenting the process in the right direction.

[safinaskar]

which are common to most such systems and not just perl 6.

And this is bad. Whole GNU/Linux should be boostrappable from C and C++

[jonathanstowe]

I'm confused, Perl 6 is nothing to do with GNU/Linux.

[pmichaud]

Out of curiosity, can you cite a published reference for "Whole GNU/Linux should be bootstrappable from C and C++"? Is this personal opinion, or is it a widely adopted philosophy that is itself documented somewhere and that I've been largely unaware of?

[stmuk]

These are technical issues not political/licence issues.

https://en.wikipedia.org/wiki/Bootstrapping_(compilers)#The_chicken_and_egg_problem

explains some of the issues of self hosting compilers.

As for UNIX-type systems (of which Linux is one) a small % of kernel code
is written in assembly language!

On 16 June 2016 at 23:53, Askar Safin notifications@github.com wrote:

which are common to most such systems and not just perl 6.
And this is bad. Whole GNU/Linux should be boostrappable from C and C++

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#599 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AEUuY3kliEeXQtKOuNyzlovUkOUehEEKks5qMdP-gaJpZM4I27NB
.

4096R/EA75174B Steve Mynott steve.mynott@gmail.com

[pmichaud]

To return to the original poster's request for a FAQ entry, here's a short blurb about Rakudo Perl 6's source code:

Q: What language is Rakudo written in?
A1: The glib answer is that Rakudo is written almost entirely in Perl 6, with a small bit of C or Java glue code thrown in depending on what virtual machine backend Rakudo is being compiled to use (e.g., MoarVM or JVM). If you look at the files in the "src/" directory in the Rakudo repository, you'll see that just about all of them are Perl 6 source code files

A2: The more detailed answer is that Rakudo is written as a mixture of Perl 6 and NQP ("Not Quite Perl"). NQP is a lightweight Perl 6-like environment for virtual machines; it's designed to be a high-level way to create compilers and libraries for virtual machines (such as MoarVM and JVM) using Perl 6 syntax. Thus programs written for NQP are also (theoretically) valid Perl 6. Then the question naturally becomes "What language is NQP written in?" NQP source is a mixture of (1) NQP code, (2) whatever language the underlying virtual machine is using, (3) some third-party C and Java libraries, and (4) some bootstrapping files created by earlier runs of the build process.

[AlexDaniel]

Does anybody want to improve http://doc.perl6.org/language/faq#What_language_is_Rakudo_written_in%3F or we can say that it's OK?

[safinaskar]

I'm confused, Perl 6 is nothing to do with GNU/Linux

Well, yes, but it is used on GNU/Linux.

Out of curiosity, can you cite a published reference for "Whole GNU/Linux should be bootstrappable from C and C++"? Is this personal opinion, or is it a widely adopted philosophy that is itself documented somewhere and that I've been largely unaware of?

First of all, this is my opinion. :) Also, I will provide some references.

The page https://wiki.debian.org/UpstreamGuide#Bootstrapping.2FCrossbuilding_Language_Compilers.2FInterpreters from Debian wiki says:

There should be a way to build the compiler or interpreter for a certain programming language without that compiler or interpreter

Please, read whole this section.

Then (bold is mine):

If parts of the core of Debian is written in a certain programming language, then the compiler for that language must additionally support cross compilation so that either compiled binaries for the new architecture can be created on an existing architecture or so that the interpreter is available on the new architecture. This condition is met for the C language with gcc but is not met for Perl which currently cannot be fully cross compiled.

It seems page authors mean Perl 5 here, and this is bad for Perl 5.

Then that wiki page says "There are a number of ways to achieve above requirements" and presents 3 options. They imply that current Perl 6 situation (i. e. we need some bootstrapping files from previous builds) is not acceptable. One of this options is the following:

identify the very early versions of your compiler/interpreter which were still built using another already existing programming language

Do you see? "Identify". You should not just say "we have some snapshots, I believe this is possible to bootstrap Perl 6 from them". You should identify what snapshots one can use to build trust path.

Also, let me point to "Reflections on Trusting Trust" ( http://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf ) and @david-a-wheeler's "Fully Countering Trusting Trust through Diverse Double-Compiling" ( http://www.dwheeler.com/trusting-trust/ ), which broadly discusses trusting trust attack. (I was not able to find "everything should be bootstrappable from C and C++" guideline in this texts, through.)

[david-a-wheeler]

Thanks for the mention.

Sadly, it is possible to slip malicious code into compilers and make the problem self-perpetuating. The good news is that there is a countermeasure, as explained in my paper "Fully Countering Trusting Trust through Diverse Double-Compiling" ( http://www.dwheeler.com/trusting-trust/ ).

However, the counter measure requires a second compiler that you have reason to believe is unlikely to be subverted the same way the first compiler might be. It doesn't require C or C++ specifically, but it does require a second compiler that is relatively independent. The good news is that the second compiler does not have to be fast or have lots of extra functionality, it just has to be good enough to provide a check on the other compiler.

Take a look at my paper for details.

[pmichaud]

I think I that Rakudo most likely fits the second of the 3 options presented, namely:

define a mini version of your language and then write a compiler or interpreter for that language in another language which is known to either already meet these requirements or which has a path down to a language that does. This mini language can then be used to compile the core of the bigger comiler/interpreter which can then in turn be used to compile more in as many steps as needed.

This is exactly what NQP is -- a mini version of the Perl 6 language that Rakudo is written in. NQP is a "mini" version of Perl 6; more notably, it's explicitly designed to enable cross-compilation and facilitate porting to other platforms/backends. The MoarVM and JVM flavors of NQP and Rakudo were created by cross-compiling from the Parrot one.

So while there will likely be a few pedantic nits, I think there's a good case to be made that Rakudo already meets much of Debian's guide here -- both by taking the mini version path and by supporting (and having repeatedly demonstrated) a viable form of cross-compilation for Rakudo.

Also, to follow-on to David Wheeler's comment: This is an excellent example of a reason to avoid labeling Rakudo as the "official" or "only" Perl 6 compiler. We explicitly recognize that it's valuable, useful, and important to have multiple independent implementations of Perl 6 (e.g., Niecza).

[david-a-wheeler]

Good to hear about NQP. I would urge you to include bootstrapping and the "compiler bootstrap test" as part of your automated test suite. It's not at all unusual to forget to test it, and over time the compiler loses its ability to be bootstrapped by anything else. If you go that far, it would be great if you would also do the diverse double compiling test... it requires that the compiler be deterministic, but you want that anyway, since non-deterministic compilers are a pain to debug. If you decide to apply diverse double compiling and have problems, or complete it, let me know.

[coke]

This is an interesting conversation, but the take away for the docs is @pmichaud's Q/A at #599 (comment)

I think we can close this ticket if that's added to the faq - and I think A1 is a reasonable level of detail for the docs site.

[AlexDaniel]

I think A1 is not enough as it opens more questions than it answers. But we can start with A1 and see how it goes.

[coke]

This question is already answered with a minimal amount of detail in the FAQ, and has been since 2776b19 (though it's been updated since then.)

Closing this ticket.