A Rust bootloader?

[Legorooj]

Thoughts/opinions/ideas on porting the bootloader to rust?

As an experiment I've started to write a rust copy of the current bootloader.

[Legorooj]

@rokm @bwoodsend @htgoebel @bjones1 @BoboTiG @rs357

[bwoodsend]

My thoughts are still the same: I'd want to see a proof of concept before we inject a 3^rd programming language which only one of us has any knowledge of. I really don't know if it'll simplify everything or require an intermediate layer of void pointer anarchy.

I'd be more for it if were still just you, me, Harmut and Bryan but we have quite a few competent C programmers now.

[bwoodsend]

@Legorooj Just do it. Try and write a bootloader. Forget onefile / cross-compiling / Apple events / whatever other shenanigans we have in there for now. Either you'll come back and say it was easy, in which case I'll be taking a crash course in Rust and joining your making it a permanent switch. Or you'll stew on it for weeks and come back with something just as unpleasant as our current bootloaders. Either way, we'll have our answer.

[BoboTiG]

Seems a good path. I would learn Rust too :)

[Legorooj]

I've already started work on one, as I mentioned 🙃. Got a little confused about how ownership worked, so tried a sideproject to test that out, but got to grasps with it now. Hell of a lot easier than pointers in C. Should, in theory, have a prototype soonish. Biggest barrier is going to be figuring out how the current bootloader does stuff - half the code is undocumented and hard to understand.

[rokm]

I think this might be an interesting experiment, but I don't have any experience with Rust at all, so I cannot predict how it will turn out.

I guess my main concern is maintenance in terms of manpower; among the developers on the team, I suspect we have more people who are at least familiar with C but I'm not sure how far our practical experience with Rust goes. So moving to a language that we are even less familiar with will likely have a negative impact on overall bootloader development (e.g., reviews of bootloader-related PRs) - at least initially. Then again, the transition probably won't happen overnight, either...

Currently, the bootloader is doing a lot of low-level interaction with native APIs, which will probably look ugly in Rust and where all its
safety benefits will go out of the window. But on the other hand, if the equivalent functionality is available in Rust's standard library,
then things might actually improve on that front.

As others have said, the best way to go is probably to make a prototype with limited functionality, just to see how things work out, to demonstrate the benefits, and uncover some (potential) pitfalls.

[Legorooj]

Currently, the bootloader is doing a lot of low-level interaction with native APIs

A lot of this can be abstracted into Rust's standard library. For instance, one of the first thing the bootloader does is resolve the path to the executable being run. This can be done in rust with the following code:

pub fn executable_path() -> path::PathBuf {
    // Get the current executable name. This may be an unresolved symlink on some platforms.
    let filename = env::current_exe().expect("Unable to determine current executable path.");
     
    // Check if filename is a symlink.
    if fs::symlink_metadata(&filename).expect("Unable to read symlink.").file_type().is_symlink() {
        // Resolve the symlink, return the value
        fs::read_link(&filename.as_path()).expect("Unable to resolve symlink").canonicalize().unwrap()
    } else {
        // Convert the path to absolute and return it.
        filename.canonicalize().unwrap()
    }
}

Whereas our current equivalent in C is much longer and uglier:

bool
pyi_path_executable(char *execfile, const char *appname)
{
#ifdef _WIN32
    wchar_t modulename_w[PATH_MAX];

    /* GetModuleFileNameW returns an absolute, fully qualified path
     */
    if (!GetModuleFileNameW(NULL, modulename_w, PATH_MAX)) {
        FATAL_WINERROR("GetModuleFileNameW", "Failed to get executable path.");
        return false;
    }

    if (!pyi_win32_utils_to_utf8(execfile, modulename_w, PATH_MAX)) {
        FATALERROR("Failed to convert executable path to UTF-8.");
        return false;
    }

#elif __APPLE__
    char buffer[PATH_MAX];
    uint32_t length = sizeof(buffer);

    /* Mac OS X has special function to obtain path to executable.
     * This may return a symlink.
     */
    if (_NSGetExecutablePath(buffer, &length) != 0) {
        FATALERROR("System error - unable to load!\n");
        return false;
    }

    if (pyi_path_fullpath(execfile, PATH_MAX, buffer) == false) {
        VS("LOADER: Cannot get fullpath for %s\n", execfile);
        return false;
    }

#else /* ifdef _WIN32 */
    /* On Linux, FreeBSD, and Solaris, we try these /proc paths first
     */
    size_t name_len = -1;

    #if defined(__linux__)
    name_len = readlink("/proc/self/exe", execfile, PATH_MAX-1);  /* Linux */
    #elif defined(__FreeBSD__)
    name_len = readlink("/proc/curproc/file", execfile, PATH_MAX-1);  /* FreeBSD */
    #elif defined(__sun)
    name_len = readlink("/proc/self/path/a.out", execfile, PATH_MAX-1);  /* Solaris */
    #endif

    if (name_len != -1) {
        /* execfile is not yet zero-terminated. result is the byte count. */
        *(execfile + name_len) = '\0';
    } else {
        if (strchr(appname, PYI_SEP)) {
            /* Absolute or relative path: Canonicalize directory path,
             * but keep original basename.
             */
            if (pyi_path_fullpath_keep_basename(execfile, appname) == false) {
                VS("LOADER: Cannot get fullpath for %s\n", execfile);
                return false;
            }
        }
        else {
            /* No absolute or relative path, just program name: search $PATH.
             */
            char buffer[PATH_MAX];
            if (! pyi_search_path(buffer, appname)) {
                /* Searching $PATH failed, user is crazy. */
                VS("LOADER: Searching $PATH failed for %s", appname);
                if (snprintf(buffer, PATH_MAX, "%s", appname) >= PATH_MAX) {
                    VS("LOADER: Full path to application exceeds PATH_MAX: %s\n", appname);
                    return false;
                }
            }
            if (pyi_path_fullpath_keep_basename(execfile, buffer) == false) {
                VS("LOADER: Cannot get fullpath for %s\n", execfile);
                return false;
            }
        }
    }
#endif /* ifdef _WIN32 */
    VS("LOADER: executable is %s\n", execfile);
    return true;
}

So in a lot of the common cases, interation with the native APIs is much easier from Rust.

Also, things like file opening, threads, etc, that has vastly different APIs in C, have one consistent API in Rust.

[BoboTiG]

Ah yes, Rust is way less verbose and more readable in your example. Having one code path for several OSes is quite attractive too.

[Legorooj]

Yes, it is.

I should also note that Rust's unrecoverable error handling system is better than C's - the biggest factor here being that C doesn't have one. Rust's panic! system to which you can add custom handlers, can give a traceback similarish to python's - making errors much easier to debug.

[htgoebel]

To be frank: I'm not conviced this is a good idea.

While the rust language seems to be a real step forward, rustc and the rust-environment is a crap: Rust does not have any notion of a library to be shared between builds - not even static ones like ".a". So each any every project has to compile all dependencies from source again. This consumes time and electric power - and is a thread for he climate.

This is esp. true for the CI: all dependencies have to be build from scratch on each CI-run. Maybe we could work around this using sophisticated caching - anyhow this is another piece of work to be done.

Anyhow, nobody can/will hinder you implementing the bootloader in rust :-)