[RFC]: Bareflank 3.0

[rianquinn]

We have been working on "2.0" for some time now, and I think it is time we make it more official. Moving forward, there are a number of pretty large changes coming to Bareflank which I would like to mark as "3.0". Here is a list of things we are working on which you should see over the next year or so.

If you have any questions, concerns, or features you would like to see, please comment below. Our main goal here in making these changes is to support the community better as a project, and if you feel like these changes will negatively effect how you are using Bareflank, we want to know about it so that we can address any of your concerns.

Finally, if you are interested in helping with these changes, please let us know!!! We are always looking for more help.

BSL

One of the biggest things that has prevented native support for Windows (i.e., the need for cygwin), prevented easy inclusion of ARM, and has generally made things more complex than needed is the use of Libc++. Originally, We thought that more of Libc++ would be needed, but after doing an analysis of our use of Libc++, and whether or not some of the features we are using are actually needed, it turns out we really use the C++17 language features and not the standard library. Removing the need for Libc++ removes the need for Newlib, which solves a LOT of problems. The BSL is a subset of the standard C++ library, containing the features that we use, without the added dependencies. It is NOT a replacement to the standard library, most of the APIs will not be supported, and a lot of the APIs will be different to address some Bareflank specific needs. But... with that said, our goal is to stay as close to the standard library spec as possible and where it makes sense to reduce the added learning curve we would be introducing.

For those who are used to using the standard library and want to continue to have these features, we will be working to ensure the Standalone C++ project can still be used if desired, but you would be limited to compiling the hypervisor on Linux (or the WSL). We do not want to continue to support cygwin.

AUTOSAR

The new coding standard for the project will be AUTOSAR C++14. We already adhere to the C++ Core Guidelines. There are some things that will look different, but the number of changes required to support AUTOSAR are minimal (and mostly pedantic). Adhering to AUTOSAR C++14 provides the project with additional applicability in real-world use-cases, based on requests we have seen over the past couple of years. Downstream users are obviously not required to code to these standards, and thus, this change should not effect downstream projects. To ensure upstream contributions adhere to this spec, we will include SonarCloud in our PR analysis, and the maintainers that work for AIS will work to ensure that any contributions that are made that have issues with AUTOSAR are fixed (either in the PR, or in additional PRs after a merge) as we have licenses to Perforce Helix QAC.

Exceptions

Our goal is to move away from exceptions in-favor of something similar to:
https://zajo.github.io/leaf/

Exceptions require an unwinder, and the unwinder has been a giant problem for the project WRT adding support for ARM and AMD. We could use something like libunwind instead (with some heavy mods), but this collides with our goals of adding support for AUTOSAR. The original design of Bareflank didn't have support for exceptions. We added this support because we wanted to use Libc++ which requires exception support if you want to use any of the APIs with Core Guideline support. Since we are moving away from Libc++, it removes the need for exceptions.

Dynamic Memory

Another goal is to remove the need for dynamic memory where possible. Specifically, we want to remove the need for malloc, free, new and delete in favor of static allocation and a basic page pool. The use of dynamic memory has over-complicated the hypervisor. For example, how much memory do you need to support 128 vCPUs... I have no idea, and trying to determine this would not be easy. Instead of using an unordered map of vCPUs allocated on the heap, we could simply pre-allocate, statically, the vCPUs we plan to support, instead of pre-allocating heap memory and hoping the amount of memory we allocated is enough for the vCPUs we need. A page pool would still be needed to support page tables and some other small needs, but that type of memory allocation is easy and deterministic.

S.O.L.I.D and Static Interfaces

The biggest, breaking change outside of dropping support for Libc++ by default is the move towards SOLID design principles. Our goal here is to write our APIs in a way that makes it painful to change moving forward. It has taken a number of years to understand what kinds of APIs we need to support everything from reverse-engineering, hacking, research and even full-blown guest VM support. Internally at AIS, we are using Bareflank, successfully for all of these, and as a result, we have much better understanding of what is needed, providing us with a good starting point to define a set of APIs we would like to support moving forward. Although there would be a number of changes made, most of them, at this point, are changes in functions names and types (as we are not using Libc++). We do not expect a lot of changes to the functions themselves. Porting to the new interfaces should be as painless as possible. The pain will be felt more on AIS and the internal implementation of the existing APIs to ensure they adhere to SOLID.

Documentation

Lets face it... documentation sucks at the moment. Our goal is to use doxygen for our main website moving forward. How to theme doxygen so that it doesn't look like a website that wasn't written in the 90's is something we are still exploring, but it can be done, and there are a number of good options. To ensure documentation is more complete, every API will have it's own set of examples, and the website will be easy to navigate and find everything you need. Bareflank is an SDK... its time we document it like a real SDK.

Native Windows Support

As stated above, our goal is to ensure working on Windows systems is as first-class as working on Linux systems. This means, no more cygwin. To ensure this is possible, all development of 3.0 will be done in Windows, testing on Linux instead of the other way around. This will ensure everything works on Windows.

AMD Support

AMD is killing it right now, and as such, we need to support it. 3.0 will add official support for AMD. Which versions of AMD CPUs we support is still being determined. You might be asking... what about ARM support? We have a working implementation of ARM internally, and if our move away from Exceptions and Libc++ really does make working ARM easier, ARM might also be added to 3.0.

[JaredWright]

A few of my opinions:

1. The project should adopt a calendar versioning model instead of arbitrary release numbers. For example, if the project is ready for a new release on March 15, 2020 then the release could be named either of the following:
   a) 20.03.15 (using the format YY.MM.DD)
   b) 20.03.1 (using the format YY.MM.minor_version)

2. "Our goal here is to write our APIs in a way that makes it painful to change moving forward."
   I would state this differently: the goal of adopting SOLID design principles is to more clearly define a boundary between the project's public facing APIs and the project's implementation details. This should make it easier for the project's implementation details to change under the hood, and enable traceability for public facing API changes between releases (not prevent them from changing in the future)

3. Tackle each of the categories above as a separate release. Scoping everything into one release prohibits developers from contributing, makes reviewing changes difficult, and leaves the release susceptible to scope-creep. I would propose that these changes be addressed in the following order:
   a) S.O.L.I.D and Static Interfaces
   b) Dynamic Memory
   c) BSL
   d) Exceptions
   e) Native Windows Support
   f) Documentation
   g) AUTOSAR compliance
   h) AMD support

[rianquinn]

I agree on all points... minus how you laid out releases. A couple of these have to be done simultaneously.

• For example, we are required to adopt AUTOSAR at the start of the project. Using afterwords is not allowed by the spec (this is pretty common with most critical systems specs).
• Removal of dynamic memory requires the removal of exceptions (cannot have one without the other).
• Native windows support will come for free as this is done.

So with all of that said, I would propose the following order instead (all of this assumes on Windows with AUTOSAR):

• Implement the BSL. This removes Libc++, Exceptions and Dynamic Memory
• Port Bareflank to the BSL with the existing APIs.
• Implement SOLID and Static Interfaces
• Add AMD support
• Implement all unit tests
• Implement documentation
• Complete website and testing

[cjams]

I like the idea of calendar releases instead of packing in bunch of changes and putting a number to it..We decide what the cadence will be and cut a release accordingly. If changes are ready, they're ready and they make it in; if not then maybe they'll get in the next one, maybe they wont.

[rianquinn]

I second that as well. We are in some ways doing that now, minus the naming part.

[rianquinn]

I was thinking about this last night, and I would like to swap adding AMD support so that we have support for this prior to redoing the interfaces. This way, if adding AMD support causing some APIs to change, these changes can be accounted for. I also added PAL to the list.

• Implement the BSL. This removes Libc++, Exceptions and Dynamic Memory
• Port Bareflank to the BSL with the existing APIs.
• Add AMD support
• Implement SOLID and Static Interfaces
• Switch to PAL
• Implement all unit tests
• Implement documentation
• Complete website and testing

As for switching to something like Zulip or Stack, we should do that soon.

[rianquinn]

Based on some internal discussions, here is the updated schedule for how we plan to roll this out:

Step 1:

• Implement the BSL (@rianquinn). This will provide enough scaffolding to ensure interfaces can be developed and compiled
• Implement the Hypervisor Directory/File Structure (@JaredWright). This will provide a complete layout of all of the anticipated files for Bareflank so that we can see how the project will be laid out.

Step 2:

• Implement the BSL (@rianquinn). This will finish the core components of the BSL to support Hypervisor development. A complete list of the components will be provided in a different RFC.
• Implement Static Interfaces (@JaredWright). This will implement all of scaffolding for the Bareflank Hypervisor including all public static interfaces, private (empty) classes, and basic build system with package/extension support

Step 3:

• Implement AMD support (@rianquinn). This will prototype AMD support using the existing Bareflank project. This will likely be very messy, and will be developed in its own branch. The goal is simply to understand what is needed for AMD support by implementing some basic support.
• Complete Static Interfaces (@JaredWright). This will complete all of scaffolding for the Bareflank Hypervisor including all public static interfaces, private (empty) classes, and basic build system with package/extension support, including support for the build system targets, documentation, CI, etc... This will also include support for PAL.

Step 4:

• Implement Intel support (@rianquinn). This will add Intel support to the new project using the existing scaffolding.
• Implement AMD PAL (@JaredWright). This will implement PAL for AMD.

Step 5:

• Implement AMD support (@rianquinn). This will add AMD support to the new project using the existing scaffolding
• Implement ARM support (@JaredWright). This will add ARM support to the new project using the existing scaffolding. Please note that this is optional as a stretch goal.

Step 6:

• Implement unit tests, documentation and website (everyone).

Also, having played with Zulip, I still think that Stack is a better option.

[JaredWright]

I'm planning to create a branch named "solid_foundation" to get started on the file/directory structure part of step 1 above. I'll try to group logical points of discussion into PRs toward that branch so we don't clutter up this issue here.

[rianquinn]

I cut the release for 2.0, and I will work on Slack today to get that setup and on our main page. From all of the testing I have done, it is the best option as it has good mobile support, good integration with GitHub, and most developers already have an account.

[tklengyel]

The project should adopt a calendar versioning model instead of arbitrary release numbers.

If the release numbers are meant to be "checkpoints" but not something you support separately longer term and/or backport patches and fixes to, then yes, +1.

[taodaqiao]

When will version 3.0 be released

[rianquinn]

@taodaqiao I just finished up Intel support this past week. There is a number of things that I still need to clean up, but I should be pushing Intel and AMD beta support for Linux in the next week. So the short answer is, we are really close. Its needed too because the current version of Bareflank doesn't compile on later versions of Linux.

As for Windows and UEFI support, I think that should take a couple more weeks to complete. From there, its just lots of little things before we can get an official release out the door, but you should be able to start playing with the new stuff real soon. Would certainly love your feedback.

[rianquinn]

FYI, the new code will be pushed sometime today or tomorrow, depending on how long it takes to get CI working.