New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] Platform dependencies #5523

Open
skyne98 opened this Issue Mar 9, 2018 · 24 comments

Comments

Projects
None yet
7 participants
@skyne98

skyne98 commented Mar 9, 2018

Hey there, fellows!
First of all, I will describe the scenario. Let's take Rust into consideration. It's a systems programming language, so it has a standard library, that doesn't depend on platform it's running on, which allows it to be used as an OS building language, running without infrastructure.

With projects such as CoreRT and CosmOS being developed in parallel, I started to wonder: maybe there is another way to solve this problem? Maybe we can use CoreRT to write platform independent code, so it will be able to run on any foundation, will it be RedoxOS, Fuchsia, or straight up bootstrapped and run by assembly.

So, is it possible to do now? What are the dependencies of CoreRT and can I write such code by just avoiding some parts of standard libraries, such as threads and networking? Also, as C# has a possibility to work straight up with memory, it may be very capable.

Thanks!

@skyne98

This comment has been minimized.

skyne98 commented Mar 9, 2018

Just realized that WASM might be a very similar target, hmm.

@jkotas

This comment has been minimized.

Member

jkotas commented Mar 9, 2018

The CoreRT depends on C/C++ runtime (libc and libstdc++, or equivalent) today at minimum. We will be happy to take contributions that work towards having an option to build without the dependency on C/C++ runtime.

A good first step may be to eliminate dependency on C++ runtime: #3564

@skyne98

This comment has been minimized.

skyne98 commented Mar 10, 2018

But what about WASM builds? Aren't they platform independent?

Also, can't I just avoid using parts of standard library with dependencies and just make syscalls, wherever possible?

@jkotas

This comment has been minimized.

Member

jkotas commented Mar 10, 2018

WASM builds depend on C/C++ runtime flavor that comes with Emsripten.

@morganbr

This comment has been minimized.

Contributor

morganbr commented Mar 10, 2018

@skyne98 , the answer for WASM depends a little on how you think of a "platform". It's true that it runs on lots of different processors and operating systems because browsers provide an abstraction. I tend to think of WASM more like being dependent on the WebAssembly/JavaScript "platform" that browsers provide.

CoreRT (and CoreFX) are factored with running on multiple platforms in mind, so you can get a pretty good idea of what's required to add a new platform from the structure of the source trees. CoreRT does make porting somewhat easier by only compiling what your program needs, so if you don't use (say) networking, you won't end up with dependencies on it.

@skyne98

This comment has been minimized.

skyne98 commented Mar 10, 2018

@morganbr, so you basically say, that, if I just make a program, that adds 2 + 2 and stores it into memory it will perfectly run everywhere? What about usage of raw pointers in C# and using FFI, are they platform dependent?

@skyne98

This comment has been minimized.

skyne98 commented Mar 10, 2018

@jkotas, yeah, I had to see it coming

@morganbr

This comment has been minimized.

Contributor

morganbr commented Mar 12, 2018

@skyne98, that's putting it a bit more strongly than I intended. There are a base set of capabilities that CoreRT assumes every application will need like memory allocation (provided by the OS and possibly passed through something like libc++) and threading (though I know @MichalStrehovsky is interested in making some of those, such as reflection not necessarily part of every application). We occasionally have to modify some of those assumptions. For instance, WASM doesn't support accessing the execution stack and doesn't have a concept of registers, so we had to develop our own stack for some scenarios.

I'm not sure I totally understand your second question. Pointers are hardware dependent, but not generally OS dependent. Interop with a platform can depend both on hardware (how big are pointers?) and OS (what's the callee's calling convention?).

@zarlo

This comment has been minimized.

zarlo commented Mar 15, 2018

its Cosmos not CosmOS

@MichalStrehovsky

This comment has been minimized.

Member

MichalStrehovsky commented Mar 19, 2018

So, is it possible to do now? What are the dependencies of CoreRT and can I write such code by just avoiding some parts of standard libraries, such as threads and networking?

There are two aspects of this:

  • What the runtime depends on to execute type safe code and provide basic services like garbage collection and exception handling
  • What the class libraries that .NET developers are used to need from the platform to work

At the core of CoreRT, there's a runtime that provides basic services for the code to run (think: garbage collection, exception handling, stack walking). This runtime is pretty small and mostly depends on C/C++ runtime (even the C++ runtime dependency is not a hard requirement as Jan pointed out - #3564). This code mostly lives in src/Native/Runtime, src/Native/gc, and src/Runtime.Base. It's structured so that the places that do require interacting with the underlying platform (allocating native memory, threading, etc.) go through a platform abstraction layer (PAL). We have a PAL for Windows, Linux, and macOS, but others can be added.

Then there's class libraries. CoreRT consumes class libraries (such as System.Console) built out of the CoreFX repo. CoreRT also provides a lot of runtime glue code in the System.Private assemblies that ship with it (things like globalization, resource management, reflection, and other classically "runtime" services) - code from CoreFX directly or indirectly calls into this code.

When thinking of CoreRT and C# for systems programming, one would be mostly interested in the services the "core" runtime provides (so that you can have type safety, but still stay "low level"); not much in things like HTTP, WCF, System.Environment, etc (because those really need a lot of platform support to begin with). The class libraries a systems programming framework provides would probably look different from what CoreFX provides.

CoreRT is structured in a way that other class libraries (ones that might look quite different from what regular .NET has) can sit on top of the small runtime. You can see an example of such different class library here (there's a README.md with instructions on how to compile against this library). A "hello world"-style app with this class library compiles down to about 480 kB EXE on Windows (most of it is the C runtime, and garbage collector), as opposed to 4 MB EXE with the rich application framework that has a lot of dependencies on things "systems programmers" don't particularly need/care about.

@fanoI

This comment has been minimized.

fanoI commented Mar 20, 2018

A clarification regarding Cosmos it does not uses CoreFx / CoreRt directly, it loads them as libraries and during compilations any native methods encountered is replaced with managed code written by us (we call this "Plug") or the compilation fails... in Cosmos C/C++ cannot exist.

The thing will be interesting for us will be to reduce in CoreFx / CoreRt the native methods so we have to plug less. It does exist a list of classes that could be changed to not have native methods anymore, String.cs and Number.cs I think could be totally managed code.

@jkotas jkotas added the Question label Mar 21, 2018

@fanoI

This comment has been minimized.

fanoI commented Mar 22, 2018

@jkotas does exists an "umbrella" issue on the native methods that you think can be replaced with managed code?

@jkotas

This comment has been minimized.

Member

jkotas commented Mar 22, 2018

CoreRT has class library methods like String.cs and Number.cs replaced with 100% managed code. We are incrementally integrating this goodness to CoreCLR as well. dotnet/coreclr#9474 is the uber issue for that.

The C/C++ parts in CoreRT are centered around GC. We can use this issue to discuss that.

@fanoI

This comment has been minimized.

fanoI commented Mar 22, 2018

decimal is all managed code in CoreRT too, right?

@jkotas

This comment has been minimized.

Member

jkotas commented Mar 22, 2018

Right

@fanoI

This comment has been minimized.

fanoI commented Mar 22, 2018

It simply means that one would have to copy CoreRt Decimal.cs and Decimal.DecCalc.cs to the CoreFx / CoreRt shared partition? After having ensured that there are not differences in CoreFx (not easier to do a diff I fear as probably they are totally different).

@jkotas

This comment has been minimized.

Member

jkotas commented Mar 22, 2018

Yes, that is our plan for decimal. There are some performance and ABI issues to sort out before it can happen. @pentp has been working through them: https://github.com/dotnet/corert/search?q=Decimal&type=Issues

@fanoI

This comment has been minimized.

fanoI commented Mar 22, 2018

Regarding the GC you plan to rewrite it all in managed code?
Seeing how much performance improvement you have got respect to C++ in managed Decimal it seems really possible.

The GC is too much tiered to the compiler to be usable by other projects right?

@jkotas

This comment has been minimized.

Member

jkotas commented Mar 23, 2018

Regarding the GC you plan to rewrite it all in managed code?

Not anytime soon. There would be no return on investment for us on that. The same goes for RyuJIT. We are not planning to rewrite the JIT in C# anytime soon either.

If you want to have a system with no C/C++, you can write a simple (non-generational, non-parallel, non-background, etc.) GC in C# to get it off the ground.

The GC is too much tiered to the compiler to be usable by other projects right?

GC is pretty general purpose. For example, Midori project was using the .NET GC as well at the time. And we have made the interface between the GC and the rest of the system even cleaner in the LocalGC project https://github.com/dotnet/coreclr/projects/3

@fanoI

This comment has been minimized.

fanoI commented Mar 23, 2018

So Midori had the GC as native code?

@jkotas

This comment has been minimized.

Member

jkotas commented Mar 23, 2018

They had several GCs for different purposes. The CLR GC was one of the options. There are clues about that in the code, e.g. here: https://github.com/dotnet/coreclr/blob/master/src/gc/gcpriv.h#L40

They have used a similar strategy for other algorithm heavy well-isolated components. For example, I do not think they have rewritten the JavaScript engine in C# for their browser. They just used https://github.com/Microsoft/ChakraCore.

Rewriting everything in C# is a noble goal, but the law of physics kick in: It becomes much cheaper to build the bridge for C/C++ code so that you can pick up the components with minimal changes where rewriting in C# gives you little extra benefit; and invest your energy only in places where the rewrite matters and makes a difference.

@fanoI

This comment has been minimized.

fanoI commented Mar 23, 2018

Well in Cosmos we have not an OS, neither a lot of C++ code that we can reuse so to use C# for all makes sense in our case.

@sergiy-k

This comment has been minimized.

Contributor

sergiy-k commented Mar 23, 2018

Right. Midori used the CLR GC for application processes.
Midori kernel, on the other hand, had a concurrent copying collector written in C#.

@fanoI

This comment has been minimized.

fanoI commented Mar 24, 2018

I think it is a pity that Midori remained only a research project... I would loved to have Windows 8 based on that technology! In the end Windows NT kernel is year '90s technology...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment