Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding Linux targets that don't depend on libc #2610

Open
japaric opened this issue Dec 9, 2018 · 18 comments

Comments

@japaric
Copy link
Member

commented Dec 9, 2018

e.g. x86_64-linux (or x86_64-unknown-linux-rust).

These would be the spiritual successor of steed: a standard library, free of C dependencies, for Linux systems. Steed implemented (or planned to) the Rust standard library using raw Linux system calls instead of libc's API. These libc-less targets would also implement std using raw system calls.

Even though steed has been inactive for over a year people continue to star it on GitHub and currently has over 500 stars so it seems there's still interest for something like it.

What we learned during development is that maintaining a standard library out of tree is a lot of work because things get quickly out sync so if there's still interest for something like steed I would recommend writing an RFC to add support for libc-less targets (e.g. x86_64-linux) to rust-lang/rust; this would be equivalent to importing and developing steed as part of rust-lang/rust.

An RFC is also a good way to (re-)evaluate the ROI of making a libc-less standard library. One of the goals of steed was hassle-free cross compilation but today that's solved by the Linux/MUSL targets + rust-lld (works on stable). The other goal was better optimizations (plain LTO doesn't optimize across FFI) but cross-language LTO is a now thing (-Z cross-lang-lto). There may be less costly ways to achieve the same results.

The rest of this post describes the status of steed as of 2017-10-20 (date of last commit).

What we had working:

  1. standard I/O (std::io::{stdin,stderr,stdout})
  2. filesystem operations (std::fs)
  3. collections (std::collections) (but see (a) below)
  4. std::sync::Mutex (courtesy of parking_lot)
  5. std::env
  6. UDP and TCP sockets (std::net) (but see (d) below)
  7. minimal support for threads and TLS
  8. #[test] support (but see (c) below)

You can check the examples directory to get an idea of what you could write with it.

What was missing:

a. a proper allocator. steed used a bump pointer allocator that never freed memory
b. all the math stuff (e.g. f32::sin). These days one can use libm.
c. unwinding, all the steed targets used -C panic=abort
d. hostname lookup
e. errno. It was unclear whether we should implement it or not. It seems to only be required by std::io::Error::last_os_error. Linux system calls have a Result-like API so errno is not required to propagate errors from syscalls to the std API.

and much more stuff; you can check the issue tracker for the full list.


cc @tbu- @briansmith

@mark-i-m

This comment has been minimized.

Copy link
Member

commented Dec 9, 2018

This sounds really cool! However, I think you may be underestimating how much work this is. There would also need to be collaboration with kernel developers about system calls and stuff.

@lorenz

This comment has been minimized.

Copy link

commented Jan 1, 2019

@mark-i-m I agree that it is quite a bit of work, but there is no need for collaborating with kernel devs. The syscall interface is considered stable and they treat any change that'll break userspace as a bug. Go has successfully implemented this strategy on all OSs they support. So I'd argue it is pretty doable and in my opinion desirable.

@tbu-

This comment has been minimized.

Copy link
Contributor

commented Jan 1, 2019

This sounds really cool! However, I think you may be underestimating how much work this is. There would also need to be collaboration with kernel developers about system calls and stuff.

Most of the stuff was already in place, as you can see in @japaric's comment. :)

@mark-i-m

This comment has been minimized.

Copy link
Member

commented Jan 2, 2019

@lorenz Yes, you are right, but what I mean is that if we want to influence the syscall interfaces to make them more Rust-friendly we would need to interact more with the kernel devs...

@lorenz

This comment has been minimized.

Copy link

commented Jan 4, 2019

@mark-i-m What do you have in mind that you want changed? The syscall interface is totally usable from Rust and is in various ways a better fit than libc itself.

@mark-i-m

This comment has been minimized.

Copy link
Member

commented Jan 4, 2019

Admittedly, I haven't yet gotten to thinking extensively about this, but mainly my thinking is that syscalls often don't even attempt to by safe, so adding good Rust wrappers around them is either painful or inefficient.

For example, it is hard to write a good safe wrapper for mmap because it can do so many things; encoding all of them into efficient zero-cost abstractions is hard. There are also a bunch of syscalls that depend on C-defined structs/types (and their layouts), e.g. clock_get*, sched_setaffinity. (EDIT: actually, I'm not sure if these structs/types are just parts of libc... so I might be wrong there...)

@lorenz

This comment has been minimized.

Copy link

commented Jan 4, 2019

But this is not about implementing all possible safe uses of (for example) mmap, but just about reimplementing the standard library on top of syscalls which limits the scope to these specific operations. Also structs with C layout are no problem in Rust (we have #[repr(C)]) and are a sensible choice for a kernel since they are interoperable with everything. The kernel people will never maintain a separate syscall interface just for a specific programming language.

@mark-i-m

This comment has been minimized.

Copy link
Member

commented Jan 4, 2019

The kernel people will never maintain a separate syscall interface just for a specific programming language.

No, but they do talk with e.g. libc maintainers to discuss interfaces, libc support, etc.

But this is not about implementing all possible safe uses of (for example) mmap, but just about reimplementing the standard library on top of syscalls which limits the scope to these specific operations.

That's true, but this is also an opportunity to expose the kernel ABI in a safer way, although that doesn't seem to be the intent of this issue...

@SimonSapin

This comment has been minimized.

Copy link
Contributor

commented Jan 5, 2019

Any newly-proposed syscall will at best be present in the next kernel version, but it’ll be a number of years before you can usefully ship a program that just assumes it is present.

@briansmith

This comment has been minimized.

Copy link

commented Jan 5, 2019

Any newly-proposed syscall will at best be present in the next kernel version, but it’ll be a number of years before you can usefully ship a program that just assumes it is present.

Yes, but Rust can take advantage of them right away by falling back to something else when it receives ENOSYS (or whatever), without having to rely on convincing glibc/musl maintainers to do so.

@SimonSapin

This comment has been minimized.

Copy link
Contributor

commented Jan 5, 2019

Sure but if a fallback is needed anyway there is no point in doing all this only "to make them more Rust-friendly", is there? (As opposed to, say, a performance optimization.)

@joshtriplett

This comment has been minimized.

Copy link
Member

commented Feb 15, 2019

@mark-i-m It sounds like you're interested in something different than this is proposing. @japaric is proposing a standard library that's equivalent to the current std but without linking to the platform libc, by calling existing syscalls directly. That's a useful goal that many people want, and it doesn't involve changes to syscalls in any way.

If someone wanted to add new syscalls to Linux, for whatever reason, that would be outside the scope of this proposal.

@joshtriplett

This comment has been minimized.

Copy link
Member

commented Mar 15, 2019

As a separate thought, for a target like this, we could theoretically have a libc-compatible C library implemented in Rust, and then link C programs to that library. We don't need that, but it would make for an interesting future addition to this target.

@Darkspirit

This comment has been minimized.

Copy link

commented Mar 15, 2019

libc-compatible C library implemented in Rust

https://gitlab.redox-os.org/redox-os/relibc#relibc- "but it also supports linux syscalls via the sc crate."

@newpavlov

This comment has been minimized.

Copy link

commented May 18, 2019

Looking forward to playing with this libc-free target! I wonder if it will be possible to later remove x86_64-unknown-linux reliance on libc after experimenting on a separate target?

@newpavlov

This comment has been minimized.

Copy link

commented May 30, 2019

a proper allocator. steed used a bump pointer allocator that never freed memory

IIUC elfmalloc can be used here.

@zmlzm

This comment has been minimized.

Copy link

commented Jun 26, 2019

There is a news : A libc in LLVM

@dvc94ch

This comment has been minimized.

Copy link

commented Aug 23, 2019

Does adding new targets require a rfc? Or were large changes required to the codebase?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.