Skip to content

/rfcs forked from rust-lang/rfcs

Permalink
Branch: target-extensi…
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
514 lines (365 sloc) 21.7 KB
Raw Blame History
  • Feature Name: target extension
  • Start Date: 2017-06-27
  • RFC PR: (leave this empty)
  • Rust Issue: (leave this empty)

Summary

Extend Rust target specification to follow more closely LLVM triple specification.

The underlined purpose is to allow Rust compiler to target more easily BSD OS version.

Motivation

LLVM triple specification is more precise than the current Rust target specification we have.

In particular, the following elements are missing from the Rust target definition:

Rust language is aimed to be used on different operating systems following themselves their own rules. In particular, each operating systems have proper way to deal with breaking changes: if Linux tends to forbid breaking changes by policy, all others systems doesn't have such rule. As Rust language tends to be a stable language, having a stable way to describe breaking changes on the OS would be very valuable and could become necessary as time passes.

LLVM deals with such changes on OS by having a different triple per OS version, like for the following triples:

  • x86_64-apple-darwin16.0.0
  • x86_64-unknown-freebsd12.0
  • x86_64-unknown-freebsd11.0
  • i386-unknown-openbsd5.8
  • x86_64-unknown-netbsd7.99

As examples, consider the following changes in several operating systems (some are ABI changes, others API changes) and how a crate like libc would have to deal with them. Please note that some are quite old but could be considered as representative of something that already occurred in the past.

In the current situation, libc crate has no way to deal in a stable way with these changes. It could only support two incompatible OS version together by only defining the common subset. Depending on the breaking part, it could result in removed feature in rustc (removing si_addr for OpenBSD would break stack overflow detection), or even breaking rustc itself (removing ino_t for FreeBSD).

Additionally, in order to switch libc from one OS version to another, it would be required to do a breaking change at libc level (incrementing major version of libc itself) which is undesirable for this purpose.

The purpose of extending Rust Target type to follow LLVM Triple definition is to be able to deal with such changes at Rust language level. As the target will be able to distinguish between particular OS or environment versions, it would be possible to export the information in the same way we export target_os, target_arch, target_endian or target_pointer_width.

This way, a crate like libc could export raw bindings of platform specifically for the targeted version.

It should be noted that if this motivation section presents globally the underlined problem of breaking changes in OS, which could be present in a large broad of OS, the RFC will focus on a fewer set of OS. The RFC will target only BSD systems where breaking changes are part of the process OS developpment.

Detailed design

Language level: what the user will see ?

At language level, new attributes for conditional compilation would be added:

  • target_os_version
  • target_env_version

There could be empty ("").

extern {
    // encrypt() function doesn't exist in freebsd12

    #[cfg(all(target_os="freebsd", not(target_os_version="12")))]
    pub fn encrypt(block *mut ::c_char, flag ::c_int) -> ::c_int;
}

Another complete (and simple) example: in OpenBSD 6.2, the structure siginfo_t changed:

pub struct siginfo_t {
    pub si_signo: ::c_int,
    pub si_code: ::c_int,
    pub si_errno: ::c_int,

    // A type correction occured in 6.2.
    // Before it was a `char *` and now it is a `void *`.
    #[cfg(not(any(target_os_version = "6.2", target_os_version = "6.3"))]
    pub si_addr: *mut ::c_char,
    #[cfg(any(target_os_version = "6.2", target_os_version = "6.3"))]
    pub si_addr: *mut ::c_void,

    #[cfg(target_pointer_width = "32")]
    __pad: [u8; 112],
    #[cfg(target_pointer_width = "64")]
    __pad: [u8; 108],
}

It would be possible to target x86_amd64-unknown-openbsd6.1 and x86_amd64-unknown-openbsd6.2 whereas with current libc code only one version is possible, and switching from one to the other version would be a breaking change in libc (and we would lose OpenBSD 6.1 supported version).

Backend level

Target structure

At the backend level, the Target structure gains two new members:

  • target_os_version: String
  • target_env_version: String

to represent the (possibly empty) versions of the OS and environment.

See librustc_back/target/.

Specifics compilations options

It could be noted that some platforms could require additionnal compilation options, like macOS and -mmacosx-version-min for specifying the minimal version (it affects e.g. dynamic library loader).

No additional changes is required for this support: TargetOptions structure already contains array for such options: pre_link_args, late_link_args and post_link_args.

Having a per version target permits to have different options per target.

Implication on targets number

It should be noted it will implied a new target per OS version (for each architecture), as soon as a breaking change occurs (new target required), or on each major release (as it could be more simple for the end user to know which target to use).

As example, FreeBSD has currently 3 targets (one per supported architecture: x86_64, i686 and aarch64). If we want to be able to express targets for 3 releases (two currently supported and one upcoming), the number of targets will grow to 9 targets.

Version tracking per OS

The exact way to tracking the OS version (creating a new target) should be done per OS, because OS has different expectations regarding when a breaking change could occur accross versions.

As example, FreeBSD keep ABI/API accross minor versions, and a breaking change should only occur at major version (but not necessary).

So, the targets should be (for x86_64 architecture):

  • x86_64-unknown-freebsd10 (currently supported)
  • x86_64-unknown-freebsd11 (currently supported)
  • x86_64-unknown-freebsd12 (in development)

At the opposite, OpenBSD only release major versions (even if expressed with two digits version), and a breaking change could occur at each version:

  • x86_64-unknown-openbsd6.0 (currently supported)
  • x86_64-unknown-openbsd6.1 (currently supported)
  • x86_64-unknown-openbsd6.2 (in development)

Default OS version for a target

It could be noted that the current unversioned target (like x86_64-unknown-openbsd) could be still used as an alias of some versioned target.

If so, the semantic have to be defined (tracking the oldest or most recent supported version).

It could be convenient thing for compiler users, but any serious work should rely on versioned OS target (as compiling for one target version could mean unusable binary on other OS version).

Keeping the unversioned target would avoid a breaking change in command-line. But the change could be useful too as it permits to downstream to be aware that targeting particular OS doesn't mean the binary will work on other version.

Session level

At the session level, rustc should populate and export the new attributes (values taken from targeted backend) in the default build configuration.

See librustc/session/config.rs.

Migration path

FreeBSD will be taken as example for the migration path. But it would apply for others BSD OS.

Currently, rustc has 3 known targets regarding FreeBSD:

  • aarch64-unknown-freebsd
  • i686-unknown-freebsd
  • x86_64-unknown-freebsd

Assuming we want to support FreeBSD 10 and FreeBSD 11 (current supported production releases), and FreeBSD 12 (upcoming release), the following steps will need consideration.

Extending Target definition

  • add target_os_version and target_env_version in Target definition. The default value for all existing targets (including FreeBSD) will be "" (empty string).

  • make rustc to export new target_os_version and target_env_version attributes.

  • make cargo to export new TARGET_OS_VERSION and TARGET_ENV_VERSION environment variables.

At this point:

  • it should be no impact. the underline mecanism is implemented but nothing use it.

Add new targets specifically for FreeBSD 10, 11 and 12

  • add new targets, by duplicating the target code (using a function):

    • FreeBSD 10

      • aarch64-unknown-freebsd10
      • i686-unknown-freebsd10
      • x86_64-unknown-freebsd10

    • FreeBSD 11

      • aarch64-unknown-freebsd11
      • i686-unknown-freebsd11
      • x86_64-unknown-freebsd11

    • FreeBSD 12

      • aarch64-unknown-freebsd12
      • i686-unknown-freebsd12
      • x86_64-unknown-freebsd12

  • in these new targets, only llvm_target and target_os_version will be changed

At this point:

  • the freebsd target is still built and distributed. freebsd10, freebsd11 and freebsd12 are available for use using usual --target argument of rustc. there is still no changes in the Rust ecosystem.
  • the number of targets will grow: for FreeBSD: from 3 to 12 targets.
  • no changes in libc. the four targets are as functional as before and all uses the same libc code (at this point, there is no specific code using target_os_version).
  • the generated code for versioned freebsdXX could be different as the LLVM backend is now aware of the targeted version.
  • downstream projects could start using versioned targets (the code is still the same), but it would not be recommanded for production, but only for testing.
  • tests should be done to ensure the compatibility of the ecosystem with versioned targets as it could break some assumption: for example, library path will be i686-unknown-freebsd11 instead of i686-unknown-freebsd in freebsd11 target.

Specific changes for FreeBSD 12 could start to occurs

  • changes libc to produce different code if target_os_version="12", specifically regarding struct changes that occured in FreeBSD 12.

At this point:

  • when explicitly targeting freebsd12, the produced code will have different structure and functions. The produced binary targets FreeBSD 12, and will not work on FreeBSD 11 (due to the use of breaking changes that occured in FreeBSD 12).
  • the rest of the Rust ecosystem (still using freebsd as default build target) is unaffected by these changes.

Smoothly remove unversioned Target for FreeBSD

  • replace the Target freebsd (unversioned) by an alias on freebsd11. It ensures that all FreeBSD targets will have target_os_version sets to a specific version.
  • alternatively, more complex code could be used for the alias:
    • on a FreeBSD host: freebsd will point to the host version.
    • on any other host: freebsd will point to predefined default version.

At this point:

  • some code could break. it will only concern untested code from the previous steps.
  • it is a transition period: downstream projects should start using versioned Target for production code. For example, Rust infrastructure should stop building freebsd in favor of freebsd11, and rustup should distribute this specific version too (freebsd11 has been taken only as example).
  • as transition period, it should be long enough.

Completely remove unversioned Target for FreeBSD

  • remove the alias freebsd
  • only keeping freebsd10, freebsd11 and freebsd12 targets

At this point:

  • some code could break. Any downstream projects still using unversioned target will break (as the target has been removed).
  • this step isn't strictly necessary. an unversioned target freebsd could have sens too. But care will be need when the alias will move (from freebsd11 to freebsd12 for example). Removing it let downstream projects to deal themself with such transition.

How We Teach This

If modifying the Target struct is a low-level change by itself, the current RFC proposes it in order to change an implicit paradigm in targets (the targeting OS will be stable accross version, which is false).

With the RFC, Rust become able to express this lack of stability on the OS, in a stable way. In a sens, it extents the ability of Rust to targets several OS by refining to targets several OS version.

From downstream perspective, it permits to use Rust to target several OS versions whereas the versions are incompatibles.

From Rust developer perspective, it adds a new attributes for conditional compilation.

Regarding documentation, additions have to be done on Rust Reference in order to mention new attributes in conditional compilation attribute section.

Visible changes should also occur on main rust-lang.org site on pages about rustc distribution: rustup will be able to distribute binaries for more platforms (per OS version), or about platform support. If only one version is distributed (what it is expected), the binary will be runnable only on one particular OS version (but will be able to produce binary for another version).

Drawbacks

Do not providing a simple way to target a range of versions (for example, "from version 12 to current version") could imply a long enumeration of version, that could only grow with time. But two things has to be noted: first it could help to limit the number of supported and tested versions ; secondly, the fact this particular RFC doesn't require such construct do not forbid to propose a new RFC later, once the necessity of such mecanism would be more evident (or not).

At backend level, the number of targets will grow a lot. It means that not all targets will be testable (too much required ressources and it would require a particular OS version for testing too).

It will require to regulary deprecate old targets (for unsupported OS version) in order to not keep too much old stuff. The end-user has still the possibility to use flexible target using external JSON file for these targets, if the corresponding code for this particular version is still in libc crate.

Projects using Rust with binary distribution will have to update in order to cover a more important number of platforms. In particular rustc itself (with and without rustup). It will mean more ressources to build more targets. But the resulting binaries will work on all targets.

Alternatives

Simply appending the version in the target name

The more simple approch is to use target_os with the OS version inside (freebsd12). But it would require to duplicate all libc code (for only small differences) at each release. Having a separated attribute is more simple.

But without some way to express breaking changes existence at OS level, Rust is unable to targeting simultaneous several OS version. Regarding issue #42681 for FreeBSD 12, it means Rust should either deprecating older FreeBSD versions support (whereas FreeBSD itself still support them) or only partially supporting FreeBSD 12.

Runtime detection

Runtime detection is already in use for Android. It has the advantage to permit rustc to target several OS with the same binary.

It is based on OS version detection (with symbol existence for example) and on providing fallback or alternative for function calls.

But it couldn't cover all aspects of ABI breaking, specially changes in structures (member size or offset change). It would require to replace structure's member access by function calls doing runtime detection. The possible overhead could be removed by using lazy detection and caching.

Dynamic bindings generation

A possible alternative is to replace libc with FFI bindings generation at compile-time (using rust-bindgen for example). But it isn't suitable for cross-building.

Adding cfg attribute using build.rs

It is possible to do some compile time detection (or using cargo feature) to select a particular OS version in libc. It has been proposed for resolving the FreeBSD 12 ABI issue.

With such code, the libc code is right regarding the selected version.

The drawback is such detection is fragile, and crosscompilation more complex (it requires cargo feature usage).

Additionally, a larger problem is mixing code with different OS version would be possible (no error at compile time): for example using libstd from rustup targeting one version, and using with crate locally compiled for another version. It would produce bad code and crash could occurs at runtime.

Unresolved questions

As unresolved-question, the question about the unversioned target on command-line is open. Does it makes sens to have it or not ?

Related previous discussions
You can’t perform that action at this time.