Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Target tier policy #2803

Open
wants to merge 1 commit into
base: master
from

Conversation

@joshtriplett
Copy link
Member

joshtriplett commented Nov 4, 2019

Rust developers regularly implement new targets in the Rust compiler,
and reviewers of pull requests for such new targets would like a clear,
consistent policy to cite for accepting or rejecting such targets.
Currently, individual reviewers do not know what overall policy to
apply, and whether to apply solely their own judgment or defer to a Rust
governance team.

Rust developers regularly ask how they can raise an existing target to
tier 2 (and in particular how they can make it available via rustup),
and occasionally ask what it would take to add a new tier 1 target. The
Rust project has no clear official policy for target tiers. People not
only don't know, they don't know who to ask or where to start.

This proposal documents an official policy for adding new (tier 3)
targets, and for raising targets to tier 2 (with rustup builds) or to
tier 1.

Based on discussions with the compiler team and representatives from
other teams.

Rendered

Rust developers regularly implement new targets in the Rust compiler,
and reviewers of pull requests for such new targets would like a clear,
consistent policy to cite for accepting or rejecting such targets.
Currently, individual reviewers do not know what overall policy to
apply, and whether to apply solely their own judgment or defer to a Rust
governance team.

Rust developers regularly ask how they can raise an existing target to
tier 2 (and in particular how they can make it available via rustup),
and occasionally ask what it would take to add a new tier 1 target. The
Rust project has no clear official policy for target tiers. People not
only don't know, they don't know who to ask or where to start.

This proposal documents an official policy for adding new (tier 3)
targets, and for raising targets to tier 2 (with rustup builds) or to
tier 1.

Based on discussions with the compiler team and representatives from
other teams.
@joshtriplett joshtriplett requested a review from nikomatsakis Nov 4, 2019
@joshtriplett

This comment has been minimized.

Copy link
Member Author

joshtriplett commented Nov 4, 2019

# Unresolved questions
[unresolved-questions]: #unresolved-questions

- Where, precisely, should we post the official policy?

This comment has been minimized.

Copy link
@Centril

Centril Nov 4, 2019

Member

The forge, probably?

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 4, 2019

Yeah I think PR might be better suited as a PR directly to forge with a RFC decision like rust-forge#266.

This comment has been minimized.

Copy link
@Centril

Centril Nov 4, 2019

Member

I prefer to have the RFC here rather than on the forge for the sake of visibility and community participation.

- Where, precisely, should we post the official policy?
- How should we track the maintainers of a target, so that we can page them if
we need an issue addressed involving that target? Can we do so using github
teams, so that we can mention them in an issue or PR to get their attention?

This comment has been minimized.

Copy link
@Centril

Centril Nov 4, 2019

Member

GH teams seems good yeah; any drawbacks?

This comment has been minimized.

Copy link
@pietroalbini

pietroalbini Nov 4, 2019

Member

The drawback of GitHub membership is the "Member" badge, which might give external contributors' comments an "official" appearance. Instead we could add them (like icebreakers-llvm) as a "marker team" in the team repo, and ping them with:

@rustbot ping team-name

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 4, 2019

I would hesitate on adding a whole bunch of teams without being sure that it's worth the overhead. At least initially I think putting a table at the end of this document containing the platforms and the people who maintain them, would be enough for most cases, and we can add teams for platforms that have a large enough group of maintainers.

post a PR to remove it; any such PR will be CCed to people who have
previously worked on the platform, to check potential interest.

## Tier 2 target policy

This comment has been minimized.

Copy link
@Centril

Centril Nov 4, 2019

Member

One concerning aspect of e.g. tier 2 targets is that they sometimes may use unimplemented!() in the standard library and then place a burden on PRs such as rust-lang/rust#56410. It would be good to discuss the use of unimplemented!().

Note: some tier 2 platforms additionally have binaries built to run on them as
a host (such as `rustc` and `cargo`). Such a platform must meet all the
requirements above, and must additionally get the compiler and infrastructure
team to approve the building of host tools.

This comment has been minimized.

Copy link
@pietroalbini

pietroalbini Nov 4, 2019

Member

Do we want to mention what happens with other tools as well? For example, we build Clippy, rustfmt and such on ARM, but they've been broken for a while.

@pietroalbini

This comment has been minimized.

Copy link
Member

pietroalbini commented Nov 4, 2019

Thanks for writing this Josh! 🎉

Left a few comments and added T-infra and T-release.

do not post comments (automated or manual) on a PR that suggests a block on
the PR based on the target. (A PR author may choose to help with a tier 3
target but is not required to.)
- Patches adding or updating tier 3 targets must not break any existing target.

This comment has been minimized.

Copy link
@Ixrec

Ixrec Nov 4, 2019

Contributor

Is "any existing target" supposed to mean just tier 1+2 targets, or does it include other tier 3 targets too? (are there any groups of closely related tier 3 targets where this could plausibly become a practical issue?)

This comment has been minimized.

Copy link
@farnz

farnz Nov 6, 2019

I would hope it includes other tier 3 targets. For example, if your target implements RV32IMC, you shouldn't be allowed to break the behaviour of atomics on a target that implements RV32GC by using target-specific features that let you emulate atomics on your platform for all RISC-V users.

This comment has been minimized.

Copy link
@Aaron1011

Aaron1011 Nov 9, 2019

Does this mean that platform-independent changes to rustc (e.g. changing an internal libstd method) are allowed to break tier 3 targets, but platform-specific changes for a specific tier 3 target may not break other tier 3 targets?

policy](https://release.debian.org/bullseye/arch_policy.html) and [archive
criteria](https://ftp-master.debian.org/archive-criteria.html).

# Unresolved questions

This comment has been minimized.

Copy link
@rylev

rylev Nov 4, 2019

This document seems to aim to give high level guidance vs specifying specific procedure when it comes to who exactly is responsible for the final decision for promoting and demoting a target to different tiers and how exactly that decision should be made. Should there be specific guidance as to who gets ultimate say if there is open disagreement with a decision? Is the specific policy of how such a decision is made left up to the discretion of the responsible team and is this document meant to be the only guidance to those teams for how they should make those decisions?

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 4, 2019

@rylev That is an interesting question, since to my knowledge there haven't been any intentional promotions or demotions of a platform since 1.0. To me platform promotions are a core team decision as it involves coordinating with multiple teams to provide the stability to the target.

This comment has been minimized.

Copy link
@pietroalbini

pietroalbini Nov 4, 2019

Member

The way I see it, each of the three teams involved in this (compiler, infra, release) has a clearly defined stake in the decision, and if any of them is not comfortable promoting the target they should be able to veto the promotion:

  • If the compiler team can't ensure "tier 1 quality" for the target and nobody is available to do the development work there is little point in promoting the target.
  • If the infrastructure team can't provide CI for the platform the only guarantee we give for tier 1 platforms (they're tested on CI) goes out of the window.
  • If the release team doesn't think the platform is viable in the long term or that shipping releases for it is going to be too cumbersome why are we doing the promotion?

I don't see how the core team needs to get involved in the decision.

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 4, 2019

@pietroalbini I would call a decision made between the compiler, infra, and release leads a core team decision, and I don't see why it couldn't involve other team's if they had concerns, and we already have the core team meetings which are recorded and have the public portion published.

This comment has been minimized.

Copy link
@Centril

Centril Nov 4, 2019

Member

The leads are not the teams tho. That's a difference between like 3 and 30 people. We currently don't give special significance to team leaders this way and I don't see why we would start with a decision of such importance as tier 1 status.

This comment has been minimized.

Copy link
@Centril

Centril Nov 5, 2019

Member

No one is saying that three people are making the full decision, however there won't be a RFC that requires 30 people to approve.

I don't see anything wrong with 30 people (probably less as some people are on several teams) approving a decision as big as adding a new tier 1 target. It's not like we do that every day, in fact, it hasn't happened for years.

The point of a team lead is to represent their team and its members, it doesn't require everyone to be personally involved.

I think we can let the team decide, if they want, to delegate to someone on the team if it is a regular task that needs delegation for time-resource reasons. As for tier 3 targets, they are added regularly by anyone familiar with the compiler internals of adding target on a regular basis; it doesn't even require approval from any team.

RFC 1068 states that the core team "Sets the priorities and release schedule", and "Focuses on broad, cross-cutting concerns". Now that document is quite old, however I don't think that section has changed much, and to me platform target support falls under both those responsibilities.

I don't think it does; The domain expertise required for making decisions about targets mainly resides in the compiler, infra, and release teams. It may not reside in the team leads of those teams but it should in the collective knowledge of each whole team.

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 5, 2019

I don't think it does; The domain expertise required for making decisions about targets mainly resides in the compiler, infra, and release teams. It may not reside in the team leads of those teams but it should in the collective knowledge of each whole team.

The platform support promotion isn't solely a technical decision. Tiers are a form of guarantee not a statement of intent. To discuss whether a platform should be promoted we'd have to already have the compiler support and release infrastructure required in place and functioning. The technical requirements are a prerequisite to even having the discussion of promotion.

This comment has been minimized.

Copy link
@Centril

Centril Nov 5, 2019

Member

Judging whether we have e.g. the CI budget or whether e.g. whether there aren't any unimplemented!()s are indeed some sort of technical requirements. However, the purpose of team review here is to make sure that problems don't slip through the cracks. Having the watchful eye of many people rather than just 3 team leads, who are typically some of the most busy people on the team, makes it more likely that things don't slip through.

Also, I don't really understand why non-technical decisions would better be made by leads rather than by the whole team (again, within the discretion of the team to delegate, sometimes to someone who isn't the lead).

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 6, 2019

@Centril Sorry I don't think I'm being clear. What I'm trying to say is that in order to get to the point where we could have a discussion about promoting a platform, those technical problems would have to already be solved. For example; we wouldn't make the decision to promote x86_64-unknown-redox from tier 2 to tier 1 and then try to make that platform tier 1 quality. That would be putting the cart before the horse. To me a platform should already be de-facto tier 1 before we would commit to making that happen, and that is something that part is something that is up to the teams to agree to work together to provide that.

This comment has been minimized.

Copy link
@Centril

Centril Nov 8, 2019

Member

@XAMPPRocky Oh sure; that seems right, but the function of team review would be to do a final verification that it is indeed the case that a tier 2 platform satisfies the wrt. the technical requirements of libstd impl maturity, CI budget, maintainers, etc. are met for tier 1 and can be promoted.

- In addition, the release team must approve the long-term viability of the
target, and the additional work of supporting the target.
- Tier 1 targets must have substantial, widespread interest within the
developer community, and must serve the ongoing needs of multiple production

This comment has been minimized.

Copy link
@rylev

rylev Nov 4, 2019

What are the risks associated with a high profile (i.e., well known and well funded) user of Rust needing support for a target that (at least in the short/medium term) is not deemed useful by the wider community? Does this risk a split of the ecosystem or an exit of that user from the Rust community? Presumably targets can be de-facto tier 1 without having official status, and it would be up to the high profile user to work with the community to ensure the target meets their needs without inconveniencing wider Rust development.

This comment has been minimized.

Copy link
@Centril

Centril Nov 5, 2019

Member

Do you have anything specific in mind here?

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 6, 2019

What are the risks associated with a high profile (i.e., well known and well funded) user of Rust needing support for a target that (at least in the short/medium term) is not deemed useful by the wider community

Would you expand on what you would mean by support? It's not likely that the Rust community would implement and maintain platform that they don't deem useful, a user is however free maintain a platform themselves and submit patches to Rust (as along as they also follow the rules for that tier).

Does this risk a split of the ecosystem or an exit of that user from the Rust community?

I don't think platform support alone is controversial enough for a split or exit. It would require other external factors as well. (e.g. providing a level of support within a tight timeline, or politics.) Is your concern that to develop the target to a tier 1 quality might cause too much churn for Rust developers to review? If this was the case it wouldn't be unreasonable for the user to have a fork for them to work on, and upstream patches at a longer pace (kind of like how rust treats LLVM) until the platform is more stable in rustc, though I suspect this wouldn't be needed in reality.

This comment has been minimized.

Copy link
@rylev

rylev Nov 7, 2019

Let's take Fuchsia as an example. This is unlikely to be important to anyone outside of Google in the near time. Are there risks associated with Google not having wider community support for that target?

This comment has been minimized.

Copy link
@Centril

Centril Nov 8, 2019

Member

@rylev That's a bit of an open-ended question. =P

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 8, 2019

@rylev Well I think firstly it's important to know whether you the user want the platform to be a host or target platform, e.g. Do you want developers to be able to build and run their code on the platform itself or is it satisfactory if they only compile to that platform and use another for development? A target platform doesn't need support everything a host platform does, so it would be lower impact. At the cost of not being able to develop on the platform itself, which may be fine depending on your use case.

With this there's the risk of having to put in the initial work investment and continued maintenance to ensure that the platform continues to compile, build, and work for the end software being built with new compilers. This cost increases with the amount of tooling you want to offer on the host platform and how much you want the platform to be tier 1 quality. There's also the work to implement the standard library for your platform if it's radically different than others.

@pietroalbini

This comment has been minimized.

Copy link
Member

pietroalbini commented Nov 5, 2019

Something we might want to clarify is the distinction between being tier 1/2/3 as a cross-compilation target and being tier 1/2/3 as a host platform, possibly including even tools in the last definition. I think formalizing that (and adding the categorization to the forge) will reduce the confusion about what's actually supported.

Copy link
Contributor

jethrogb left a comment

I support bringing clarity to the tier system. Has there been any effort of notifying existing tier 2 & 3 target owners of this ongoing RFC?

- A tier 2 target may be demoted or removed if it no longer meets these
requirements. Any proposal for demotion or removal will be CCed to people who
have previously worked on the platform, and will be communicated widely to
the Rust community before being dropped from a stable release.

This comment has been minimized.

Copy link
@jethrogb

jethrogb Nov 6, 2019

Contributor

@alexcrichton has previously made this comment about a tier 2 target:

As with all other platforms we reserve the right to turn this off at any point in time to land otherwise critical changes to the standard library. In practice we've only done this once or twice (adding u128 and disabling some targets is what comes to mind), but I just want to make sure that it's explicit that we're not going to provide a long-term guarantee that this target is available one every single nightly in the future.

Is this stance still intended to be part of the tier 2 policy? If so, is that intended to be captured by this point? I feel like it could be made more clear.

Note: some tier 2 platforms additionally have binaries built to run on them as
a host (such as `rustc` and `cargo`). Such a platform must meet all the
requirements above, and must additionally get the compiler and infrastructure
team to approve the building of host tools.

This comment has been minimized.

Copy link
@jethrogb

jethrogb Nov 6, 2019

Contributor

Related: As discussed when resolving rust-lang/rust#65335, some people including myself think it's a good idea to build "hello world" with the dist artifacts for all tier 2 targets. Should we capture this in this RFC?

@ehuss

This comment has been minimized.

Copy link
Contributor

ehuss commented Nov 6, 2019

This may be off-topic: but I think it would be nice to be clearer about the definition of a tier and a platform.

For example, tier 1 says "pass tests", but can I just add #[cfg(not(my_platform))] in front of a test to make it pass? Can I do that to a few tests? A lot of tests? There is also some amount of graduated support. For example, some features may be disabled or degraded on older versions of a platform. Usually that's because the platform doesn't fundamentally support it, but what if it is because I'm not in the mood to load a Windows 7 VM to fix it, so I just disable something because it is "too hard" to support?

Also, which version of a platform is supported. The forge page currently lists macos 10.7+ and Windows 7+ as "tier 1" status, but I would be surprised if macos 10.7 still works. And very few tier 2 platforms mention which version is built. Should targets list their version? Should that version be kept up-to-date with reality? For tier 1, that would be macos 10.13, Windows Server 2016, linux is still kernel 2.6.18. And only those versions are tested, not newer ones. Should it be documented which version is tested, and which versions are intended to be supported?

Is there any expectation about when newer versions of a platform will be supported? If 1.39 was released without supporting macos 10.15, is that OK? What about 1.40? How soon should newer versions be supported?

Platforms also have a suite of tooling that may be required, such as linkers. Are there any guarantees about which versions of those are required? Can I decide to stop supporting an old version of xcode? Can I say "you must use clang, gcc is not supported"?

Feel free to ignore if this is too out of scope. Most of these are judgement calls, but some guidance might help making those decisions and communicating what is actually supported and tested.

architectures. https://forge.rust-lang.org/release/platform-support.html
documents some of that policy.

Future expansions of such policy may find requirements from other communities

This comment has been minimized.

Copy link
@ehuss

ehuss Nov 6, 2019

Contributor

I assume Rust's tiered approach is derived from firefox's tiers, is that true? I tried to find other languages/tools using a similar tiered approach, but only found node had something similar. I know sometimes in web development projects will have "grades" for browser support.

This comment has been minimized.


# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

This comment has been minimized.

Copy link
@farnz

farnz Nov 6, 2019

I think this is a good point to make it clear that you should meet the requirements for a tier before opening the asoociated promotion PR.

Something like:

You should normally ensure that your target meet the tier requirements before posting a PR.
In general, we would prefer to delay adding a target to a tier until it is clear that the target
has been meeting the standards set for the tier for a significant time period.

This comment has been minimized.

Copy link
@carols10cents

carols10cents Nov 8, 2019

Member

for a significant time period.

If we add this language, we should define in this RFC what the time period is. A release cycle? 2 release cycles?

This comment has been minimized.

Copy link
@Centril

Centril Nov 8, 2019

Member

That time should ideally also be a function of the tier, with more time for tier 1.

This comment has been minimized.

Copy link
@Lokathor

Lokathor Nov 8, 2019

I would expect something more, like 6 months, which means yes we need to be more specific it two people are picking very different assumptions.

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 8, 2019

What would a PR promoting a tier look like? As far as I know there's no tight coupling between a tier and the code & infrastructure. It's not like we have a TOML file that enforces the tier structure, it's mainly a social commitment. You could make a PR updating the documentation on forge, but this doesn't seem like best place to have the promotion discussion.

It would also be reasonable for the procedure for promoting or demoting to be out of scope of this RFC. Since we've never done this before I think it might be best to create the procedure when we actually want to promote a platform, rather than discuss and try to reason around a hypothetical promotion.

This comment has been minimized.

Copy link
@farnz

farnz Nov 8, 2019

What would a PR promoting a tier look like? As far as I know there's no tight coupling between a tier and the code & infrastructure. It's not like we have a TOML file that enforces the tier structure, it's mainly a social commitment. You could make a PR updating the documentation on forge, but this doesn't seem like best place to have the promotion discussion.

It would also be reasonable for the procedure for promoting or demoting to be out of scope of this RFC. Since we've never done this before I think it might be best to create the procedure when we actually want to promote a platform, rather than discuss and try to reason around a hypothetical promotion.

The bit of my comment I care about is telling people that promotion to a higher tier (or addition to tier 3) trails meeting the requirements.

In other words, let's call out that you not only can meet the standards for any of the support tiers without being promoted into that tier, but that you should expect to meet the standards first, for long enough that people watching the process can look at your historic behaviour, and then go into the tier promotion process, whatever that might be.

This comment has been minimized.

Copy link
@Lokathor

Lokathor Nov 8, 2019

In the past I've been told that the bar for Tier 3 is as low as "file a PR to get your target specification into rustc's files, and make sure core can build".

This comment has been minimized.

Copy link
@jethrogb

jethrogb Nov 9, 2019

Contributor

What would a PR promoting a tier look like?

Isn't that obvious? It's a PR that changes the CI such that it now gates changes to master in accordance with the tier policy. Like rust-lang/rust#57130.

This comment has been minimized.

Copy link
@XAMPPRocky

XAMPPRocky Nov 9, 2019

@jethrogb Your answer doesn't actually describe what that PR would be though. What does a person need to use, where do they need to add it, how can they test it, where do they document it, who do they ask to review it?

Your PR example for instance is actually a good example of it's not so obvious.

As I understand that PR only makes fortanix a tier 2 target platform, if I wanted to make it so I could build rustc, cargo, or tools on fortanix, what would I need to do?

Elsewhere in this thread, team members have said that promotion should go through the infra, release, and compiler teams. However that wasn't the process that, that PR went through, so who should I be asking for a review for promotion?

I think the answers to these questions are only obvious if you have picked up on this information through osmosis, which is why it should be documented. Though as I said I don't think this needs to be included with this policy though, it could be added later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.