Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce the ASM project group #2836

Open
wants to merge 3 commits into
base: master
from
Open

Conversation

@Amanieu
Copy link
Contributor

Amanieu commented Dec 10, 2019

Rendered

@Amanieu

This comment has been minimized.

Copy link
Contributor Author

Amanieu commented Dec 10, 2019

The reposity link is currently broken, could someone with the appropriate permissions create rust-lang/project-asm?

@Ekleog

This comment has been minimized.

Copy link

Ekleog commented Dec 10, 2019

@Amanieu Amanieu force-pushed the Amanieu:project-asm branch from 6bd89bc to 9b40dec Dec 11, 2019
@joshtriplett joshtriplett self-assigned this Dec 13, 2019
@joshtriplett joshtriplett added the T-lang label Dec 13, 2019
@joshtriplett

This comment has been minimized.

Copy link
Member

joshtriplett commented Dec 13, 2019

This looks great to me. We already discussed this in last week's language team meeting, and this proposal is just to charter the team to work on the problem rather than endorsing any particular solution, so let's go ahead and get consensus and get this rolling!

@rfcbot merge

@rfcbot

This comment has been minimized.

Copy link

rfcbot commented Dec 13, 2019

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

Concerns:

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@gnzlbg
gnzlbg approved these changes Dec 13, 2019
Copy link
Contributor

gnzlbg left a comment

Sounds great!

@fintelia

This comment has been minimized.

Copy link

fintelia commented Dec 15, 2019

Certain elements are definitively out of scope:

  • The chosen asm! syntax should not involve rustc "understanding" the asm code itself, since this would require a huge amount of work in the compiler to support the full assembly syntax of multiple architectures. Instead, interpretation of the assembly code should be left to the compiler backend (LLVM) or an external assembler (GAS).

Is this saying that the resulting design must have rustc treat the asm code as an opaque string, or just that rustc can't be expected to compile the code itself? There is vast middle ground between these two requirements. For instance, the project group might want to define a single comment syntax so higher level tools (like say syntax highlighters) can at least partially interpret the code

@Lokathor

This comment has been minimized.

Copy link

Lokathor commented Dec 15, 2019

That sounds neat, but also that sounds like something that should wait for the project group to be accepted and then have that discussion of the details there.

@fintelia

This comment has been minimized.

Copy link

fintelia commented Dec 15, 2019

I'm not super familiar with how Rust project groups operate, but my expectation would be that if the charter says something is out of scope, then the project group isn't "allowed" to discuss it. That's why I'm hoping to clarify the group scope before it is accepted/started. (This is of course the wrong place to debate my hypothetical comment syntax proposal, I just wanted to give an example of an issue that I wasn't sure if it was germane or not)

@Amanieu

This comment has been minimized.

Copy link
Contributor Author

Amanieu commented Dec 15, 2019

The scope is intentionally a bit vague here. The intent is that we do not want to have to re-implement a whole assembler ourselves, for every supported architecture.

@Ekleog

This comment has been minimized.

Copy link

Ekleog commented Dec 15, 2019

I had the exact same reaction while reading this sentence, and hesitated commenting on it. Maybe it'd make sense to just remove that “this is out of scope” line, so that the ambiguity is not there and the project group can just reasonably end up with the conclusion that the solution is not going to be re-implementing a whole assembler?

For instance, I seem to remember some pretty convincing arguments saying that all stable asm should actually be at least completely parsed by rustc, if not more, which comes close enough to “re-implement a whole assembler ourselves” to be in the gray zone, as far as I understand.

@Ixrec

This comment has been minimized.

Copy link
Contributor

Ixrec commented Dec 15, 2019

For instance, I seem to remember some pretty convincing arguments saying that all stable asm should actually be at least completely parsed by rustc, if not more

I'm guessing this is https://internals.rust-lang.org/t/inline-assembly-syntax/239/5, which was brought up in the recent irlo inline asm thread here: https://internals.rust-lang.org/t/pre-rfc-2-inline-assembly/11310/102

Although I disagree with these arguments, I do think it's reasonable to expect that some future RFC produced by the ASM project group will explicitly say "we didn't go with asm-as-syntax for reasons X, Y, Z", and "because the charter said so" would be a pretty unsatisfying reason. So I'd also be fine with removing that "out of scope" qualifier for now. It's not like there's another Rust project group working on reimplementing assemblers or assembly syntax, and assembly syntax is certainly relevant to the whole discussion of inline asm (as opposed to e.g. how two-phase borrows are technically irrelevant to NLL, and that was worth highlighting on NLL-related issues since it's not obvious).

@elichai

This comment has been minimized.

Copy link

elichai commented Dec 17, 2019

Love it :)

@joshtriplett

This comment has been minimized.

Copy link
Member

joshtriplett commented Jan 2, 2020

@Amanieu

This comment has been minimized.

Copy link
Contributor Author

Amanieu commented Jan 12, 2020

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Jan 13, 2020

We discussed this some in our lang team meeting last Thursday. @Centril expressed some concerns, though it would be helpful if he could elaborate them, since I don't think I have a great grasp of what they are. One thing that he raised I know is a question of whether we'll have the bandwidth available to tackle this.

I personally feel pretty good about this, since I think that

  • the interaction of inline assembly with the compiler is generally broad but shallow. That is, I feel like inline assembly touches a lot of the compiler, but it doesn't usually require very complex modifications to any one piece.
  • @Amanieu is (as I understand it) volunteering to do a lot of the implementation work, and @joshtriplett is happy to help summarize and bring things to lang team.

In particular I'd like for the @rust-lang/lang team to get better at being able to "delegate" projects like this. The hope would be that the group can handle figuring out the right path, and keep the team updating both via RFCs and regular updates.

As far as priority, I feel that this fits our current project goals in a few ways:

  • I would like us to be taking unfinished features with no particular path to stability and either removing them or finishing them off. Prioritization in these cases can be hard, because most individual features may not be that high priority one by one, but I think that reducing their number is valuable.
  • As for this individual feature, it falls in a few categories ("systems programming", "C parity") that I think are important, and I think that it is clearly of high importance to many potential adopters (especially given the interest we've seen). I think it's important for Rust to both have capabilities that rival C in terms of enabling low-level, systems applications, but also to do a good job on their design.

One thing that would be helpful is if we had some folks from @rust-lang/compiler (or @rust-lang/compiler-contributors) who was excited to serve as a dedicated reviewer here.

@rfcbot

This comment has been minimized.

Copy link

rfcbot commented Jan 13, 2020

🔔 This is now entering its final comment period, as per the review above. 🔔

@Amanieu

This comment has been minimized.

Copy link
Contributor Author

Amanieu commented Jan 13, 2020

Incidentally, we've just finished the RFC for inline assembly and it is now up at #2850.

@Centril

This comment has been minimized.

Copy link
Member

Centril commented Jan 22, 2020

We discussed this some in our lang team meeting last Thursday. @Centril expressed some concerns, though it would be helpful if he could elaborate them, since I don't think I have a great grasp of what they are. One thing that he raised I know is a question of whether we'll have the bandwidth available to tackle this.

I don't have a lot of time to elaborate, and I am sorry that it took so long to get to it, but I do indeed have concerns. In particular:

  • I think getting this over the finish-line is a a huge project (as large as async/await), which will take a long time, especially to fix all the bugs (which there are a lot of for the old asm! macro). Our resources (not just people who implement) are already stretched. Those resources that we do have should, in my view, be going into already accepted but unfinished features and fixing the many bugs we have. For 2019, our roadmap was under the headline "maturity". I think this should continue for 2020. I don't think it fits with "maturity" to be starting new grand projects. The bandwidth here is also not just about the compiler engineering, but also about design work. The design space here is huge, and it is quite noticeable in the RFC discussion. I don't believe the language team has the bandwidth either for answering the project groups queries at various points.

  • Inline assembly is a complex feature in the compiler that does not carry its weigh. To assess the existing complexity, I removed it from the compiler in an experimental branch. In my view, the complexity here is both large and pervasive, in all parts of the compiler (I don't think that's a good thing.) Moreover, this is implementation is buggy. I fully expect that an implementation we could stabilize would be substantially more complicated (in LOC, but also the spec). I think this is clear if you read the proposal. A lot of new static restrictions are added (as they should) in that RFC.

    • I think this will mean continual maintenance and complexity beyond the initial feature. That is, I think this is not a feature that you implement in a small corner and then there are not a lot of follow-ups (based on the discussion in the other RFC hitherto). Moreover, I expect there will be a stream of bugs that will need to be fixed over time and which are difficult to reduce & fix (similar to linker bugs). I think the compiler team already lacks time to work on the incoming bugs today, so we do not have a budget for more.

      • Including #[naked] and global_asm! is also part and indicative of the scope creep here and the
    • Based on the discussion thus far, I am also concerned that unless we reach full parity with the existing asm! macro, it will be difficult to fully remove existing one, leading to double complexity.

    • See also bytecodealliance/cranelift#444.

  • I think this is a niche feature that is mostly subsumed by intrinsics and external assembly (which we could make more ergonomic cargo-side.) From the discussion thus far, I surmise that what isn't subsumed is primarily about improved performance. While niche features and perf improvements are sometimes acceptable, this must also be in proportion with the complexity for the language and the compiler that they bring. In my view, function call overhead is also acceptable in proportion. Rust is already plenty fast enough, and I think we do not have to squeeze out every last ounce of performance when the trade-off is not right (of course that's something people can disagree on).

    • In particular because the feature is niche, I'm also concerned about the desire for lots of extensions.
  • With respect to alternative backends (or less immediately a problem, alternative compilers), I still believe that this non-trivially complicates what the requirements for a Rust compiler are. The proposal says that we should find a way to handle alternative backends and take Cranelift into account specially. However, I have not seen a willingness to participate in this from Cranelift's side. I think this should be Cranelift's call whether they are willing to implement / accept the necessary infrastructure to accommodate inline assembly.

  • I think inline assembly will result in people making more assumptions about concrete implementation details of rustc (and e.g. the memory model, more so than today's underspecified language). For example: assumptions about #[inline(...)]. Given our strong stability commitments (including when they are overstated), I think this will create problems for changes to the compiler.

  • The way generics generally works in Rust is based on checking and requiring the necessary constraints at definition time as opposed to when substituting. With inline assembly however, given that we don't have control over the backend, this will necessarily result in post-monomorphization errors. While we can forbid generic types (and we should, although again, this adds more too the complexity) being used in asm!(...), it is unclear to me how we prevent post-monomorphization errors for types we don't know whether they would work with the register. I think this makes inline assembly a poor fit.

I would like us to be taking unfinished features with no particular path to stability and either removing them or finishing them off. Prioritization in these cases can be hard, because most individual features may not be that high priority one by one, but I think that reducing their number is valuable.

Unlike other unfinished features, asm! was previously never RFCed and does indeed not have a path to stabilization (at least not yet). So I do not consider this in the same category as e.g. finishing slice_patterns

With this said:

@rfcbot concern i-dont-think-we-should-do-this
@rfcbot concern particularly-not-now

@cramertj

This comment has been minimized.

Copy link
Member

cramertj commented Jan 22, 2020

@Centril

I think this is a niche feature that is mostly subsumed by intrinsics and external assembly

I share your concerns about complexity, but I'm skeptical of the conclusion that we don't need inline assembly. There are many developers for which external assembly is hugely impractical and who cannot afford to pay the cost of function calls. Perhaps there's some way to provide appropriate access to a linker and plumb things through a procedural macro / build.rs to make a user-defined asm!-like-thing, but I have a hard time understanding how that would work properly and avoid introducing function calls. Such a solution would also likely require complex build system hacks, which are often not an option for bare-metal development environments where build system simplicity may be a requirement due to security or isolation principles.

I know of at least a couple projects at Google that have run into these issues (see @dancrossnyc's post for information about one such case). I would love to see us reaching out and providing great support for using Rust in these domains in which performance, security, and correctness are critical, and which previously have been dominated almost exclusively by C.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Jan 22, 2020

Thanks @Centril for writing that post. Before I write anything else, I do have one clarifying question:

With inline assembly however, given that we don't have control over the backend, this will necessarily result in post-monomorphization errors. While we can forbid generic types (and we should, although again, this adds more too the complexity) being used in asm!(...), it is unclear to me how we prevent post-monomorphization errors for types we don't know whether they would work with the register. I think this makes inline assembly a poor fit.

Can you give me an example where we get a post-monomorphization error of the kind you are referring to? One thing I could imagine is that you have the type of an operand be some kind of generic type, like T, but you have some requirement on the IR that the result is (say) fit into an integer register. This would work fine for (say) T = u32 but less fine for T = String. Is this the sort of error you meant? Presumably not, since you said we could ban operands with generic types. So what is an example of the sort of error you meant?

I guess I could also imagine that, since we are delegating a lot of the responsibility for generating the IR to LLVM and to an assembler, there might be other errors that arise at code generation time which are difficult for us to detect earlier, which is indeed post-monomorphization, but that doesn't seem to have anything to do with generic types.

@comex

This comment has been minimized.

Copy link

comex commented Jan 23, 2020

This is probably the wrong thread to be having this discussion, since there's already an RFC for the feature itself, whereas this RFC just sets up a project group. But going point by point:

With respect to alternative backends (or less immediately a problem, alternative compilers), I still believe that this non-trivially complicates what the requirements for a Rust compiler are. The proposal says that we should find a way to handle alternative backends and take Cranelift into account specially. However, I have not seen a willingness to participate in this from Cranelift's side. I think this should be Cranelift's call whether they are willing to implement / accept the necessary infrastructure to accommodate inline assembly.

The plan is to implement 'inline assembly in terms of external assembly' on the rustc side, so that Cranelift doesn't have to do anything at all. I agree this must be done before the new inline asm can be stabilized.

Those resources that we do have should, in my view, be going into already accepted but unfinished features and fixing the many bugs we have.

I also hope that the ones proposing the feature will be the ones doing most of the implementation work. That may or may not include me.

Inline assembly is a complex feature in the compiler that does not carry its weigh. To assess the existing complexity, I removed it from the compiler in an experimental branch. In my view, the complexity here is both large and pervasive, in all parts of the compiler

You only removed ~1000 lines of code...

Moreover, this is implementation is buggy. I fully expect that an implementation we could stabilize would be substantially more complicated (in LOC, but also the spec). I think this is clear if you read the proposal. A lot of new static restrictions are added (as they should) in that RFC.

But yes, the new one will require more code.

Including #[naked] and global_asm! is also part and indicative of the scope creep here and the

The RFC as currently written does not mention either #[naked] or global_asm!. But it's hardly surprising that a replacement for asm! should also (eventually) cover global_asm!, and #[naked] is a feature that must be paired with asm! to be used correctly.

Based on the discussion thus far, I am also concerned that unless we reach full parity with the existing asm! macro, it will be difficult to fully remove existing one, leading to double complexity.

I agree, but I believe full or very-near-full parity is achievable.

I think this is a niche feature that is mostly subsumed by intrinsics and external assembly (which we could make more ergonomic cargo-side.)

I disagree that it's niche, and I think you're subjectively underestimating the number of people this is important to.

In my view, function call overhead is also acceptable in proportion. Rust is already plenty fast enough, and I think we do not have to squeeze out every last ounce of performance when the trade-off is not right (of course that's something people can disagree on).

Strongly disagree that Rust should not "squeeze out every last ounce of performance". There should never be a case where C is faster than Rust (with no plan to fix it).

@matklad

This comment has been minimized.

Copy link
Member

matklad commented Jan 23, 2020

Strongly disagree that Rust should not "squeeze out every last ounce of performance". There should never be a case where C is faster than Rust (with no plan to fix it).

I'd like to explicitly +1 this. I'd phrase it this way:

If Rust is not at least as fast as any memory-unsafe high-level language, some critical systems will be written in a memory-unsafe language.

@joshtriplett

This comment has been minimized.

Copy link
Member

joshtriplett commented Jan 23, 2020

@Centril
Alright, let's take this point by point.

* I think getting this over the finish-line is a a huge project (as large as async/await), which will take a long time, especially to fix all the bugs (which there are a lot of for the old `asm!` macro).

I'm not going to claim that this will be trivial, but part of the point of this design was to create something we felt was more manageable, and which insulates us from issues in LLVM's assembly support. The existing inline assembly mechanism is a fairly "raw" export of LLVM's assembly support, along with any fragility it has. The new proposal avoids that fragility.

Our resources (not just people who implement) are already stretched.

I have specifically volunteered to handle the language team side of this, as the liaison. Language team time will not be taken up by the day-to-day activities of this project group. That's part of the point of the project group model.

Those resources that we do have should, in my view, be going into already accepted but unfinished features and fixing the many bugs we have. For 2019, our roadmap was under the headline "maturity". I think this should continue for 2020.

I agree, which is why we're taking the existing feature of inline assembly and finishing it, turning it into something mature that we can feel comfortable stabilizing. I'll respond to your claim that we shouldn't do this at all later in this post; in the absence of that claim, this is absolutely a feature we currently have, continue to need, and should work on the maturity and stabilization path of.

The bandwidth here is also not just about the compiler engineering, but also about design work. The design space here is huge, and it is quite noticeable in the RFC discussion.

Which is why we're chartering a project group to do the design work, rather than doing it directly in the language team.

I don't believe the language team has the bandwidth either for answering the project groups queries at various points.

I will handle any such inquiries myself; that's the job I'm volunteering for as liaison.

* Inline assembly is a complex feature in the compiler that does not carry its weigh.

It's clear that you don't see the value in inline assembly. Many others do, and this was extensively discussed on internals. We can talk about the complexity below, but many people find inline assembly essential for their work.

To assess the existing complexity, I removed it from the compiler in an experimental branch. In my view, the complexity here is both large and pervasive, in all parts of the compiler (I don't think that's a good thing.) Moreover, this is implementation is buggy.

Yes, that implementation is buggy, and regularly generates internal compiler errors. That's why we're building an easier-to-maintain version.

Furthermore, as @comex observed, that really isn't much code. And half the files affected only have an import and a single match branch that just says "handle inline assembly too". Far fewer files actually do something specific to inline assembly. That hardly seems "pervasive".

I fully expect that an implementation we could stabilize would be substantially more complicated (in LOC, but also the spec). I think this is clear if you read the proposal. A lot of new static restrictions are added (as they should) in that RFC.

On the contrary, the point of the implementation we want to work towards stabilizing is that it's simpler than the existing implementation, because it doesn't expose all of LLVM's assembly mechanism. The spec should be substantially simpler. (We've never really had a full specification for the existing raw-LLVM-assembly mechanism, and people have had to go read LLVM's documentation for it instead. We do have a full specification for the new mechanism.)

  * I think this will mean _continual_ maintenance and complexity beyond the initial feature. That is, I think this is not a feature that you implement in a small corner and then there are not a lot of follow-ups (based on the discussion in the other RFC hitherto). Moreover, I expect there will be a stream of bugs that will need to be fixed over time and which are difficult to reduce & fix (similar to linker bugs). I think the compiler team already lacks time to work on the incoming bugs today, so we do not have a budget for more.

Inline assembly today regularly generates ICEs; I expect the new version to generate less bugs over time, and require less maintenance.

I also expect the implementation to be more self-contained than you're suggesting, but in any case, yes, I'm sure there will be at least some ongoing maintenance. And there's a large set of people interested in the feature, some of whom could step up to help with such maintenance. That's part of what we can evaluate in nightly.

    * Including `#[naked]` and `global_asm!` is also part and indicative of the scope creep here and the

This thought seems unfinished. But in any case, you seem to be objecting to the existence of a feature on the basis of potential future features that are not being proposed today, rather than on the basis of what's actually being proposed. (And for that matter, you seem to be objecting to the formation of a project group to work on it, no matter what that group proposes.)

  * Based on the discussion thus far, I am also concerned that [unless we reach full parity with the existing `asm!` macro](https://github.com/rust-lang/rfcs/pull/2843#discussion_r365023029), it will be difficult to fully remove existing one, leading to double complexity.

The comment you're linking to very specifically says that they don't think it'll be much work to get to full parity, and explains why.

I have no problem committing, up front, to removing llvm_asm! when we have asm! implemented in nightly, after a reasonable transition period. We should not maintain both long-term.

(I would also point out that we're doing such a transition at all precisely because so many people use the existing asm!, which you should take as evidence against your claims that it isn't important.)

  * See also [bytecodealliance/cranelift#444](https://github.com/bytecodealliance/cranelift/issues/444).

First, I expect that the more controlled interface we're providing will be much easier for cranelift to support than today's "support everything LLVM does" interface.

Second, the current inline assembly proposal can be implemented with zero support from cranelift, albeit with a performance hit. And given that the proposed cranelift backend is intended to optimize for compilation speed rather than performance of compiled code, that seems fine.

* I think this is a niche feature that is mostly subsumed by intrinsics and external assembly (which we could make more ergonomic `cargo`-side.)

This is incorrect, and there have been multiple counterexamples across various internals threads.

It's also insulting and dismissive to claim that features you don't personally care about are "niche features". Please stop.

This is not a niche feature. Many people have been looking for this for some time, and in some cases this is the feature that keeps them on nightly.

From the discussion thus far, I surmise that what isn't subsumed is primarily about improved performance. While niche features and perf improvements are sometimes acceptable, this must also be in proportion with the complexity for the language and the compiler that they bring. In my view, function call overhead is also acceptable in proportion. Rust is already plenty fast enough, and I think we do not have to squeeze out every last ounce of performance when the trade-off is not right (of course that's something people can disagree on).

And do, as several responses have already said.

Regarding performance, I certainly don't think that we should make tradeoffs in favor of performance over all else in all cases. I absolutely think the tradeoff makes sense in this case.

This is not exclusively about performance; it's also about capabilities, and simplicity, and maintainability. There are not intrinsics for every instruction that people need, and there never will be. (And adding them would be more work than this feature.) And external assembly alone is not quite sufficient, for reasons other than just performance. It's much simpler to write an instruction or two via inline assembly, rather than creating a separate file containing a separate named assembly function with a function-call-based interface.

  * In particular because the feature is niche,

See above regarding this problematic and inaccurate phrasing.

I'm also concerned about the desire for lots of extensions.

I'd hardly characterize the overall picture as having a "desire for lots of extensions". On the contrary, I expect us to have fewer extensions than the existing asm! in nightly does.

The particular message you link to is pretty much exclusively a matter of syntax, in order to be able to provide other things via libs rather than language.

* With respect to alternative backends (or less immediately a problem, alternative compilers), I still believe that this non-trivially complicates what the requirements for a Rust compiler are. The proposal says that we should find a way to handle alternative backends and take Cranelift into account specially. However, I have not seen a willingness to participate in this from Cranelift's side. I think this should be Cranelift's call whether they are willing to implement / accept the necessary infrastructure to accommodate inline assembly.

The current proposal very specifically shows how we can implement this without any support from the backend.

And while I very much like the Cranelift project, I don't think that it's the call of a backend that isn't part of the Rust project right now to decide whether this feature is important, particularly given that we can implement a compatible-but-slower solution for the Cranelift backend with no changes to Cranelift.

  * See also [rust-lang/rust#29722 (comment)](https://github.com/rust-lang/rust/issues/29722#issuecomment-313146254) and [rust-lang/rust#29722 (comment)](https://github.com/rust-lang/rust/issues/29722#issuecomment-334544675).

The first of those comments (from 2017) specifically mentions asm! as one of the more widely used nightly-only features, and states that we should find a path forward for it.

The second of those comments makes the point that the current nightly version of inline assembly is not something we could reasonably stabilize, which I agree with. Which is why we wrote a whole new spec for a simpler version of inline assembly that we believe we can reasonably stabilize.

* The way generics generally works in Rust is based on checking and requiring the necessary constraints at definition time as opposed to when substituting. With inline assembly however, given that we don't have control over the backend, this will necessarily result in post-monomorphization errors. While we can forbid generic types (and we should, although again, this adds more too the complexity) being used in `asm!(...)`, it is unclear to me how we prevent post-monomorphization errors for types we don't know whether they would work with the register. I think this makes inline assembly a poor fit.

This has already been answered by other comments. In short, this has nothing to do with monomorphization, post-monomorphization, or generics. This is simply an inherent as

I would like us to be taking unfinished features with no particular path to stability and either removing them or finishing them off. Prioritization in these cases can be hard, because most individual features may not be that high priority one by one, but I think that reducing their number is valuable.

Unlike other unfinished features, asm! was previously never RFCed and does indeed not have a path to stabilization (at least not yet). So I do not consider this in the same category as e.g. finishing slice_patterns

asm! is one of the most widely used nightly-only features, and the current implementation creates problems, which sounds like the very definition of a feature we should try to improve nd get on a path towards stabilization.

With this said:

@rfcbot concern i-dont-think-we-should-do-this
@rfcbot concern particularly-not-now

So, to be clear, by blocking the project group RFC (as opposed to any specific proposal), you are saying you wish to block people from working on a proposal to improve inline assembly from its curent state and stabilize inline assembly. You're not raising specific concerns; you're blanket-rejecting any possible proposal, to improve something that's already part of nightly Rust. (Which means that the current nightly-only less-stable implementation will be staying around longer. Blocking this will not make the existing implementation go away, it will keep that implementation longer.)

I believe that by doing so, you are not only blocking productive work by others that does not ask anything of you, you are also blocking progress towards goals you are otherwise stating that you favor (maturity, stability). And I further believe that your post demonstrates a lack of regard or empathy for points of view other than your own.

EDIT: In the interests of clarification, I'd like to make it clear that this is a statement of the feelings that the previous post brought up. It feels like the previous posts were pushing back on the existence of inline assembly at all. It feels like we've been over and over the reasons that inline assembly are important, and it feels like these arguments are not being heard. I wanted to make that frustration clear, and to re-present summaries of information from previous responses and discussions in one place.

@Ekleog

This comment has been minimized.

Copy link

Ekleog commented Jan 26, 2020

@joshtriplett

I wanted to refrain from speaking on this thread or on the associated RFC, for reasons that should become clear soon, but here it is.

I come forward speaking as someone who did embedded work as a dayjob, and will likely in the near future, despite the fact it's not my present right now.

I really, really want the code that currently uses inline assembly to be possible to compile on stable. But I agree with @Centril's concern that we should not do this now.

The reason why I have been refraining from speaking was that I believed that this would consume time that would have been better used at other things. In particular, finishing off all the in-flight features that are currently waiting (off the top of my head, const generics and async/await, at least). I was also thinking that the current RFC was not too bad, and did not deserve more time spent on it than it was having.

However, I do believe that having this project group now is trying to rush inline assembly to stabilization, and that the resources allocated to it (including your time off lang-team spent in it, the development work done by people in the project group, etc.), that would be better allocated finishing these other features first, and then when they are finished, switched back to finding a proper path to inline assembly.

For instance, having one intrinsic per instruction, despite being more work, would probably make more sense to me -- at least, off the top of my head, I think it would probably be able to handle all the inline assembly I've been doing, except for one particular snippet (a stack switching one, that probably would have been better done as external assembly anyway, as AFAIR it was in a #[naked] function). And, as a big advantage, it would not be relying on the current RFC's “whatever gas does”.

I am not trying to argue in favor or against that specific RFC. The current one is good enough, and if we want to get something done now, with the resources currently available, it is probably the best bet.

However, I do agree with @Centril that the project group should not be chartered for the time being, until the resources available to it allow basically any solution to be implemented, and in particular so that the language does not unnecessarily definitely end up tied to the current implementation details of the compiler, that happens to be gas-compatible. And I believe that, despite the fact that I've been longing for stable inline assembly for a few years -- the currently available resources just appear, to me, insufficient to reasonably encompass the full design space available.

EDIT: I will probably not submit further messages to these threads, as that would go against my “let's try to focus on other work” objective. I hope this message's content is clear, and will let other people decide of what to do with the information therein.

@joshtriplett

This comment has been minimized.

Copy link
Member

joshtriplett commented Jan 26, 2020

@Ekleog

I do believe that having this project group now is trying to rush inline assembly to stabilization

I'm not sure where that impression would come from. I expect the new inline assembly to spend a long time in nightly for experimentation before we consider stabilizing it. We're trying to implement a version of inline assembly that we have a hope of stabilizing in the future, but we have no interest in rushing that process.

For instance, having one intrinsic per instruction, despite being more work, would probably make more sense to me

Part of the point of this effort is to reduce the surface area of the feature to make it more maintainable, which this would not do.

Also, many kinds of assembly cannot work via intrinsics, as they require control not just over the instructions used but the exact block of instructions emitted.

the resources allocated to it (including your time off lang-team spent in it, the development work done by people in the project group, etc.), that would be better allocated finishing these other features first

My time in the language team, and the time of the other people in the project group, isn't a generic "resource" that can be "better allocated". We're spending time on this because we need inline assembly support specifically.

We'd love to finish other features, too, but that isn't mutually exclusive.

@Lokathor

This comment has been minimized.

Copy link

Lokathor commented Jan 26, 2020

EDIT: I will probably not submit further messages to these threads, as that would go against my “let's try to focus on other work” objective. I hope this message's content is clear, and will let other people decide of what to do with the information therein.

Throwing out concerns and then immediately saying "also I won't come back and discuss this further" is unfortunate to say the least.

@Ekleog

This comment has been minimized.

Copy link

Ekleog commented Jan 26, 2020

@Lokathor I simply think there's nothing more I could say, should questions arise on my concerns I'll answer them, but trying to argue further would mean spend even more time from the people reading me, and that goes against the entire objective my post was striving towards ;)

@joshtriplett Sorry if the choice of the word “rush” was bad, what I meant was “later could be a better moment to do it, with fewer inflight projects, and it is not particularly more urgent now than a year ago” (this is not an argument towards never doing it and always pushing it back to later, either, I think once either async/await or const generics get completed then it could be a great project, but three simultaneous huge projects sounds like too many to me)

@Lokathor

This comment has been minimized.

Copy link

Lokathor commented Jan 26, 2020

I think that the end of Josh's most recent reply is the most important fact to hold in mind when talking about any of this: Just because it's decided "We're not working on A, we're working on B", that doesn't mean people who would have worked on A will have the skills or interest level to work on B instead. Volunteer time is not fungible like that.

Specifically you mentioned async/await twice now, and while I'm sure it's very nice to some portion of Rust users, it is also totally useless to a different portion of Rust users. Telling the embedded development people of Rust "We don't want to even let you discuss how you could improve your usage of Rust, because the web app developers need to get more attention instead" is not a very good argument. It really doesn't make sense at all. The people working on one area aren't the same people working in the other. They should be totally separate efforts, each allowed to progress at their own pace according to how much interest and support they're able to have people put in.

Similarly for const generics. Very cool language feature, and people are excited for it, but the tracking issue which was opened 2 years ago still has only 1 of 15 boxes checked (including "design" being unchecked), and there's 3 pages of related issues in the issue tracker. It's simply unreasonable to imagine that it would all suddenly get sorted out in, say, 6 months or less. So saying that discussion of inline assembly is going to be blocked until const generics is entirely sorted out doesn't make any sense.

@Diggsey

This comment has been minimized.

Copy link
Contributor

Diggsey commented Jan 26, 2020

There are people with the requisite knowledge and skills, eager to invest their time into this feature today. That may not be the case in six months time, or whenever inline assembly reaches the top of everyone else's priority list.

If there's no question that Rust should support inline assembly at some point, that the people in the proposed group are qualified, and (most importantly) that inline assembly is entirely orthogonal to/won't conflict with any other features in the works, then the language team should be able to accommodate that.

The Rust teams exist to create some kind of order out of the chaos that is open source, and prioritising features is a tool to do that, but if taken to the extreme then it just becomes the Rust teams telling people what they can or cannot work on, and that's a sure-fire way to kill people's enthusiasm for open source because it just becomes an unpaid job.

@fintelia

This comment has been minimized.

Copy link

fintelia commented Jan 26, 2020

The way I see it, the current direction is the least bad option that still meets the needs of embedded/low level developers. It would be really nice if adding a bunch of intrinsics was enough for everyone. It would be really nice if linking to external assembly was enough. Or if there were the resources to design and write our own assember. Or if we had the expertise to write a custom DSL. Or...

But the reality is that none of those fully solve the issue within the constrains we have. Inline assembly may be inferior to each of them in some dimension, but only it is viable solution.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Jan 27, 2020

I think this conversation about resourcing -- and how to think about it -- is very important. I've been advocating for some time now that the Rust project needs to think more about followthrough and resourcing. This was the point of my unbounded queues blog post, for example.

Still, I strongly agree with @Diggsey that we have to balance this sort of planning with a respect for serendipity and momentum (and indeed the post talked about this, too). I also think we need to think about the balance of skills. I don't think that folks pushing on const generics will necessarily slow down progress on const generics, for example.

In my view, one of the main purposes for the Rust organization is to help people help themselves. In some ways, we're really great at that, with RFCs, mentoring, and the like. But for larger efforts, it's harder. There are also a lot of challenging problems that have "half-finished" designs that we just can't seem to get over the finish line. I would put inline assembly into this bucket, but also things like custom test frameworks, const generics, custom allocators, specialization, etc. I frequently get asked by people "how can I make feature X happen?" and the bottom line is that I usually don't have a good answer for them.

Indeed, the whole project group concept is basically a response to these two concerns: balancing the dangers of unbounded queues with the desire to help people help themselves. (And adding in the idea of creating a structure that can help with mentoring and getting people involved in the Rust project.)

My hope is that the answer to someone who says "I want to see feature X happen", from now on, will be:

  • You can do it! But you're going to have to shepherd a project group
  • Find a liaison or sponsor on the team(s) in question who has some spare time
  • Working with them, form a project group to pursue the idea
  • But make sure you're ready to see it through to the end

It's absolutely true though that we can't do all the things at once. This is part of the reason to ensure that we have specific team liaisons and sponsors: so long as nobody is liaison for more than 1 or 2 projects at a time, this should naturally tie the number of active projects to the bandwidth of the team. I also think it's the right model for our teams: team members should be serving as "multipliers", helping to enable people.

Anyway, enough waxing philosophical. I will just close by noting that -- of course -- we also have to be careful about what we do. Just because people are excited to push some design doesn't make it a good idea. But I think that in the case of inline assembly (and many of the examples I listed above) there have already been a number of excellent comments about why it's important, so I don't have that much to add there.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Jan 27, 2020

I was thinking about what it will take to move this conversation towards consensus. I thought it would be useful to start with what are the key things under contention.

  • Is inline assembly an important feature and/or a good fit for Rust?
  • How big is the job of designing and implementing this feature and do we have the bandwidth?

Does this sound about right? Looking at the second question in particular, I would say that we might consider several kinds of work. Here is my first stab at a list:

  • Language:
    • The design itself -- how ergonomic is it? Can it express the things people want?
    • Interactions with the rest of the language, like borrow checker, unsafety rules, and so forth
  • Compiler:
    • Parsing the directives, checking that static conditions are satisfied
    • Integrating with borrow checker and other bits of code
    • Mapping to LLVM
    • Compatibility with other backends (implement the "inline-assembly-as-function" shim)
    • Writing thorough and exhaustive tests
      • Testing e.g. all the options, the different kinds of errors, that we get the right LLVM IR
  • Documentation
    • Explainers
    • Documentation that shows how to map from GNU/LLVM inline assembly to Rust
@comex

This comment has been minimized.

Copy link

comex commented Jan 27, 2020

Another necessary task is to design and implement the inline-assembly-as-external-assembly shim.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Jan 27, 2020

Another necessary task is to design and implement the inline-assembly-as-external-assembly shim.

Yes, I meant for that to be "compatibility with other backends" -- though it's worth asking ourselves how important we think this is. In any case, I updated the list of work items.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Jan 27, 2020

One thing I am wondering about -- are there folks on @rust-lang/compiler or @rust-lang/compiler-contributors who are interested in (and feel they would have time for) serving as the "main reviewer" for this work? (I imagine some aspects of it, like integration into the borrow checker, might want double-checking by @matthewjasper and/or myself, but I think the majority of this is going to be about checking that the constraints are met, that things are parsed etc correctly, and that we translate to LLVM and/or the backend correctly)

As @Centril said, there will always be maintenance work to consider, and I definitely think we'd want to be growing multiple people with knowledge of how the code works. (Ideally, this would also happen as part of the project group.)

I thought @nagisa might be into this, but I'm not sure how much time they've got. =)

Edit: Oh, I see I asked this earlier.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Jan 27, 2020

I'm not sure how much time they've got

I always appreciate some good code to read and should be available to review inline assembly PRs too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.