New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for test suites emulated in QEMU #39400

Merged
merged 1 commit into from Feb 8, 2017

Conversation

Projects
None yet
8 participants
@alexcrichton
Member

alexcrichton commented Jan 29, 2017

This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the arm-unknown-linux-gnueabihf target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes #33114

@rust-highfive

This comment has been minimized.

Show comment
Hide comment
@rust-highfive

rust-highfive Jan 29, 2017

Collaborator

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

Collaborator

rust-highfive commented Jan 29, 2017

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

Add support for test suites emulated in QEMU
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes #33114
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jan 29, 2017

Member

I believe this is ready to land in the sense that tests should pass as-is, but I think we should discuss a bit before landing this. For example I think we should answer questions like:

  1. This essentially makes arm-unknown-linux-gnueabihf a tier 1 target. Are we ready to do that?
  2. Are we comfortable not running gdb tests? I don't really know how we'd do that at all.
  3. Is the method of building the emulator suitable? Right now it's kinda crazy how it's set up:
    • a custom kernel is compiled
    • the kernel boots into busybox
    • busybox is ad-hocly configured to spawn a server on startup
    • the server runs inside of a bare-bones ubuntu chroot
    • the emulator runs entirely in ram, no virtual hard drive
    • a random dtb file (I don't know what this is) is required to run the emulator
    • I have no idea how easy it'd be to stand up other architectures, I suspect very hard (took me forever here)
  4. We're running low on capacity on Travis, can we afford adding another builder? I'm clarifying with Travis about the precise capacity numbers we have hear, but we're definitely running near the limit. Is this the right place to spend another builder?
  5. If we are going to add another tier 1 target effectively, is this the right target? Should we choose another perhaps if we're running near builder capacity?

Curious to hear others' thoughts!

cc @rust-lang/tools
cc @japaric

Member

alexcrichton commented Jan 29, 2017

I believe this is ready to land in the sense that tests should pass as-is, but I think we should discuss a bit before landing this. For example I think we should answer questions like:

  1. This essentially makes arm-unknown-linux-gnueabihf a tier 1 target. Are we ready to do that?
  2. Are we comfortable not running gdb tests? I don't really know how we'd do that at all.
  3. Is the method of building the emulator suitable? Right now it's kinda crazy how it's set up:
    • a custom kernel is compiled
    • the kernel boots into busybox
    • busybox is ad-hocly configured to spawn a server on startup
    • the server runs inside of a bare-bones ubuntu chroot
    • the emulator runs entirely in ram, no virtual hard drive
    • a random dtb file (I don't know what this is) is required to run the emulator
    • I have no idea how easy it'd be to stand up other architectures, I suspect very hard (took me forever here)
  4. We're running low on capacity on Travis, can we afford adding another builder? I'm clarifying with Travis about the precise capacity numbers we have hear, but we're definitely running near the limit. Is this the right place to spend another builder?
  5. If we are going to add another tier 1 target effectively, is this the right target? Should we choose another perhaps if we're running near builder capacity?

Curious to hear others' thoughts!

cc @rust-lang/tools
cc @japaric

@cuviper

This comment has been minimized.

Show comment
Hide comment
@cuviper

cuviper Jan 29, 2017

Member

It would be nice to have big-endian test results, even if they're not blocking / tier-1.

Member

cuviper commented Jan 29, 2017

It would be nice to have big-endian test results, even if they're not blocking / tier-1.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jan 29, 2017

Member

Yeah definitely! If I could I'd add a QEMU-emulated builder for basically every platform we support. It's not just a question of desires thought, but also:

  • Build capacity
  • Doing the legwork of getting an image working

We're running quite close to capacity and it took quite awhile to get this image going, so we probably can't add just everything yet. This is what I meant above with (5) of whether we want to add this target, or perhaps another as CI. (e.g. which we'll get the most mileage out of if we're resource constrained)

Member

alexcrichton commented Jan 29, 2017

Yeah definitely! If I could I'd add a QEMU-emulated builder for basically every platform we support. It's not just a question of desires thought, but also:

  • Build capacity
  • Doing the legwork of getting an image working

We're running quite close to capacity and it took quite awhile to get this image going, so we probably can't add just everything yet. This is what I meant above with (5) of whether we want to add this target, or perhaps another as CI. (e.g. which we'll get the most mileage out of if we're resource constrained)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jan 30, 2017

Member

According to Travis we can have 35 concurrent jobs total. We currently have 26, this PR adds one more, and #38847 adds another, bringing the total to 28.

(just FYI)

Member

alexcrichton commented Jan 30, 2017

According to Travis we can have 35 concurrent jobs total. We currently have 26, this PR adds one more, and #38847 adds another, bringing the total to 28.

(just FYI)

@cuviper

This comment has been minimized.

Show comment
Hide comment
@cuviper

cuviper Jan 30, 2017

Member

Yeah, I get the resource constraints. It's nice that there's still a little headroom.

I suggest some big-endian testing because this is a common case where code that otherwise appears platform independent may actually have portability issues. So adding any BE platform to the mix will improve coverage for all of them. In librustc_back that's currently mips, mip64, powerpc, powerpc64, s390x, and sparc64. I don't know where sparc64 stands, but the rest are all Tier 2 platforms.

Member

cuviper commented Jan 30, 2017

Yeah, I get the resource constraints. It's nice that there's still a little headroom.

I suggest some big-endian testing because this is a common case where code that otherwise appears platform independent may actually have portability issues. So adding any BE platform to the mix will improve coverage for all of them. In librustc_back that's currently mips, mip64, powerpc, powerpc64, s390x, and sparc64. I don't know where sparc64 stands, but the rest are all Tier 2 platforms.

@michaelwoerister

This comment has been minimized.

Show comment
Hide comment
@michaelwoerister

michaelwoerister Jan 31, 2017

Contributor

So adding any BE platform to the mix will improve coverage for all of them.

Yes, please, we had a number of actual issues around type hashes being accidentally different between big- and little-endian platforms, which broke cross-compilation.

Contributor

michaelwoerister commented Jan 31, 2017

So adding any BE platform to the mix will improve coverage for all of them.

Yes, please, we had a number of actual issues around type hashes being accidentally different between big- and little-endian platforms, which broke cross-compilation.

@nagisa

This comment has been minimized.

Show comment
Hide comment
@nagisa

nagisa Jan 31, 2017

Contributor

If the BE platform also disallowed unaligned access, we’d benefit doubly.

Contributor

nagisa commented Jan 31, 2017

If the BE platform also disallowed unaligned access, we’d benefit doubly.

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Feb 6, 2017

Contributor

@bors r+ Looks awesome.

Long-term it would be better if all this logic was in cargo-cross or some other tool that others can benefit from.

I would not say this automatically makes the platform tier 1. We can have coverage even without offering guarantees. I would like to rework our tier system slightly to decouple from our CI setup - just because we add tests shouldn't mean a platform is automatically in a different support tier.

Contributor

brson commented Feb 6, 2017

@bors r+ Looks awesome.

Long-term it would be better if all this logic was in cargo-cross or some other tool that others can benefit from.

I would not say this automatically makes the platform tier 1. We can have coverage even without offering guarantees. I would like to rework our tier system slightly to decouple from our CI setup - just because we add tests shouldn't mean a platform is automatically in a different support tier.

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Feb 6, 2017

Contributor

📌 Commit 1747ce2 has been approved by brson

Contributor

bors commented Feb 6, 2017

📌 Commit 1747ce2 has been approved by brson

frewsxcv added a commit to frewsxcv/rust that referenced this pull request Feb 7, 2017

Rollup merge of #39400 - alexcrichton:arm-cross-test, r=brson
Add support for test suites emulated in QEMU

This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes #33114

bors added a commit that referenced this pull request Feb 7, 2017

Auto merge of #39605 - frewsxcv:rollup, r=frewsxcv
Rollup of 18 pull requests

- Successful merges: #38764, #39361, #39400, #39426, #39431, #39482, #39512, #39523, #39557, #39561, #39582, #39583, #39586, #39587, #39598, #39599, #39601, #39604
- Failed merges:

frewsxcv added a commit to frewsxcv/rust that referenced this pull request Feb 7, 2017

Rollup merge of #39400 - alexcrichton:arm-cross-test, r=brson
Add support for test suites emulated in QEMU

This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes #33114

bors added a commit that referenced this pull request Feb 7, 2017

Auto merge of #39609 - frewsxcv:rollup, r=frewsxcv
Rollup of 18 pull requests

- Successful merges: #38764, #39361, #39400, #39426, #39431, #39482, #39523, #39557, #39561, #39582, #39583, #39586, #39587, #39598, #39599, #39601, #39602, #39604
- Failed merges:

frewsxcv added a commit to frewsxcv/rust that referenced this pull request Feb 8, 2017

Rollup merge of #39400 - alexcrichton:arm-cross-test, r=brson
Add support for test suites emulated in QEMU

This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes #33114

bors added a commit that referenced this pull request Feb 8, 2017

bors added a commit that referenced this pull request Feb 8, 2017

frewsxcv added a commit to frewsxcv/rust that referenced this pull request Feb 8, 2017

Rollup merge of #39400 - alexcrichton:arm-cross-test, r=brson
Add support for test suites emulated in QEMU

This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes #33114

bors added a commit that referenced this pull request Feb 8, 2017

Auto merge of #39638 - frewsxcv:rollup, r=frewsxcv
Rollup of 13 pull requests

- Successful merges: #38764, #39361, #39372, #39374, #39400, #39426, #39431, #39459, #39482, #39545, #39593, #39620, #39621
- Failed merges:

@bors bors merged commit 1747ce2 into rust-lang:master Feb 8, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@alexcrichton alexcrichton deleted the alexcrichton:arm-cross-test branch Feb 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment