Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DoS issue when using virtio with rust-vmm/vm-memory #93

Closed
andreeaflorescu opened this issue May 28, 2020 · 1 comment
Closed

DoS issue when using virtio with rust-vmm/vm-memory #93

andreeaflorescu opened this issue May 28, 2020 · 1 comment

Comments

@andreeaflorescu
Copy link
Member

We have identified an issue in the rust-vmm vm-memory crate that leads to a denial-of-service (DoS) issue if the crate is used in a VMM in conjunction with virtio. The issue affects both vm-memory releases (v0.1.0 and v0.2.0). In our environment, we reproduced this with musl builds on x86_64, and with all aarch64 builds.

Issue Description

In vm-memory, the functions read_obj and write_obj are not doing atomic accesses for all combinations of platform and libc implementations. These reads and writes translate to memcpy, which may be performing byte-by-byte copies. Using vm-memory in the virtio implementation can cause undefined behavior, as descriptor indexes require 2-byte atomic accesses.

Impact

The issue can affect any virtio/emulated device which expects atomic writes for base types longer than 1 byte.

Observed impact: When the network stack is under load, the driver will try to clear a used descriptor before the index of the descriptor is fully written by the device. When this issue is triggered, the virtio-net device will be unable to transmit packets. This leads to VMs using rust-vmm/vm-memory having their network effectively disconnected by outside network traffic, resulting in both a DoS vector and an availability issue under normal at-load operations.

Affected Systems

For a VMM to be affected, it must run on aarch64 (built with either musl or glibc), or on x86_64 with a musl build. All VMMs using rust-vmm/vm-memory (any release) in a production scenario, and that take arbitrary traffic over the virtio-net device, are confirmed to be at risk of a DOS. All VMMs using rust-vmm/vm-memory (any release) in a production scenario with a virtio-net deice are under availability risk. All VMMs using rust-vmm/vm-memory (any release) in a production scenario using other devices that expect atomic reads for more than 1-byte values may also be affected, but we are unaware of any risk for other devices (beyond the guest freezing its own virtio stack).

andreeaflorescu added a commit to andreeaflorescu/firecracker that referenced this issue May 28, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
sandreim pushed a commit to firecracker-microvm/firecracker that referenced this issue May 29, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
gbionescu pushed a commit to jabedude/firecracker that referenced this issue Jun 5, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
gbionescu pushed a commit to gbionescu/firecracker that referenced this issue Jun 11, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
@andreeaflorescu
Copy link
Member Author

Fix merged in master and in patch releases.

dianpopa pushed a commit to dianpopa/firecracker that referenced this issue Aug 4, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
dianpopa pushed a commit to dianpopa/firecracker that referenced this issue Aug 5, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
dianpopa pushed a commit to dianpopa/firecracker that referenced this issue Aug 5, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
dianpopa pushed a commit to firecracker-microvm/firecracker that referenced this issue Aug 5, 2020
In rust-vmm/vm-memory 0.2.0, the functions read_obj and write_obj
are not doing atomic accesses for all combinations of platform and
libc implementations. This is caused by everything being written
at a one byte granularity. The reads and writes eventually translate
to memcpy, which in some cases are not optimized to write at a higher
granularity. Since accesses are not guaranteed to be atomic, using
vm-memory in the virtio implementation causes undefined behavior
because the requirement of 2-byte aligned accesses of descriptor
indexes cannot be fulfilled.

The patch that fixes this issue is included in the 0.2.1 vm-memory
update.

There were no Firecracker releases after 2020-03-03; all existing
releases, including those used by AWS Lambda and AWS Fargate, are
not affected.

Related to: rust-vmm/vm-memory#93

Signed-off-by: Andreea Florescu <fandree@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant