Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDE Linux struct page object aliasing #4809

Closed
cnuke opened this issue Apr 13, 2023 · 7 comments
Closed

DDE Linux struct page object aliasing #4809

cnuke opened this issue Apr 13, 2023 · 7 comments
Labels

Comments

@cnuke
Copy link
Member

cnuke commented Apr 13, 2023

Due to the way the memory-allocation code is currently implemented and the struct page object handling is integrate aliasing that has unexpected side-effects can occur. Since struct page objects are created for a page-size aligned address [1] when requested, allocations that fall within the same virtual page address range share the same object.

Such objects are created lazy and and life-time management is only performed via ref-counting when the page API [2] (alloc_pages, free_pages etc.) is used. Since the current implementation removes struct page objects [3] when performing allocations still referenced objects can become invalid and can be corrupted as the memory is re-used.

[1]

struct page *lx_emul_virt_to_pages(void const *virt, unsigned long count)
{
/* sanitize argument */
void * const page_aligned_virt = (void *)((uintptr_t)virt & PAGE_MASK);
struct page *page = lx_emul_associated_page(page_aligned_virt, 1);
if (!page) {
unsigned long i;
struct page * p = kzalloc(sizeof(struct page)*count, 0);
page = p;
for (i = 0; i < count; i++, p++) {
p->virtual = (void*)((uintptr_t)page_aligned_virt + i*PAGE_SIZE);
init_page_count(p);
lx_emul_associate_page_with_virt_addr(p, p->virtual);
}
}

[2] shadow/mmc/page_alloc.c

[3]

extern "C" void * lx_emul_mem_alloc_aligned(unsigned long size, unsigned long align)
{
void * const ptr = Lx_kit::env().memory.alloc(size, align);
lx_emul_forget_pages(ptr, size);
return ptr;
};

@chelmuth chelmuth added the bug label May 9, 2023
chelmuth added a commit to chelmuth/genode that referenced this issue Jul 11, 2023
USB devio splits large transaction into 16 KiB buffers in scatter-gather
lists. Unfortunately, this mechanism seems unreliable most certainly because
of issue genodelabs#4809 "DDE Linux struct page object aliasing".
chelmuth added a commit that referenced this issue Oct 27, 2023
USB devio splits large transaction into 16 KiB buffers in scatter-gather
lists. Unfortunately, this mechanism seems unreliable most certainly because
of issue #4809 "DDE Linux struct page object aliasing".

Issue #5036
chelmuth added a commit that referenced this issue Oct 27, 2023
USB devio splits large transaction into 16 KiB buffers in scatter-gather
lists. Unfortunately, this mechanism seems unreliable most certainly because
of issue #4809 "DDE Linux struct page object aliasing".

Issue #5036
chelmuth added a commit that referenced this issue Nov 2, 2023
USB devio splits large transaction into 16 KiB buffers in scatter-gather
lists. Unfortunately, this mechanism seems unreliable most certainly because
of issue #4809 "DDE Linux struct page object aliasing".

Issue #5036
chelmuth added a commit to chelmuth/genode that referenced this issue Nov 3, 2023
USB devio splits large transaction into 16 KiB buffers in scatter-gather
lists. Unfortunately, this mechanism seems unreliable most certainly because
of issue genodelabs#4809 "DDE Linux struct page object aliasing".

Issue genodelabs#5036
chelmuth added a commit to chelmuth/genode that referenced this issue Nov 9, 2023
USB devio splits large transaction into 16 KiB buffers in scatter-gather
lists. Unfortunately, this mechanism seems unreliable most certainly because
of issue genodelabs#4809 "DDE Linux struct page object aliasing".

Issue genodelabs#5036
chelmuth added a commit to chelmuth/genode that referenced this issue Nov 28, 2023
USB devio splits large transaction into 16 KiB buffers in scatter-gather
lists. Unfortunately, this mechanism seems unreliable most certainly because
of issue genodelabs#4809 "DDE Linux struct page object aliasing".

Issue genodelabs#5036
@nfeske
Copy link
Member

nfeske commented Nov 30, 2023

Fixed in Genode 23.11.

@nfeske nfeske closed this as completed Nov 30, 2023
@chelmuth
Copy link
Member

chelmuth commented Dec 1, 2023

The real fix is still not merged.

@chelmuth chelmuth reopened this Dec 1, 2023
chelmuth added a commit to genodelabs/genode-allwinner that referenced this issue Dec 5, 2023
chelmuth added a commit to genodelabs/genode-imx that referenced this issue Dec 5, 2023
chelmuth added a commit to genodelabs/genode-rpi that referenced this issue Dec 5, 2023
chelmuth added a commit to genodelabs/genode-zynq that referenced this issue Dec 5, 2023
chelmuth added a commit that referenced this issue Dec 5, 2023
Allocator for not-DMA-capable meta data buffers.

Issue #4809
chelmuth added a commit that referenced this issue Dec 5, 2023
The management of Linux page structs is now tied to the life time of DMA
buffers. Thus, page structs are created when a buffer is allocated and
deallocated only when a buffer is freed - not on lx_emul_mem_free()
because DMA buffers are cached. Page struct refcounting was entirely
reworked in lx_emul/shadow/mm/page_alloc.c.

Fixes #4809
@chelmuth
Copy link
Member

chelmuth commented Dec 5, 2023

Commit cfe7915 implements page-struct management per DMA buffer. Repositories for allwinner, imx, rpi, and zynq were also adapted for the change. NOw I cross fingers for nightly. Also, any test results for Pinephone and MNT Reform are much appreciated.

cnuke added a commit to cnuke/genode-allwinner that referenced this issue Dec 5, 2023
cnuke added a commit to cnuke/genode-imx that referenced this issue Dec 5, 2023
@cnuke
Copy link
Member Author

cnuke commented Dec 5, 2023

@chelmuth please merge 1dadb8f and b337484 to the corresponding staging branches as they contained the needed adaptation to the used_apis file of the graphics drivers.

@cnuke
Copy link
Member Author

cnuke commented Dec 5, 2023

I briefly tested the glmark2 preset on the PinePhone with the current staging branch. It works with the page-struct commits but after the sixth scenario the gpu driver fails to upgrade the Platform session and issues a RAM resource request of 16M. When I revert cfe7915 and 06b00b5 it finishes successfully without any resource request.

I can look into that later this week.

chelmuth pushed a commit to genodelabs/genode-allwinner that referenced this issue Dec 6, 2023
chelmuth added a commit that referenced this issue Dec 6, 2023
chelmuth added a commit that referenced this issue Dec 6, 2023
@cnuke
Copy link
Member Author

cnuke commented Dec 7, 2023

@chelmuth commit 74c561f appears to solve the unexpected memory consumption. I'll will give the MNT Reform 2 a spin next.

@cnuke
Copy link
Member Author

cnuke commented Dec 7, 2023

The MNT Reform 2 also plays ball if the aforementioned fixup is applied (otherwise the imx8mq_gpu_drvrequests more caps).

nfeske pushed a commit that referenced this issue Dec 13, 2023
Allocator for not-DMA-capable meta data buffers.

Issue #4809
nfeske pushed a commit that referenced this issue Dec 13, 2023
nfeske pushed a commit to genodelabs/genode-allwinner that referenced this issue Dec 13, 2023
nfeske pushed a commit to genodelabs/genode-imx that referenced this issue Dec 13, 2023
@nfeske nfeske closed this as completed in 1e7116f Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants