Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better fix for marking buffers held on the stack #349

Merged
merged 1 commit into from
Jul 17, 2023

Conversation

casperisfine
Copy link

Followup: #347

As discussed previously, the alloca hack was just a quick fix, it's not good as it could lead to stack overflows and also could be optimized away by compilers etc.

Here instead we copy all the VALUE references into a dedicated object that is in change of holding and pinning these references.

cc @peterzhu2118 @eregon @pavel-workato @sitano

ext/msgpack/buffer_class.c Outdated Show resolved Hide resolved
ext/msgpack/buffer_class.c Outdated Show resolved Hide resolved
Followup: msgpack#347

As discussed previously, the `alloca` hack was just a quick fix,
it's not good as it could lead to stack overflows and also could
be optimized away by compilers etc.

Here instead we copy all the VALUE references into a dedicated object
that is in change of holding and pinning these references.
@byroot byroot merged commit 6dcdb39 into msgpack:master Jul 17, 2023
16 checks passed
@eregon
Copy link
Contributor

eregon commented Jul 18, 2023

Thanks!

I was thinking maybe we can just use a msgpack Buffer to hold onto the struct and the VALUEs inside.
But probably that doesn't quite work and hence the new class?

@byroot
Copy link
Member

byroot commented Jul 18, 2023

But probably that doesn't quite work and hence the new class?

Yeah that doesn't. The buffer struct does this weird thing that it's a linked list of buffer_chunk_t, but the tail of the list is a member of the butter_t. So if you copy the buffer elsewhere, the tail reference is broken...

From what I could gather, it's an optimization to avoid pointer chasing when you append to the buffer, but it's unclear how efficient that really is.

We talked with @peterzhu2118 about potentially simplyfing the buffer implementation, but not sure when we'll actually get around it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants