Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Foiling CG #43

Merged
1 commit merged into from
Jun 20, 2016
Merged

Foiling CG #43

1 commit merged into from
Jun 20, 2016

Conversation

ohler55
Copy link
Contributor

@ohler55 ohler55 commented Nov 18, 2015

Made most VALUE variables volatile to stop them from being GCed until they are off the stack.

I probably changed a couple that did not need to be changed but making the variable volatile will not hurt anything. What it does is stop the compiler from optimizing those variable into registers when Ruby does not see them. They remain on the stack and Ruby will not CG them until they are no longer on the stack.

I also left a FIXME comment related to the redisreaderi->reply use. It might be worth doing a double check.

@badboy
Copy link
Contributor

badboy commented Dec 29, 2015

I'm not so happy about using volatile for everything, especially as there is a macro available that has clearer meaning (though it does the same)

@badboy badboy added this to the 1.0.0 milestone Dec 29, 2015
@ohler55
Copy link
Contributor Author

ohler55 commented Dec 29, 2015

If you would prefer I can change to using RB_GC_GUARD. Let me explain my reasons first though. There is a good article on memory management for ruby at https://ruby-hacking-guide.github.io/gc.html. It does into more detail than most car about. Search for volatile to get skip a lot of the details.

More than once when Oj was growing up I had complaint about stack use for deeply nested objects using multiple threads with smaller stacks. I went spend some time trying to keep the stack as tight as I could for each iteration. The RB_GC_GUARD was one of the places I could get rid of an extra expansion of the stack. Pretty minor but every bit helped. Since coming to understand what Ruby is doing with the GC I've always thought it was more clear to be direct and use volatile. I understand that not everyone has the same understanding though so I can also see your point about the macro. It does spell out in bold letters what is being done.

@badboy
Copy link
Contributor

badboy commented Dec 29, 2015

Thanks for the link. I'm gonna read it later, but you're comment already makes some stuff better to understand.
If it indeed fixes the problem and we can keep other bloat out it's probably the right thing to do (and we can always leave a comment in the code or commit why it was done)

@ohler55
Copy link
Contributor Author

ohler55 commented Dec 29, 2015

Have you been able to test it yet? That was the hard part I think.

@badboy
Copy link
Contributor

badboy commented Dec 29, 2015

Nope, no chance reproducing it so far, so basically this will be a shot in the dark anyway

@badboy
Copy link
Contributor

badboy commented Dec 30, 2015

@findchris Did you already try running hiredis-rb off this PR? Would be nice to know if at least it seems to fix your issue.

@badboy
Copy link
Contributor

badboy commented Dec 30, 2015

Relevant: https://github.com/ruby/ruby/blob/trunk/doc/extension.rdoc#appendix-e-rb_gc_guard-to-protect-from-premature-gc

There volatile is explicitely discouraged. I pushed a change using RB_GC_GUARD here: 72ba42c
From what I understand these are the only places where the value could be optimized away. Then again compilers are hard and do weird things.

@ohler55
Copy link
Contributor Author

ohler55 commented Dec 30, 2015

There might be a couple more places where the compiler could use a register instead but without a way to test it is tough to know.

Interesting the note about buggy compilers. Something to keep in mind as a possibility.

Anyway, if you want to drop this PR thats okay by me. I mostly trying to help out and you seem to have it under control.

@badboy
Copy link
Contributor

badboy commented Dec 30, 2015

@ohler55 I'm very thankful for the PR and your help here, I'm still undecided what way to go.

@findchris
Copy link

I can try running this PR against production, but it doesn't appear that it would contain the RB_GC_GUARDs you added @badboy. Are you against adding those to this PR?

@badboy
Copy link
Contributor

badboy commented Dec 31, 2015

Well, mixing them wouldn't give us a better idea whether one or the other solution works, so it would be nice if you try the volatile version (this PR).

@badboy
Copy link
Contributor

badboy commented Jan 6, 2016

@findchris Any word from production yet?
I decided to pull in this change in the hope it fixes it. I guess there's no easy way to ever reproduce it in tests.

@findchris
Copy link

findchris commented Jun 17, 2016

Hi there.

I am sorry for the delay in responding. I was waiting to deploy and monitor for a little while. It's been a long time since I've seen this error crop up, and so I'm likely to say this patch was helpful in that regard.

I appreciate all of the time and contribution @ohler55 and @badboy!

👍

@badboy badboy closed this Jun 20, 2016
@badboy badboy reopened this Jun 20, 2016
@badboy
Copy link
Contributor

badboy commented Jun 20, 2016

@findchris Thanks, I'm pulling this in now.

@badboy
Copy link
Contributor

badboy commented Jun 20, 2016

@not-A-robot r+

@ghost
Copy link

ghost commented Jun 20, 2016

📌 Commit 0fb7284 has been approved by badboy

@ghost
Copy link

ghost commented Jun 20, 2016

⌛ Testing commit 0fb7284 with merge f646aad...

ghost pushed a commit that referenced this pull request Jun 20, 2016
Foiling CG

Made most VALUE variables volatile to stop them from being GCed until they are off the stack.

I probably changed a couple that did not need to be changed but making the variable volatile will not hurt anything. What it does is stop the compiler from optimizing those variable into registers when Ruby does not see them. They remain on the stack and Ruby will not CG them until they are no longer on the stack.

I also left a FIXME comment related to the redisreaderi->reply use. It might be worth doing a double check.
@ghost
Copy link

ghost commented Jun 20, 2016

☀️ Test successful - status-travis
Approved by: badboy
Pushing f646aad to master...

@ghost ghost merged commit 0fb7284 into redis:master Jun 20, 2016
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants