Run GC if fiber pool expansion fails.

[samuel-williams-shopify]

Bug #21964

[ioquatix]

I confirmed the failure path works on Linux.

[Copilot]

Pull request overview

This PR addresses Bug #21964 by making fiber stack acquisition more resilient when the fiber pool cannot be expanded (e.g., due to VMA/map limits), triggering a GC + retry before ultimately raising.

Changes:

• Refactors fiber_pool_expand to return NULL (with errno) on failure instead of raising immediately, allowing callers to decide recovery behavior.
• Adds a slow-path in fiber_pool_stack_acquire to run rb_gc() and retry pool expansion when expansion fails.
• Adds a regression test intended to exercise fiber creation under vm.max_map_count pressure.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
cont.c	Changes fiber pool expansion/acquisition flow to GC+retry on expansion failure and adjusts error handling paths.
test/ruby/test_fiber.rb	Adds a regression test to ensure fiber creation doesn’t raise when pool expansion hits mapping limits.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ioquatix]

LGTM.