Missing comprehensive docs on worker pool types

[micahjsmith]

Checklist

• I have checked the issues list
  for similar or identical bug reports.
• I have checked the pull requests list
  for existing proposed fixes.
• I have checked the commit log
  to find out if the bug was already fixed in the main branch.
• I have included all related issues and possible duplicate issues in this issue
  (If there are none, check this box anyway).

Description

The docs are missing a comprehensive discussion of the worker pool types.

The docs I have found:

• https://docs.celeryq.dev/en/stable/userguide/workers.html#concurrency which only mentions prefork and eventlet
• https://docs.celeryq.dev/en/latest/reference/cli.html#cmdoption-celery-worker-P which mentions the available pool types but not what they are or how a user should choose
• https://docs.celeryq.dev/en/stable/userguide/configuration.html#worker-pool which is not comprehensive at all

There are also various changelog entries (hard to find, usually announce the very initial version of the pool type but not subsequent changes/improvements) and docstrings exposed from celery.concurrency like https://docs.celeryq.dev/en/latest/internals/reference/celery.concurrency.thread.html (not very descriptive)

Suggestions

I suggest that a new section be written in userguide/workers titled Pools which has a more comprehensive user-facing discussion of pool types including:

• how the pool type works and any limitations it has
• considerations for choosing among pool types

An excellent (but outdated) example of this type of documentation is https://distributedpython.com/posts/celery-execution-pools-what-is-it-all-about/

I'm happy to make a PR with a draft of this but would like to collaborate with someone who has more extensive understanding the different pool types to make sure it is more comprehensive

[open-collective-bot]

Hey @micahjsmith 👋,
Thank you for opening an issue. We will get back to you as soon as we can.
Also, check out our Open Collective and consider backing us - every little helps!

We also offer priority support for our sponsors.
If you require immediate assistance please consider sponsoring us.