GOBBLIN-1524: Limit the number of container launch threads used by YarnService

[sv2000]

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-1524

Description

• Here are some details about my PR, including screenshots (if applicable):
  Currently, there are no limits on the number of threads used, which can result in OOMEs with "unable to create a new native thread". This PR makes the number of threads configurable with a default of 1024.

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:
  Trivial change. Existing tests should be sufficient.

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 46.42%. Comparing base (44c7316) to head (e9f45b1).
⚠️ Report is 715 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3375      +/-   ##
============================================
- Coverage     46.43%   46.42%   -0.01%     
+ Complexity    10159    10155       -4     
============================================
  Files          2056     2056              
  Lines         80080    80083       +3     
  Branches       8947     8947              
============================================
- Hits          37186    37181       -5     
- Misses        39448    39453       +5     
- Partials       3446     3449       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[autumnust]

+1