AGC integration fixes

[adamnovak]

Changelog Entry

To be copied to the draft changelog by merger:

• Toil no longer stores state in S3 at keys with an extra leading /. (S3 State Store for WES server sticks a leading slash into key names instead of storing state where it is supposed to go #4109)
• Added more tests for Toil's server mode and its integration with AGC components.
• Reduced the time to time out CANCELING WES workflows to CANCELED.
• Actually handle the case where the Toil workflow takes longer than our timeout to clean up after the Celery task is canceled. (Celery tasks end with an exception when canceled instead of finishing properly #4111).

Reviewer Checklist

• Make sure it is coming from issues/XXXX-fix-the-thing in the Toil repo, or from an external repo.
  • If it is coming from an external repo, make sure to pull it in for CI with:

    contrib/admin/test-pr otheruser theirbranchname issues/XXXX-fix-the-thing
    

  • If there is no associated issue, create one.
• Read through the code changes. Make sure that it doesn't have:
  • Addition of trailing whitespace.
  • New variable or member names in camelCase that want to be in snake_case.
  • New functions without type hints.
  • New functions or classes without informative docstrings.
  • Changes to semantics not reflected in the relevant docstrings.
  • New or changed command line options for Toil workflows that are not reflected in docs/running/{cliOptions,cwl,wdl}.rst
  • New features without tests.
• Comment on the lines of code where problems exist with a review comment. You can shift-click the line numbers in the diff to select multiple lines.
• Finish the review with an overall description of your opinion.

Merger Checklist

• Make sure the PR passes tests.
• Make sure the PR has been reviewed since its last modification. If not, review it.
• Merge with the Github "Squash and merge" feature.
  • If there are multiple authors' commits, add Co-authored-by to give credit to all contributing authors.
• Copy its recommended changelog entry to the Draft Changelog.
• Append the issue number in parentheses to the changelog entry.

[adamnovak]

This should fix #4109.

[adamnovak]

Everything passed except the Cactus integration test, which my fix in #4107 turns out not to have fixed. I am going to merge this.

[adamnovak]

Oh no I let the release panic get ahead of me; this hasn't been reviewed yet.

[adamnovak]

I walked the merge back; please review these changes in #4113 so we can merge just one thing.