[AutomationAPI/Python] Confirm Parallelism Flag Respected

[RobbieMcKinstry]

What happened?

NOTE: This issue was originally discovered and patched the CLI only. See #11116.

This issue tracks the work to confirm that the bug identified in #1116 does not apply to AutoAPI. The hardcoded thread count in the runtime code implies to me that the Automation API will have the same bug as in #1116, but I wasn't able to trace the code deeply enough to be sure. The hardcoding occurs in preview, up, and refresh. If the bug is legit, I don't think the fix is as simple as replacing the hardcoded 4 with the value of parallel because that implies to me the total number of threads will now be twice parallel: set once when constructing the Settings object and once again for handling gRPC requests: there would be two ThreadPoolExecutor allocated, one attached to the default system executor and one provided to the gRPC server. The solution might be to cache the ThreadPoolExecutor created in Settings and passing that to the gRPC server.

From the original issue:

---

Python programs do not respect the --parallel flag. While the --parallel flag is plumbed through to the language host, and even into the the SDK at pulumi/runtime/settings.py, the value is never used.

Certain rpc calls generate blocking futures that the runtime offloads to a worker thread. For example, any use of run_in_executor throws the future into a worker thread. (Here's where we make that call.)

The default executor sets the max number of worker threads using a fixed value scaling based on the number of CPUs on the machine. This is the value that's currently employed.

This issue is to replace the default executor with one that respects the --parallel flag.

Steps to reproduce

Copying the steps from #11116, but deploy the code with the Automation API.

Run the following program. Even though parallelism is unbounded by default, the maximum number of concurrently created resources is limited.

"""An AWS Python Pulumi program"""
import pulumi
import pulumi_aws as aws

# SQS
for i in range(100):
    name = f'pulumi-{str(i).rjust(3,"0")}'
    aws.sqs.Queue(
        name
    )
    
# SNS
for i in range(100):
    name = f'pulumi-{str(i).rjust(3,"0")}'
    aws.sns.Topic(
        name
    )

Expected Behavior

All resources should be created concurrently.

Actual Behavior

Only about 20 resources are created at a time.

Output of pulumi about

CLI          
Version      3.43.2-dev.0
Go Version   go1.19
Go Compiler  gc

Plugins
NAME    VERSION
aws     5.18.0
python  unknown

Host     
OS       darwin
Version  12.6
Arch     arm64

This project is written in python: executable='/Users/robbiemckinstry/workspace/pulumi/pulumi/sdk/python/venv/bin/python3' version='3.10.6
'

Current Stack: dev

TYPE                  URN
pulumi:pulumi:Stack   urn:pulumi:dev::ls::pulumi:pulumi:Stack::ls-dev
pulumi:providers:aws  urn:pulumi:dev::ls::pulumi:providers:aws::default_5_18_0
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-001
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-000
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-003
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-002
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-004
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-006
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-005
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-007
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-009
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-008
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-010
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-011
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-012
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-013
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-014
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-015
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-016
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-017
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-018
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-019
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-020
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-021
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-022
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-023
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-024
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-025
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-026
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-027
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-029
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-028
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-030
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-031
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-032
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-033
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-034
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-035
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-036
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-037
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-038
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-039
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-040
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-041
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-042
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-043
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-044
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-045
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-046
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-047
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-048
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-049
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-050
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-051
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-052
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-053
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-054
aws:sqs/queue:Queue   urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-055


OPP TYPE  URN
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-056
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-057
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-058
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-059
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-060
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-061
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-062
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-063
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-064
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-065
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-066
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-067
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-068
creating  urn:pulumi:dev::ls::aws:sqs/queue:Queue::pulumi-069


Backend        
Name           pulumi.com
URL            https://app.pulumi.com/thesnowmancometh
User           thesnowmancometh
Organizations  thesnowmancometh, whiterabbit, pulumi

Dependencies:
NAME        VERSION
pip         22.3.0
pulumi-aws  5.18.0
setuptools  65.5.0
wheel       0.37.1

Pulumi locates its logs in /var/folders/8n/y4s73s2d4rnbn7d85clz9pz40000gn/T/ by default

Additional context

Original PR:
#11122

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[RobbieMcKinstry]

Preliminary results indicate there isn't a bug here, and the spurious code isn't impacting performance.

[RobbieMcKinstry]

After a sizable experiment, the results indicate the automation API performs similarly to the CLI. I'm going to close this issue as a result.

In this graph, the x-axis represents the value of the --parallel flag, and the y-axis represents time in seconds. One benchmark was used, which creates 200 resources concurrently using AWS Classic.

[pulumi-bot]

Cannot close issue without required labels: resolution/