fix: resolve e2e import test concurrency races

[Hweinstock]

Description

Two issues in running the e2e tests in parallel:

naming conflict

botocore.errorfactory.ConflictException: An error occurred (ConflictException) when calling the CreateAgentRuntime operation: An agent with the specified name already exists. Use a different name and try again.

Resource names are using time (in seconds) to suffix, but this creates an issue when they create at the same time. Switch to UUID or millisecond precision in resource name suffix.

list then get race condition

[error] Online evaluation configuration not found

The code performs a check that the given evaluator isn't already in an eval config, because if it is, the import will fail.
When iterating through all online eval configs in the account, we call list then get, but its possible an eval config is deleted between the list and get. This will cause the promise to reject with a ResourceNotFoundError. Therefore we can ignore these errors.

Related Issue

Closes ##1066

Documentation PR

N/A

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Other (please describe):

Testing

How have you tested the change?

• I ran npm run test:unit and npm run test:integ
• I ran npm run typecheck
• I ran npm run lint
• If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

[agentcore-cli-automation]

Reviewed the changes — looks good to merge.

The two fixes are well-scoped and correct:

1. Name collision fix: Switching from int(time.time()) to a per-process UUID (uuid.uuid4().hex[:12]) correctly addresses the ConflictException when parallel shards call CreateAgentRuntime within the same second. The fallback to RESOURCE_SUFFIX when set preserves the existing override mechanism, and 48 bits of randomness is plenty for CI collision avoidance. Each shard runs on its own GitHub Actions runner, so there's no concern about bugbash-resources.json being shared across shards (only the AWS account is shared, which was the actual source of the collision).

2. TOCTOU race fix: Swallowing ResourceNotFoundException between listAllOnlineEvaluationConfigs and getOnlineEvaluationConfig is the right call — a config that was deleted mid-iteration can't be holding a lock on the evaluator being imported. The comment in the code explains the rationale clearly. Using instanceof ResourceNotFoundException is a bit stylistically different from the rest of the codebase, which tends to use err.name === 'ResourceNotFoundException', but it's equivalent and arguably more robust.

Very minor nit (not blocking): import time is now unused in setup_evaluator.py, setup_gateway.py, setup_memory_full.py, and setup_runtime_basic.py. Feel free to clean those up if you touch these files again.

[Hweinstock]

removed unused time import. Good call review agent!

[tejaskash]

LGTM

[jesseturner21]

lgtm