shared-compute implementation to support multiple GPU SKUs

[mgmachado]

Description

Adds multi-tenant shared GPU Batch pools so many environments can share scarce GPU quota instead of fragmenting it across one pool per environment, plus the v2.1.0 hastelib routing + per-job SAS changes that make it work.

• Shared pools (IaC): a parameterized batchPool.bicep (fixed/autoscale × dedicated/low-priority × optional VNet) and a new standalone shared-pools.bicep + shared-pools.bicepparam that provision an H100 + T4 pool pair per group (dev, demo, …), autoscaling on low-priority nodes and scaling to zero when idle.
• Data isolation: each Batch job mints a short-lived user-delegation SAS scoped to its own storage container; the pool holds no standing storage access (ACR-pull-only identity). A tenant's task cannot read another tenant's data — enforced by Azure at the credential boundary, not by app code.
• Capacity-aware routing: hastelib now takes ordered candidate pool lists and binds a pool at submit time (select_pool) — preferred tier first, spillover to a tier with a free node.
• Reusability: IaC defaults are genericized (HASTE_RESOURCE_PREFIX → neutral haste, BYO account/ACR), so other partners can deploy without editing the templates.

All new behavior is behind per-environment flags that default to the existing single-pool, pool-identity path, so existing environments are unaffected until they explicitly opt in.

Full design: spec/features/batch-compute-expansion/.

Fixes #

Type of change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that changes existing behavior)
• Documentation update
• Infrastructure / CI change

Checklist

• I have read CONTRIBUTING.md
• My changes follow the project's coding standards (PEP 8 for Python, ESLint rules for JS/TS)
• I have added or updated tests that cover my changes
• Python tests pass locally (cd hastelib && hatch run test:pytest) and the UI lints clean (cd ui && npm run lint) — new unit tests pass (9/9); full suite to run in CI (conda env)
• I have updated the relevant documentation (README, docs/, inline comments)
• I have added an entry to CHANGELOG.md if this is a user-facing change

Testing

• Unit (new): hastelib/tests/core/runners/test_azure_batch_routing.py — 9 tests covering select_pool (single / spillover-to-idle / preferred fallback), the SAS-vs-identity blob-credential toggle, and config candidate-list parsing. All pass.
• Isolation PoC (real Azure Batch): proved SAS-based ResourceFile/OutputFile I/O works with no identity_reference; a cross-tenant SAS was denied (BlobAccessDenied); the ACR-pull-only shared identity authenticated and pulled the training image.
• IaC: batchPool.bicep + shared-pools.bicep compile clean; what-if shows no drift against the live pools.
• Not yet run: end-to-end on two live environments — gated on a low-priority GPU quota increase (tracked in rollout.md).

Additional context

• No behavior change for existing envs. Opt-in per env via AZURE_BATCH_USE_SAS=true, AZURE_BATCH_MANAGE_POOLS=false, and AZURE_BATCH_{TRAINING,INFERENCE,IMAGERYPREP}_POOL_IDS; all default to legacy.
• Storage Blob Delegator is granted to the api/queues identity (in functionApp.bicep) so it can mint the SAS.
• azure-batch pinned to ==14.2.0 — the 15.x track-2 rewrite restructures the models this code uses; migration tracked separately.
• Shared-dev pools are already provisioned (0 nodes, scale-to-zero) and idle-cost-free while waiting on the quota bump.
• Spec is template-conformant: README, plan, impact-analysis, user-stories, design, test-plan, rollout.