Repository navigation
Releases: OLC-LOC-Bioinformatics/AzureBatch
Release list
v0.0.33
AzureBatch v0.0.33
This release fixes Azure Batch task submissions that could be incorrectly reported as failures when the Azure SDK returned a successful task status together with an empty error object.
Fixed
- Normalize Azure Batch task-add status values across string and enum representations.
- Recognize success, succeeded, and enum-style success statuses correctly.
- Ignore empty error models attached to otherwise successful task-add results.
- Preserve meaningful task-level errors returned by Azure Batch.
- Prevent successful submissions from triggering automatic pool and job rollback.
- Return clearer Batch API failure information when task submission genuinely fails.
Improved - Log each task-add result, including task ID, status, normalized status value, and error object.
- Log submission exceptions with their complete traceback.
- Log job and pool deletion when submission rollback is performed.
- Report whether cleanup was performed in failed worker responses.
- Preserve configurable output containers and destination prefixes.
- Preserve Nanopore input and output SAS environment variables.
- Preserve Nanopore runtime path, micromamba, GPU VM, and Trusted Launch configuration.
v0.0.32
AzureBatch v0.0.32 improves support for long-running, manifest-driven
Nanopore workflows.
Key changes include scoped input and output SAS credentials, separate result
containers, run-specific output prefixes, Azure SDK task-result validation,
and best-effort cleanup after partial submission failures.
Worker-mode orchestration now returns plain dictionaries rather than Flask
response objects. Existing workflows remain compatible through default output
container and prefix values.
Focused Azure CLI and Nanopore tests: 61 passed.
v0.0.31
AzureBatch Runtime and Trusted Launch Configuration
Added
- Added analysis-specific Batch VM-size defaults.
- Added configurable task runtime paths through
FOODPORT_RUNTIME_PATH. - Added Nanopore micromamba runtime support.
- Added
MAMBA_ROOT_PREFIXto Nanopore Batch tasks. - Added support for placing the PoreSippR environment, micromamba, and Dorado on the Batch task
PATH.
Fixed
- Restored Trusted Launch for COWBAT, AmpliSeq, and COWSNPhR pools created from Trusted Launch-only Compute Gallery images.
- Preserved the Nanopore Trusted Launch configuration with Secure Boot and vTPM disabled.
- Prevented the CLI default CPU VM size from overriding the configured Nanopore A10 GPU size.
- Updated Batch task submission to pass runtime configuration into each task.
Compatibility
- Existing Miniconda workflows continue to receive:
CONDA=/usr/bin/miniconda/binFOODPORT_RUNTIME_PATH=/usr/bin/miniconda/bin
- Nanopore workflows receive:
CONDA=/opt/micromamba/binMAMBA_ROOT_PREFIX=/opt/micromamba/root- the PoreSippR micromamba environment and Dorado directories on
PATH
Testing
- Expanded unit-test coverage for Trusted Launch, UEFI overrides, VM-size selection, legacy Conda tasks, Nanopore micromamba tasks, and CLI orchestration.
- All 68 tests pass.
v0.0.30
AzureBatch
Fixed
- Restored Trusted Launch for all Azure Batch pool types.
- Fixed allocation failures affecting COWBAT, AmpliSeq, and COWSNPhR pools created from Trusted Launch-only Compute Gallery images.
- Preserved the Nanopore pool configuration using Trusted Launch with Secure Boot and vTPM disabled for NVIDIA GRID driver compatibility.
- Expanded unit-test coverage for common and Nanopore-specific security profiles.
v0.0.29
v0.0.29
This patch release adds explicit Azure Batch UEFI security configuration for Nanopore GPU pools.
Added
- Added support for configuring Secure Boot separately for Nanopore Batch nodes.
- Added support for configuring the virtual Trusted Platform Module separately for Nanopore Batch nodes.
- Added strict parsing for boolean settings supplied as strings, booleans, or numeric-style values.
- Added validation for invalid boolean configuration values.
- Added comprehensive tests for Nanopore UEFI defaults, explicit overrides, and generated Azure Batch security profiles.
Changed
Nanopore Batch pools continue to use the Trusted Launch security type while explicitly disabling Secure Boot and vTPM by default.
The resulting security configuration is:
Security type: trustedLaunch
Secure Boot: false
vTPM: false
This configuration supports the Trusted Launch-compatible Generation 2 image definition while preparing the Nanopore pool for the NVIDIA GRID driver required by the NVadsA10_v5 VM family.
Configuration
The following settings are now supported:
NANOPORE_SECURITY_TYPE=trustedLaunch
NANOPORE_SECURE_BOOT_ENABLED=false
NANOPORE_VTPM_ENABLED=false
If the Secure Boot and vTPM settings are omitted for a Nanopore analysis, both default to false.
Existing analysis types continue to omit an explicit security profile unless one is configured.
Boolean values
The new boolean parser accepts the following case-insensitive values:
True values:
true
1
yes
on
False values:
false
0
no
off
Native Python boolean values are also accepted.
Unsupported values produce a clear configuration error instead of being interpreted implicitly.
Azure Batch integration
Nanopore pool creation now generates an Azure Batch security profile equivalent to:
security_type = trustedLaunch
secure_boot_enabled = false
v_tpm_enabled = false
The implementation uses the UefiSettings model provided by the pinned azure-batch 14.2.0 package.
Existing COWBAT, AmpliSeq, and COWSNPhR pool behavior remains unchanged.
Tests
The test suite now covers:
- boolean values supplied as native booleans
- lowercase and uppercase string values
- yes and no values
- numeric-style values
- on and off values
- default values
- invalid configuration values
- Nanopore Secure Boot defaults
- Nanopore vTPM defaults
- explicit UEFI overrides
- Trusted Launch security-profile construction
- compatibility with existing non-Nanopore workflows
- Nanopore VM-size defaults and explicit overrides
Test result:
61 passed
Compatibility
- No public AzureBatch API changes are included.
- No changes are required for COWBAT, AmpliSeq, or COWSNPhR configurations.
- Existing non-Nanopore pools continue to omit an explicit security profile.
- The Conda package continues to support Python 3.12 and Python 3.13.
- Automated package builds and tests use Python 3.12.
- The implementation remains compatible with
azure-batch14.2.0.
Deployment notes
After publishing the updated Conda package:
- Update the Batch container image to install
azure_batch=0.0.29. - Add the new Nanopore UEFI settings to the FoodPort configuration.
- Rebuild and restart the Batch service container.
- Confirm that version 0.0.29 is installed in the container.
- Continue with the Nanopore image 0.0.2 NVIDIA GRID layer.
- Validate the completed image on a
Standard_NV18ads_A10_v5Azure Batch node.
Upgrade notes
Add the following values to the settings supplied to AzureBatch:
NANOPORE_SECURITY_TYPE=trustedLaunch
NANOPORE_SECURE_BOOT_ENABLED=false
NANOPORE_VTPM_ENABLED=false
No other application configuration changes are required when upgrading from v0.0.28.
v0.0.28
v0.0.28
This patch release fixes local Conda channel indexing during post-build package validation.
Fixed
- Fixed the post-build installation test failing because the
conda-indexcommand was unavailable in the build environment. - Added
conda-indexas an explicitly pinned build-tool dependency. - Updated local channel indexing to use the documented Python module entry point.
- Ensured that the newly built AzureBatch package is added to the local channel metadata before installation testing.
Changed
The automated release environment now installs:
Python 3.12
conda-build 25.1.1
conda-index 0.12.1
anaconda-client 1.14.1
Local channel metadata is now generated using:
python -m conda_index <local-channel-root>
The workflow continues to derive the local channel root from the generated package path.
For a generated package such as:
/usr/share/miniconda/envs/azure_batch_build/conda-bld/noarch/azure_batch-0.0.28-py_0.conda
the local channel root is:
/usr/share/miniconda/envs/azure_batch_build/conda-bld
and the channel URL is:
file:///usr/share/miniconda/envs/azure_batch_build/conda-bld
Validation
The release workflow now performs the following post-build checks:
- Confirms that the expected
.condapackage exists. - Displays the package path and SHA-256 checksum.
- Determines the local package directory and channel root.
- Displays available package and channel metadata files.
- Regenerates local channel metadata using
conda-index. - Confirms that the requested AzureBatch version is visible through
conda search. - Creates a clean Python 3.12 test environment.
- Installs the locally built package with full dependency resolution.
- Confirms that the
azure_batchmodule can be imported. - Runs
AzureBatch -h. - Displays the installed AzureBatch package record.
Channel configuration
The post-build installation test uses the following channel order:
explicit local build channel
conda-forge
bioconda
olcbioinformatics
The locally built AzureBatch package is selected from the explicit file channel, while its runtime dependencies are resolved from the configured remote channels.
Network resilience
The workflow retains timeout and retry handling for transient failures involving:
- Conda build-tool installation
- recipe rendering
- package output-path calculation
- package building and recipe tests
- dependency resolution
- local package installation
- source archive downloads
- package publication
Conda index caches are cleared between applicable retry attempts.
Test status
The complete AzureBatch source test suite passes using Python 3.12:
43 passed
The Conda recipe also validates:
- package dependency resolution
- wheel generation
- Conda package creation
- package installation
- Python module imports
- command-line help
- packaged unit tests
Compatibility
- No AzureBatch API changes are included in this release.
- No Azure Batch pool, job, or task behavior is changed.
- No Azure Storage runtime behavior is changed.
- No deployment configuration changes are required.
- Existing AzureBatch workflows remain compatible.
- The Conda package continues to support Python 3.12 and Python 3.13.
- Automated package builds and installation tests use Python 3.12.
Upgrade notes
No application configuration changes are required when upgrading from v0.0.27.
This release supersedes v0.0.27 for automated Conda publication. Version 0.0.27 included explicit local-channel testing, but the build environment did not include the separate conda-index package required to regenerate that channel's metadata.
v0.0.27
v0.0.27
This patch release includes the corrected Conda publication workflow for testing installation of the newly built package.
Fixed
- Fixed post-build installation tests failing because the generated AzureBatch package could not be found through Conda's generic
localchannel. - Replaced implicit
--use-localpackage discovery with an explicit local channel derived from the generated package path. - Ensured the corrected publication workflow is included in the release tag.
Changed
The workflow now derives the local Conda channel from PACKAGE_PATH.
For a generated package such as:
/usr/share/miniconda/envs/azure_batch_build/conda-bld/noarch/azure_batch-0.0.27-py_0.conda
the workflow uses the following channel root:
/usr/share/miniconda/envs/azure_batch_build/conda-bld
and supplies it to Conda as:
file:///usr/share/miniconda/envs/azure_batch_build/conda-bld
Validation
The publication workflow now:
- Confirms that the expected
.condapackage exists. - Determines the package directory and local channel root.
- Rebuilds the local channel metadata with
conda-index. - Confirms that the requested AzureBatch version is visible through
conda search. - Creates a clean Python 3.12 test environment.
- Installs the locally built package with full dependency resolution.
- Confirms that the installed package can be imported.
- Runs
AzureBatch -hto validate the installed command-line interface. - Displays the installed AzureBatch package record.
Dependency resolution
The post-build installation test uses the following channel order:
explicit local build channel
conda-forge
bioconda
olcbioinformatics
The AzureBatch package is selected from the explicit local channel, while its runtime dependencies are resolved from the configured remote channels.
Test status
The complete source test suite passes using Python 3.12:
43 passed
The Conda recipe also successfully:
- resolves its build and test environments
- builds the AzureBatch wheel
- creates the
.condapackage - installs the package into the recipe test environment
- imports the
azure_batchmodule - runs the AzureBatch command-line help
- executes all packaged unit tests
Network resilience
The workflow retains retry handling for transient failures involving:
- Conda build-tool installation
- recipe rendering
- package output-path calculation
- package building and recipe tests
- dependency resolution
- local installation testing
- source archive downloads
- package publication
Conda index caches are cleared between applicable retry attempts.
Compatibility
- No AzureBatch API changes are included in this release.
- No Azure Batch pool, job, or task behavior is changed.
- No Azure Storage runtime behavior is changed.
- No deployment configuration changes are required.
- Existing AzureBatch workflows remain compatible.
- The Conda package continues to support Python 3.12 and Python 3.13.
- Automated package builds and installation tests use Python 3.12.
Upgrade notes
No application configuration changes are required when upgrading from v0.0.26.
This release supersedes v0.0.26 for Conda publication. The previous release tag did not contain the committed explicit local-channel workflow changes.
v0.0.26
v0.0.26
This patch release fixes the Conda publication workflow's post-build installation test.
Fixed
- Fixed local package installation failing with
PackagesNotFoundInChannelsError. - Fixed the workflow relying on
--use-local, which searched Conda's default local build channel rather than the build directory used by the isolatedazure_batch_buildenvironment. - Updated the workflow to derive the actual local Conda channel from the generated package path.
- Ensured the newly built
azure_batchpackage is discoverable before the installation test begins.
Changed
- The workflow now determines the local channel root from
PACKAGE_PATH. - The local build channel is provided to Conda using an explicit
file://URL. - Channel metadata is refreshed with
conda-indexbefore package installation. - The workflow verifies that the requested AzureBatch version is visible in the local channel using
conda search. - The installation test continues to resolve all runtime dependencies rather than installing the package archive with dependency checking disabled.
- Added diagnostic output showing:
- the generated package path
- the local package directory
- the local channel root
- the local channel URL
- package and channel metadata files
- the installed AzureBatch package record
Local channel resolution
For a generated package such as:
/usr/share/miniconda/envs/azure_batch_build/conda-bld/noarch/azure_batch-0.0.26-py_0.conda
the workflow now uses the following local channel root:
/usr/share/miniconda/envs/azure_batch_build/conda-bld
and supplies it to Conda as:
file:///usr/share/miniconda/envs/azure_batch_build/conda-bld
This allows Conda to discover the package under the channel's noarch directory while resolving its dependencies from the configured remote channels.
Dependency resolution
The installation test uses the following channel order:
local build channel
conda-forge
bioconda
olcbioinformatics
The locally built AzureBatch package is selected from the explicit local channel, while its runtime dependencies continue to be resolved normally.
Validation
The workflow now validates the locally built package by:
- Confirming that the expected
.condapackage exists. - Refreshing the local channel metadata.
- Confirming that
azure_batch=0.0.26is visible in the local channel. - Creating a clean Python 3.12 test environment.
- Installing the locally built package with full dependency resolution.
- Importing the
azure_batchPython package. - Running
AzureBatch -h. - Displaying the installed package record.
Network resilience
Existing retry handling remains enabled for transient failures involving:
- Conda build-tool installation
- recipe rendering
- package output-path calculation
- package building
- dependency resolution
- installation testing
- source archive downloads
- package publication
The local package visibility check now fails immediately with useful diagnostics if the generated package is absent from the expected build channel.
Compatibility
- No AzureBatch application programming interface changes are included.
- No Azure Batch pool, job, or task behavior is changed.
- No Azure Storage runtime behavior is changed.
- No deployment configuration changes are required.
- Existing AzureBatch workflows remain compatible.
- The Conda package continues to support Python 3.12 and Python 3.13.
- Automated package builds and installation tests use Python 3.12.
Upgrade notes
No application configuration changes are required when upgrading from v0.0.25.
This release supersedes v0.0.25 for automated Conda publication. Version 0.0.25 built and passed its recipe tests successfully, but its post-build installation check could not locate the generated package through Conda's generic local channel.
v0.0.25
v0.0.25
This patch release ensures that the corrected credential-independent Azure Storage tests are included in the published source archive and Conda package validation.
Fixed
- Fixed the published release source containing the earlier credential-dependent
test_prep_output_containerimplementation. - Replaced real container SAS signing in the packaged test suite with a mocked
generate_container_sascall. - Removed the output-container test's dependency on
AZURE_ACCOUNT_NAMEandAZURE_ACCOUNT_KEYenvironment variables. - Prevented isolated Conda package tests from failing when Azure Storage credentials are unavailable.
Added
- Added assertions confirming that the expected Blob Storage container is created.
- Added assertions confirming that SAS generation receives the configured account name, account key, and container name.
- Added assertions confirming that read and write SAS permissions are requested.
- Added validation that a SAS expiry time is supplied.
- Added validation of the final container SAS URL.
- Added validation that existing non-Nanopore pools omit an explicit security profile.
- Added validation that Batch pools use one task slot per node.
Test results
The complete test suite passes using Python 3.12:
43 passed
The updated unit tests do not require:
- live Azure resources
- Azure Storage credentials
- network connectivity
- a local environment file
Compatibility
- No AzureBatch API changes are included.
- No Azure Storage runtime behavior is changed.
- No Azure Batch pool, job, or task behavior is changed.
- No deployment configuration changes are required.
- Existing AzureBatch workflows remain compatible.
- The Conda package continues to support Python 3.12 and Python 3.13.
- Automated Conda package builds and tests use Python 3.12.
Upgrade notes
No application configuration changes are required when upgrading from v0.0.24.
This release supersedes v0.0.24, whose release tag contained the earlier credential-dependent Azure Storage test.
v0.0.24
v0.0.24
This patch release fixes the Azure Storage unit tests used during Conda package validation.
Fixed
- Fixed
test_prep_output_containerfailing in isolated Conda test environments when Azure Storage credentials were unavailable. - Removed the test's dependency on
AZURE_ACCOUNT_NAMEandAZURE_ACCOUNT_KEYenvironment variables. - Replaced real SAS token generation with a mocked
generate_container_sascall. - Updated the legacy pool test to provide the explicit security-profile and VM-size attributes expected by the current pool creation implementation.
Added
- Added assertions confirming that output-container creation requests the expected container name.
- Added assertions confirming that SAS generation receives the configured storage account name and account key.
- Added assertions confirming that the generated SAS token requests read and write permissions.
- Added validation that an expiry time is supplied when generating the container SAS token.
- Added validation of the final Blob Storage container SAS URL.
- Added coverage confirming that existing non-Nanopore workflows omit the VM security profile.
- Added coverage confirming that Batch pools use one task slot per node.
Changed
- Azure Storage unit tests now use explicit non-secret test values rather than local environment files or CI credentials.
test_prep_output_containernow tests application behavior without invoking Azure SDK token-signing logic.- The existing
create_pooltest now reflects configuration-driven security profiles and GPU task-slot behavior.
Test results
The updated test suite validates:
- output-container creation
- SAS token argument construction
- read and write SAS permissions
- SAS token expiry configuration
- container SAS URL formatting
- explicit Batch VM security profiles
- compatibility with existing workflows
- one task slot per Batch node
The expected Conda package test result is:
43 passed
Compatibility
- No AzureBatch application programming interface changes are included in this release.
- No Azure Storage runtime behavior is changed.
- No Azure Batch pool, job, or task behavior is changed.
- No deployment configuration changes are required.
- Existing AzureBatch workflows remain compatible.
- The Conda package continues to support Python 3.12 and Python 3.13.
- Automated package builds and tests use Python 3.12.
Upgrade notes
No application configuration changes are required when upgrading from v0.0.23.
This release is recommended because it removes the final dependency on live Azure Storage credentials from the Conda package unit tests.