Improve Azure deployment stability #26
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix formatting in cloud.datahub_template_info Add retry to address intermittent failure in datahub_template_info listing of datahub templates in CDP 7.2.10 Add explicit test for Azure Storage Account being unavailable for use in this deployment remove duplicate usage of __azure_netapp_vol_info as variable Handle edge cases for preparation of Netapp NFS Mount when deploying ML automatically on Azure Introduce wait and retry controls for handling Azure eventual consistency when negotiating between Ansible Controller, Azure Control Plane, and CDP Control Plane Move default Azure minimal policy json from private gist to cloudera-labs snippets Correct Azure App name where sometimes referred to with http:// header and sometimes not, resulting in idempotence failures. Oops. Introduce more robust validations that Service principals and other objects created by az CLI are populated as expected Ensure that Azure objects are consistently bound to the Azure namespace created from the name_prefix Provide more user friendly errors when Azure App and Service Principal creation doesn't go as planned Add tunnel and public endpoint control support to Azure Environment creation in line with AWS offering Fix ML submission preparation to include nfs information following existing combination patterns Swap order of Runtime initialization tasks to handle provider-specific tasks before general tasks, to allow Azure-specific values to be populated. No impact on other implementations Explicitly derive Azure NFS Mount information in Runtime deployment from earlier Infrastructure deployment steps Fix purge teardown edgecase where child services are not deleted if at least one child service is not present in the Definition. Now they are removed if purge is set and any are found regardless of provided Definition plan. cdpy now recognises that CML deployments may fail during provisioning and responds accordingly cdpy will now automatically retry Azure cross-account credential creation in CDP when receiving eventual consistency errors Signed-off-by: Daniel Chaffelson <chaffelson@gmail.com>
This was referenced Jul 27, 2021
wmudge
approved these changes
Jul 30, 2021
acsjumpi
pushed a commit
to acsjumpi/cdpy
that referenced
this pull request
Sep 21, 2021
* Handle eventual consistency errors during Datahub list cluster definitions in CDP * Handle eventual consistency errors during Azure cross-account credential creation in CDP Signed-off-by: Daniel Chaffelson <chaffelson@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No collateral changes to collections
cdpy now recognises that CML deployments may fail during provisioning and responds accordingly
cdpy will now automatically retry Azure cross-account credential creation in CDP when receiving eventual consistency errors
Meta-changelist:
Fix formatting in cloud.datahub_template_info
Add retry to address intermittent failure in datahub_template_info listing of datahub templates in CDP 7.2.10
Add explicit test for Azure Storage Account being unavailable for use in this deployment
remove duplicate usage of __azure_netapp_vol_info as variable
Handle edge cases for preparation of Netapp NFS Mount when deploying ML automatically on Azure
Introduce wait and retry controls for handling Azure eventual consistency when negotiating between Ansible Controller, Azure Control Plane, and CDP Control Plane
Move default Azure minimal policy json from private gist to cloudera-labs snippets
Correct Azure App name where sometimes referred to with http:// header and sometimes not, resulting in idempotence failures. Oops.
Introduce more robust validations that Service principals and other objects created by az CLI are populated as expected
Ensure that Azure objects are consistently bound to the Azure namespace created from the name_prefix
Provide more user friendly errors when Azure App and Service Principal creation doesn't go as planned
Add tunnel and public endpoint control support to Azure Environment creation in line with AWS offering
Fix ML submission preparation to include nfs information following existing combination patterns
Swap order of Runtime initialization tasks to handle provider-specific tasks before general tasks, to allow Azure-specific values to be populated. No impact on other implementations
Explicitly derive Azure NFS Mount information in Runtime deployment from earlier Infrastructure deployment steps
Fix purge teardown edgecase where child services are not deleted if at least one child service is not present in the Definition. Now they are removed if purge is set and any are found regardless of provided Definition plan.
Signed-off-by: Daniel Chaffelson chaffelson@gmail.com