Skip to content

Start Data Explorer when stopped #1637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: dev
Choose a base branch
from
Open

Start Data Explorer when stopped #1637

wants to merge 13 commits into from

Conversation

DUeffing
Copy link
Contributor

@DUeffing DUeffing commented May 28, 2025

Fixes #1371 start ADX cluster when it is stopped

📷 Screenshots

added new if activity to check status with web activity to check cluster and to start if necessary
image

🔬 How did you test this change?

  • 🤏 Lint tests
  • 🤞 PS -WhatIf / az validate
  • 👍 Manually deployed + verified
  • 💪 Unit tests
  • 🙌 Integration tests

🙋‍♀️ Do any of the following that apply?

  • 🚨 This is a breaking change.
  • 🤏 The change is less than 20 lines of code.

📑 Did you update docs/changelog.md?

  • ✅ Updated changelog (required for dev PRs)
  • ➡️ Will add log in a future PR (feature branch PRs only)
  • ❎ Log not needed (small/internal change)

📖 Did you update documentation?

  • ✅ Public docs in docs (required for dev)
  • ✅ Internal dev docs in src (required for dev)
  • ➡️ Will add docs in a future PR (feature branch PRs only)
  • ❎ Docs not needed (small/internal change)

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs: Review 👀 PR that is ready to be reviewed label May 28, 2025
@DUeffing DUeffing changed the title Dev start ADX cluster when it is stopped - fix #1371 May 28, 2025
@flanakin flanakin added this to the 2025-05 - May milestone May 28, 2025
typeProperties: {
method: 'POST'
url: {
value: '@if(equals(variables(\'dataExplorerStatus\'),\'Stopped\'),\'${environment().resourceManager}/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Kusto/clusters/${dataExplorerCluster.name}/start?api-version=2024-04-13\',\'${environment().resourceManager}/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Kusto/clusters/${dataExplorerCluster.name}/start?api-version=2024-04-13\')\n \n\n '
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I missing something or is this if() block setting the same value in both conditions? Is that intentional? If so, it seems like we don't need an if() block, right?

Suggested change
value: '@if(equals(variables(\'dataExplorerStatus\'),\'Stopped\'),\'${environment().resourceManager}/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Kusto/clusters/${dataExplorerCluster.name}/start?api-version=2024-04-13\',\'${environment().resourceManager}/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Kusto/clusters/${dataExplorerCluster.name}/start?api-version=2024-04-13\')\n \n\n '
value: '@if(equals(variables(\'dataExplorerStatus\'),\'Stopped\'),\'${environment().resourceManager}${dataExplorerCluster.id}/start?api-version=2024-04-13\',\'${environment().resourceManager}${dataExplorerCluster.id}/start?api-version=2024-04-13\')'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it is intentional at the time being as the if clause is expecting a value for the false case. I could not yet determine a good alternative "Post" command to be executed if the the status of the cluster is stopped. Additionally same as above, the API URL is in the expected format

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if we attempt to start a running cluster? Does it fail or does it noop? I'm guessing it succeeds and does nothing. If so, we could just always attempt to start the cluster and leave it at that.

Perhaps a bigger question is, what happens if starting fails? Do we need to loop to wait for the cluster to start? Do we need to try again if it fails?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I redefined the validation by the following:
image

  1. attempt to start the cluster
  2. the only errors where we have to do a restart are if the cluster is in a not defined state (while starting or stopping),
  3. if so we wait and attempting a new start of the cluster
  4. if this still not work, then we post the error

@flanakin flanakin changed the title start ADX cluster when it is stopped - fix #1371 Start Data Explorer when stopped May 29, 2025
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to update the changelog

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, will be added after all comments are resolved in this PR

@flanakin flanakin modified the milestones: 2025-05 - May, 2025-06 - June Jun 1, 2025
DataExplorer status check and start description for the ingestion_ExecuteETL pipeline

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
DUeffing and others added 11 commits June 2, 2025 09:25
DataExplorer Identity role assignment, adjustment to adhere to Naming conventions

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
remove spaced

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
rename adxStatus variable to adhere to naming conventions

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
change activity name to adhere to naming conventions

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
activity name changed to adhere to naming conventions

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
remove of parameter as not needed

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
remove additional variable to track ADX deployment as an existing one can be used for this

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
Adjust of Activity name to adhere to naming conventions

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
remove parameter for Data Explorer deployment validation as now backed into directly

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
change activity name to adhere to naming conventions

Co-authored-by: Michael Flanakin <flanakin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs: Review 👀 PR that is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Hubs] Start the ADX cluster when shut down
3 participants