Skip to content

[Modules] Add Synapse Workspace module#928

Merged
AlexanderSehr merged 87 commits into
mainfrom
users/repajta/852_synapse_workspace_module
Jul 20, 2022
Merged

[Modules] Add Synapse Workspace module#928
AlexanderSehr merged 87 commits into
mainfrom
users/repajta/852_synapse_workspace_module

Conversation

@renepajta
Copy link
Copy Markdown
Member

@renepajta renepajta commented Jan 25, 2022

Change

Add support for deploying Synapse Workspace. The PR implements basic configuration and more advanced configuration will come in future PRs.

Pipeline reference

Build Status

Dependencies

  • ADLS Gen 2 storage account and file system
  • Azure KeyVault and Encryption key for enabling encryption

Type of Change

  • New feature (non-breaking change which adds functionality)

Checklist

  • I'm sure there are no other open Pull Requests for the same update/change
  • My corresponding pipelines / checks run clean and green without any errors or warnings
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (readme)
  • I did format my code

renepajta and others added 2 commits January 20, 2022 11:35
* Added support for Synapse Private Link Hubs

* Fixed typo in Github workflow

* Create new workflow for deploying Synapse Private Link Hub

* Added support for Locks, RBAC and Private Endpoints

* Updated ReadMe file

* Fixed outputs & updated ReadMe

* Switched back to main branch

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/.bicep/nested_rbac.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/.bicep/nested_rbac.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/deploy.bicep

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Update arm/Microsoft.Synapse/privateLinkHubs/readme.md

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>

* Fixed naming, added ADO pipeline, updated README

* Removed TODOs in ReadMe

* Reversed changes in auto-generated Readme to avoid conflicts

* Remove lock settings from parameter file

* Updated based on comments

* Updated readme.md

* Initial commit for Synapse Workspace module

* Fixed failed Pester tests

* Fixes for failed Pester tests

* Fix: Workspace cannot be created with only UserAssigned identity.

* The default data lake storage account url property should be in the format: https://accountname.dfs.core.windows.net

* Added Diagnostics settings and additional set of parameters

* Add param file to pipeline

* Updated readme file

* Added test cases for encryption configuration

* Added test case for managed vnet

* Updated workspace name to be unique

* Removed unsupported setting - AzureADOnlyAuth

* Added ADO pipeline for Synapse Workspace module

* Created unique names

* Updated resource names, switched from name to resource id for storage account

* Parameter file should not contain the Subscription ID guid

* Changed from resource id to name for default storage account, provided tenant id instead of tenant name

* Updated param reference to Key Vault keys

* Fixed diagnostics account id

* Switched GitHub Workflow to trigger on main branch

* Removed fetch from upstream repo

* Switched back to trigger on current branch for testing

* Switched back to trigger on current branch for testing

* Added missing UAI to parameters file

* Updated process for MSI encryption

* Fixed Pester errors

* Fixed deployment errors

* Deployment fixes

* Deployment fixes

* Deployment fixes

* Deployment fixes

* Deployment fixes

* Slim down the tests

* Deployment fixes

* Deployment fixes

* Deployment fixes 1

* Deployment fixes 2

* Removed testing deployment configuration

* Removed unused code blocks

* Fixed Linter issues

* Updated README

* Added support for Encryption Key Vault with Access Policies

* Trigger on branch to test before merge

* Trigger GH action

* Added all validation steps

* Updated README

* Trigger workflow

* Removed changes

* Revert "Removed changes"

This reverts commit 8cf78a0.

* Revert "Added all validation steps"

This reverts commit 8f95b10.

* Revert "Updated README"

This reverts commit a0b4c49.

* Fixing weird issues

* Updated messsages from Test function

* Fixed missing if condition

* Removed scope fro RBAC assignment

* Removed table formatting

* Added missing parent to access policy

* Resolved code formatting issues

* Updated README

* Moved to dependency keyvault

Co-authored-by: Alexander Sehr <ASehr@hotmail.de>
Comment thread arm/Microsoft.Synapse/workspaces/.bicep/nested_cmk.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/readme.md Outdated
Comment thread arm/Microsoft.Synapse/workspaces/readme.md Outdated
Comment thread arm/Microsoft.Synapse/workspaces/keys/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/keys/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/keys/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/keys/readme.md Outdated
Comment thread arm/Microsoft.Synapse/workspaces/readme.md Outdated
Comment thread arm/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread arm/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread .azuredevops/modulePipelines/ms.synapse.workspaces.yml Outdated
@AlexanderSehr AlexanderSehr added [cat] modules category: modules [cat] pipelines category: pipelines labels Jan 25, 2022
@AlexanderSehr AlexanderSehr added this to the v 0.4 milestone Jan 25, 2022
@AlexanderSehr AlexanderSehr linked an issue Jan 25, 2022 that may be closed by this pull request
Comment thread arm/Microsoft.Synapse/workspaces/.bicep/nested_cmkRbac.bicep
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 25, 2022

Unit Test Results

  1 files  ±    0    1 suites  ±0   15s ⏱️ -27s
11 tests  -   19  11 ✔️  - 19    0 💤 ±    0  0 ±0 
76 runs   - 256  11 ✔️  - 19  65 💤  - 237  0 ±0 

Results for commit c65cedf. ± Comparison against base commit 80087b3.

This pull request removes 30 and adds 11 tests. Note that renamed tests count towards both.
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/authorizationRules] used resource type [namespaces/AuthorizationRules] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/disasterRecoveryConfigs] used resource type [namespaces/disasterRecoveryConfigs] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/ipFilterRules] used resource type [namespaces/ipfilterrules] should use one of the recent API version(s). Currently using [2018-01-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/migrationConfigurations] used resource type [namespaces/migrationConfigurations] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues/authorizationRules] used resource type [namespaces/queues/authorizationRules] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [locks] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [namespaces/queues/authorizationRules] should use one of the recent API version(s). Currently using [2017-04-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [namespaces/queues] should use one of the recent API version(s). Currently using [2021-06-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/queues] used resource type [roleassignments] should use one of the recent API version(s). Currently using [2021-04-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.ServiceBus/namespaces/topics/authorizationRules] used resource type [namespaces/topics/authorizationRules] should use one of the recent API version(s). Currently using [2021-06-01-preview]
…
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces/keys] used resource type [diagnosticsettings] should use one of the recent API version(s). Currently using [2021-05-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces/keys] used resource type [workspaces/keys] should use one of the recent API version(s). Currently using [2021-06-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [diagnosticsettings] should use one of the recent API version(s). Currently using [2021-05-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [locks] should use one of the recent API version(s). Currently using [2016-09-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [privateEndpoints/privateDnsZoneGroups] should use one of the recent API version(s). Currently using [2021-05-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [privateEndpoints] should use one of the recent API version(s). Currently using [2021-05-01]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [roleassignments] should use one of the recent API version(s). Currently using [2020-04-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [roleassignments] should use one of the recent API version(s). Currently using [2021-04-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [vaults/accessPolicies] should use one of the recent API version(s). Currently using [2021-06-01-preview]
/home/runner/work/ResourceModules/ResourceModules/arm/.global/global.module.tests.ps1 ‑ API version tests [All apiVersions in the template should be 'recent'].In [Microsoft.Synapse/workspaces] used resource type [workspaces/keys] should use one of the recent API version(s). Currently using [2021-06-01]
…

♻️ This comment has been updated with latest results.

renepajta and others added 3 commits January 25, 2022 18:21
Co-authored-by: Alexander Sehr <ASehr@hotmail.de>
Co-authored-by: Alexander Sehr <ASehr@hotmail.de>
Co-authored-by: Alexander Sehr <ASehr@hotmail.de>
@AlexanderSehr AlexanderSehr requested a review from eriqua July 18, 2022 12:09
Comment thread .github/workflows/platform.dependencies.yml Outdated
Comment thread modules/Microsoft.Synapse/workspaces/keys/deploy.bicep
Comment thread modules/Microsoft.Synapse/workspaces/keys/deploy.bicep Outdated
Comment thread modules/Microsoft.Synapse/workspaces/deploy.bicep Outdated
Comment thread modules/Microsoft.Synapse/workspaces/deploy.bicep Outdated
AlexanderSehr and others added 10 commits July 18, 2022 15:54
Co-authored-by: Erika Gressi <56914614+eriqua@users.noreply.github.com>
Co-authored-by: Erika Gressi <56914614+eriqua@users.noreply.github.com>
Co-authored-by: Erika Gressi <56914614+eriqua@users.noreply.github.com>
Co-authored-by: Erika Gressi <56914614+eriqua@users.noreply.github.com>
@AlexanderSehr AlexanderSehr requested a review from eriqua July 18, 2022 17:26
Copy link
Copy Markdown
Contributor

@eriqua eriqua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

@eriqua eriqua dismissed stale reviews from AlexanderSehr and MariusStorhaug July 20, 2022 19:03

All comments addressed

@AlexanderSehr AlexanderSehr merged commit 2a3a325 into main Jul 20, 2022
@AlexanderSehr AlexanderSehr deleted the users/repajta/852_synapse_workspace_module branch July 20, 2022 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[cat] modules category: modules [cat] pipelines category: pipelines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement template for Synapse Workspace

5 participants