[Storage] Support Oauth/StaticWeb in Powershell official release #7081

blueww · 2018-08-28T10:02:46Z

Description

Oauth design pass review in: https://github.com/Azure/azure-powershell-cmdlet-review-pr/issues/37 , milestone 1
StaticWeb design pass review in Azure/azure-powershell-cmdlet-review-pr#88

Since the 2 features both need XSCL upgrade to 9.3, and most of the changes are on XSCL upgrade, so raise them in same PR. (actually the code change al ready pass review and merge to storage branch in #5960 and #6518)

Checklist

I have read the Submitting Changes section of CONTRIBUTING.md
The title of the PR is clear and informative
The appropriate change log has been updated
The PR does not introduce breaking changes

[Wei] The breaking change is remove.

If applicable, the changes made in the PR have proper test coverage

[Wei] Will add cases after #7037 is in.

For public API changes to cmdlets:
- the changes have gone through a cmdlet design review
- the cmdlet markdown files were generated using the platyPS module

blueww · 2018-08-29T10:21:24Z

@markcowl ， @MiYanni
The netcore build fail with following error, I even tried to upgrade the XSCL to 9.3 to align with storage project, but still fail. Any idea?

[xUnit.net 00:00:07.89] Microsoft.Azure.Commands.Compute.Test.ScenarioTests.ComputeCloudExceptionTests.RunComputeCloudExceptionTests [FAIL]

Failed Microsoft.Azure.Commands.Compute.Test.ScenarioTests.ComputeCloudExceptionTests.RunComputeCloudExceptionTests

Error Message:

System.Management.Automation.RuntimeException : Expected exception is not like the expected text 'Resourcenot found*OperationID : *', the actual message is 'The term 'Get-AzVM' is not recognized as the name of a cmdlet, function, script file, or operable program.

maddieclayton · 2018-08-29T21:32:52Z

@blueww Looks like it is related to this error:
/usr/share/dotnet/sdk/2.1.400/Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3277: Found conflicts between different versions of "Microsoft.WindowsAzure.Storage" that could not be resolved. These reference conflicts are listed in the build log when log verbosity is set to detailed. [/home/travis/build/Azure/azure-powershell/src/ResourceManager/Compute/Commands.Compute.Test/Commands.Compute.Test.Netcore.csproj]

blueww · 2018-08-30T10:32:22Z

@maddieclayton

I have upgrade both Commands.Storage.Test.Netcore.csproj and Commands.Storage.Netcore.csproj to XSCL 9.3.0, but the error still happen.

in .net, we only need to upgrade XSCL for storage project; but in netcore, seems we need to upgrade to many projects. Could we make it also only need upgrade XSCL for storage project?
And for this PR, could you please point out which projects we need to upgrade XSCL when upgrade XSCL for storage?

Thanks!

MiYanni · 2018-08-30T16:48:44Z

@blueww In NetCore, every module has to rely on the same version of any dependency. The Storage dependency used in one place needs to be updated everywhere it is used.

blueww · 2018-08-31T10:25:50Z

@MiYanni , @maddieclayton
I have searched for all netcore projects which reference "WindowsAzure.Storage" 9.0, and make them all reference 9.3, but the netcore build (test) still fail with following error, any idea?

PowerShell Error Record: Could not load file or assembly 'Microsoft.WindowsAzure.Storage, Version=9.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

Exception:System.IO.FileLoadException: Could not load file or assembly 'Microsoft.WindowsAzure.Storage, Version=9.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

at System.Management.Automation.Runspaces.InitialSessionState.Bind_LoadAssemblies(ExecutionContext context)

at System.Management.Automation.Runspaces.InitialSessionState.Bind(ExecutionContext context, Boolean updateOnly, PSModuleInfo module, Boolean noClobber, Boolean local)

at System.Management.Automation.Runspaces.InitialSessionState.Bind(ExecutionContext context, Boolean updateOnly)

at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModuleManifest(String moduleManifestPath, ExternalScriptInfo scriptInfo, Hashtable data, Hashtable localizedData, ManifestProcessingFlags manifestProcessingFlags, Version minimumVersion, Version maximumVersion, Version requiredVersion, Nullable`1 requiredModuleGuid, ImportModuleOptions& options, Boolean& containedErrors)

The projects has been upgraded to XSCL 9.3 are following, any missing

?
C:\code\PSH_Dev\src\Common\Commands.Common.Storage\Common.Storage.Netcore.csproj(29):
C:\code\PSH_Dev\src\Common\Commands.Common.Storage.Management\Common.Storage.Management.Netcore.csproj(25):
C:\code\PSH_Dev\src\ResourceManager\ApiManagement\Commands.ApiManagement\Commands.ApiManagement.Netcore.csproj(33):
C:\code\PSH_Dev\src\ResourceManager\AzureBatch\Commands.Batch\Commands.Batch.Netcore.csproj(34):
C:\code\PSH_Dev\src\ResourceManager\AzureBatch\Commands.Batch.Test\Commands.Batch.Test.Netcore.csproj(30):
C:\code\PSH_Dev\src\ResourceManager\Compute\Commands.Compute\Commands.Compute.Netcore.csproj(37):
C:\code\PSH_Dev\src\ResourceManager\RedisCache\Commands.RedisCache\Commands.RedisCache.Netcore.csproj.orig(35):
C:\code\PSH_Dev\src\ResourceManager\Storage\Commands.Management.Storage\Commands.Management.Storage.Netcore.csproj(37):
C:\code\PSH_Dev\src\Storage\Commands.Storage\Commands.Storage.Netcore.csproj(32):
C:\code\PSH_Dev\src\Storage\Commands.Storage.Test\Commands.Storage.Test.Netcore.csproj(27):

MiYanni · 2018-08-31T10:52:57Z

@blueww Please do not change any Common files in this repo as they are not used. The equivalent files for the ones you changed are here in our azure-powershell-common repo:
https://github.com/Azure/azure-powershell-common/blob/develop/src/Storage/Storage.Netcore.csproj
https://github.com/Azure/azure-powershell-common/blob/develop/src/Storage.Management/Storage.Management.Netcore.csproj

I will be removing Common code from this codebase within the next couple business days. I also plan on consolidating references to the Common packages in this codebase. If you make the changes to those files in azure-powershell-common, and create a build here, https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/ps-common-sign/, you can drop the packages into the tools/LocalFeed folder, and replace the references to 1.0.94-preview in this codebase with whichever version you get from the packages. Then, you should be good.

blueww · 2018-08-31T11:29:13Z

@MiYanni
I am not sure why the following files are common file, they seems should be storage project file, and should not be common.

https://github.com/Azure/azure-powershell-common/blob/develop/src/Storage/Storage.Netcore.csproj
https://github.com/Azure/azure-powershell-common/blob/develop/src/Storage.Management/Storage.Management.Netcore.csproj

And it seems I don't have direct write access to them. should I raise a PR for it?
Would you please point out all files that I should change in azure-powershell-common, and I will send a PR for it together?

And would you please point out all Packages that I should get from https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/ps-common-sign/, for XSCL , or "Microsoft.Azure.Management.Storage" upgrade.
And is there any way that I can test it before the PR to azure-powershell-common is merged.

And is there any formal doc for this process? I still have many questions. I will schedule a meeting to discuss it with you.

MiYanni · 2018-08-31T11:59:31Z

@blueww These are common SDK dlls for the Azure PowerShell runtime. The only difference is we moved the projects to a separate repo and are going to make them provided by a MyGet feed for this repo. Storage and Storage.Management are dataplane and management DLLs respectively of 'locked' versions of those SDKs that are shared by the modules within this repo. Otherwise, if someone wanted to update an SDK and several modules referenced that SDK, every time the SDK version was changed, it would require them to re-record all their tests. This way, we lock a shared version of the SDK so that other modules can still use the functionality there without worrying about SDK updates. This has been how anything in the Common folder in this repo has worked for a long time. Again, that folder will be removed very soon.

For the azure-powershell-common repo, simply use https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/ps-common-sign/ to make the packages, and follow the steps I mentioned above. You don't need a PR, just a fork to that repo for now. Prior to this PR being accepted, those changes would need to be accepted into azure-powershell-common. There is no documentation yet as this process is in-flux, as we are transitioning to making Az (NetCore/NetStandard) our primary focus. All documentation will be updated as we transition to Az.

Additionally, the intent of azure-powershell-common is for us to use to manage the Azure PowerShell runtime of this repo. Services should not need to modify this repo often (if ever). The only reason you need to now is that WindowsAzure.Storage is referenced there. We have an open issue to remove WindowsAzure.Storage usage entirely from (almost) everything for Az. So many modules show not rely on it, or this situation happens. Only Storage modules should use this dll. It is an issue from when we were converting modules that we haven't resolved yet. Sorry for the inconvenience.

blueww · 2018-08-31T12:12:55Z

Thanks @MiYanni for the reply!

Just to confime:

For XSCL upgrade, I should

Upgrade XSCL in my own branch of azure-powershell-common
https://github.com/Azure/azure-powershell-common/blob/develop/src/Storage/Storage.Netcore.csproj
https://github.com/Azure/azure-powershell-common/blob/develop/src/Storage.Management/Storage.Management.Netcore.csproj
Get a build on https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/ps-common-sign/
Update the 2 packages to PSH code, could you point out which project should be update (Any sample PR)? Currently only update .net project for Storage and Profile.

For SRP SDK ("Microsoft.Azure.Management.Storage") upgrade, should:

Also update it in the 2 files in my own branch of azure-powershell-common
Get build on https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/ps-common-sign/
Update Powershell code, which project should update? Only storage and profile?

MiYanni · 2018-08-31T13:16:41Z

@blueww 1. The current version number of the common packages in this repo, I believe, is 1.0.94. Right now, from the build from your fork, simply replace all common packages in LocalFeed. Then, for all of azure-powershell, just do a find/replace for 1.0.94 with your build number (it is created automatically from the sign build). We will have individual package replacement, but we don't do that yet (I'm working on it).
2. I don't believe any Netcore builds in azure-powershell-common use Microsoft.Azure.Management.Storage. Double check that as I might be wrong.

blueww · 2018-09-04T10:23:00Z

@MiYanni

I have tried to change all azure-powershell-common packages to 1.0.96, but build still fail.
I think we still need a meeting to go through the change process.
I have schedule a meeting on Tuesday 6:00pm Redmond time, propose a new time if the time is not proper for you.

MiYanni · 2018-09-04T20:44:47Z

demand: https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/powershell-demand/1078/
ps-sign: https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/ps-sign/1155/
psnetsign: https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/psnetsign/151/

# Conflicts: # src/Common/Commands.Common.Storage.Management/Common.Storage.Management.Netcore.csproj # src/Common/Commands.Common.Storage/Common.Storage.Netcore.csproj

MiYanni · 2018-09-05T00:34:42Z

@azuresdkci Retest this please

blueww · 2018-09-05T09:10:42Z

@MiYanni
Thanks for fixing this!
I have add another feture to this PR (static Web), since the feature also need upgrade XSCL to 9.3. This feature also pass review already in the storage branch.
On Demand: https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/powershell-demand/1082/
ps-sign: https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/ps-sign/1159/
psnetsign: https://azuresdkci.westus2.cloudapp.azure.com/view/PowerShell/job/psnetsign/153/

Please help to review and merge it ASAP. Let me know if any thing need change.

MiYanni · 2018-09-05T17:49:08Z

src/Storage/Commands.Storage/Common/Cmdlet/DisableAzureStorageStaticWebsite.cs

+    /// <summary>
+    /// Disable azure storage service StaticWebsite, currently only available on Blob service
+    /// </summary>
+    [Cmdlet(VerbsLifecycle.Disable, StorageNouns.ServiceStaticWebsite, SupportsShouldProcess = true),


Needs to be named Azure.Commands.ResourceManager.Common.AzureRMConstants.AzurePrefix + "StorageStaticWebsite"

MiYanni · 2018-09-05T17:52:44Z

src/Storage/Commands.Storage/Common/Cmdlet/EnableAzureStorageStaticWebsite.cs

+    /// <summary>
+    /// Enable azure storage service StaticWebsite, currently only enabled on Blob service
+    /// </summary>
+    [Cmdlet(VerbsLifecycle.Enable, StorageNouns.ServiceStaticWebsite, SupportsShouldProcess = true),


Needs to be named Azure.Commands.ResourceManager.Common.AzureRMConstants.AzurePrefix + "StorageStaticWebsite"

MiYanni · 2018-09-05T18:19:27Z

src/Storage/Commands.Storage/Common/StorageNouns.cs


+        /// Azure storage Service Delete Retention Policy
+        /// </summary>
+        public const string ServiceStaticWebsite = "AzureStorageStaticWebsite";


These constants are not used for cmdlet names. No need to add a new constant.

blueww force-pushed the oauth-preview2 branch from c36860b to 7e9fac4 Compare August 28, 2018 10:06

azuresdkci1x added the needs-review label Aug 28, 2018

maddieclayton added needs-revision and removed needs-review labels Aug 29, 2018

maddieclayton assigned blueww Aug 29, 2018

blueww force-pushed the oauth-preview2 branch from f2b104f to 9aea6ab Compare August 31, 2018 03:24

blueww added 2 commits August 31, 2018 13:10

[Storage] Support Oauth

38ba017

Upgrade XSCL tp 9.3.0 to align with storage project and make build pass

476e7f7

blueww force-pushed the oauth-preview2 branch from 9aea6ab to 476e7f7 Compare August 31, 2018 05:14

blueww added 2 commits August 31, 2018 13:19

[Stroage] revert the breaking change for new-azurestoragecontext

a7afbeb

Fix a merge error

757a27a

blueww mentioned this pull request Aug 31, 2018

Upgrade to XSCL9.3 Azure/azure-powershell-common#39

Closed

Update the Microsoft.Azure.PowerShell.Storage for XSCL upgrade

f863878

Update all 1.0.94 packages to 1.0.96 with XSCL upgrade to 9.3

861e5a5

Update common version to 1.0.96

c4f61ca

MiYanni mentioned this pull request Sep 4, 2018

Upgrade to XSCL9.3 Azure/azure-powershell-common#40

Merged

MiYanni added 2 commits September 4, 2018 16:20

Merge branch 'preview' into oauth-preview2

559dd18

# Conflicts: # src/Common/Commands.Common.Storage.Management/Common.Storage.Management.Netcore.csproj # src/Common/Commands.Common.Storage/Common.Storage.Netcore.csproj

Additional 1.0.94 to 1.0.96 changes after preview merge

4ce8571

Add StaticWebSite Dataplane Support

4bb32a3

blueww changed the title ~~[Storage] Support Oauth in Powershell official release~~ [Storage] Support Oauth/StaticWeb in Powershell official release Sep 5, 2018

Update the changelog of Azure.Storage

0f64f34

MiYanni suggested changes Sep 5, 2018

View reviewed changes

Changes based on review comments.

83ba8d1

MiYanni approved these changes Sep 5, 2018

View reviewed changes

MiYanni merged commit d47ebfe into Azure:preview Sep 5, 2018

blueww mentioned this pull request Sep 10, 2018

[Storage] Disable “StaticWebSite” cmdlets as feature team decide not GA at Ignite. #7174

Merged

8 tasks

blueww mentioned this pull request Nov 21, 2018

[Storage] Support Static Web blob service configuration #7879

Merged

8 tasks

[Storage] Support Oauth/StaticWeb in Powershell official release #7081

[Storage] Support Oauth/StaticWeb in Powershell official release #7081

Uh oh!

Conversation

blueww commented Aug 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

blueww commented Aug 29, 2018

Uh oh!

maddieclayton commented Aug 29, 2018

Uh oh!

blueww commented Aug 30, 2018

Uh oh!

MiYanni commented Aug 30, 2018

Uh oh!

blueww commented Aug 31, 2018

Uh oh!

MiYanni commented Aug 31, 2018

Uh oh!

blueww commented Aug 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MiYanni commented Aug 31, 2018

Uh oh!

blueww commented Aug 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MiYanni commented Aug 31, 2018

Uh oh!

blueww commented Sep 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MiYanni commented Sep 4, 2018

Uh oh!

MiYanni commented Sep 5, 2018

Uh oh!

blueww commented Sep 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MiYanni Sep 5, 2018

Choose a reason for hiding this comment

Uh oh!

MiYanni Sep 5, 2018

Choose a reason for hiding this comment

Uh oh!

MiYanni Sep 5, 2018

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

blueww commented Aug 28, 2018 •

edited

Loading

blueww commented Aug 31, 2018 •

edited

Loading

blueww commented Aug 31, 2018 •

edited

Loading

blueww commented Sep 4, 2018 •

edited

Loading

blueww commented Sep 5, 2018 •

edited

Loading