(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases #339

jpogran · 2018-02-07T01:02:19Z

This PR updates the DSC Resources listed below to the latest versions from the October, November, December, and February releases.

~~This PR also fixes the issue detailed in MODULES-6592 where the rename of the xSQLServer DSC Resource to SqlServerDSC causes problems with our build process.~~ This has been avoided by the xSQLServer submodule reference being removed entirely and only the SqlServerDsc submodule remaining

This PR also includes several commits that fix minor issues in the build process that historically haven't been a large enough problem, but now prevented building DSC Resources for this release.

Normally this would be one commit with all changes included, but because of the SqlServerDSC rename each DSC Resource had to be committed separately to maintain the rename structure reflected correctly in git.

The following DSC Resources have been updated:

StorageDsc
SqlServerDsc
OfficeOnlineServerDsc
SecurityPolicyDsc
SharePointDsc
SystemLocaleDsc
xActiveDirectory
xAdcsDeployment
xCertificate
xComputerManagement
xDatabase
xDnsServer
xExchange
xFailoverCluster
xHyper-V
xNetworking
xPSDesiredStateConfiguration
xRemoteDesktopSessionHost
xSharePoint
xStorage
xTimeZone
xWebAdministration
~~Update dsc.rake with comments describing that it is a workaround until we move to a solution that does not use the powershell/dscresources repo~~ Not needed
Add comment to dsc_resources_tags.yml that states xPowerShellExecutionPolicy needs to be pinned until BREAKING CHANGE: New Scope property breaks DSC Resource dsccommunity/xPowerShellExecutionPolicy#14 is fixed
Test the tar with puppet module install, validate that all DSC files are present

Iristyle · 2018-02-07T01:23:34Z

build/dsc.rake

@@ -80,6 +80,8 @@ eod
        Rake::Task['dsc:resources:checkout'].invoke(
          official_dsc_resources_root, update_versions, composite_resources)

+        FileUtils.mv("#{dsc_resources_path_tmp}/xDscResources/xSQLServer", "#{dsc_resources_path_tmp}/xDscResources/SqlServerDsc", :verbose => true, :force => true)


Is there another spot this could go? Seems like it at least belongs in a parameterized list and with some sort of comment about why it's necessary.

You mean like i noted in the PR description in the TODO?

jpogran · 2018-02-07T15:37:07Z

Inside lib/puppet_x/dsc_resources:

PS C:\src\puppetlabs\modules\puppetlabs-dsc> gci -rec .\lib\puppet_x\dsc_resources\ | group extension

Count Name                      Group
----- ----                      -----
  690                           {AuditPolicyDsc, OfficeOnlineServerDsc, SecurityPolicyDsc, SharePointDsc...}
    8 .ResourceHelper           {SystemLocaleDsc.ResourceHelper, AdcsDeploymentDsc.ResourceHelper, CertificateDsc.ResourceHelper, Co...
  179 .psd1                     {AuditPolicyDsc.psd1, AuditPolicyResourceHelper.strings.psd1, MSFT_AuditPolicyCsv.strings.psd1, MSFT...
  464 .psm1                     {AuditPolicyResourceHelper.psm1, MSFT_AuditPolicyCsv.psm1, MSFT_AuditPolicyOption.psm1, MSFT_AuditPo...
  389 .mof                      {MSFT_AuditPolicyCsv.schema.mof, MSFT_AuditPolicyOption.schema.mof, MSFT_AuditPolicySubcategory.sche...
    2 .Util                     {OfficeOnlineServerDsc.Util, SharePointDsc.Util}
    1 .ContentDatabase          {SharePointDsc.ContentDatabase}
    1 .Farm                     {SharePointDsc.Farm}
    1 .ProjectServer            {SharePointDsc.ProjectServer}
    1 .Search                   {SharePointDsc.Search}
    1 .WebApplication           {SharePointDsc.WebApplication}
    1 .WebAppPolicy             {SharePointDsc.WebAppPolicy}
    1 .dll                      {ProjectServerServices.dll}
    1 .config                   {output.config}
   23 .cs                       {wcf.Admin.cs, wcf.Archive.cs, wcf.Calendar.cs, wcf.CubeAdmin.cs...}
   25 .ps1                      {Assert-HADC.ps1, Assert-ParentChildDomains.ps1, MSFT_xADCommon.ps1, GeneratexADRecycleBinSchema.ps1...
    5 .Common                   {CertificateDsc.Common, ComputerManagementDsc.Common, NetworkingDsc.Common, StorageDsc.Common...}
    1 .PDT                      {CertificateDsc.PDT}
   14 .mfl                      {MSFT_xPowerPlan.schema.mfl, MSFT_xArchive.schema.mfl, MSFT_xEnvironmentResource.schema.mfl, MSFT_xG...

Installed using puppet module install

PS C:\ProgramData\PuppetLabs\code\modules\dsc> gci -rec .\lib\puppet_x\dsc_resources\ | group extension

Count Name                      Group
---- ----                      -----
 690                           {AuditPolicyDsc, OfficeOnlineServerDsc, SecurityPolicyDsc, SharePointDsc...}
   8 .ResourceHelper           {SystemLocaleDsc.ResourceHelper, AdcsDeploymentDsc.ResourceHelper, CertificateDsc.Re...
 179 .psd1                     {AuditPolicyDsc.psd1, AuditPolicyResourceHelper.strings.psd1, MSFT_AuditPolicyCsv.st...
 464 .psm1                     {AuditPolicyResourceHelper.psm1, MSFT_AuditPolicyCsv.psm1, MSFT_AuditPolicyOption.ps...
 389 .mof                      {MSFT_AuditPolicyCsv.schema.mof, MSFT_AuditPolicyOption.schema.mof, MSFT_AuditPolicy...
   2 .Util                     {OfficeOnlineServerDsc.Util, SharePointDsc.Util}
   1 .ContentDatabase          {SharePointDsc.ContentDatabase}
   1 .Farm                     {SharePointDsc.Farm}
   1 .ProjectServer            {SharePointDsc.ProjectServer}
   1 .Search                   {SharePointDsc.Search}
   1 .WebApplication           {SharePointDsc.WebApplication}
   1 .WebAppPolicy             {SharePointDsc.WebAppPolicy}
   1 .dll                      {ProjectServerServices.dll}
   1 .config                   {output.config}
  23 .cs                       {wcf.Admin.cs, wcf.Archive.cs, wcf.Calendar.cs, wcf.CubeAdmin.cs...}
  25 .ps1                      {Assert-HADC.ps1, Assert-ParentChildDomains.ps1, MSFT_xADCommon.ps1, GeneratexADRecy...
   5 .Common                   {CertificateDsc.Common, ComputerManagementDsc.Common, NetworkingDsc.Common, StorageD...
   1 .PDT                      {CertificateDsc.PDT}
  14 .mfl                      {MSFT_xPowerPlan.schema.mfl, MSFT_xArchive.schema.mfl, MSFT_xEnvironmentResource.sch...

Iristyle · 2018-02-12T07:50:47Z

build/dsc.rake

+        # is stop using the powershell/dscresources git repo as a source of truth, which is part of a larger
+        # project that planned.
+        # This is hardcoded and done in a single line right now because premature optimization
+        # is the root of all evil. While we can predict there will be more renamed


My PR comment at #339 (comment) was just about maintenability - i.e. a small well-named method that would be called like this:

resource_renames = [{:from => 'xSQLServer', :to => 'SqlServerDsc'}] rename_resources(resource_renames)

Premature optimization is historically in reference to performance, so that doesn't apply here. YAGNI is closer (the Fowler def at https://martinfowler.com/bliki/Yagni.html is good), but doesn't really apply either since YAGNI is more about unused features than organizing code for maintainability.

The first few lines of the explanation about why xSQLServer requires the disk shuffling is good, but much of the rest of the commentary doesn't seem necessary.

michaeltlombardi

A few (very) minor nits, but we need the updated warning about WaitFor resources in the README, I think.

michaeltlombardi · 2018-02-14T16:16:31Z

build/dsc.rake

@@ -79,7 +79,7 @@ eod
          community_dsc_resources_root, update_versions, composite_resources)
        Rake::Task['dsc:resources:checkout'].invoke(
          official_dsc_resources_root, update_versions, composite_resources)
-
+        


Nit: Extra whitespace, but this shouldn't hold up the review.

michaeltlombardi · 2018-02-14T16:19:46Z

build/dsc.rake

@@ -224,8 +224,6 @@ eod
    task :build, [:module_path] do |t, args|
      module_path = args[:module_path] || default_dsc_module_path
      m = Dsc::Manager.new
-      wait_for_resources = Dir["#{module_path}/**/MSFT_WaitFor*"]
-      fail "MSFT_WaitFor* resources found - aborting type building! Please remove the following MSFT_WaitFor* DSC Resources and run the build again.\n\n#{wait_for_resources}\n" if !wait_for_resources.empty?


The code is removed here, but this commit doesn't include adding the warning to the README. I don't see the warning in the README when looking at master or the rest of this PR.

Addressed in readme

michaeltlombardi · 2018-02-14T16:21:52Z

CHANGELOG.md

@@ -6,6 +6,39 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a

 ## [Unreleased]

+### Adds
+
+- *BREAKING*: Update SqlServerDsc to 11.0.0.0 ([MODULES-6592](https://tickets.puppetlabs.com/browse/MODULES-6592))


Nit: We're currently wrapping this as BREAKING, the line here will be emphasized instead of strong.

michaeltlombardi · 2018-02-14T16:22:01Z

CHANGELOG.md

+
+### Removed
+
+- *BREAKING*: Removed xSQLServer ([MODULES-6592](https://tickets.puppetlabs.com/browse/MODULES-6592))


Same nit as above.

michaeltlombardi · 2018-02-14T16:51:46Z

Built the module locally on my mac, copied to a test vm, installed and verified. Everything is 🎉

RandomNoun7 · 2018-02-14T16:17:08Z

build/dsc.rake

@@ -79,7 +79,7 @@ eod
          community_dsc_resources_root, update_versions, composite_resources)
        Rake::Task['dsc:resources:checkout'].invoke(
          official_dsc_resources_root, update_versions, composite_resources)
-
+        


NIT, Whitespace.

RandomNoun7 · 2018-02-14T16:17:17Z

build/dsc.rake

-      Dir["#{dsc_resources_path}/*"].each do |dsc_resource_path|
-        dsc_resource_name = Pathname.new(dsc_resource_path).basename
-        FileUtils.cd(dsc_resource_path) do
+


NIT, Whitespace

RandomNoun7 · 2018-02-14T16:17:57Z

build/dsc.rake

-      File.open("#{dsc_resources_file}", 'w+') { |f| f.write resource_tags.to_yaml }
+
+      # We use YAML.dump here to update the file instead of overwriting it. This ensures
+      # we can write both HQ DSC Resources and well as Expertimental ones to the same yml


Typo? Should be "Resources 'as' well as.."?

RandomNoun7 · 2018-02-14T16:21:32Z

dsc_resource_release_tags.yml

@@ -41,7 +42,7 @@ xSCVMM: 1.2.4.0-PSGallery
 xSharePoint: 1.8.0.0-PSGallery
 xSmbShare: 2.0.0.0-PSGallery
 xSqlPs: 1.4.0.0-PSGallery
-xSQLServer: 7.0.0.0-PSGallery # Keep at version 7.0.0 due to PUP-7888 (long file names)
+xSQLServer: 7.0.0.0-PSGallery


Just curious, does this change really belong in this commit?

RandomNoun7 · 2018-02-14T16:24:39Z

lib/puppet/type/dsc_opticaldiskdriveletter.rb

+    For more information about built-in DSC Resources, please visit
+    https://technet.microsoft.com/en-us/library/dn249921.aspx.
+
+    For more information about xDsc Resources, please visit


Should this instance of xDsc retain the x? This question applies to most or all of the documentation blocks in this commit, and perhaps any others that change from xDsc to a normal Dsc module.

We decided this was either valid or not important for now.

When this line in the template was written HQ DSC Resources didn't exist, so there wasn't a distinction.

Admittantly the language in the line is problematic without the 'Experimental' reference, as it should say something more like '...information about Microsoft DSC Resources we vendor...'.

FWIW I don't this this doc block is helpful at all, and should really be in the readme more than in each type.

In this PR, to 'fix' this would require touching every template file, whereas the point of this ticket is to only update the DSC Resources that have new versions. I have created MODULES-6642 to address whether to change this part of the template or not

This commit fixes building both HQ DSC Resources as well as Experimental DSC Resources by fixing the method to update the dsc_resource_tags.yml and updating the dsc_resource_tags.yml with the current list of HQ DSC Resources.

In commit 44f2bd4 we added PSDscResources to the blacklist because it is a side-by-side release of the same DSC Resources that are built into DSC as well as some DSC Resources included in xPSDscResources, and we can't build all of those without overwritting all of them. This commit fixes this attempt by including PSDscResources in the filter that ignores certain DSC Resources when adding them to the dsc_resource_tags.yml file.

In commit a5147a6 we added a warning to remove any DSC Resources that match the name 'MSFT_WaitFor`. This did not prevent some DSC Resources that do perform a `WaitFor` type function from being included over time, and now presents a problem with updating official DSC Resources (the warning prevents us from including parts of updated DSC Resource modules). Since it would potentially cause unintended problems to pick and choose which parts of a DSC Resource were included in our build process, it was decided to remove the warning altogether and allow `WaitFor` DSC Resources to be included, and warn the user in the READEME that these DSC Resources may not work as expected.

This commit updates StorageDsc to the latest gallery versions~

This commit updates SqlServerDsc to the latest gallery versions. Note: This is a BREAKING change. In commit dsccommunity/SqlServerDsc@afca929 the xSQLServer DSC Resource was renamed from xSQLServer to SqlServerDsc (everything from repo name to PS module) and then all files and folder names were shortened (i.e. xSQLServerSetup to 'SqlSetup', etc). In commit PowerShell/DscResources@20b9f00 the xSQLServer git submodule was removed, and the SqlServerDsc submodule was added in PowerShell/DscResources@27bedc3. This means that anyone using xSQLServer has to migrate their manifests to SqlServerDsc when they update to this dsc module release.

This commit updates OfficeOnlineServerDSc to the latest gallery version

This commit updates SecurityPolicyDsc to the latest gallery version

This commit updates SharePointDsc to the latest gallery version

This commit updates SystemLocaleDsc to the latest gallery version

This commit updates xActiveDirectory to the latest gallery version

This commit updates SqlServerDsc to the latest gallery versions

This commit updates xCertificate to the latest gallery versions

This commit updates xComputerManagement to the latest gallery version 4.0.0.0

This commit updates xDatabase to the latest gallery version 1.7.0.0

This commit updates xDnsServer to the latest gallery version 1.9.0.0

This commit updates xExchange to the latest gallery version 1.19.0.0

This commit updates xFailOverCluster to the latest gallery version 1.9.0.0

This commit updates xHyper-V to the latest gallery version 3.11.0.0

This commit updates xNetworking to the latest gallery version 5.5.0.0

This commit updates xPSDesiredStateConfiguration to the latest gallery version 8.0.0.0

This commit updates xRemoteDesktopSessionHost to the latest gallery version 1.5.0.0

This commit updates xSharePoint to the latest gallery version 2.1.0.0

This commit updates xTimeZone to the latest gallery version 1.7.0.0

This commit updates xWebAdministration to the latest gallery version 1.19.0.0

This commit updates the changlog to reflect the changes for the October, November, December and February DSC Resource Kit releases. It also lists the fixes necessary to add those releases.

jpogran changed the title ~~(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases~~ (wip)(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases Feb 7, 2018

Iristyle reviewed Feb 7, 2018

View reviewed changes

jpogran force-pushed the MODULES-6592-update-dsc-resources-to-oct2017-nov2017-dec2017-releases branch 2 times, most recently from 8aa2336 to a2623a0 Compare February 7, 2018 04:39

jpogran changed the title ~~(wip)(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases~~ (MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases Feb 7, 2018

jpogran changed the title ~~(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases~~ (WIP)(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases Feb 8, 2018

Iristyle reviewed Feb 12, 2018

View reviewed changes

jpogran force-pushed the MODULES-6592-update-dsc-resources-to-oct2017-nov2017-dec2017-releases branch 5 times, most recently from 0a50e2c to 7474184 Compare February 14, 2018 03:37

jpogran changed the title ~~(WIP)(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases~~ (MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases Feb 14, 2018

michaeltlombardi suggested changes Feb 14, 2018

View reviewed changes

RandomNoun7 reviewed Feb 14, 2018

View reviewed changes

jpogran force-pushed the MODULES-6592-update-dsc-resources-to-oct2017-nov2017-dec2017-releases branch 2 times, most recently from 1456c3f to fce0858 Compare February 14, 2018 18:30

jpogran added 11 commits February 14, 2018 13:33

(maint) Fix including HQ DSC Resources in yml

08c8fc0

This commit fixes building both HQ DSC Resources as well as Experimental DSC Resources by fixing the method to update the dsc_resource_tags.yml and updating the dsc_resource_tags.yml with the current list of HQ DSC Resources.

(MODULES-6592) Update StorageDsc to 4.0.0.0

cf6e00e

This commit updates StorageDsc to the latest gallery versions~

(MODULES-6592) Update OfficeOnlineServerDsc to 1.2.0.0

7a13ce8

This commit updates OfficeOnlineServerDSc to the latest gallery version

(MODULES-6592) Update SecurityPolicyDsc to 2.2.0.0

38b4f22

This commit updates SecurityPolicyDsc to the latest gallery version

(MODULES-6592) Update SharePointDsc to 2.1.0.0

fd8f622

This commit updates SharePointDsc to the latest gallery version

(MODULES-6592) Update SystemLocaleDsc to 1.2.0.0

6a36353

This commit updates SystemLocaleDsc to the latest gallery version

(MODULES-6592) Update xActiveDirectory to 2.17.0.0

2d7f065

This commit updates xActiveDirectory to the latest gallery version

(MODULES-6592) Update xAdcsDeployment to 1.4.0.00

b694129

This commit updates SqlServerDsc to the latest gallery versions

jpogran added 14 commits February 14, 2018 13:33

(MODULES-6592) Update xCertificate to 4.2.0.0

03018ad

This commit updates xCertificate to the latest gallery versions

(MODULES-6592) Update xComputerManagement to 4.0.0.0

1a04393

This commit updates xComputerManagement to the latest gallery version 4.0.0.0

(MODULES-6592) Update xDatabase to 1.7.0.0

803c414

This commit updates xDatabase to the latest gallery version 1.7.0.0

(MODULES-6592) Update xDnsServer to 1.9.0.0

c817e06

This commit updates xDnsServer to the latest gallery version 1.9.0.0

(MODULES-6592) Update xExchange to 1.19.0.0

5dfef8d

This commit updates xExchange to the latest gallery version 1.19.0.0

(MODULES-6592) Update xFailOverCluster to 1.9.0.0-

a633159

This commit updates xFailOverCluster to the latest gallery version 1.9.0.0

(MODULES-6592) Update xHyper-V to 3.11.0.0

60bb77c

This commit updates xHyper-V to the latest gallery version 3.11.0.0

(MODULES-6592) Update xNetworking to 5.5.0.0

e598776

This commit updates xNetworking to the latest gallery version 5.5.0.0

(MODULES-6592) Update xPSDesiredStateConfiguration to 8.0.0.0

64857a7

This commit updates xPSDesiredStateConfiguration to the latest gallery version 8.0.0.0

(MODULES-6592) Update xRemoteDesktopSessionHost to 1.5.0.0

02f3959

This commit updates xRemoteDesktopSessionHost to the latest gallery version 1.5.0.0

(MODULES-6592) Update xSharePoint to 2.1.0.0

0a78e5d

This commit updates xSharePoint to the latest gallery version 2.1.0.0

(MODULES-6592) Update xTimeZone to 1.7.0.0

2bfa5d2

This commit updates xTimeZone to the latest gallery version 1.7.0.0

(MODULES-6592) Update xWebAdministration to 1.19.0.0

e7d270c

This commit updates xWebAdministration to the latest gallery version 1.19.0.0

(maint) Update changelog for Oct,Nov,Dec,Feb Releases

7af9dbe

This commit updates the changlog to reflect the changes for the October, November, December and February DSC Resource Kit releases. It also lists the fixes necessary to add those releases.

jpogran force-pushed the MODULES-6592-update-dsc-resources-to-oct2017-nov2017-dec2017-releases branch from fce0858 to 7af9dbe Compare February 14, 2018 18:34

michaeltlombardi approved these changes Feb 14, 2018

View reviewed changes

michaeltlombardi merged commit 0880987 into puppetlabs-toy-chest:master Feb 14, 2018

jpogran deleted the MODULES-6592-update-dsc-resources-to-oct2017-nov2017-dec2017-releases branch May 10, 2018 14:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases #339

(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases #339

jpogran commented Feb 7, 2018 •

edited

Iristyle Feb 7, 2018

jpogran Feb 7, 2018

jpogran commented Feb 7, 2018

Iristyle Feb 12, 2018

michaeltlombardi left a comment

michaeltlombardi Feb 14, 2018

michaeltlombardi Feb 14, 2018

jpogran Feb 14, 2018

michaeltlombardi Feb 14, 2018

michaeltlombardi Feb 14, 2018

michaeltlombardi commented Feb 14, 2018

RandomNoun7 Feb 14, 2018

RandomNoun7 Feb 14, 2018

RandomNoun7 Feb 14, 2018

RandomNoun7 Feb 14, 2018

RandomNoun7 Feb 14, 2018

RandomNoun7 Feb 14, 2018

jpogran Feb 14, 2018


		### Removed

		- BREAKING: Removed xSQLServer ([MODULES-6592](https://tickets.puppetlabs.com/browse/MODULES-6592))

(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases #339

(MODULES-6592) Update DSC Resources to Oct-Nov-Dev 2017 Releases #339

Conversation

jpogran commented Feb 7, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpogran commented Feb 7, 2018

Choose a reason for hiding this comment

michaeltlombardi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaeltlombardi commented Feb 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpogran commented Feb 7, 2018 •

edited