Skip to content
This repository has been archived by the owner on Aug 18, 2022. It is now read-only.

(maint) Pay down tech debt #349

Merged

Conversation

jpogran
Copy link
Contributor

@jpogran jpogran commented Mar 15, 2018

The PR is purely maintenance work and introduces no new features, but it does remove an unused task and extracts a lot of code to ruby classes that were previously sitting free flowing inside rake tasks. This will reduce pain in the future when we modify this to pull DSC Resources from sources other than git.

Commit 7f0876e is most important, and could be considered a bugfix all on it's own, as it standardizes the variables that control folder placement inside both the rake task and inside the manager.rb class. Previously these were hardcoded, but gave the impression that they could be separately controlled when in fact it was possible to specify a new path in the rake task but have it ignored by the manager.rb class.

This PR also renames some global variables and some local variables defined in the dsc namespace in the rake tasks. Previous variable names were confusing and often similarly named with contrary purposes. This rename leans towards more clarity in naming at the expense of being a little verbose. Proper use of these variables is important, as misuse results in a failed build or a corrupted build that still 'succeeds'.

In summation, this PR:

  • Extract document building, type cleaning, and type building to separate classes
  • Removes unused global variables from rake tasks
  • Renames global and local variables in rake task for clarity
  • Removes dsc:module:skeleton unused rake task

To test:

Criteria 1:

  • Clean checkout
  • bundle exec rake dsc:clean && bundle exec rake dsc:build
  • No files should change

Criteria 2:

  • Clean checkout
  • Delete dsc_resource_release_tags.yml
  • bundle exec rake dsc:clean && bundle exec rake dsc:build
  • New DSC Resources should be vendored.

Note: If PSExecutionPolicy still has issue dsccommunity/xPowerShellExecutionPolicy#14, then it will fail parsing that DSC Resource. This is not related to this PR

Note: PR #352 addresses problems in SqlServerDsc module manifest parsing

This commit extracts cleaning the types and specs from the build out of
the Manager class and rake tasks into seperate classes. This allows
testing in isolation as well as seperation of concerns.
This commit extracts out building the types.md file out of the Manager
class and the rake task into a seperate class. This allows testing in
isolation as well as seperation of concerns.
This commit extracts building types from the Manager class and rake
tasks to a seperate class. This allows testing in isolation and further
refactoring work (namely global variable use) to be done later.

This commit refactors the build_dsc_types method into seperate methods
that handle building the types and the specs seperately.
This commit extracts the type import code from the Manager class and
rake task to a seperate class. This allows for testing in isolation as
well as further refactor work to move to a PSGallery solution and move
the git checkout workflow to something not dependent on rake tasks.
This commit cleans up the decription blocks in the dsc.rake file. Some
of these used variables that did not reduce any code, others used
herestrings that didn't add any useful information. Each desc block was
reduced to one line that clearly stated intent.
This commit removes the dsc:module:skeleton rake task. This task creates
a new puppet module directory that fills out a Gemfile, git files, spec
files, and many other files that do not follow Puppet practices and are
not relevant to building the DSC resources now.
This commit standardizes the folder paths used in both dsc.rake and
manager.rb.

The dsc.rake task lead you to believe you could change most folder paths
used to build the module using rake task arguments. This was mostly
untrue, as the folder paths were hardcoded in both dsc.rake and
manager.rb. Changing the path in the rake task would not change it for
the manager.rb class, resulting in inconsistent builds. After this
commit, changing the folder paths in the rake task will propagate down
to the manager and inherited classes.
This commit renames some global variables defined in the dsc namespace
in the rake tasks. Previous variable names were confusing and often
similarly named with contrary purposes. This rename moves for more
clarity in naming, if a little verbose. Proper use of these variables is
important, as misuse results in a failed built or a corrupted build that
still 'succeeds'.
@glennsarti
Copy link
Contributor

I tried running this and I'm getting an error on the xActiveDirectory module saying it has no PS Gallery tags ...

Cloning into 'xDscResources/xWindowsUpdate'...
remote: Counting objects: 345, done.
remote: Total 345 (delta 0), reused 0 (delta 0), pack-reused 345
Receiving objects: 100% (345/345), 86.27 KiB | 0 bytes/s, done.
Resolving deltas: 100% (169/169), done.
Checking connectivity... done.
Submodule path 'xDscResources/xWindowsUpdate': checked out 'cd90e8015c30a0f50631dcf8025be3a94bc98857'
Cloning into 'xDscResources/xWordPress'...
remote: Counting objects: 96, done.
remote: Total 96 (delta 0), reused 0 (delta 0), pack-reused 96
Unpacking objects: 100% (96/96), done.
Checking connectivity... done.
Submodule path 'xDscResources/xWordPress': checked out '8fa349ca5d37f505f5a1ea8cdd6956c71d665181'
Getting latest release tags for DSC resources in /project/import/dsc_resources_tmp/xDscResources...
rake aborted!
xActiveDirectory does not have any '*-PSGallery' tags. Appears it has not been released yet. Tags found %D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
%D
/project/build/dsc.rake:100:in `block (5 levels) in <top (required)>'
/project/build/dsc.rake:90:in `block (4 levels) in <top (required)>'
/project/build/dsc.rake:88:in `each'
/project/build/dsc.rake:88:in `block (3 levels) in <top (required)>'
/project/build/dsc.rake:58:in `block (3 levels) in <top (required)>'
/project/build/dsc.rake:17:in `block (2 levels) in <top (required)>'

Although there clearly are tags in the that repo

root@35adf0a20cfa:/project/import/dsc_resources_tmp/xDscResources/xActiveDirectory# git tag
2.10.0.0-PSGallery
2.11.0.0-PSGallery
2.12.0.0-PSGallery
2.13.0.0-PSGallery
2.14.0.0-PSGallery
2.15.0.0-PSGallery
2.16.0.0-PSGallery
2.17.0.0-PSGallery
2.4.0.0-PSGallery
2.5.0.0-PSGallery
2.6.0.0-PSGallery
2.7.0.0-PSGallery
2.8.0.0-PSGallery
2.9.0.0-PSGallery

@jpogran
Copy link
Contributor Author

jpogran commented Mar 19, 2018

@glennsarti your git version is old, it can't see the tags: https://github.com/puppetlabs/puppetlabs-dsc/blob/master/build/dsc.rake#L157

Copy link
Contributor

@glennsarti glennsarti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm +1 for merge after comments are addressed!

Awesome work!

end

def clean_folder(folders)
type_pathes = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: typo. Won't stop a merge though

class TypeBuilder < Manager

def build_dsc_types
type_pathes = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: typo. Won't stop a merge though

build/dsc.rake Outdated
@@ -58,7 +58,7 @@ eod
update_versions = args[:update_versions] || false
is_custom_resource = (dsc_resources_path != default_dsc_resources_path)

m = Dsc::Manager.new
m = Dsc::TypeImporter.new
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of single letter variables, but may be outside the scope of this?

@glennsarti
Copy link
Contributor

Will try this on a fresh VM

@glennsarti
Copy link
Contributor

Hrmm...So I'm 99.99% sure this is not the fault of these changes...

...
ma.mof [pragma]
rake aborted!
module manifest /home/glennsarti/project/puppetlabs-dsc/import/dsc_resources/SqlServerDsc/SqlServerDsc.psd1 not found
/home/glennsarti/project/puppetlabs-dsc/build/dsc/resource.rb:190:in `ps_module'
(erb):79:in `block in build_dsc_type'
/home/glennsarti/project/puppetlabs-dsc/build/dsc/typebuilder.rb:26:in `block in build_dsc_type'
/home/glennsarti/project/puppetlabs-dsc/build/dsc/typebuilder.rb:25:in `open'
...

Looks like there's a case sensitivity issue in the SQL Server DSC module as the file is called SQLServerDsc.psd1 Note the Q and L;

drwxrwxr-x  5 glennsarti glennsarti  4096 Apr 24 20:17 .
drwxrwxr-x 55 glennsarti glennsarti  4096 Apr 24 20:17 ..
drwxrwxr-x 32 glennsarti glennsarti  4096 Apr 24 20:17 DSCResources
drwxrwxr-x  2 glennsarti glennsarti  4096 Apr 24 20:17 en-US
-rw-rw-r--  1 glennsarti glennsarti  1089 Apr 24 20:17 LICENSE
-rw-rw-r--  1 glennsarti glennsarti 46312 Apr 24 20:17 SqlServerDscHelper.psm1
-rw-rw-r--  1 glennsarti glennsarti 11440 Apr 24 20:17 SQLServerDsc.psd1
drwxrwxr-x  2 glennsarti glennsarti  4096 Apr 24 20:17 sv-SE

@jpogran
Copy link
Contributor Author

jpogran commented Apr 25, 2018

@glennsarti Fix already implemented in #352

@glennsarti
Copy link
Contributor

glennsarti commented Apr 25, 2018

This errors the same as master so no change in behaviour.

@glennsarti glennsarti merged commit 408fc74 into puppetlabs-toy-chest:master Apr 25, 2018
@jpogran jpogran deleted the maint-pay-down-tech-debt branch May 10, 2018 14:55
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants