Restores ability to run processors standalone on the commandline #621

n8felton · 2020-02-27T13:39:20Z

Sample usage:

defaults write /tmp/processor.plist ARTICLE_NUMBER '1888'
defaults write /tmp/processor.plist verbose -int 3
plutil -convert xml1 /tmp/processor.plist

# Change to be your git dev branch of AutoPkg
export PYTHONPATH=${HOME}/src/autopkg/autopkg/Code 

/Library/AutoPkg/Python3/Python.framework/Versions/Current/bin/python3 ${HOME}/src/autopkg/n8felton-recipes/SharedProcessors/AppleSupportDownloadInfoProvider.py < /tmp/processor.plist

Sample Usage 2:

# Change to be your git dev branch of AutoPkg
export PYTHONPATH=${HOME}/src/autopkg/autopkg/Code 

/Library/AutoPkg/Python3/Python.framework/Versions/Current/bin/python3 ${HOME}/src/autopkg/n8felton-recipes/SharedProcessors/AppleSupportDownloadInfoProvider.py ARTICLE_NUMBER=1888 verbose=3

# Press CTRL-D to send EOF to Python waiting for stdin input. This is a fix for another day....

Sample Output in both cases:

AppleSupportDownloadInfoProvider: Article URL: https://support.apple.com/kb/DL1888
AppleSupportDownloadInfoProvider: Download URL: https://support.apple.com/downloads/DL1888/en_US/&
AppleSupportDownloadInfoProvider: Full URL: https://updates.cdn-apple.com/2019/cert/041-88763-20191011-6e70f498-9d39-420c-b11b-b252b17233e2/HewlettPackardPrinterDrivers.dmg
AppleSupportDownloadInfoProvider: Article title: HP Printer Drivers v5.1 for OS X
AppleSupportDownloadInfoProvider: Version: 5.1
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>ARTICLE_NUMBER</key>
	<string>1888</string>
	<key>article_url</key>
	<string>https://support.apple.com/kb/DL1888</string>
	<key>url</key>
	<string>https://updates.cdn-apple.com/2019/cert/041-88763-20191011-6e70f498-9d39-420c-b11b-b252b17233e2/HewlettPackardPrinterDrivers.dmg</string>
	<key>verbose</key>
	<integer>3</integer>
	<key>version</key>
	<string>5.1</string>
</dict>
</plist>

This reverts commit 5cc89b5.

nmcspadden · 2020-02-27T14:57:49Z

This looks safe enough. I think you are probably the first person to ask for this.

nmcspadden · 2020-02-27T14:58:29Z

Can you also, just for peace of mind, show that it doesn't affect normal autopkg operation? i.e. still running an existing recipe "the usual way"?

gregneagle · 2020-02-27T15:03:22Z

Since it "restores" an ability, I suspect this ability was used in the past. I think I used it 5+ years ago when working on AutoPkg processors (both core and custom). I know I made use of the functionality implied by the PROCESSOR.execute_shell() code.

It might be helpful for the future and for other people if Nate were to write a wiki page on using this functionality to help test processors as you are creating/modifying/debugging them.

n8felton · 2020-02-29T19:45:30Z

Can you also, just for peace of mind, show that it doesn't affect normal autopkg operation? i.e. still running an existing recipe "the usual way"?

${HOME}/src/autopkg/autopkg/Code/autopkg run AppleHPPrinterDrivers.download -vvvvv                                                               
Processing AppleHPPrinterDrivers.download...
WARNING: AppleHPPrinterDrivers.download is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
{'AUTOPKG_VERSION': '2.0.3',
 'MUNKI_REPO': '/Users/Shared/munki_repo',
 'NAME': 'AppleHPPrinterDrivers',
 'PARENT_RECIPES': [],
 'RECIPE_CACHE_DIR': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers',
 'RECIPE_DIR': '/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes/ApplePrinterDrivers',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes/ApplePrinterDrivers/AppleHPPrinterDrivers.download.recipe',
 'RECIPE_REPOS': {'/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes': {'URL': 'https://github.com/autopkg/n8felton-recipes'}},
 'RECIPE_SEARCH_DIRS': ['.',
                        '~/Library/AutoPkg/Recipes',
                        '/Library/AutoPkg/Recipes',
                        '/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes'],
 'verbose': 5}
com.github.n8felton.shared/AppleSupportDownloadInfoProvider
{'Input': {'ARTICLE_NUMBER': '1888'}}
AppleSupportDownloadInfoProvider: Article URL: https://support.apple.com/kb/DL1888
AppleSupportDownloadInfoProvider: Download URL: https://support.apple.com/downloads/DL1888/en_US/&
AppleSupportDownloadInfoProvider: Curl command: ['/usr/bin/curl', '--compressed', '--location', '--silent', '--head', '--write-out', '%{url_effective}', '--url', 'https://support.apple.com/downloads/DL1888/en_US/&', '--output', '/dev/null']
AppleSupportDownloadInfoProvider: Full URL: https://updates.cdn-apple.com/2019/cert/041-88763-20191011-6e70f498-9d39-420c-b11b-b252b17233e2/HewlettPackardPrinterDrivers.dmg
AppleSupportDownloadInfoProvider: Curl command: ['/usr/bin/curl', '--compressed', '--location', 'https://support.apple.com/kb/DL1888']
AppleSupportDownloadInfoProvider: Article title: HP Printer Drivers v5.1 for OS X
AppleSupportDownloadInfoProvider: Version: 5.1
{'Output': {'article_url': 'https://support.apple.com/kb/DL1888',
            'url': 'https://updates.cdn-apple.com/2019/cert/041-88763-20191011-6e70f498-9d39-420c-b11b-b252b17233e2/HewlettPackardPrinterDrivers.dmg',
            'version': '5.1'}}
URLDownloader
{'Input': {'url': 'https://updates.cdn-apple.com/2019/cert/041-88763-20191011-6e70f498-9d39-420c-b11b-b252b17233e2/HewlettPackardPrinterDrivers.dmg'}}
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: No value supplied for CHECK_FILESIZE_ONLY, setting default value of: False
URLDownloader: Curl command: ['/usr/bin/curl', '--silent', '--show-error', '--no-buffer', '--dump-header', '-', '--speed-time', '30', '--location', '--url', 'https://updates.cdn-apple.com/2019/cert/041-88763-20191011-6e70f498-9d39-420c-b11b-b252b17233e2/HewlettPackardPrinterDrivers.dmg', '--fail', '--output', '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/tmplb8lxrad']
URLDownloader: Storing new Last-Modified header: Tue, 15 Oct 2019 05:50:51 GMT
URLDownloader: Storing new ETag header: "eb70ff5ad0ceb9f8a64e23836bcb2e9e"
URLDownloader: Downloaded /Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg
{'Output': {'download_changed': True,
            'etag': '"eb70ff5ad0ceb9f8a64e23836bcb2e9e"',
            'last_modified': 'Tue, 15 Oct 2019 05:50:51 GMT',
            'pathname': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg',
            'url_downloader_summary_result': {'data': {'download_path': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg'},
                                              'summary_text': 'The following '
                                                              'new items were '
                                                              'downloaded:'}}}
EndOfCheckPhase
{'Input': {}}
{'Output': {}}
CodeSignatureVerifier
{'Input': {'expected_authority_names': ['Software Update',
                                        'Apple Software Update Certification '
                                        'Authority',
                                        'Apple Root CA'],
           'input_path': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg/*PrinterDrivers.pkg'}}
CodeSignatureVerifier: Mounted disk image /Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg
CodeSignatureVerifier: Using path '/private/tmp/dmg.YQWDaL/HewlettPackardPrinterDrivers.pkg' matched from globbed '/private/tmp/dmg.YQWDaL/*PrinterDrivers.pkg'.
CodeSignatureVerifier: Verifying installer package signature...
CodeSignatureVerifier: Package "HewlettPackardPrinterDrivers.pkg":
CodeSignatureVerifier:    Status: signed Apple Software
CodeSignatureVerifier:    Certificate Chain:
CodeSignatureVerifier:     1. Software Update
CodeSignatureVerifier:        SHA1 fingerprint: 75 86 00 B2 79 B3 ED 1D B5 52 46 5B 17 63 E5 89 87 85 D5 73
CodeSignatureVerifier:        -----------------------------------------------------------------------------
CodeSignatureVerifier:     2. Apple Software Update Certification Authority
CodeSignatureVerifier:        SHA1 fingerprint: E3 30 E5 04 00 4B D2 5C 45 80 0A F2 D5 1B 03 D5 77 27 B7 01
CodeSignatureVerifier:        -----------------------------------------------------------------------------
CodeSignatureVerifier:     3. Apple Root CA
CodeSignatureVerifier:        SHA1 fingerprint: 61 1E 5B 66 2C 59 3A 08 FF 58 D1 4A E2 24 52 D1 98 DF 6C 60
CodeSignatureVerifier: 
CodeSignatureVerifier: Signature is valid
CodeSignatureVerifier: Authority name chain is valid
{'Output': {}}
{'ARTICLE_NUMBER': '1888',
 'AUTOPKG_VERSION': '2.0.3',
 'CHECK_FILESIZE_ONLY': False,
 'MUNKI_REPO': '/Users/Shared/munki_repo',
 'NAME': 'AppleHPPrinterDrivers',
 'PARENT_RECIPES': [],
 'RECIPE_CACHE_DIR': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers',
 'RECIPE_DIR': '/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes/ApplePrinterDrivers',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes/ApplePrinterDrivers/AppleHPPrinterDrivers.download.recipe',
 'RECIPE_REPOS': {'/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes': {'URL': 'https://github.com/autopkg/n8felton-recipes'}},
 'RECIPE_SEARCH_DIRS': ['.',
                        '~/Library/AutoPkg/Recipes',
                        '/Library/AutoPkg/Recipes',
                        '/Users/n8felton/Library/AutoPkg/RecipeRepos/com.github.autopkg.n8felton-recipes'],
 'article_url': 'https://support.apple.com/kb/DL1888',
 'download_changed': True,
 'etag': '"eb70ff5ad0ceb9f8a64e23836bcb2e9e"',
 'expected_authority_names': ['Software Update',
                              'Apple Software Update Certification Authority',
                              'Apple Root CA'],
 'input_path': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg/*PrinterDrivers.pkg',
 'last_modified': 'Tue, 15 Oct 2019 05:50:51 GMT',
 'pathname': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg',
 'prefetch_filename': False,
 'url': 'https://updates.cdn-apple.com/2019/cert/041-88763-20191011-6e70f498-9d39-420c-b11b-b252b17233e2/HewlettPackardPrinterDrivers.dmg',
 'url_downloader_summary_result': {'data': {'download_path': '/Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg'},
                                   'summary_text': 'The following new items '
                                                   'were downloaded:'},
 'verbose': 5,
 'version': '5.1'}
Receipt written to /Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/receipts/AppleHPPrinterDrivers-receipt-20200229-142211.plist

The following new items were downloaded:
    Download Path                                                                                                                        
    -------------                                                                                                                        
    /Users/n8felton/Library/AutoPkg/Cache/com.github.n8felton.download.AppleHPPrinterDrivers/downloads/HewlettPackardPrinterDrivers.dmg

It might be helpful for the future and for other people if Nate were to write a wiki page on using this functionality to help test processors as you are creating/modifying/debugging them.

Threw together https://github.com/autopkg/autopkg/wiki/Developing-Custom-Processors

n8felton added 5 commits February 26, 2020 09:31

Restores displaying output to stdout

474ecaa

Restores reading variables from a plist via stdin

5cc89b5

Ensure that the verbose variable is an int

52bbb5c

Revert "Restores reading variables from a plist via stdin"

70a3f27

This reverts commit 5cc89b5.

Restores reading variables from a plist via stdin

8dc10e4

nmcspadden changed the base branch from dev to dev_fetch_parents May 4, 2020 20:39

nmcspadden merged commit 8b8a008 into autopkg:dev_fetch_parents May 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restores ability to run processors standalone on the commandline #621

Restores ability to run processors standalone on the commandline #621

n8felton commented Feb 27, 2020

nmcspadden commented Feb 27, 2020

nmcspadden commented Feb 27, 2020

gregneagle commented Feb 27, 2020

n8felton commented Feb 29, 2020

Restores ability to run processors standalone on the commandline #621

Restores ability to run processors standalone on the commandline #621

Conversation

n8felton commented Feb 27, 2020

nmcspadden commented Feb 27, 2020

nmcspadden commented Feb 27, 2020

gregneagle commented Feb 27, 2020

n8felton commented Feb 29, 2020