Skip to content
This repository has been archived by the owner on Oct 19, 2022. It is now read-only.

Xcode.munki.recipe fails at xip extract stage #40

Closed
mikethehill opened this issue Nov 27, 2019 · 5 comments
Closed

Xcode.munki.recipe fails at xip extract stage #40

mikethehill opened this issue Nov 27, 2019 · 5 comments

Comments

@mikethehill
Copy link

Running AutoPkg v1.4

XcodeXIPUnpacker: Extracting xip archive, please be patient, this could take a long time... Traceback (most recent call last): File "/usr/local/bin/autopkg", line 2810, in <module> sys.exit(main(sys.argv)) File "/usr/local/bin/autopkg", line 2804, in main exit(subcommands[verb]["function"](argv)) File "/usr/local/bin/autopkg", line 2324, in run_recipes autopackager.process(recipe) File "/Library/AutoPkg/autopkglib/__init__.py", line 698, in process log_err(unicode(err)) # noqa TODO UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 24: ordinal not in range(128)
@nmcspadden
Copy link
Contributor

I can't reproduce this. Can you please provide the full output of autopkg run -vvvv Xcode.munki, and sanitize out your Apple ID?

@nmcspadden
Copy link
Contributor

I'm unable to reproduce this on either 1.4 or 2.0b3:

XcodeXIPUnpacker
{'Input': {'PKG': u'/Users/Shared/AutoPkg/Cache/com.facebook.autopkg.xcode.downloader/downloads/Xcode_11.3_beta.xip'}}
XcodeXIPUnpacker: Extracting xip archive, please be patient, this could take a long time...
XcodeXIPUnpacker: Finished xip unpack.

@mikethehill
Copy link
Author

Output below:

Processing Xcode.munki...
Use of undefined key in variable substitution: u'RECIPE_CACHE_DIR'
{u'APPLE_ID': u'email@domain.com',
 'AUTOPKG_VERSION': u'1.4',
 u'BETA': u'',
 u'DESTINATION_APP_NAME': u'Xcode.app',
 u'ICON_NAME': u'Xcode.png',
 u'MUNKI_REPO': u'/Volumes/Data',
 u'MUNKI_REPO_SUBDIR': u'apps/apple/xcode/',
 u'NAME': u'Xcode',
 u'NOSKIP': u'',
 'PARENT_RECIPES': [u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode/Xcode.munki.recipe',
                    u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode/Xcode.extract.recipe',
                    u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode/Xcode.download.recipe'],
 u'PASSWORD': u'',
 u'PASSWORD_FILE': u'secret.txt',
 u'PATTERN': u'((?!.*beta).*\\/Xcode_.*\\/Xcode.*.xip)',
 'RECIPE_CACHE_DIR': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode',
 'RECIPE_DIR': '/Users/autopkg/Library/AutoPkg/RecipeOverrides',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/autopkg/Library/AutoPkg/RecipeOverrides/Xcode.munki.recipe',
 u'RECIPE_REPOS': {u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.HobbitHardcase-recipes': {u'URL': u'https://github.com/autopkg/HobbitHardcase-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.apettinen-recipes': {u'URL': u'https://github.com/autopkg/apettinen-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.apizz-recipes': {u'URL': u'https://github.com/autopkg/apizz-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.arubdesu-recipes': {u'URL': u'https://github.com/autopkg/arubdesu-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.chilcote-recipes': {u'URL': u'https://github.com/autopkg/chilcote-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.dataJAR-recipes': {u'URL': u'https://github.com/autopkg/dataJAR-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.foigus-recipes': {u'URL': u'https://github.com/autopkg/foigus-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.hansen-m-recipes': {u'URL': u'https://github.com/autopkg/hansen-m-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes': {u'URL': u'https://github.com/autopkg/hjuutilainen-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.homebysix-recipes': {u'URL': u'https://github.com/autopkg/homebysix-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.its-unibas-recipes': {u'URL': u'https://github.com/autopkg/its-unibas-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.jessepeterson-recipes': {u'URL': u'https://github.com/autopkg/jessepeterson-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.jlehikoinen-recipes': {u'URL': u'https://github.com/autopkg/jlehikoinen-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.keeleysam-recipes': {u'URL': u'https://github.com/autopkg/keeleysam-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.killahquam-recipes': {u'URL': u'https://github.com/autopkg/killahquam-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.munkireport-recipes': {u'URL': u'https://github.com/autopkg/munkireport-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes': {u'URL': u'https://github.com/autopkg/recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.rtrouton-recipes': {u'URL': u'https://github.com/autopkg/rtrouton-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.scriptingosx-recipes': {u'URL': u'https://github.com/autopkg/scriptingosx-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.valdore86-recipes': {u'URL': u'https://github.com/autopkg/valdore86-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.wardsparadox-recipes': {u'URL': u'https://github.com/autopkg/wardsparadox-recipes'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg': {u'URL': u'https://github.com/facebook/Recipes-for-AutoPkg.git'},
                   u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.gitlab.techahoy.autopkg': {u'URL': u'https://gitlab.com/techahoy/autopkg'}},
 u'RECIPE_SEARCH_DIRS': [u'.',
                         u'~/Library/AutoPkg/Recipes',
                         u'/Library/AutoPkg/Recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.gitlab.techahoy.autopkg',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.jessepeterson-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.jlehikoinen-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.arubdesu-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.killahquam-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.munkireport-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.apizz-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.wardsparadox-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.foigus-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.its-unibas-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.dataJAR-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.homebysix-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.hansen-m-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.keeleysam-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.scriptingosx-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.chilcote-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.HobbitHardcase-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.rtrouton-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.valdore86-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.autopkg.apettinen-recipes',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg',
                         '/Users/autopkg/Library/AutoPkg/RecipeOverrides',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode',
                         u'/Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.facebook.Recipes-for-AutoPkg/Xcode'],
 u'VERSION_EMIT_PATH': u'%RECIPE_CACHE_DIR%/xcode_tag',
 u'pkginfo': {
    catalogs =     (
        testing
    );
    description = Xcode;
    "display_name" = Xcode;
    name = Xcode;
    "unattended_install" = 1;
},
 'verbose': 4}
AppleDataGatherer
{'Input': {'appID_key': u'891bd3417a7776362562d2197f89480a8547b108fd934911bcbea0110d07f757',
           'apple_id': u'email@domain.com',
           'password_file': u'secret.txt'}}
AppleDataGatherer: Writing data to file
{'Output': {'data_pathname': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/login_data'}}
AppleCookieDownloader
{'Input': {'login_data': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/login_data'}}
AppleCookieDownloader: No value supplied for CURL_PATH, setting default value of: /usr/bin/curl
AppleCookieDownloader: Getting login cookie
AppleCookieDownloader: Getting download cookie
AppleCookieDownloader: Unzipping download list
{'Output': {'download_cookies': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/download_cookies'}}
AppleURLSearcher
{'Input': {'re_pattern': u'((?!.*beta).*\\/Xcode_.*\\/Xcode.*.xip)'}}
AppleURLSearcher: No value supplied for result_output_var_name, setting default value of: match
AppleURLSearcher: Beta flag not set, searching More downloads list...
AppleURLSearcher: Sorted list of possible filenames: [u'Xcode_8', u'Xcode_8.1', u'Xcode_8.2', u'Xcode_8.2.1', u'Xcode_8.3', u'Xcode_8.3.1', u'Xcode_8.3.2', u'Xcode_8.3.3', u'Xcode_9', u'Xcode_9.0.1', u'Xcode_9.1', u'Xcode_9.2', u'Xcode_9.3', u'Xcode_9.3.1', u'Xcode_9.4', u'Xcode_9.4.1', u'Xcode_10', u'Xcode_10.1', u'Xcode_10.2', u'Xcode_10.2.1', u'Xcode_10.3', u'Xcode_11', u'Xcode_11.1', u'Xcode_11.2', u'Xcode_11.2.1']
AppleURLSearcher: Found matching item: Xcode_11.2.1
AppleURLSearcher: Full URL: https://download.developer.apple.com/Developer_Tools/Xcode_11.2.1/Xcode_11.2.1.xip
{'Output': {'match': u'https://download.developer.apple.com/Developer_Tools/Xcode_11.2.1/Xcode_11.2.1.xip'}}
AppleCookieDownloader
{'Input': {'CURL_PATH': '/usr/bin/curl',
           'login_data': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/login_data'}}
AppleCookieDownloader: Getting login cookie
AppleCookieDownloader: Getting download cookie
AppleCookieDownloader: Unzipping download list
{'Output': {'download_cookies': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/download_cookies'}}
XcodeVersionEmitter
{'Input': {'dont_skip': u'',
           'output_filepath': u'%RECIPE_CACHE_DIR%/xcode_tag',
           'url': u'https://download.developer.apple.com/Developer_Tools/Xcode_11.2.1/Xcode_11.2.1.xip'}}
XcodeVersionEmitter: dont_skip is false, so skipping this Processor.
{'Output': {}}
EndOfCheckPhase
{'Input': {}}
{'Output': {}}
URLDownloader
{'Input': {'CURL_PATH': '/usr/bin/curl',
           'curl_opts': (
    "--request",
    GET,
    "--cookie",
    "/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/download_cookies"
),
           'url': u'https://download.developer.apple.com/Developer_Tools/Xcode_11.2.1/Xcode_11.2.1.xip'}}
URLDownloader: No value supplied for CHECK_FILESIZE_ONLY, setting default value of: False
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: Item at URL is unchanged.
URLDownloader: Using existing /Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/Xcode_11.2.1.xip
{'Output': {'pathname': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/Xcode_11.2.1.xip'}}
PathDeleter
{'Input': {'path_list': (
    "/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/login_data"
)}}
PathDeleter: Deleted /Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/login_data
{'Output': {}}
XcodeXIPUnpacker
{'Input': {'PKG': u'/Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/Xcode_11.2.1.xip'}}
XcodeXIPUnpacker: Extracting xip archive, please be patient, this could take a long time...
Traceback (most recent call last):
  File "/usr/local/bin/autopkg", line 2810, in <module>
    sys.exit(main(sys.argv))
  File "/usr/local/bin/autopkg", line 2804, in main
    exit(subcommands[verb]["function"](argv))
  File "/usr/local/bin/autopkg", line 2324, in run_recipes
    autopackager.process(recipe)
  File "/Library/AutoPkg/autopkglib/__init__.py", line 684, in process
    log_err(unicode(err))  # noqa TODO
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 24: ordinal not in range(128)```

@nmcspadden
Copy link
Contributor

What happens if you manually run xip --expand /Users/autopkg/Library/AutoPkg/Cache/local.munki.xcode/downloads/Xcode_11.2.1.xip? Sounds like there's an issue extracting the package, but it doesn't seem to be related to the recipe itself. The recipe failed because it tried to log a unicode string incorrectly (which I can fix).

@nmcspadden
Copy link
Contributor

I can't reproduce this on any machine, so I'm closing this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nmcspadden @mikethehill