Don't add etag/last_modified headers if CHECK_FILESIZE_ONLY == True #866

MLBZ521 · 2023-07-15T10:01:06Z

So, this may be an extremely rare issue...but came across it on the recipe com.github.moofit-recipes.download.MirrorOp and I described it in autopkg/moofit-recipes#126.

To re-cap here:

The web server is not accepting, or allowing, the etag If-None-Match and last modified If-Modified-Since header checks. If these are included, then the web server will return a 400, but if they are not, it processes the request as expected. Not quite sure why they're not supporting this as it literally should be saving them bandwidth...

When setting CHECK_FILESIZE_ONLY as True in the URLDownloader processor step, the parent class URLGetter processor still adds the etag and last modified headers to the call.

I'm not sure if this was an oversight or by design, but as far as I know, it hasn't affected anything else for all these years...

Also not sure if this is something that the maintainers would want to merge considering how little it seems to be affecting...but I'll at least propose it here and let you decide.

homebysix · 2023-07-15T12:42:28Z

Seems reasonable. Could you include verbose output showing an affected recipe running before and after the change?

Code/autopkglib/URLGetter.py

MLBZ521 · 2023-07-15T19:03:31Z

Yeah, I can do this as well. 👍

MLBZ521 · 2023-07-17T05:15:48Z

Here are three verbose (-vvvv) runs demonstrating the issue. Please note, that I've customized the com.github.moofit-recipes.download.MirrorOp recipe (from it's current state) to demonstrate this issue (it currently has a "fix" which is deleting the entire recipe's directory to work around this issue).

First run: successfully downloads the file and stores the etag/last modified date as expected.

$ autopkg run com.github.moofit-recipes.download.MirrorOp -vvvv
Processing com.github.moofit-recipes.download.MirrorOp...
WARNING: com.github.moofit-recipes.download.MirrorOp is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
{'AUTOPKG_VERSION': '2.7.2',
 'NAME': 'MirrorOp',
 'PARENT_RECIPES': [],
 'RECIPE_CACHE_DIR': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp',
 'RECIPE_DIR': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco/MirrorOp.download.recipe',
 'RECIPE_REPOS': {'/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes': {'URL': 'https://github.com/autopkg/moofit-recipes'}},
 'RECIPE_SEARCH_DIRS': ['.',
                        '~/Library/AutoPkg/Recipes',
                        '/Library/AutoPkg/Recipes',
                        '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes'],
 'verbose': 4}
URLTextSearcher
{'Input': {'re_pattern': 'downloadUrl\\":\\"(.*)\\",',
           'result_output_var_name': 'match',
           'url': 'https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=R33050100&tdeType=3'}}
URLTextSearcher: Curl command: ['/usr/bin/curl', '--compressed', '--location', 'https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=R33050100&tdeType=3']
URLTextSearcher: Found matching text (match): https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?5tsYTTV108n7NwOwoJqu2jU6oBtYvhsJaxBV5xlqsoZGO0ACDVIg4H-8IpFYTfy5kz6uTwPWpWYfFACUQEmfuVdlOZrn
{'Output': {'match': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?5tsYTTV108n7NwOwoJqu2jU6oBtYvhsJaxBV5xlqsoZGO0ACDVIg4H-8IpFYTfy5kz6uTwPWpWYfFACUQEmfuVdlOZrn'}}
URLDownloader
{'Input': {'CHECK_FILESIZE_ONLY': True,
           'filename': 'MirrorOp.dmg',
           'url': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?5tsYTTV108n7NwOwoJqu2jU6oBtYvhsJaxBV5xlqsoZGO0ACDVIg4H-8IpFYTfy5kz6uTwPWpWYfFACUQEmfuVdlOZrn'}}
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: Curl command: ['/usr/bin/curl', '--silent', '--show-error', '--no-buffer', '--dump-header', '-', '--speed-time', '30', '--location', '--url', 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?5tsYTTV108n7NwOwoJqu2jU6oBtYvhsJaxBV5xlqsoZGO0ACDVIg4H-8IpFYTfy5kz6uTwPWpWYfFACUQEmfuVdlOZrn', '--fail', '--output', '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/tmplvslu6sp']
URLDownloader: Storing new Last-Modified header: Thu, 22 Jul 2021 08:01:06 GMT
URLDownloader: Storing new ETag header: "0x8D94CE6DB989671"
URLDownloader: Downloaded /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg
{'Output': {'download_changed': True,
            'etag': '"0x8D94CE6DB989671"',
            'last_modified': 'Thu, 22 Jul 2021 08:01:06 GMT',
            'pathname': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg',
            'url_downloader_summary_result': {'data': {'download_path': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg'},
                                              'summary_text': 'The following '
                                                              'new items were '
                                                              'downloaded:'}}}
EndOfCheckPhase
{'Input': {}}
{'Output': {}}
CodeSignatureVerifier
{'Input': {'input_path': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg/MirrorOp.app',
           'requirement': 'identifier "com.MirrorOp.Vitali" and anchor apple '
                          'generic and certificate '
                          '1[field.1.2.840.113635.100.6.2.6] /* exists */ and '
                          'certificate leaf[field.1.2.840.113635.100.6.1.13] '
                          '/* exists */ and certificate leaf[subject.OU] = '
                          'UWHQ4AWKZ8'}}
CodeSignatureVerifier: Mounted disk image /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg
CodeSignatureVerifier: Verifying code signature...
CodeSignatureVerifier: Deep verification enabled...
CodeSignatureVerifier: Strict verification not defined. Using codesign defaults...
CodeSignatureVerifier: /private/tmp/dmg.9LURlM/MirrorOp.app: valid on disk
CodeSignatureVerifier: /private/tmp/dmg.9LURlM/MirrorOp.app: satisfies its Designated Requirement
CodeSignatureVerifier: /private/tmp/dmg.9LURlM/MirrorOp.app: explicit requirement satisfied
CodeSignatureVerifier: Signature is valid
{'Output': {}}
{'AUTOPKG_VERSION': '2.7.2',
 'CHECK_FILESIZE_ONLY': True,
 'NAME': 'MirrorOp',
 'PARENT_RECIPES': [],
 'RECIPE_CACHE_DIR': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp',
 'RECIPE_DIR': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco/MirrorOp.download.recipe',
 'RECIPE_REPOS': {'/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes': {'URL': 'https://github.com/autopkg/moofit-recipes'}},
 'RECIPE_SEARCH_DIRS': ['.',
                        '~/Library/AutoPkg/Recipes',
                        '/Library/AutoPkg/Recipes',
                        '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes'],
 'download_changed': True,
 'etag': '"0x8D94CE6DB989671"',
 'filename': 'MirrorOp.dmg',
 'input_path': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg/MirrorOp.app',
 'last_modified': 'Thu, 22 Jul 2021 08:01:06 GMT',
 'match': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?5tsYTTV108n7NwOwoJqu2jU6oBtYvhsJaxBV5xlqsoZGO0ACDVIg4H-8IpFYTfy5kz6uTwPWpWYfFACUQEmfuVdlOZrn',
 'pathname': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg',
 'prefetch_filename': False,
 're_pattern': 'downloadUrl\\":\\"(.*)\\",',
 'requirement': 'identifier "com.MirrorOp.Vitali" and anchor apple generic and '
                'certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ '
                'and certificate leaf[field.1.2.840.113635.100.6.1.13] /* '
                'exists */ and certificate leaf[subject.OU] = UWHQ4AWKZ8',
 'result_output_var_name': 'match',
 'url': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?5tsYTTV108n7NwOwoJqu2jU6oBtYvhsJaxBV5xlqsoZGO0ACDVIg4H-8IpFYTfy5kz6uTwPWpWYfFACUQEmfuVdlOZrn',
 'url_downloader_summary_result': {'data': {'download_path': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg'},
                                   'summary_text': 'The following new items '
                                                   'were downloaded:'},
 'verbose': 4}
Receipt written to /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/receipts/com.github.moofit-recipes.download-receipt-20230716-215442.plist

The following new items were downloaded:
    Download Path                                                                                             
    -------------                                                                                             
    /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg

Second run, without fix: Even though CHECK_FILESIZE_ONLY is True, the headers 'If-None-Match: "0x8D94CE6DB989671"' and 'If-Modified-Since: Thu, 22 Jul 2021 08:01:06 GMT' are still sent, which cause this webserver to return an HTTP 400 error.

$ autopkg run com.github.moofit-recipes.download.MirrorOp -vvvv
Processing com.github.moofit-recipes.download.MirrorOp...
WARNING: com.github.moofit-recipes.download.MirrorOp is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
{'AUTOPKG_VERSION': '2.7.2',
 'NAME': 'MirrorOp',
 'PARENT_RECIPES': [],
 'RECIPE_CACHE_DIR': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp',
 'RECIPE_DIR': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco/MirrorOp.download.recipe',
 'RECIPE_REPOS': {'/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes': {'URL': 'https://github.com/autopkg/moofit-recipes'}},
 'RECIPE_SEARCH_DIRS': ['.',
                        '~/Library/AutoPkg/Recipes',
                        '/Library/AutoPkg/Recipes',
                        '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes'],
 'verbose': 4}
URLTextSearcher
{'Input': {'re_pattern': 'downloadUrl\\":\\"(.*)\\",',
           'result_output_var_name': 'match',
           'url': 'https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=R33050100&tdeType=3'}}
URLTextSearcher: Curl command: ['/usr/bin/curl', '--compressed', '--location', 'https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=R33050100&tdeType=3']
URLTextSearcher: Found matching text (match): https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?j6k2otaPuPTOgmiLnkVC-RzeqXJEdkVIDFt9rB3IKiIH4jVvgsvy-CmJ__s-vz9OhCaWgJvmHl7qEjgMaPvkHib1HaKa
{'Output': {'match': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?j6k2otaPuPTOgmiLnkVC-RzeqXJEdkVIDFt9rB3IKiIH4jVvgsvy-CmJ__s-vz9OhCaWgJvmHl7qEjgMaPvkHib1HaKa'}}
URLDownloader
{'Input': {'CHECK_FILESIZE_ONLY': True,
           'filename': 'MirrorOp.dmg',
           'url': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?j6k2otaPuPTOgmiLnkVC-RzeqXJEdkVIDFt9rB3IKiIH4jVvgsvy-CmJ__s-vz9OhCaWgJvmHl7qEjgMaPvkHib1HaKa'}}
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: ERROR: (22) The requested URL returned error: 400

Traceback (most recent call last):
  File "/Library/AutoPkg/autopkglib/URLGetter.py", line 172, in execute_curl
    result = subprocess.run(
  File "/Library/AutoPkg/Python3/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/curl', '--silent', '--show-error', '--no-buffer', '--dump-header', '-', '--speed-time', '30', '--location', '--url', 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?j6k2otaPuPTOgmiLnkVC-RzeqXJEdkVIDFt9rB3IKiIH4jVvgsvy-CmJ__s-vz9OhCaWgJvmHl7qEjgMaPvkHib1HaKa', '--fail', '--output', '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/tmpts6df5sy', '--header', 'If-None-Match: "0x8D94CE6DB989671"', '--header', 'If-Modified-Since: Thu, 22 Jul 2021 08:01:06 GMT']' returned non-zero exit status 22.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/AutoPkg/autopkglib/__init__.py", line 840, in process
    self.env = processor.process()
  File "/Library/AutoPkg/autopkglib/__init__.py", line 626, in process
    self.main()
  File "/Library/AutoPkg/autopkglib/URLDownloader.py", line 331, in main
    raw_headers = self.download_with_curl(curl_cmd)
  File "/Library/AutoPkg/autopkglib/URLGetter.py", line 186, in download_with_curl
    proc_stdout, proc_stderr, retcode = self.execute_curl(curl_cmd, text)
  File "/Library/AutoPkg/autopkglib/URLGetter.py", line 181, in execute_curl
    raise ProcessorError(e.stderr) from e
autopkglib.ProcessorError: curl: (22) The requested URL returned error: 400

  File "/Library/AutoPkg/autopkglib/__init__.py", line 840, in process
    self.env = processor.process()
curl: (22) The requested URL returned error: 400

Failed.
Receipt written to /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/receipts/com.github.moofit-recipes.download-receipt-20230716-215555.plist

The following recipes failed:
    com.github.moofit-recipes.download.MirrorOp
        Error in com.github.moofit-recipes.download.MirrorOp: Processor: URLDownloader: Error: curl: (22) The requested URL returned error: 400

Nothing downloaded, packaged or imported.

Third run, with fix applied: etag/last modified headers are not sent, but file size is checked and confirmed the same -- no errors, existing file is used, and no reported "new download."

$ autopkg run com.github.moofit-recipes.download.MirrorOp -vvvv
Processing com.github.moofit-recipes.download.MirrorOp...
WARNING: com.github.moofit-recipes.download.MirrorOp is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
{'AUTOPKG_VERSION': '2.7.2',
 'NAME': 'MirrorOp',
 'PARENT_RECIPES': [],
 'RECIPE_CACHE_DIR': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp',
 'RECIPE_DIR': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco/MirrorOp.download.recipe',
 'RECIPE_REPOS': {'/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes': {'URL': 'https://github.com/autopkg/moofit-recipes'}},
 'RECIPE_SEARCH_DIRS': ['.',
                        '~/Library/AutoPkg/Recipes',
                        '/Library/AutoPkg/Recipes',
                        '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes'],
 'verbose': 4}
URLTextSearcher
{'Input': {'re_pattern': 'downloadUrl\\":\\"(.*)\\",',
           'result_output_var_name': 'match',
           'url': 'https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=R33050100&tdeType=3'}}
URLTextSearcher: Curl command: ['/usr/bin/curl', '--compressed', '--location', 'https://www.barco.com/bin/barco/tde/downloadUrl.json?fileNumber=R33050100&tdeType=3']
URLTextSearcher: Found matching text (match): https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?qNPNCZLQPnFgBvjvQ5dMSzYlQQx6nur7iLSPifgqFHGLBoccgj0hBrzXSJB5NQmP6CGWSJRjjEC8VVdEZpXXsmFAg1Ok
{'Output': {'match': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?qNPNCZLQPnFgBvjvQ5dMSzYlQQx6nur7iLSPifgqFHGLBoccgj0hBrzXSJB5NQmP6CGWSJRjjEC8VVdEZpXXsmFAg1Ok'}}
URLDownloader
{'Input': {'CHECK_FILESIZE_ONLY': True,
           'filename': 'MirrorOp.dmg',
           'url': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?qNPNCZLQPnFgBvjvQ5dMSzYlQQx6nur7iLSPifgqFHGLBoccgj0hBrzXSJB5NQmP6CGWSJRjjEC8VVdEZpXXsmFAg1Ok'}}
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: Curl command: ['/usr/bin/curl', '--silent', '--show-error', '--no-buffer', '--dump-header', '-', '--speed-time', '30', '--location', '--url', 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?qNPNCZLQPnFgBvjvQ5dMSzYlQQx6nur7iLSPifgqFHGLBoccgj0hBrzXSJB5NQmP6CGWSJRjjEC8VVdEZpXXsmFAg1Ok', '--fail', '--output', '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/tmpvq40qv3p']
URLDownloader: File size returned by webserver matches that of the cached file: 19432522 bytes
URLDownloader: WARNING: Matching a download by filesize is a fallback mechanism that does not guarantee that a build is unchanged.
URLDownloader: Using existing /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg
{'Output': {'pathname': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg'}}
EndOfCheckPhase
{'Input': {}}
{'Output': {}}
CodeSignatureVerifier
{'Input': {'input_path': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg/MirrorOp.app',
           'requirement': 'identifier "com.MirrorOp.Vitali" and anchor apple '
                          'generic and certificate '
                          '1[field.1.2.840.113635.100.6.2.6] /* exists */ and '
                          'certificate leaf[field.1.2.840.113635.100.6.1.13] '
                          '/* exists */ and certificate leaf[subject.OU] = '
                          'UWHQ4AWKZ8'}}
CodeSignatureVerifier: Mounted disk image /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg
CodeSignatureVerifier: Verifying code signature...
CodeSignatureVerifier: Deep verification enabled...
CodeSignatureVerifier: Strict verification not defined. Using codesign defaults...
CodeSignatureVerifier: /private/tmp/dmg.fTgID3/MirrorOp.app: valid on disk
CodeSignatureVerifier: /private/tmp/dmg.fTgID3/MirrorOp.app: satisfies its Designated Requirement
CodeSignatureVerifier: /private/tmp/dmg.fTgID3/MirrorOp.app: explicit requirement satisfied
CodeSignatureVerifier: Signature is valid
{'Output': {}}
{'AUTOPKG_VERSION': '2.7.2',
 'CHECK_FILESIZE_ONLY': True,
 'NAME': 'MirrorOp',
 'PARENT_RECIPES': [],
 'RECIPE_CACHE_DIR': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp',
 'RECIPE_DIR': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco',
 'RECIPE_OVERRIDE_DIRS': ['~/Library/AutoPkg/RecipeOverrides'],
 'RECIPE_PATH': '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes/Barco/MirrorOp.download.recipe',
 'RECIPE_REPOS': {'/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes': {'URL': 'https://github.com/autopkg/moofit-recipes'}},
 'RECIPE_SEARCH_DIRS': ['.',
                        '~/Library/AutoPkg/Recipes',
                        '/Library/AutoPkg/Recipes',
                        '/Users/<redacted>/Library/AutoPkg/RecipeRepos/com.github.autopkg.moofit-recipes'],
 'download_changed': False,
 'etag': '',
 'filename': 'MirrorOp.dmg',
 'input_path': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg/MirrorOp.app',
 'last_modified': '',
 'match': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?qNPNCZLQPnFgBvjvQ5dMSzYlQQx6nur7iLSPifgqFHGLBoccgj0hBrzXSJB5NQmP6CGWSJRjjEC8VVdEZpXXsmFAg1Ok',
 'pathname': '/Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/downloads/MirrorOp.dmg',
 'prefetch_filename': False,
 're_pattern': 'downloadUrl\\":\\"(.*)\\",',
 'requirement': 'identifier "com.MirrorOp.Vitali" and anchor apple generic and '
                'certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ '
                'and certificate leaf[field.1.2.840.113635.100.6.1.13] /* '
                'exists */ and certificate leaf[subject.OU] = UWHQ4AWKZ8',
 'result_output_var_name': 'match',
 'url': 'https://barcoprdwebsitefs.azureedge.net/barcoprdfs/Data/secure/downloads/tde/Active/SoftwareFiles/ApplicationSoftware/R33050100_11_ApplicationSw.dmg?qNPNCZLQPnFgBvjvQ5dMSzYlQQx6nur7iLSPifgqFHGLBoccgj0hBrzXSJB5NQmP6CGWSJRjjEC8VVdEZpXXsmFAg1Ok',
 'verbose': 4}
Receipt written to /Users/<redacted>/Library/AutoPkg/Cache/com.github.moofit-recipes.download.MirrorOp/receipts/com.github.moofit-recipes.download-receipt-20230716-215814.plist

Nothing downloaded, packaged or imported.

Don't add etag/last_modified headers if CHECK_FILESIZE_ONLY == True

d70ca22

homebysix reviewed Jul 15, 2023

View reviewed changes

Code/autopkglib/URLGetter.py Outdated Show resolved Hide resolved

Do this more safely

fbfc104

nmcspadden merged commit 84b011e into autopkg:dev Aug 10, 2023
0 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't add etag/last_modified headers if CHECK_FILESIZE_ONLY == True #866

Don't add etag/last_modified headers if CHECK_FILESIZE_ONLY == True #866

MLBZ521 commented Jul 15, 2023

homebysix commented Jul 15, 2023

MLBZ521 commented Jul 15, 2023

MLBZ521 commented Jul 17, 2023

Don't add etag/last_modified headers if CHECK_FILESIZE_ONLY == True #866

Don't add etag/last_modified headers if CHECK_FILESIZE_ONLY == True #866

Conversation

MLBZ521 commented Jul 15, 2023

homebysix commented Jul 15, 2023

MLBZ521 commented Jul 15, 2023

MLBZ521 commented Jul 17, 2023