Unicode parsing failure

[tonkapango]

Is this a request for help?:
yes

Is this a BUG REPORT or a FEATURE REQUEST? (choose one):
BUG

Version of Anchore Engine and Anchore CLI if applicable:

anchore-cli, version 0.8.1
engine: v0.9.1
What happened:
parsing failure on a unicode field when trying to catalogue python packages.

What did you expect to happen:
proper parsing and scan to complete without this uncaught failure.

Any relevant log output from /var/log/anchore:
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"cataloger 'ruby-gemspec-cataloger' discovered '0' packages","time":"2021-02-11 22:11:03"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"encountered package.json file without a name and/or version field, ignoring this file","time":"2021-02-11 22:11:05"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"cataloger 'javascript-package-cataloger' discovered '11' packages","time":"2021-02-11 22:11:05"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"cataloger 'dpkgdb-cataloger' discovered '332' packages","time":"2021-02-11 22:11:06"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"cataloger 'rpmdb-cataloger' discovered '0' packages","time":"2021-02-11 22:11:06"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"cataloger 'java-cataloger' discovered '2' packages","time":"2021-02-11 22:11:09"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"cataloger 'apkdb-cataloger' discovered '0' packages","time":"2021-02-11 22:11:09"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"debug","msg":"cataloger 'go-cataloger' discovered '0' packages","time":"2021-02-11 22:11:10"}
2021-02-11 22:11:11+0000 [-] [Thread-764] [anchore_engine.utils/run_check()] [ERROR] {"level":"error","msg":"failed to catalog input: 1 error occurred:\n\t* unable to catalog python package=/opt/mxnet/python/mxnet.egg-info/PKG-INFO: cannot parse field from line: '\u003c!--- or more contributor license agreements. See the NOTICE file --\u003e'\n\n","time":"2021-02-11 22:11:11"}
2021-02-11 22:11:13+0000 [-] Traceback (most recent call last):
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/clients/localanchore_standalone.py", line 1100, in analyze_image
2021-02-11 22:11:13+0000 [-] analyzer_report = run_anchore_analyzers(
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/clients/localanchore_standalone.py", line 903, in run_anchore_analyzers
2021-02-11 22:11:13+0000 [-] analyzer_report = analyzer_manager.run(
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/analyzers/manager.py", line 18, in run
2021-02-11 22:11:13+0000 [-] _run_syft(analyzer_report, copydir)
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/analyzers/manager.py", line 91, in _run_syft
2021-02-11 22:11:13+0000 [-] results = syft.catalog_image(imagedir=copydir)
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/analyzers/syft/init.py", line 16, in catalog_image
2021-02-11 22:11:13+0000 [-] all_results = run_syft(imagedir)
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/clients/syft_wrapper.py", line 20, in run_syft
2021-02-11 22:11:13+0000 [-] stdout, _ = run_check(shlex.split(cmd), env=proc_env)
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/utils.py", line 277, in run_check
2021-02-11 22:11:13+0000 [-] raise CommandException(cmd, code, stdout, stderr)
2021-02-11 22:11:13+0000 [-] anchore_engine.utils.CommandException: Non-zero exit status code when running subprocess: cmd=syft -vv -o json oci-dir:/analysis_scratch/acbdc3f6-29e6-4756-be55-4e6752cf0ccf/raw, rc=1
2021-02-11 22:11:13+0000 [-]
2021-02-11 22:11:13+0000 [-] During handling of the above exception, another exception occurred:
2021-02-11 22:11:13+0000 [-]
2021-02-11 22:11:13+0000 [-] Traceback (most recent call last):
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/services/analyzer/analysis.py", line 333, in process_analyzer_job
2021-02-11 22:11:13+0000 [-] image_data = perform_analyze(
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/services/analyzer/analysis.py", line 193, in perform_analyze
2021-02-11 22:11:13+0000 [-] analyzed_image_report, manifest_raw = localanchore_standalone.analyze_image(
2021-02-11 22:11:13+0000 [-] File "/usr/local/lib/python3.8/site-packages/anchore_engine/clients/localanchore_standalone.py", line 1125, in analyze_image
2021-02-11 22:11:13+0000 [-] raise AnalysisError(
2021-02-11 22:11:13+0000 [-] anchore_engine.clients.localanchore_standalone.AnalysisError: failed to download, unpack, analyze, and generate image export (nvcr.io/nvidian/prodsec/sridevi@sha256:104a0b3f3cc2f7f315b60390685450a0355dd6a76f43ea3ad8d046183658cc57) - exception: Non-zero exit status code when running subprocess: cmd=syft -vv -o json oci-dir:/analysis_scratch/acbdc3f6-29e6-4756-be55-4e6752cf0ccf/raw, rc=1
2021-02-11 22:11:13+0000 [-] [Thread-764] [anchore_engine.services.analyzer.analysis/process_analyzer_job()] [ERROR] problem analyzing image - exception: failed to download, unpack, analyze, and generate image export (nvcr.io/nvidian/prodsec/sridevi@sha256:104a0b3f3cc2f7f315b60390685450a0355dd6a76f43ea3ad8d046183658cc57) - exception: Non-zero exit status code when running subprocess: cmd=syft -vv -o json oci-dir:/analysis_scratch/acbdc3f6-29e6-4756-be55-4e6752cf0ccf/raw, rc=1
2021-02-11 22:11:17+0000 [-] [Thread-8] [anchore_engine.services.analyzer.service/handle_image_analyzer()] [INFO] worker thread completed

What docker images are you using:
anchore/anchore-engine v0.9.1 aec8da91351f 8 days ago 599MB

How to reproduce the issue:
anchore-cli image add basimentos/poc:anchore_unicode_bug

added a slimmed down image, with the problem. the file can be found /opt/python/egg-info

Anything else we need to know:
thanks!!!!!!!!

[zhill]

HI, @tonkapango , thanks for reporting this. We're looking into it. This looks like an issue within Syft itself. If it proves to be that we'll open an issue in that repo as well to track the fix there, which will then be included in engine after it gets released in Syft.

To reproduce it:

❯ syft docker:basimentos/poc:anchore_unicode_bug
 ✔ Pulled image
 ✔ Loaded image
 ✔ Parsed image
 ⠋ Cataloging image     [packages 0]

[0025] ERROR failed to catalog input: 1 error occurred:
	* unable to catalog python package=/opt/python/egg-info/PKG-INFO: cannot parse field from line: '<!--- or more contributor license agreements.  See the NOTICE file -->'

[zhill]

Fix will be delivered as a dependency update on the Syft version.

[dland]

A report from the field:

I was having a variant of the same problem, python analysis aborting on a multi-line string. Upgraded from 0.9.0 to 0.9.2 and the problem went away. Thanks!