Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scancode license scan produces invalid yaml #3219

Closed
AyanSinhaMahapatra opened this issue Jan 23, 2023 · 3 comments · Fixed by #3220
Closed

scancode license scan produces invalid yaml #3219

AyanSinhaMahapatra opened this issue Jan 23, 2023 · 3 comments · Fixed by #3220

Comments

@AyanSinhaMahapatra
Copy link
Member

Description

Please leave a brief description of the bug or feature request:

scancode license scan now has license references by default, and as license texts
often has whitespace to the left, this leads to invalid yaml output when --license
option is used.

How To Reproduce

Tell us how to reproduce the issue.

From scancode latest develop:

./configure --clean && ./configure --dev
scancode -clipeu --classify --summary samples --yaml output.yaml
python
> import saneyaml
> path = "path/to/output.yaml"
>  data = saneyaml.load(open(path).read())

This results in:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ayansinha/nexB/write_access/scancode-toolkit/venv/lib/python3.8/site-packages/saneyaml.py", line 62, in load
    return yaml.load(s, Loader=loader)
  File "/home/ayansinha/nexB/write_access/scancode-toolkit/venv/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/home/ayansinha/nexB/write_access/scancode-toolkit/venv/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "yaml/_yaml.pyx", line 673, in yaml._yaml.CParser.get_single_node
  File "yaml/_yaml.pyx", line 687, in yaml._yaml.CParser._compose_document
  File "yaml/_yaml.pyx", line 731, in yaml._yaml.CParser._compose_node
  File "yaml/_yaml.pyx", line 845, in yaml._yaml.CParser._compose_mapping_node
  File "yaml/_yaml.pyx", line 729, in yaml._yaml.CParser._compose_node
  File "yaml/_yaml.pyx", line 806, in yaml._yaml.CParser._compose_sequence_node
  File "yaml/_yaml.pyx", line 731, in yaml._yaml.CParser._compose_node
  File "yaml/_yaml.pyx", line 847, in yaml._yaml.CParser._compose_mapping_node
  File "yaml/_yaml.pyx", line 860, in yaml._yaml.CParser._parse_next_event
yaml.parser.ParserError: while parsing a block mapping
  in "<unicode string>", line 561, column 9
did not find expected key
  in "<unicode string>", line 600, column 40

System configuration

For bug reports, it really helps us to know:

  • What OS are you running on? (Windows/MacOS/Linux) ubuntu 20.04
  • What version of scancode-toolkit was used to generate the scan file? 32.0.0rc1
  • What installation method was used to install/run scancode? (pip/source download/other) source checkout
@AyanSinhaMahapatra
Copy link
Member Author

AyanSinhaMahapatra commented Jan 23, 2023

This also happens with the --license-text option enabled, which adds the attribute matched_text.

AyanSinhaMahapatra added a commit that referenced this issue Jan 23, 2023
- Add tests verifying JSON and YAML outputs are valid and
  can be loaded without crashing.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
AyanSinhaMahapatra added a commit that referenced this issue Apr 4, 2023
License texts were producing invalid YAML files when there were
different indentations in the text along with empty newlines, and
to make these texts YAML safe we add a single space to empty
newlines to license texts with different indentation levels.

Reference: #3219
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Copy link
Member Author

See #3220 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant