No license detected in a docker scans of docker://redis:buster

[pombredanne]

I created a project and scanned docker://redis:buster with a docker pipeline.
No license was detected at all using the tip of the main branch

[tdruez]

Let's take a data example of an installed_packages in https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/docker.py#L182

{'api_data_url': None,
 'bug_tracking_url': None,
 'code_view_url': None,
 'copyright': None,
 'declared_license': None,
 'source_packages': ['pkg:deb/zlib'],
 'subpath': None,
 'type': 'deb',
 'vcs_url': None,
 'version': '1:1.2.11.dfsg-1'
 'description': 'compression library - runtime\n'
                ' zlib is a library implementing the deflate compression '
                'method found\n'
                ' in gzip and PKZIP.  This package includes the shared '
                'library.',
 'homepage_url': 'http://zlib.net/',
 'keywords': ['libs'],
 'license_expression': None,
 'name': 'zlib1g',
 'namespace': None,
 'notice_text': None,
 'package_uid': 'pkg:deb/zlib1g@1:1.2.11.dfsg-1?architecture=amd64&uuid=fec22812-0d35-49e2-a0d9-7c9d381138a2',
 'primary_language': None,
 'purl': 'pkg:deb/zlib1g@1:1.2.11.dfsg-1?architecture=amd64',
 'qualifiers': {'architecture': 'amd64'},
 'resources': [{'for_packages': ['pkg:deb/zlib1g@1:1.2.11.dfsg-1?architecture=amd64&uuid=fec22812-0d35-49e2-a0d9-7c9d381138a2',
                                 'pkg:deb/zlib1g@1:1.2.11.dfsg-1?architecture=amd64&uuid=fec22812-0d35-49e2-a0d9-7c9d381138a2'],
                'package_data': [{'api_data_url': None,
                                  'bug_tracking_url': None,
                                  'code_view_url': None,
                                  'copyright': '1995-2013 Jean-loup Gailly and '
                                               'Mark Adler\n'
                                               '1998 by Andreas R. Kleinert\n'
                                               '1998-2010 Gilles Vollant\n'
                                               '2007-2008 Even Rouault\n'
                                               '2009-2010 Mathias Svensson\n'
                                               '2000-2017 Mark Brown',
                                  'datasource_id': 'debian_copyright_in_package',
                                  'declared_license': ['Zlib',
                                                       'Zlib',
                                                       'Zlib',
                                                       'Zlib',
                                                       'Zlib'],
                                  'dependencies': [],
                                  'description': None,
                                  'download_url': None,
                                  'extra_data': {},
                                  'file_references': [],
                                  'homepage_url': None,
                                  'keywords': [],
                                  'license_expression': 'zlib AND zlib AND '
                                                        'zlib AND zlib',
                                  'md5': None,
                                  'name': 'zlib1g',
                                  'namespace': None,
                                  'notice_text': None,
                                  'parties': [],
                                  'primary_language': None,
                                  'purl': 'pkg:deb/zlib1g',
                                  'qualifiers': {},
                                  'release_date': None,
                                  'repository_download_url': None,
                                  'repository_homepage_url': None,
                                  'sha1': None,
                                  'sha256': None,
                                  'sha512': None,
                                  'size': None,
                                  'source_packages': [],
                                  'subpath': None,
                                  'type': 'deb',
                                  'vcs_url': None,
                                  'version': None}],
                'path': '476baebdfbf7a68c50e979971fcd47d799d1b194bcf1f03c1c979e9262bcd364/usr/share/doc/zlib1g/copyright',
                'scan_errors': [],
                'type': 'file'},
 }

1. copyright, declared_license, and license_expression are empty at the package level so during package creation in https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/docker.py#L186 nothing is set for those fields.

2. The resources.package_data contains scan values but those are never used in the current code, as the package is not created from those, and the codebase resource is never updated with those neither when assigned to the package instance https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/docker.py#L219

3. For some reason entries are duplicated in for_packages.

[pombredanne]

Thanks for the inputs
I tracked the issue of 1. to a bug in SCTK... working on a fix!

For 2., resources.package_data are no longer used correctly but the data should still be processed correctly. I will submit a fix

An 3. is another SCTK bug

[pombredanne]

See nexB/scancode-toolkit#3036 for the fixes on the SCTK side