Duplicate package output when scanning alpine images

[JamieMagee]

Describe the bug
When scanning alpine based images I see duplicate packages in the output, one of which has an empty version.

To Reproduce
Steps to reproduce the behavior:

1. tern -q --driver fuse report -f json -i node:14.15.3-alpine3.12
2. Outputs duplicate packages, one of which has an empty version

Expected behavior
A single instance of each package, with a version listed

Environment you are running Tern on
Enter all that apply

• Output of 'tern --version'
  • 2.3.0
• Operating System
  • Windows 10 running Docker in WSL2
• Vagrant file
• Container OS
• Python version (3.6 or higher)
• Cloud environment (AWS, Azure, GCP)

Please attach files if they exist
JSON output: https://gist.github.com/JamieMagee/c9423ee6bbc9adb7334e3f1ff18f8252

[rnjudge]

@JamieMagee for debug purposes, what is name the duplicate package with an empty version?

[JamieMagee]

@rnjudge I should have specified that all packages are duplicated. You can use musl as an example for debugging.

[rnjudge]

Indeed, I can see the package versions missing in the second layer.

        Packages found in Layer:  musl-, busybox-, alpine-baselayout-, alpine-keys-, libcrypto1.1-,
 libssl1.1-, ca-certificates-bundle-, libtls-standalone-, ssl_client-, zlib-, apk-tools-, scanelf-,
 musl-utils-, libc-utils-, libgcc-, libstdc++-

The missing package versions are the reason that they are being reported twice, since musl-1.1.24-r10 != musl- when Tern tries to check if the package has already been reported in the previous layer. Seems to be an issue with the apk collection utility in that second layer not finding the package versions but I need to do more digging. Thanks for reporting!

[JamieMagee]

Thanks for the quick response, and confirmation, on this bug :)

[rnjudge]

Update for myself: The issue is due to a mismatch in elements when running the command_lib scripts. When I use Tern's debug scripts the number of elements for the packages found versus the versions are different:

Output list: musl busybox alpine-baselayout alpine-keys libcrypto1.1 libssl1.1 ca-certificates-bundle libtls-standalone ssl_client zlib apk-tools scanelf musl-utils libc-utils libgcc libstdc++
Error messages: 
Number of elements: 16


Output list: 1.1.24-r10 1.31.1-r19 3.2.0-r7 2.2-r0 1.1.1i-r0 1.1.1i-r0 20191127-r4 2.9.1-r1 1.31.1-r19 1.2.11-r3 2.10.5-r1 1.2.6-r0 1.1.24-r10 0.7.2-r3 9.3.0-r2
Error messages: 
Number of elements: 15

The apk version parsing for libstdc++ is the culprit. The apk command_lib script is running echo $lic | awk -F "${p}-" '{print $2}' where lic is libstdc++-9.3.0-r2 and p is libstdc++ and returns nothing when it should return a version. I suspect this is due to the ++ in the package name but need to investigate more. If anyone is an awk or parsing wizard, feel free to jump in as to what the solution may be :)

[JamieMagee]

I think it might be easier with sed (That's not a sentence I ever thought I'd say!)

I took your example strings for $lic and $p, and constructed a small regex that removes the prefix $p- from the input $lic

$ export lic=libstdc++-9.3.0-r2
$ export p=libstdc++
$ echo $lic | sed -e "s/^$p-//"
9.3.0-r2

[rnjudge]

I think it might be easier with sed (That's not a sentence I ever thought I'd say!)

I took your example strings for $lic and $p, and constructed a small regex that removes the prefix $p- from the input $lic

$ export lic=libstdc++-9.3.0-r2
$ export p=libstdc++
$ echo $lic | sed -e "s/^$p-//"
9.3.0-r2

Thanks @JamieMagee. Let me test this out with all the other package strings to make sure it works more generically... stay tuned.

[rnjudge]

@JamieMagee looks good! When I run with the sed parsing instead of awk I get the following summary for the layers in the output report (this is a hobbled together copy and paste for simplicity's sake):

Layer1:
        File licenses found in Layer:  None
        Packages found in Layer:  musl-1.1.24-r10, busybox-1.31.1-r19, alpine-baselayout-3.2.0-r7, alpine-keys-2.2-r0, libcrypto1.1-1.1.1i-r0, libssl1.1-1.1.1i-r0, ca-certificates-bundle-20191127-r4, libtls-standalone-2.9.1-r1, ssl_client-1.31.1-r19, zlib-1.2.11-r3, apk-tools-2.10.5-r1, scanelf-1.2.6-r0, musl-utils-1.1.24-r10, libc-utils-0.7.2-r3
        Licenses found in Layer:  MIT, GPL-2.0-only, OpenSSL, MPL-2.0 GPL-2.0-or-later, ISC, Zlib, MIT BSD GPL2+, BSD-2-Clause AND BSD-3-Clause


Layer2:
        File licenses found in Layer:  None
        Packages found in Layer:  libgcc-9.3.0-r2, libstdc++-9.3.0-r2
        Licenses found in Layer:  GPL-2.0-or-later LGPL-2.1-or-later

Layer 3:
        File licenses found in Layer:  None
        Packages found in Layer:  None
        Licenses found in Layer:  None

Layer 4:
        File licenses found in Layer:  None
        Packages found in Layer:  None
        Licenses found in Layer:  None

I'll open a PR for the change now.