Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scan failing with 2.9.0 release on .split #1101

Closed
jasona7 opened this issue Jan 13, 2022 · 2 comments · Fixed by #1105
Closed

Scan failing with 2.9.0 release on .split #1101

jasona7 opened this issue Jan 13, 2022 · 2 comments · Fixed by #1105
Assignees
Labels
bug Something went wrong

Comments

@jasona7
Copy link

jasona7 commented Jan 13, 2022

Describe the bug
A clear and concise description of what the bug is.

We run weekly Tern scans against hundreds of images on a Harbor registry (v2.1.3) with the previous Tern release v2.8.0. We upgraded to the v2.9.0 release and are experiencing Traceback split errors from /usr/local/lib/python3.9/re.py. Not all scans are failing, about 40% of the scans fail.

To Reproduce
Steps to reproduce the behavior:

  1. Execute:

sudo ./docker_run.sh tern-debug "report -i %REGISTRY%/library/fluentd-syslog:v2"

  1. See error
    2022-01-06 13:48:11,205 - DEBUG - rootfs - Running command: skopeo copy docker://%REGISTRY%/library/fluentd-syslog:v2 dir:/root/.tern/temp
    Traceback (most recent call last):
    File "/usr/local/bin/tern", line 8, in
    sys.exit(main())
    File "/usr/local/lib/python3.9/site-packages/tern/main.py", line 286, in main
    do_main(args)
    File "/usr/local/lib/python3.9/site-packages/tern/main.py", line 116, in do_main
    crun.execute_image(args)
    File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/run.py", line 70, in execute_image
    full_image = cimage.load_full_image(
    File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/image.py", line 39, in load_full_image
    image.load_image(load_until_layer)
    File "/usr/local/lib/python3.9/site-packages/tern/classes/oci_image.py", line 105, in load_image
    repo_dict = general.parse_image_string(
    File "/usr/local/lib/python3.9/site-packages/tern/utils/general.py", line 296, in parse_image_string
    tokens = re.split(r'[@:]', image_string)
    File "/usr/local/lib/python3.9/re.py", line 231, in split
    return _compile(pattern, flags).split(string, maxsplit)
    TypeError: expected string or bytes-like object

Expected behavior
Successful Scan

docker pull & inspect using the same image works

Environment you are running Tern on

  • Tern version 2.9.0
    python version = 3.9 (main, Dec 21 2021, 10:35:05)

  • Operating System (Linux Distro and version or Mac or Windows)
    Description: Ubuntu 20.04.3 LTS
    Release: 20.04
    Codename: focal

  • Image Registry
    Harbor (v2.1.3)

  • Vagrant file
    NA

  • Container OS
    Multiple

  • Python version (3.6 or higher)
    Using Tern Docker build. Python version is 3.9
    Virtualenv running Tern (docker_run.sh): Python 3.8.10

  • Cloud environment (AWS, Azure, GCP)
    NA

@nishakm nishakm added the bug Something went wrong label Jan 13, 2022
@nishakm nishakm self-assigned this Jan 13, 2022
@vEpiphyte
Copy link

I've also experienced this with Python 3.10.1 and tern 2.9.0.

$ tern report -i vertexproject/synapse-base-image3:py38
2022-01-18 14:44:53,239 - DEBUG - __main__ - Starting...
2022-01-18 14:44:53,240 - DEBUG - prep - Setting up...
2022-01-18 14:44:53,284 - DEBUG - rootfs - Running command: sudo chmod +x /home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/tools/fs_hash.sh
2022-01-18 14:44:53,288 - DEBUG - run - Starting analysis...
2022-01-18 14:44:53,289 - DEBUG - skopeo - Attempting to pull image "vertexproject/synapse-base-image3:py38"
2022-01-18 14:44:53,289 - DEBUG - rootfs - Running command: skopeo copy docker://vertexproject/synapse-base-image3:py38 dir:/home/user/.tern/temp
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/__main__.py", line 286, in main
    do_main(args)
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/__main__.py", line 116, in do_main
    crun.execute_image(args)
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/analyze/default/container/run.py", line 70, in execute_image
    full_image = cimage.load_full_image(
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/analyze/default/container/image.py", line 39, in load_full_image
    image.load_image(load_until_layer)
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/classes/oci_image.py", line 105, in load_image
    repo_dict = general.parse_image_string(
  File "/home/user/.pyenv/versions/3.10.1/envs/fp3101/lib/python3.10/site-packages/tern/utils/general.py", line 296, in parse_image_string
    tokens = re.split(r'[@:]', image_string)
  File "/home/user/.pyenv/versions/3.10.1/lib/python3.10/re.py", line 230, in split
    return _compile(pattern, flags).split(string, maxsplit)
TypeError: expected string or bytes-like object

@nishakm
Copy link
Contributor

nishakm commented Jan 19, 2022

I'm able to reproduce this. A fix is coming up, followed by a 2.9.1 release once the bug fixes are merged.

nishakm pushed a commit to nishakm/tern that referenced this issue Jan 19, 2022
The config object is the incorrect method of retrieving an image
digest as the config and layers itself creates a manifest which
is hashed to get the image digest. Instead, we will use skopeo
to retireve it. For this, we introduce a function in
tern/load/skopeo to do the remote inspection, and parse out
the image digest type and digest.

Fixes tern-tools#1101

Signed-off-by: Nisha K <nishak@vmware.com>
nishakm pushed a commit to nishakm/tern that referenced this issue Jan 20, 2022
The config object is the incorrect method of retrieving an image
digest as the config and layers itself creates a manifest which
is hashed to get the image digest. Instead, we will use skopeo
to retireve it. For this, we introduce a function in
tern/load/skopeo to do the remote inspection, and parse out
the image digest type and digest.

Fixes tern-tools#1101

Signed-off-by: Nisha K <nishak@vmware.com>
rnjudge pushed a commit that referenced this issue Jan 20, 2022
The config object is the incorrect method of retrieving an image
digest as the config and layers itself creates a manifest which
is hashed to get the image digest. Instead, we will use skopeo
to retireve it. For this, we introduce a function in
tern/load/skopeo to do the remote inspection, and parse out
the image digest type and digest.

Fixes #1101

Signed-off-by: Nisha K <nishak@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something went wrong
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants