Upgrade Greenbone to debian-master (Report outdated / end-of-life Scan Engine) #335

devMarc5598 · 2022-03-11T09:25:41Z

Hello,
I want to update from my current alpine version (21.4.2) to the newest debian version. Is there a guide with some simple steps to follow?
A detailed guidance would be very good!?

I would be happy about any help.

GVM versions
gsad: Greenbone Security Assistant 21.4.2
gvmd: Greenbone Vulnerability Manager 21.4.3
openvas-scanner: OpenVAS 21.4.2
gvm-libs: gvm-libs 21.4.2

Dexus · 2022-03-14T11:55:43Z

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

It use POSTGRESQL 13.

What are meaning the tags:

docker pull deineagenturug/gvm:latest               # is an not initialized basic image, that first start its initialization on the first start (short: no db & no PDF report support)
docker pull deineagenturug/gvm:latest-full          # is an not initialized basic image, that first start its initialization on the first start (short: no db, PDF report is support)
docker pull deineagenturug/gvm:latest-data          # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db, but no PDF report support)
docker pull deineagenturug/gvm:latest-data-full     # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db & PDF report is support)

It can be used as direct replacement for the here provided image, with the same arguments on docker run.

⚠️ I have not tested the migration from the alpine image, but this should work out of the box now. Maybe you can give a notice it it works as I expected.

devMarc5598 · 2022-03-16T09:53:30Z

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

It use POSTGRESQL 13.

What are meaning the tags:
docker pull deineagenturug/gvm:latest               # is an not initialized basic image, that first start its initialization on the first start (short: no db & no PDF report support)
docker pull deineagenturug/gvm:latest-full          # is an not initialized basic image, that first start its initialization on the first start (short: no db, PDF report is support)
docker pull deineagenturug/gvm:latest-data          # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db, but no PDF report support)
docker pull deineagenturug/gvm:latest-data-full     # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db & PDF report is support)
It can be used as direct replacement for the here provided image, with the same arguments on docker run.

warning I have not tested the migration from the alpine image, but this should work out of the box now. Maybe you can give a notice it it works as I expected.

Why do you switch from https://hub.docker.com/r/securecompliance/gvm to https://hub.docker.com/r/deineagenturug/gvm?

Dexus · 2022-03-16T10:33:36Z

Why do you switch from https://hub.docker.com/r/securecompliance/gvm to https://hub.docker.com/r/deineagenturug/gvm?

Because I feel like I'm the only one working on it. So I created my own fork so that I am able to push new versions and build all Docker images directly and correctly without having to wait for a merge. Unfortunately, that's what we've seen here in the past and again currently, no one cares. So someone has to do it who also has all the rights. And since I don't work for SCS LLC, I have no choice but to use my own fork and hardware so I can do my work and support best possible.

Kelvinloucosta · 2022-03-17T12:41:37Z

I'm using the "docker pull deineagenturug/gvm:latest-data-full" but when I go to check the openvas scan added, the service appears unavailable, does anyone have a link to openvas correct or have you faced this problem?

FiveOFive · 2022-03-19T01:16:24Z

I think I have had the same or a similar issue. The gvmd logs shows "osp_scanner_feed_version: failed to get scanner_feed_version. OSPd OpenVAS is still starting"

@Dexus - Any thoughts? Thanks for your efforts keeping this project going!

Dexus · 2022-03-20T14:37:27Z

I think I have had the same or a similar issue. The gvmd logs shows "osp_scanner_feed_version: failed to get scanner_feed_version. OSPd OpenVAS is still starting"

I will investigate, since It build the latest-data-full without problems I think not there is a problem.

But will check this.

I'm using the "docker pull deineagenturug/gvm:latest-data-full" but when I go to check the openvas scan added, the service appears unavailable, does anyone have a link to openvas correct or have you faced this problem?

How did you run the Image? What is your run command? While it is building the image successful I think there is a unknown problem which I can't currently reproduce.

xenago · 2022-03-21T15:34:14Z

Thanks for your efforts, @Dexus! Unfortunately I'm unable to start up deineagenturug/gvm:latest-full and deineagenturug/gvm:latest-data-full (versions published today) on top of my existing install. I will continue to use securecompliance/gvm:debian-master-data-full in the meantime.

Hopefully this info helps...

With latest-data-full, it throws repo signature errors:
E: The repository 'http://deb.debian.org/debian bullseye-backports InRelease' is not signed.
W: GPG error: http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease: At least one invalid signature was encountered.

With latest-full, it throws postgres startup errors:
2022-03-21 15:08:32,173 INFO spawned: 'postgresql' with pid 653
2022-03-21 15:08:32,188 INFO exited: postgresql (exit status 1; not expected)
....
2022-03-21 15:08:52,251 INFO gave up: postgresql entered FATAL state, too many start retries too quickly

Docker-compose:

services:
    gvm:
#        image: securecompliance/gvm:debian-master-data-full
        image: deineagenturug/gvm:latest-data-full
    restart: unless-stopped
    volumes:
      - /gvm_data/database:/opt/database
      - /gvm_data/plugins:/var/lib/openvas/plugins
      - /gvm_data/gvm:/var/lib/gvm
      - /gvm_data/ssh:/etc/ssh
    ports:
      - "443:9392"
      - "5432:5432"
      - "2222:22"
    environment:
      - DB_PASSWORD="passwordhere"
      - USERNAME="admin"
      - PASSWORD="passwordhere"
      - TZ="America/Toronto"
      - SSHD=true
      - HTTPS=false
      - AUTO_SYNC=true
      - RELAYHOST=172.17.0.1
      - SMTPPORT=25

Dexus · 2022-03-22T10:46:27Z

Thanks for your efforts, @Dexus! Unfortunately I'm unable to start up deineagenturug/gvm:latest-full and deineagenturug/gvm:latest-data-full (versions published today) on top of my existing install. I will continue to use securecompliance/gvm:debian-master-data-full in the meantime.

Hopefully this info helps...

With latest-full, it throws postgres startup errors:
2022-03-21 15:08:32,173 INFO spawned: 'postgresql' with pid 653
2022-03-21 15:08:32,188 INFO exited: postgresql (exit status 1; not expected)
....
2022-03-21 15:08:52,251 INFO gave up: postgresql entered FATAL state, too many start retries too quickly

hi @xenago i think you have to migrate your DB something like: https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/

currently we have no automatic ways for this - what makes me a bit unhappy but without full time work on it, it will take a while :(

If you use latest-data-full or latest-data together with volumes, you can simply run latest-full or latest as image tag. Because you don't will benefit from the pre-initalized images.

With latest-data-full, it throws repo signature errors:
E: The repository 'http://deb.debian.org/debian bullseye-backports InRelease' is not signed.
W: GPG error: http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease: At least one invalid signature was encountered.

When you get this message? On my local system I never get this message.

xenago · 2022-03-22T14:56:55Z

Ok thanks for the info regarding the DB upgrade, I appreciate it!

When you get this message?

I see the repo errors in the docker logs when running the docker-compose as-included in my previous comment. I'm not sure if it has anything to do with the environment variables or volumes I used or if it has something to do with the image itself.

Dexus · 2022-03-24T07:24:54Z

Ok thanks for the info regarding the DB upgrade, I appreciate it!

When you get this message?

I see the repo errors in the docker logs when running the docker-compose as-included in my previous comment. I'm not sure if it has anything to do with the environment variables or volumes I used or if it has something to do with the image itself.

are you behind an proxy? I'm not able here to reproduce this, maybe a newer image will solve this issue.

Dexus · 2022-03-24T12:18:55Z

For further support, use https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker

devMarc5598 · 2022-03-24T13:44:45Z

@Dexus
Using the image deineagenturug/gvm:latest-full worked for me. Now I have the version 21.4.3.
1)
But I cant do Scans without an error...
It shows "Task interrupted unexpectedly "and "Scan process Failure" after I run a task.
2)
I cant start the container with a preinitialized volume...
So I don't have some of the old data in the current version.

Dexus · 2022-03-24T13:57:57Z

@devMarc5598
to point 1: your feed sync is done before you start the scan?
to point 2: do you have a DB problem? did you use before the debian image with pg11/12 or the alpine image? if the debian with pg11/12 then you need to upgrade the DB. But I'm no DB Admin so I can only ref to https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/

Edit:

I will try, to provide a Docker Image to Upgrade the DB. But this may take some days.

devMarc5598 · 2022-03-24T14:18:07Z

@Dexus
no, I created a simple task and started it. -> it stoped after a view seconds and shows this errors in the report:

Before: deineagenturug/gvm:latest-data I was using the Alpine Version securecompliance/gvm:21.4.3-v1 from 7 months ago

Dexus · 2022-03-24T14:21:59Z

@Dexus no, I created a simple task and started it. -> it stoped after a view seconds and shows this errors in the report:

Before: deineagenturug/gvm:latest-data I was using the Alpine Version securecompliance/gvm:21.4.3-v1 from 7 months ago

what is the status of <url>/feedstatus are there any sync in progress?

Dexus · 2022-03-24T14:24:40Z

plase exec docker exec -ti <container_id/name> cat /var/log/gvm/* and docker exec -ti <container_id/name> cat /var/log/supoervisor/* this will showing some problems maybe.

devMarc5598 · 2022-03-24T14:25:27Z

My Main-Problem is:
I want to update from my current greenbone alpine version to the newest version because I get this issue
-> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/
But now I need to get this work with my old alpine data.

Dexus · 2022-03-24T14:29:26Z

My Main-Problem is: I want to update from my current greenbone alpine version to the newest version because I get this issue -> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/ But now I need to get this work with my old alpine data.

This should not be a problem since the alpine version was with PG 13.3 so it should work out of the box with the current debian based image with PG 13.6.

Dexus · 2022-03-24T15:36:19Z

My Main-Problem is: I want to update from my current greenbone alpine version to the newest version because I get this issue -> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/ But now I need to get this work with my old alpine data.

@devMarc5598 I have found 2 bugs, that may have impact of the things you mentioned. I have have a run open to publish a fix for it - but it only releases an amd64 image for now.

Dexus · 2022-03-24T15:37:13Z

But I cant do Scans without an error...
It shows "Task interrupted unexpectedly "and "Scan process Failure" after I run a task.

@devMarc5598 i mean this point of failure

Kelvinloucosta · 2022-03-24T17:00:29Z

I have a doubt, I just uploaded two dockers, one with openvas and the other with gvm, I now want to install an openvas on the clients so that I can manage everything in a single panel that would be GSA, is this possible? Point to the public address of the GVM?

Dexus · 2022-03-25T09:05:58Z

@Kelvinloucosta

I have a doubt, I just uploaded two dockers, one with openvas and the other with gvm, I now want to install an openvas on the clients so that I can manage everything in a single panel that would be GSA, is this possible? Point to the public address of the GVM?

you can use the gvm image as "MASTER" and the openvas-scanner image as "SENSOR" in your DMZ. Use the Images from https://hub.docker.com/u/deineagenturug

Once you registered the "SENSOR" in "MASTER" you are able to select the Scanner for your new Scans.

Kelvinloucosta · 2022-03-25T12:02:13Z

thanks @Dexus , but I would like to install openvas-scanner on another network by public ip. it's possible?

Can anyone help me explain this design to me, sorry about that but I would like to know, why use redis? what is the function of postgresql? what does ospd do?

Dexus · 2022-03-25T12:11:16Z

thanks @Dexus , but I would like to install openvas-scanner on another network by public ip. it's possible?

Yes this would work, but I would recommend a connection via VPN. if you make the SSH port from MASTER listen public, you can connect the openvas-scanner to it.

Can anyone help me explain this design to me, sorry about that but I would like to know, why use redis? what is the function of postgresql? what does ospd do?

If you need more details for this, please look around at Greenbone

Kelvinloucosta · 2022-03-25T12:12:36Z

thanks again @Dexus

Dexus · 2022-03-25T12:14:41Z

@Kelvinloucosta you're welcome. But you should know, that the performance via WAN is not the same as via LAN. So you should be know that this may have a quit big different.

Kelvinloucosta · 2022-03-25T12:15:55Z

yeah you right

gdanldo · 2022-03-31T16:46:39Z

I have the same problem.
I have found this link https://community.greenbone.net/t/report-outdated-end-of-life-scan-engine-environment-local/5430/2

"You can edit the following script to turn off the message:

/var/lib/openvas/plugins/gcf/gb_outdated_scan_engine.nasl

Insert the versions for your scanner and libs, mine is 5.1.3 and 9.0.3:

#expected_gce_ver = “6.0.7”;
expected_gce_ver = “5.1.3”;
#expected_libs_ver1 = “10.0.2”;
expected_libs_ver1 = “9.0.3”;
#expected_libs_ver2 = “11.0.1”;
expected_libs_ver2 = “9.0.3”;

Now it won’t report the scanner.

An alternative is to set an override for the script."

This workarround work but the file change after stop and restart container.
Override doesn't work for me.
I don't have find how to update scan engine :(

Dexus · 2022-03-31T17:06:15Z

@gdanldo do read the full thread
https://community.greenbone.net/t/report-outdated-end-of-life-scan-engine-environment-local/5430/21

4920441 · 2022-04-01T06:53:20Z

Hi,

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

Sorry, nice try, but there are too many problems with this container - the most recognizeable one is, that all reports are empty - only logging information is shown in the reports. There are many small problems with their container additionaly. For me, it is not useable as a drop in replacement at all :-(

Dexus · 2022-04-01T07:05:28Z

Hi,

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

Sorry, nice try, but there are too many problems with this container - the most recognizeable one is, that all reports are empty - only logging information is shown in the reports. There are many small problems with their container additionaly. For me, it is not useable as a drop in replacement at all :-(

It's up to you to report problems. But I will not work on this repo anymore, to fix issues. You should know: I was the maintainer for the last big improvements here, but also the only one.

4920441 · 2022-04-01T07:43:19Z

It's up to you to report problems. But I will not work on this repo anymore, to fix issues. You should know: I was the maintainer for the last big improvements here, but also the only one.

I didn't want to blame you or anything, but I wanted to point out it sounds like a drop in replacement, which it isn't - so I used up many hours to make it work, only to get more flaws whcih are preventing me from using it...

Dexus · 2022-04-01T08:56:19Z

It's up to you to report problems. But I will not work on this repo anymore, to fix issues. You should know: I was the maintainer for the last big improvements here, but also the only one.

I didn't want to blame you or anything, but I wanted to point out it sounds like a drop in replacement, which it isn't - so I used up many hours to make it work, only to get more flaws whcih are preventing me from using it...

It has nothing changed from the usage of the image from here to my image. And it should - yes mistakes included sometimes - work out of the box to switch.

Btw. your mentioned PDF report problem I have created at Greenbone an Issue AP-1987 for it - tacked DeineAgenturUG/greenbone-gvm-openvas-for-docker#9

markdesilva · 2022-04-01T14:04:40Z

Thank you @Dexus for all the hard work on this and the new DeineAgenturUG images, I for one am grateful all the time and effort put in!

Cheers!

willemdh · 2022-04-02T11:32:45Z

Thank a lot @Dexus for everything you did. Working for free on projects such as these is not always easy. If you are the only one working on this project then we should all start following https://hub.docker.com/r/deineagenturug/gvm imho and hope for you to find the time to keep maintaining this useful project.

Dexus · 2022-04-03T19:57:23Z

Thank you @markdesilva and @willemdh I'll do my best, it's also in my own interest.

landonstewart · 2022-04-04T19:43:13Z

Can @Secure-Compliance-Solutions simply update their docker images?

Kelvinloucosta · 2022-04-04T20:33:04Z

Hi @Dexus which gvm image and openvas are you using?
Is it running normally without any problems?

Dexus · 2022-04-04T23:38:40Z

Can @Secure-Compliance-Solutions simply update their docker images?

Sure, they can...

Hi @Dexus which gvm image and openvas are you using? Is it running normally without any problems?

https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker look there.

I use https://hub.docker.com/r/deineagenturug/openvas-scanner and https://hub.docker.com/r/deineagenturug/gvm

Kelvinloucosta · 2022-04-08T17:53:13Z

Which redis image are you guys using? Official?

Dexus · 2022-04-20T17:58:41Z

We have a "development" DB Upgrade tool for the migration from here to my deineagenturug/gvm.

Details you can read here: DeineAgenturUG/greenbone-gvm-openvas-for-docker#16 (comment)

Any feedback in the issue on my repo is welcome.

weskerjiang · 2022-10-09T05:52:05Z

Hello，i use docker pull deineagenturug/gvm:latest-data-full to pull the image and i use docker run -d -p 9390:9392 -e PASSWORD="xxx" --name=gvm deineagenturug/gvm:latest-data-full,it runs but the browser can't visit https://127.0.0.1:9390

devMarc5598 added the bug Something isn't working label Mar 11, 2022

devMarc5598 assigned Dexus and pixelsquared Mar 11, 2022

Upgrade Greenbone to debian-master (Report outdated / end-of-life Scan Engine) #335

Upgrade Greenbone to debian-master (Report outdated / end-of-life Scan Engine) #335

Comments

devMarc5598 commented Mar 11, 2022

Dexus commented Mar 14, 2022

devMarc5598 commented Mar 16, 2022

Dexus commented Mar 16, 2022

Kelvinloucosta commented Mar 17, 2022

FiveOFive commented Mar 19, 2022

Dexus commented Mar 20, 2022

xenago commented Mar 21, 2022

Dexus commented Mar 22, 2022

xenago commented Mar 22, 2022

Dexus commented Mar 24, 2022

Dexus commented Mar 24, 2022

devMarc5598 commented Mar 24, 2022

Dexus commented Mar 24, 2022 • edited Loading

devMarc5598 commented Mar 24, 2022 • edited Loading

Dexus commented Mar 24, 2022

Dexus commented Mar 24, 2022 • edited Loading

devMarc5598 commented Mar 24, 2022

Dexus commented Mar 24, 2022 • edited Loading

Dexus commented Mar 24, 2022

Dexus commented Mar 24, 2022

Kelvinloucosta commented Mar 24, 2022

Dexus commented Mar 25, 2022

Kelvinloucosta commented Mar 25, 2022

Dexus commented Mar 25, 2022

Kelvinloucosta commented Mar 25, 2022

Dexus commented Mar 25, 2022

Kelvinloucosta commented Mar 25, 2022

gdanldo commented Mar 31, 2022

Dexus commented Mar 31, 2022

4920441 commented Apr 1, 2022

Dexus commented Apr 1, 2022

4920441 commented Apr 1, 2022 • edited Loading

Dexus commented Apr 1, 2022

markdesilva commented Apr 1, 2022 • edited Loading

willemdh commented Apr 2, 2022

Dexus commented Apr 3, 2022

landonstewart commented Apr 4, 2022

Kelvinloucosta commented Apr 4, 2022

Dexus commented Apr 4, 2022 • edited Loading

Kelvinloucosta commented Apr 8, 2022

Dexus commented Apr 20, 2022

weskerjiang commented Oct 9, 2022

Dexus commented Mar 24, 2022 •

edited

Loading

devMarc5598 commented Mar 24, 2022 •

edited

Loading

Dexus commented Mar 24, 2022 •

edited

Loading

Dexus commented Mar 24, 2022 •

edited

Loading

4920441 commented Apr 1, 2022 •

edited

Loading

markdesilva commented Apr 1, 2022 •

edited

Loading

Dexus commented Apr 4, 2022 •

edited

Loading