Database update restarting

[maxenced]

What did you do? (required. The issue will be closed when not provided.)

Started last version of vuls, using vuls2 db, in a kubernetes deployment

What did you expect to happen?

The database should update correctly. Ideally, the /health route should not be OK when no database are available.

What happened instead?

When starting pod, the following occurs within first minutes after pods starts :

/vuls # ls /tmp/ -lh
total 6G
-rw-------    1 root     root      180.6M Jul 27 12:53 vuls.db.1464525509
-rw-------    1 root     root        3.7G Jul 27 12:53 vuls.db.1720303110
-rw-------    1 root     root        1.7G Jul 27 12:53 vuls.db.2612629079
/vuls # 

They do not start all at same time, but the 3 keeps growing. Logs return some (not really consistent) data :

❯ kubectl -n onevuln  logs -f vulsio-server-7bb98df54b-w7bpq
time="Jul 27 12:25:53" level=info msg="vuls-v0.39.3-build-20260609_091632_5ce5623"
time="Jul 27 12:25:53" level=info msg="Validating config..."
time="Jul 27 12:25:53" level=info msg="cveDict.type=redis, cveDict.url=redis://10.203.138.123:6379/1, cveDict.SQLite3Path="
time="Jul 27 12:25:53" level=info msg="cti.type=sqlite3, cti.url=, cti.SQLite3Path=/vuls/go-cti.sqlite3"
time="Jul 27 12:25:53" level=warning msg="cti.SQLite3Path=/vuls/go-cti.sqlite3 file not found"
time="Jul 27 12:25:53" level=info msg="Validating DBs..."
time="Jul 27 12:25:59" level=info msg="Listening on 0.0.0.0:80"
time="Jul 27 12:28:35" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠧ fetching (11 GB, 29 MB/s) [6m34s]
time="Jul 27 12:37:21" level=error msg="Failed to detect Pkg CVE: Failed to detect CVE with Vuls2:\n    github.com/future-architect/vuls/detector.DetectPkgCves\n        github.com/future-architect/vuls/detector/detector.go:306\n  - Failed to get new db connection. err:\n    github.com/future-architect/vuls/detector/vuls2.Detect\n        github.com/future-architect/vuls/detector/vuls2/vuls2.go:64\n  - Failed to fetch vuls2 db. err:\n    github.com/future-architect/vuls/detector/vuls2.newDBConfig\n        github.com/future-architect/vuls/detector/vuls2/db.go:36\n  - stream error: stream ID 5; PROTOCOL_ERROR; received from peer\n    write to /tmp/vuls.db.3236042340\n    github.com/MaineK00n/vuls2/pkg/db/fetch.(*options).writeTempDB\n    \tgithub.com/MaineK00n/vuls2@v0.0.1-alpha.0.20260428085730-652a7653079a/pkg/db/fetch/fetch.go:190\n    github.com/MaineK00n/vuls2/pkg/db/fetch.Fetch\n    \tgithub.com/MaineK00n/vuls2@v0.0.1-alpha.0.20260428085730-652a7653079a/pkg/db/fetch/fetch.go:160\n    github.com/future-architect/vuls/detector/vuls2.newDBConfig\n    \tgithub.com/future-architect/vuls/detector/vuls2/db.go:35\n    github.com/future-architect/vuls/detector/vuls2.Detect\n    \tgithub.com/future-architect/vuls/detector/vuls2/vuls2.go:62\n    github.com/future-architect/vuls/detector.DetectPkgCves\n    \tgithub.com/future-architect/vuls/detector/detector.go:305\n    github.com/future-architect/vuls/server.VulsHandler.ServeHTTP\n    \tgithub.com/future-architect/vuls/server/server.go:65\n    net/http.(*ServeMux).ServeHTTP\n    \tnet/http/server.go:2828\n    net/http.serverHandler.ServeHTTP\n    \tnet/http/server.go:3311\n    net/http.(*conn).serve\n    \tnet/http/server.go:2073\n    runtime.goexit\n    \truntime/asm_amd64.s:1771\n    write temp db\n    github.com/MaineK00n/vuls2/pkg/db/fetch.Fetch\n    \tgithub.com/MaineK00n/vuls2@v0.0.1-alpha.0.20260428085730-652a7653079a/pkg/db/fetch/fetch.go:162\n    github.com/future-architect/vuls/detector/vuls2.newDBConfig\n    \tgithub.com/future-architect/vuls/detector/vuls2/db.go:35\n    github.com/future-architect/vuls/detector/vuls2.Detect\n    \tgithub.com/future-architect/vuls/detector/vuls2/vuls2.go:62\n    github.com/future-architect/vuls/detector.DetectPkgCves\n    \tgithub.com/future-architect/vuls/detector/detector.go:305\n    github.com/future-architect/vuls/server.VulsHandler.ServeHTTP\n    \tgithub.com/future-architect/vuls/server/server.go:65\n    net/http.(*ServeMux).ServeHTTP\n    \tnet/http/server.go:2828\n    net/http.serverHandler.ServeHTTP\n    \tnet/http/server.go:3311\n    net/http.(*conn).serve\n    \tnet/http/server.go:2073\n    runtime.goexit\n    \truntime/asm_amd64.s:1771"
time="Jul 27 12:37:30" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠇ fetching (3.1 GB, 12 MB/s) [4m42s] time="Jul 27 12:42:22" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠦ fetching (1.9 GB, 7.4 MB/s) [4m58s] time="Jul 27 12:47:30" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠸ fetching (5.0 GB, 8.3 MB/s) [10m2s]
⠏ fetching (477 MB, 7.4 MB/s) [1m11s] time="Jul 27 12:49:11" level=error msg="Failed to detect Pkg CVE: Failed to detect CVE with Vuls2:\n    github.com/future-architect/vuls/detector.DetectPkgCves\n        github.com/future-architect/vuls/detector/detector.go:306\n  - Failed to get new db connection. err:\n    github.com/future-architect/vuls/detector/vuls2.Detect\n        github.com/future-architect/vuls/detector/vuls2/vuls2.go:64\n  - Failed to fetch vuls2 db. err:\n    github.com/future-architect/vuls/detector/vuls2.newDBConfig\n        github.com/future-architect/vuls/detector/vuls2/db.go:36\n  - stream error: stream ID 5; PROTOCOL_ERROR; received from peer\n    write to /tmp/vuls.db.494383713\n    github.com/MaineK00n/vuls2/pkg/db/fetch.(*options).writeTempDB\n    \tgithub.com/MaineK00n/vuls2@v0.0.1-alpha.0.20260428085730-652a7653079a/pkg/db/fetch/fetch.go:190\n    github.com/MaineK00n/vuls2/pkg/db/fetch.Fetch\n    \tgithub.com/MaineK00n/vuls2@v0.0.1-alpha.0.20260428085730-652a7653079a/pkg/db/fetch/fetch.go:160\n    github.com/future-architect/vuls/detector/vuls2.newDBConfig\n    \tgithub.com/future-architect/vuls/detector/vuls2/db.go:35\n    github.com/future-architect/vuls/detector/vuls2.Detect\n    \tgithub.com/future-architect/vuls/detector/vuls2/vuls2.go:62\n    github.com/future-architect/vuls/detector.DetectPkgCves\n    \tgithub.com/future-architect/vuls/detector/detector.go:305\n    github.com/future-architect/vuls/server.VulsHandler.ServeHTTP\n    \tgithub.com/future-architect/vuls/server/server.go:65\n    net/http.(*ServeMux).ServeHTTP\n    \tnet/http/server.go:2828\n    net/http.serverHandler.ServeHTTP\n    \tnet/http/server.go:3311\n    net/http.(*conn).serve\n    \tnet/http/server.go:2073\n    runtime.goexit\n    \truntime/asm_amd64.s:1771\n    write temp db\n    github.com/MaineK00n/vuls2/pkg/db/fetch.Fetch\n    \tgithub.com/MaineK00n/vuls2@v0.0.1-alpha.0.20260428085730-652a7653079a/pkg/db/fetch/fetch.go:162\n    github.com/future-architect/vuls/detector/vuls2.newDBConfig\n    \tgithub.com/future-architect/vuls/detector/vuls2/db.go:35\n    github.com/future-architect/vuls/detector/vuls2.Detect\n    \tgithub.com/future-architect/vuls/detector/vuls2/vuls2.go:62\n    github.com/future-architect/vuls/detector.DetectPkgCves\n    \tgithub.com/future-architect/vuls/detector/detector.go:305\n    github.com/future-architect/vuls/server.VulsHandler.ServeHTTP\n    \tgithub.com/future-architect/vuls/server/server.go:65\n    net/http.(*ServeMux).ServeHTTP\n    \tnet/http/server.go:2828\n    net/http.serverHandler.ServeHTTP\n    \tnet/http/server.go:3311\n    net/http.(*conn).serve\n    \tnet/http/server.go:2073\n    runtime.goexit\n    \truntime/asm_amd64.s:1771"
⠸ fetching (1.7 GB, 9.0 MB/s) [4m23s] time="Jul 27 12:52:22" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"

I guess the Failed to detect are because the pod is seen as ready (because /health is) and some requests are sent. But the main issue is that the vuls.db is never correctly downloaded (and seems to be downloaded multiple times).

At some point, some versions are deleted and new ones starts, exemple on the same pod after a few minutes :

/vuls # ls /tmp/ -lh
total 7G
-rw-------    1 root     root      673.8M Jul 27 13:05 vuls.db.2038384878
-rw-------    1 root     root        4.5G Jul 27 13:05 vuls.db.2612629079
-rw-------    1 root     root        1.8G Jul 27 13:05 vuls.db.2984721173
/vuls # 

Cleaned log at that same time :

❯ kubectl -n onevuln  logs -f vulsio-server-7bb98df54b-w7bpq | grep -v 'Failed to detect'
time="Jul 27 12:25:53" level=info msg="vuls-v0.39.3-build-20260609_091632_5ce5623"
time="Jul 27 12:25:53" level=info msg="Validating config..."
time="Jul 27 12:25:53" level=info msg="cveDict.type=redis, cveDict.url=redis://10.203.138.123:6379/1, cveDict.SQLite3Path="
time="Jul 27 12:25:53" level=info msg="cti.type=sqlite3, cti.url=, cti.SQLite3Path=/vuls/go-cti.sqlite3"
time="Jul 27 12:25:53" level=warning msg="cti.SQLite3Path=/vuls/go-cti.sqlite3 file not found"
time="Jul 27 12:25:53" level=info msg="Validating DBs..."
time="Jul 27 12:25:59" level=info msg="Listening on 0.0.0.0:80"
time="Jul 27 12:28:35" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠧ fetching (11 GB, 29 MB/s) [6m34s]
time="Jul 27 12:37:30" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠇ fetching (3.1 GB, 12 MB/s) [4m42s] time="Jul 27 12:42:22" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠦ fetching (1.9 GB, 7.4 MB/s) [4m58s] time="Jul 27 12:47:30" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠸ fetching (5.0 GB, 8.3 MB/s) [10m2s]
⠸ fetching (1.7 GB, 9.0 MB/s) [4m23s] time="Jul 27 12:52:22" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠏ fetching (1.3 GB, 6.2 MB/s) [4m58s] time="Jul 27 12:57:30" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠸ fetching (5.0 GB, 5.6 MB/s) [14m59s]
⠸ fetching (4.1 GB, 3.4 MB/s) [14m23s] time="Jul 27 13:02:22" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"
⠦ fetching (2.8 GB, 4.2 MB/s) [11m11s]
⠏ fetching (5.0 GB, 4.7 MB/s) [17m40s]
⠼ fetching (2.8 GB, 5.3 MB/s) [8m57s]
⠴ fetching (1.6 GB, 12 MB/s) [4m7s] time="Jul 27 13:06:39" level=info msg="Fetching vuls2 db. repository: ghcr.io/vulsio/vuls-nightly-db:nightly"

Config is :

[vuls2]
Path = "/tmp/vuls.db"
Repository = "ghcr.io/vulsio/vuls-nightly-db:nightly"
SkipUpdate = false

[maxenced]

Interesting things, it seems that the download stops at exactly 5GB, you can see this in the log :

⠏ fetching (5.0 GB, 4.8 MB/s) [17m29s]

and a while true; do ls -l /tmp; sleep 1; done confirms the behavior :

# 2 download ongoing
-rw-------    1 root     root     1634340864 Jul 27 13:24 vuls.db.1006947563
-rw-------    1 root     root     4994203648 Jul 27 13:24 vuls.db.2835122749
total 6487440
-rw-------    1 root     root     1642037248 Jul 27 13:24 vuls.db.1006947563
-rw-------    1 root     root     5001089024 Jul 27 13:24 vuls.db.2835122749    # Read 5G, the log appear
total 1610116
-rw-------    1 root     root     1648754688 Jul 27 13:24 vuls.db.1006947563   # File has been deleted
total 1618936

[maxenced]

So, I tried to move out of kubernetes to test if this could improve things.
And I found that the very same behavior was happening. Trying to diagnose, I found that :

• The first fetch of DB is lazy (it waits for the first request to download it, but given the size, the request will likely timeout)
• More important, There is no lock for db init, so if I send 4 curl to /vuls with a valid payload (I took the debian one from vuls.io) ...it will start 4 fetch of the database.

I'm not sure why this has been moved to discussion as it really sound like a bug :)