Concurrency issue in boltdb handling

[maxenced]

Note : I first opened this as a question, but finally found some time to investigate, so converted to a bug report.

Issue

When running vuls in server mode with the vuls2 BoltDB backend, concurrent requests
cause severe performance degradation. At concurrency >= 4-5, the server effectively
hangs, taking 10+ minutes to return a single response.

Analyse

The issue seems to be a compounding BoltDB file-lock contention caused by the per-request
open/close pattern.

1. Each HTTP request opens BoltDB multiple times independently

In server.go, each request sequentially calls:

• detector.DetectPkgCves() → vuls2.Detect() → newDBConfig() → opens BoltDB
• vuls2.EnrichVulnInfos() → newDBConfig() → opens BoltDB again

2. newDBConfig() opens the DB twice per call

detector/vuls2/db.go#L40-L53 opens BoltDB to validate metadata, then closes it — only for the caller to immediately open it again at vuls2/vuls2.go:74. This means 4 open/close cycles per request (2 in Detect, 2 in Enrich).

3. Most important one : bolt.Open() acquires an OS-level file lock

Even with ReadOnly: true, boltdb.go:62-78 calls bolt.Open() . Multiple concurrent Open() calls on the same file serialize against each other

From bolt documentation :

Please note that Bolt obtains a file lock on the data file so multiple processes cannot open the same database at the same time. Opening an already open Bolt database will cause it to hang until the other process closes it.

Suggested solution

Open the BoltDB database once at server startup and share the single *bolt.DB connection across all requests. BoltDB natively supports concurrent read transactions (View()) on a single open connection — the contention comes from repeatedly calling bolt.Open()/Close(), not from concurrent reads themselves.

This would also enable a shared in-memory cache across requests, eliminating redundant reads for the same CVE data.

=== Previous question ===

Hi,

I'm running vulsio in kubernetes from latest, configured vuls2 database.

I'm only using remote / server mode, no scan.

With some real-world payload (debian 9 , 841 packages, ~14k cve), I can get an answer in ~ 30s as long as this is the only request sent. 

If I does 2 request at same time for 2 servers, it looks like they often take a significant amount of time to get an answer (much more than if requested one by one). For my real world payload, it increased up to 11 minutes.

I can't really understand what's going on here. My config is cvedb + vuls2 db, basically looks like : 

[cveDict]
type = "redis"
url = "redis://${redis_host}:${redis_port}/1"
RedisTimeout = 60

[vuls2]
Path = "/tmp/vuls.db"
SkipUpdate = true


Could you let me know if : 

* I'm doing something wrong by trying multiple requests in parallel 
* There is likely some glitch/bug ? 

/label ~bug

[maxenced]

Some more information, the more I increase parallelism / async threads, the worst it is. Moving from 2 to 5 threads, it basically hangs immediately

[maxenced]

/unlabel ~question
/label ~bug

[MaineK00n]

@maxenced — before we go further on the BoltDB angle, could you try
setting GOMAXPROCS to your pod's CPU limit?

$ GOMAXPROCS=2 vuls server -config=...                                                                                                                                                                                                                                                                                                                                                                                

vuls2.Detect calls ospkg.Detect(..., runtime.NumCPU()), and
runtime.NumCPU() returns the host CPU count, not the pod's
allocation. On a 2-vCPU pod scheduled on, say, an 8-core host, every
request spawns 8 detection goroutines while the cgroup quota only lets
2 actually run — the rest get throttled, and concurrent requests
amplify this. Setting GOMAXPROCS to match the pod's allocation makes
the Go scheduler cap live workers there, avoiding the cgroup-level
throttling.

(Side note: bbolt with ReadOnly: true uses LOCK_SH, so concurrent
ReadOnly opens don't actually serialize on the file lock — the
multi-process quote in the bbolt doc applies to read-write opens only.
The throttling angle fits your symptoms better.)

Could you try and report back?

[maxenced]

Hello, tried this, but did not seems to make any real difference. Though I have other issues (see discussion about db update) so will retry when other issue will be fixed.