implement multiprocessing of CVE retrieval and massively overhaul & enhance pastebin feature

[ryran]

#15 #14

Search queries (--q-xxx) are always done by a single API (or IAVM Mapper app) request.

However, each CVE retrieval was a separate http request against the API and when querying dozens or hundreds of CVEs in serial, the time starts really adding up.

These commits implement multiprocessing for CVE retrieval.

By default, there will be NUM_CPUS + 1 separate worker processes used for making requests. Configurable via -W or --workers, e.g., on my 4-core system, the help page printed:

  -W, --workers N       Set number of concurrent worker processes to allow
                        when making CVE queries (default on this system: 5)

With these commits, the pastebin system is also massively more robust and handles lots of errors. Woo hoo! Especially important: it gracefully exits (& explains) when trying to send too much data to pastebin (i.e., >= 512 KiB). When there are exceptions with the pastebin stuff, approporiate messages are now printed but rhsecapi also actually offers to print the gathered data (that would have been posted to pastebin) to stdout instead.

Last but not least, Ctrl-c still works, even when in the middle the multi-process CVE retrieval.