db_mgmt_json.py download CVE database not stable

[elsi06]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/e-m-b-a/emba.git

2. cd emba

3. sudo ./installer.sh -d

4. Error:
   Downloading files: 0%| | 0/22 [00:00<?, ?it/s]2023-10-17 20:17:15,185 - CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html'}
   2023-10-17 20:17:15,187 - CVEDownloads - ERROR - Unhandled Content-Type encountered: text/html from url
   Downloading files: 27%|██████▊ | 6/22 [00:03<00:09, 1.65it/s]

5. I ran the script manually and after three times it was working fine:
   franky@Franky-ubuntu:/Downloads/emba/external/cve-search$ sudo ./sbin/db_mgmt_json.py -p
   2023-10-17 20:33:16,699 - CVEDownloads - INFO - CVE database population started
   2023-10-17 20:33:16,699 - CVEDownloads - INFO - Starting CVE database population starting from year: 2002
   2023-10-17 20:33:16,711 - DatabaseIndexer - INFO - Success to create index [('id', 1)] on cves
   2023-10-17 20:33:16,714 - DatabaseIndexer - INFO - Success to create index [('vulnerable_configuration', 1)] on cves
   2023-10-17 20:33:16,716 - DatabaseIndexer - INFO - Success to create index [('vulnerable_product', 1)] on cves
   2023-10-17 20:33:16,719 - DatabaseIndexer - INFO - Success to create index [('Modified', 1)] on cves
   2023-10-17 20:33:16,721 - DatabaseIndexer - INFO - Success to create index [('Published', 1)] on cves
   2023-10-17 20:33:16,724 - DatabaseIndexer - INFO - Success to create index [('last-modified', 1)] on cves
   2023-10-17 20:33:16,726 - DatabaseIndexer - INFO - Success to create index [('cvss', 1)] on cves
   2023-10-17 20:33:16,728 - DatabaseIndexer - INFO - Success to create index [('cvss3', 1)] on cves
   2023-10-17 20:33:16,731 - DatabaseIndexer - INFO - Success to create index [('summary', 'text')] on cves
   2023-10-17 20:33:16,733 - DatabaseIndexer - INFO - Success to create index [('vendors', 1)] on cves
   2023-10-17 20:33:16,736 - DatabaseIndexer - INFO - Success to create index [('products', 1)] on cves
   2023-10-17 20:33:16,738 - DatabaseIndexer - INFO - Success to create index [('vulnerable_product_stems', 1)] on cves
   2023-10-17 20:33:16,741 - DatabaseIndexer - INFO - Success to create index [('vulnerable_configuration_stems', 1)] on cves
   Downloading files: 0%| | 0/22 [00:00<?, ?it/s]2023-10-17 20:33:19,041 - CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html'}
   2023-10-17 20:33:19,042 - CVEDownloads - ERROR - Unhandled Content-Type encountered: text/html from url
   2023-10-17 20:33:19,618 - CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html'}
   2023-10-17 20:33:19,618 - CVEDownloads - ERROR - Unhandled Content-Type encountered: text/html from url
   Downloading files: 68%|████████████████▎ | 15/22 [00:08<00:03, 1.85it/s]
   franky@Franky-ubuntu:/Downloads/emba/external/cve-search$ sudo ./sbin/db_mgmt_json.py -p
   2023-10-17 20:33:40,527 - CVEDownloads - INFO - CVE database population started
   2023-10-17 20:33:40,527 - CVEDownloads - INFO - Starting CVE database population starting from year: 2002
   2023-10-17 20:33:40,539 - DatabaseIndexer - INFO - Success to create index [('id', 1)] on cves
   2023-10-17 20:33:40,542 - DatabaseIndexer - INFO - Success to create index [('vulnerable_configuration', 1)] on cves
   2023-10-17 20:33:40,544 - DatabaseIndexer - INFO - Success to create index [('vulnerable_product', 1)] on cves
   2023-10-17 20:33:40,546 - DatabaseIndexer - INFO - Success to create index [('Modified', 1)] on cves
   2023-10-17 20:33:40,548 - DatabaseIndexer - INFO - Success to create index [('Published', 1)] on cves
   2023-10-17 20:33:40,551 - DatabaseIndexer - INFO - Success to create index [('last-modified', 1)] on cves
   2023-10-17 20:33:40,553 - DatabaseIndexer - INFO - Success to create index [('cvss', 1)] on cves
   2023-10-17 20:33:40,555 - DatabaseIndexer - INFO - Success to create index [('cvss3', 1)] on cves
   2023-10-17 20:33:40,557 - DatabaseIndexer - INFO - Success to create index [('summary', 'text')] on cves
   2023-10-17 20:33:40,560 - DatabaseIndexer - INFO - Success to create index [('vendors', 1)] on cves
   2023-10-17 20:33:40,562 - DatabaseIndexer - INFO - Success to create index [('products', 1)] on cves
   2023-10-17 20:33:40,564 - DatabaseIndexer - INFO - Success to create index [('vulnerable_product_stems', 1)] on cves
   2023-10-17 20:33:40,567 - DatabaseIndexer - INFO - Success to create index [('vulnerable_configuration_stems', 1)] on cves
   Downloading files: 0%| | 0/22 [00:00<?, ?it/s]2023-10-17 20:33:42,607 - CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html'}
   2023-10-17 20:33:42,607 - CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html'}
   2023-10-17 20:33:42,608 - CVEDownloads - ERROR - Unhandled Content-Type encountered: text/html from url
   Downloading files: 0%| | 0/22 [00:02<?, ?it/s]
   2023-10-17 20:33:42,609 - CVEDownloads - ERROR - Unhandled Content-Type encountered: text/html from url
   franky@Franky-ubuntu:~/Downloads/emba/external/cve-search$ sudo ./sbin/db_mgmt_json.py -p
   2023-10-17 20:33:48,238 - CVEDownloads - INFO - CVE database population started
   2023-10-17 20:33:48,238 - CVEDownloads - INFO - Starting CVE database population starting from year: 2002
   2023-10-17 20:33:48,250 - DatabaseIndexer - INFO - Success to create index [('id', 1)] on cves
   2023-10-17 20:33:48,253 - DatabaseIndexer - INFO - Success to create index [('vulnerable_configuration', 1)] on cves
   2023-10-17 20:33:48,255 - DatabaseIndexer - INFO - Success to create index [('vulnerable_product', 1)] on cves
   2023-10-17 20:33:48,257 - DatabaseIndexer - INFO - Success to create index [('Modified', 1)] on cves
   2023-10-17 20:33:48,260 - DatabaseIndexer - INFO - Success to create index [('Published', 1)] on cves
   2023-10-17 20:33:48,262 - DatabaseIndexer - INFO - Success to create index [('last-modified', 1)] on cves
   2023-10-17 20:33:48,264 - DatabaseIndexer - INFO - Success to create index [('cvss', 1)] on cves
   2023-10-17 20:33:48,267 - DatabaseIndexer - INFO - Success to create index [('cvss3', 1)] on cves
   2023-10-17 20:33:48,269 - DatabaseIndexer - INFO - Success to create index [('summary', 'text')] on cves
   2023-10-17 20:33:48,271 - DatabaseIndexer - INFO - Success to create index [('vendors', 1)] on cves
   2023-10-17 20:33:48,273 - DatabaseIndexer - INFO - Success to create index [('products', 1)] on cves
   2023-10-17 20:33:48,276 - DatabaseIndexer - INFO - Success to create index [('vulnerable_product_stems', 1)] on cves
   2023-10-17 20:33:48,278 - DatabaseIndexer - INFO - Success to create index [('vulnerable_configuration_stems', 1)] on cves
   Downloading files: 100%|████████████████████████| 22/22 [00:09<00:00, 2.30it/s]
   Processing downloaded files: 5%|▋ | 1/22 [00:34<11:55, 34.07s/it]Processing downloaded files: 5%|▋ | 1/22 [01:06<23:15, 66.47s/it]

Expected behavior
Try to run it again when it fails

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Ubuntu 22.04.3 LTS 64bit
• EMBA version: current master branch
• Installation method: default with up to date docker image

Priority issue
Are you already a Sponsor? - N

Additional context
Add any other context about the problem here.

[m-1-k-3]

I have seen this error more often in the recent past. Probably a few tries are not a bad idea.
Thanks for pointing us to this.