How should we handle the old nvd.vulnerabilities.db?

[terriko]

In solving #1, @pdxjohnny and I made some pretty big changes to the way we load the NVD data, removing the old NVDAutoUpdate.py in favour of the new cvedb.py. One of the things I did was change the default location from

.cache/cve-bin-tool/ which included nvd.vulnerabilities.db and a bunch of zip files
to
.cache/cvedb/ which included nvd.vulnerabilities.db and a bunch of json files

That's because it made it easier fo rme to test people's incoming PRs while sitll working on the new updater. I'd intended to fix it to use the old directory (although the new filename just to be sure of not getting stale data) but I forgot in my enthusiasm for having gotten it to work.

Options

1. Do nothing. People may never notice they have old files hanging around.
2. Document so people can remove the files manually
3. Make the tool check once and remove old files.
4. Make -u now remove the old files if they exist.
5. Switch back to the old directory and do one of of the previous options to deal with both old and new files.
6. Switch back to the old directory and switch back to using zipfiles instead of unzipping them, then do one of the previous options to deal with the now obsolete nvd.vulnerabilities.db

I'm not in a huge hurry to do anything as long as I make a reasonable decision before we release 1.0, so I'm just documenting thoughts here for now. Feel free to chime in if you have a particular opinion about any of these.

[terriko]

Oh, another option:

• investigate if removing old files is something we could do in setup.py as a once-off during upgrade.

[PrajwalM2212]

@terriko It seems that we can add custom functionality to commands like install, develop.
Assuming that upgrading a package calls install command, we may be able to remove old files.

[Niraj-Kamdar]

@terriko The way i understand the problem, people who is using this tool for a long time could have 2 directories one containing old data and one for new data and we should remove old data so that it won't waste user's space.
For this I like both option 3 and 4. We can implement one of this option as part of next release and make sure that all user only have new directory for storing db.

[terriko]

I will probably be implementing something for this over the next few days. I'm thinking that we should switch back to the old directory (since .cache/cve-bin-tool is more obvious than .cache/cvedb). I want to run some data on how much space/performance tradeoff we get for going back to json but I'm guessing performance will be minimal (especially since they're only read once per day) and space less minimal so it might be good to go back to the zipped files.

If we go back to the old directory and the zipped json files, we only need to delete nvd.vulnerabilities.db (for old installs) and the .cache/cvedb files (for the current dev tree -- no big deal as they're replaced pretty much daily right now anyhow) so probably we'll just look to do it with every run of the tool. I debated for a wihle if people might need to run them concurrently, but with the bug #1 fix in the 1.0 release, the old data used in 0.3.1 is pretty terrible in comparison so I don't know that anyone should want to stay at the old release.