feat: bundle databases with package

[JamieMagee]

Closes #39

This is hard to test without running semantic-release locally, but when running the prepack script manually the databases are included in the dist directory:

$ ls packages/osv-offline/dist/
db  index.d.ts  index.js  lib


$ ls packages/osv-offline/dist/db/
crates.io.nedb  go.nedb  maven.nedb  npm.nedb  nuget.nedb  pypi.nedb  rubygems.nedb

[JamieMagee]

@Churro I'd appreciate your review here too 😄

[viceice]

If we bundle the database, we always need a new renovate release to get latest db.
this is the same issue like we currently have with the azure pipeline tasks.

I'll add a new idea to the issue

[JamieMagee]

That's not the intention of this change. The bundled database is designed to be a fallback if we're unable to download the latest database. For example if the GitHub API is down, or there are other network issues, we can fallback to a stale version of the database. It might not be the most up-to-date, but it is better than nothing.

[Churro]

I appreciate the idea, as it adds resilience to download errors due to rate limiting or unavailability.

However, I'm not yet enirely convinced it will work as intended:

• osv-offline.zip gets added whenever osv-offline is bundled
• The database is not re-fetched if modified within < 24h:
  

  osv-offline/packages/osv-offline/src/lib/download.ts

  Line 32 in b7b6a27

  DateTime.utc().diff(DateTime.fromJSDate(stats.mtime)).as('days') < 1

• Now the promise await new Octokit().repos.listReleases may be rejected, e.g. to value [HttpError: Not Found]:
  

  osv-offline/packages/osv-offline/src/lib/download.ts

  Lines 37 to 41 in b7b6a27

  	const latestRelease = (
  	await new Octokit().repos.listReleases({
  	...baseParameters,
  	})
  	).data[0];

• ... which will make osvOffline fail as well:
  

  osv-offline/packages/osv-offline/src/lib/osv-offline.ts

  Lines 14 to 17 in b7b6a27

  	const success = await tryDownloadDb();
  	if (!success) {
  	throw new Error();
  	}

Please correct me if I'm overlooking something here.

[viceice]

ok, but it'll also blow up a lot. so we should weight that too.

@rarkins WDYT?

[rarkins]

I use Renovate as a dependency downstream so would prefer to avoid any major increase in install size.

[JamieMagee]

It would add ~7MB to the install size.

[rarkins]

Really seems too much. Can you publish two packages - slim and full and Renovate uses the slim?

[JamieMagee]

No worries, I'll just close this instead.