Like deNIST except dotnet. Intended to help identify NIST files in ediscovery processing. Maintained by Elemental Discovery.
The underlying NIST list comes from the National Software Reference Library and is formally known as the Reference Data Set.
This project consists of a variety of components:
- The
Dotnistlibrary, which does lookups against a provided sqlite database. - The
Dotnist.GrpcgRPC server, which is bundled into thedotnist-grpccontainer image (db included). - The
Dotnist.Clientclient library, which is just a package that directly provides the generated gRPC client library. You could alternately copy thedotnist.protofile directly, which is what you would need to do for other languages.
Dotnist and Dotnist.Client are available as NuGet packages. The dotnist-grpc container is available both on Docker Hub and in this repository's Github Container Registry.
For usage, look at the tests and the dotnist.proto file.
The 'minimal' modern RDS database is comically large at >170 GB after extraction, so we have a manual workflow to shrink the database into just a single package reference for each unique Sha256 hash, discarding the filename and other hash types from the database as well. The library is designed to work with both a full database and the flattened database, but the Docker container bundles this handmade flattened database.
| Version | RDS Release |
|---|---|
| 1.0.x | 2025-06-01 |
| 1.1.x | 2025-09-01 |
| 1.2.x | 2025-12-01 |
In order to apply the quarterly RDS patches:
- Download the NSRL provided .sql patch for the modern minimal dataset.
- Use
update-database.shto apply the patch to a complete (non-flattened) sqlite database. - Use
flatten-sqlite.shto generate a flattened database. - Use
build-db-image.shto bundle the new flattened database and push it to GHCR.
After that the build process that runs in Github Actions will be able to use the updated .db file.
Some hypothetical improvements that aren't worth the time at the moment:
- Running tests in CI is a hassle since you need the database, one option would be to run the tests in a docker container.
- A version of this that works against a NIST list stored in a regular hosted database.
- I didn't bother to configure the image builds to run during pull requests, like you would normally want in a development workflow.
If anyone else finds this library useful definitely feel free to reach out! I'd be curious about what you are working on.