How can I find details on the binary representation of the RPM DB?

[shanebishop]

I am trying to find an RPM version agnostic way to find details on installed software on any Linux distro that uses RPM packages.

A co-worker found for one version of RPM that the magic byte array/slice []byte{0, 1, 0x43, 0} works for extracting each installed package from the RPM DB using regex (I don't know which version this works for). However, this does not work for other RPM versions.

Is there any documentation for the binary representation of the RPM DB for each RPM version I could look at?

If there is no documentation for this, where can I find the relevant part of the source code for the rpm binaries? (My thought is if there is no documentation, I could try reading the source code to figure out how I can extract the information I need from the RPM DB for each RPM version.)

[pmatilai]

That's the wrong end to be looking at, totally.

'rpm -q' with --queryformat gives you access to every single bit of data in the rpmdb.

The other option is the librpm API, whether through C or Python bindings. The API for database access is largely unchanged in the last 14-15 years.

[shanebishop]

I found that, starting with RPM 4.16, RPM uses an SQLite database (reference).

From querying my local table, I see there are the following tables:

sqlite> .tables
Basenames             Name                  Sigmd5              
Conflictname          Obsoletename          Suggestname         
Dirnames              Packages              Supplementname      
Enhancename           Providename           Transfiletriggername
Filetriggername       Recommendname         Triggername         
Group                 Requirename         
Installtid            Sha1header          

I can look at the schemas of individual tables like so:

sqlite> .schema Name
CREATE TABLE IF NOT EXISTS 'Name' (key 'TEXT' NOT NULL, hnum INTEGER NOT NULL, idx INTEGER NOT NULL, FOREIGN KEY (hnum) REFERENCES 'Packages'(hnum));
CREATE INDEX 'Name_key_idx' ON 'Name'(key ASC);

Is there any documentation for what is stored in each table?

For example, it seems the Name table stores the names of all of the installed RPM packages, and from the schema for this table I can see that it has a foreign hnum key for the Packages table. But when I look at the Packages table, I can't make sense of the data, since it stores blobs:

sqlite> .schema Packages
CREATE TABLE IF NOT EXISTS 'Packages' (hnum INTEGER PRIMARY KEY AUTOINCREMENT,blob BLOB NOT NULL);
sqlite> select * from Packages limit 10;
1|
2|
3|
4|
5|
6|
7|
8|
9|
10|

[pmatilai]

The details of the database format are OFF-LIMITS TO EXTERNAL USERS!

Sqlite is but one of the possible database formats. If you want to access the rpm database, you do so through the librpm API.

[DemiMarie]

The details of the database format are OFF-LIMITS TO EXTERNAL USERS!

Sqlite is but one of the possible database formats. If you want to access the rpm database, you do so through the librpm API.

Just because one should not access the rpmdb without going through librpm does not mean that one might not be curious about the actual rpmdb format. I know I am!

[pmatilai]

Nothing wrong with curiosity, and this being open source, obviously nobody can prevent people from looking.
However I have no incentive to help planned misuse, which is clearly the case here.

The rpmdb format is undocumented because it is a private implementation detail which rpm is free to change at will.

[pmatilai]

A bug is a bug. The database needs to be as robust as anything else in rpm, security impact or no.

[rhdesmond]

Thanks for the detailed discussion all!

@DemiMarie is correct; I understand @pmatilai's concerns about intended use and security impact. For now, we parse the db files (as other open source scanners do) as creating a runtime is prohibitively expensive as pointed out above. Appreciate the notes!

[DemiMarie]

@pmatilai I, and I suspect @rhdesmond as well, are not comfortable creating a situation where a bug is not a security vulnerability in RPM, but is a security vulnerability in the downstream project.

[pmatilai]

And that prevents you from reporting bugs? If so, the security world is even sadder place than I thought.

[DemiMarie]

It does not @pmatilai, but many people are uncomfortable with situations where reporting a bug via the proper channels (public GitHub issue) means publicizing a 0day vulnerability in their own product. They would prefer if security problems in their product caused by upstream bugs were considered security vulnerabilities in the upstream code, so that they could be handled via the private channels used for such things (secalert@redhat.com in this case).