Authenticode verification support for Windows (similar to the 'signature' table on macOS) #3716

alessandrogario · 2017-09-21T21:49:54Z

Goal

This PR aims to add Authenticode verification support to osquery in a similar way to the 'signature' virtual table that has been implemented for macOS.

Changes

I have defined a signature virtual table for Windows, containing the following columns: path, original_program_name (from the publisher), serial_number, issuer_name, subject_name, result.

The following are the possible values for the result column:

missing: Missing signature.
invalid: An invalid signature, caused by missing or broken files.
untrusted: A signature that could not be validated.
distrusted: A valid signature, explicitly distrusted by the user.
valid: A valid signature which is not explicitly trusted by the user.
trusted: A valid signature, trusted by the user.

Examples

Verifying the authenticode signature for running executables

SELECT process.pid, process.path, authenticode.result
FROM processes as process
LEFT JOIN signature AS authenticode
ON process.path = authenticode.path;

...
| 5124 | C:\Windows\system32\winlogon.exe    | missing |
| 2476 | C:\Windows\system32\dwm.exe         | missing |
| 2744 | C:\Windows\system32\fontdrvhost.exe | trusted |
| 6372 | c:\windows\system32\sihost.exe      | missing |
| 7152 | c:\windows\system32\svchost.exe     | trusted |
| 80   | c:\windows\system32\svchost.exe     | trusted |
| 6748 | c:\windows\system32\taskhostw.exe   | trusted |
...

Listing unsigned/untrusted processes listening for connections

SELECT port_info.pid, port_info.family, port_info.protocol, port_info.port, port_info.address,
       process.name, process.path,
       authenticode.result AS authenticode

FROM listening_ports AS port_info

LEFT JOIN processes
AS process ON port_info.pid = process.pid

LEFT JOIN signature AS authenticode
ON process.path = authenticode.path

WHERE authenticode <> "trusted";



+------+--------+----------+-------+---------+-------------+------------------------------------+--------------+
| pid  | family | protocol | port  | address | name        | path                               | authenticode |
+------+--------+----------+-------+---------+-------------+------------------------------------+--------------+
| 6588 | 2      | 6        | 80    | 0.0.0.0 | hfs.exe     | C:\Program Files (x86)\HFS\hfs.exe | missing      |
| 1724 | 2      | 6        | 49667 | 0.0.0.0 | spoolsv.exe | C:\Windows\System32\spoolsv.exe    | missing      |
| 1724 | 23     | 6        | 49667 | ::      | spoolsv.exe | C:\Windows\System32\spoolsv.exe    | missing      |
+------+--------+----------+-------+---------+-------------+------------------------------------+--------------+

facebook-github-bot · 2017-09-21T22:09:16Z

@alessandrogario has updated the pull request. View: changes

facebook-github-bot · 2017-09-21T22:46:11Z

@alessandrogario has updated the pull request. View: changes

theopolis

I'd recommend creating an authenticode or signature table for Windows.

alessandrogario · 2017-09-24T14:05:28Z

@theopolis sure! I'm still writing the utilities I need, as I also want to be able to list signature information like signer name etc. Once it's done, I'll create the new table; how would you like it to work? Should it just list the running executables?

theopolis · 2017-09-25T03:26:13Z

how would you like it to work? Should it just list the running executables?

Check out the signature table for macOS, if you don't mind, mimic the user experience for that.

facebook-github-bot · 2017-09-28T23:08:25Z

@alessandrogario has updated the pull request.

facebook-github-bot · 2017-09-28T23:09:17Z

@alessandrogario has updated the pull request.

alessandrogario · 2017-09-28T23:31:15Z

I have defined a 'signature' virtual table for Windows; it only sports two columns for now, but I plan on adding more. I'm using a more verbose field instead of the 'signed' boolean used in the macOS implementation.

The following are the possible states:

missing: Missing signature.
invalid: An invalid signature, caused by missing or broken files.
untrusted: A signature that could not be validated.
distrusted: A valid signature, explicitly distrusted by the user.
valid: A valid signature which is not explicitly trusted by the user.
trusted: A valid signature, trusted by the user.

EDIT: I should probably put this information in the summary!

muffins

A couple of smaller nits and a question about the << operator definition, not sure if that'll fly as it seems risky to define it in such a high scope for something used by so many places in the code base.

Otherwise super excited for this table to land!

muffins · 2017-09-29T05:55:30Z

osquery/tables/system/windows/signature.cpp

+  case SignatureInformation::Result::Untrusted: {
+    row["result"] = "untrusted";
+    break;
+  }


perhaps you could default to unknown or -1?

muffins · 2017-09-29T05:55:30Z

osquery/tables/system/windows/signature.cpp

+  return row;
+}
+
+Status toUtf16String(std::wstring& output, const std::string& s) {


We have a stringToWstring function already implemented here that you could use, but this looks good -- any chance you could take a look at our implementation to see if it could stand some improvements? ;)

They look similar! I am now using that one!

muffins · 2017-09-29T05:55:30Z

osquery/tables/system/windows/signature.cpp

+    if (status.ok()) {
+      results.push_back(r);
+    } else {
+      LOG(ERROR) << status.getMessage();


ERROR seems a bit intense for a failure to compute the signature of an executable, maybe this could be a WARNING instead? I think we try and reserve ERROR for something which would halt the daemon itself.

muffins · 2017-09-29T05:55:30Z

specs/windows/signature.table

+])
+implementation("signature@generateQueryResults")
+examples([
+  "select * from signature where path = 'C:\\Windows\\notepad.exe'",


Would you mind adding in an example that joins off of the process table just to show up the extra cool features? :D

Done! I've added a simple example in the table specs, and also edited the PR summary to include a couple of more (that we can probably add to the docs once this is merged).

muffins · 2017-09-29T05:55:30Z

osquery/tables/system/windows/signature.cpp

+  Result result;
+};
+
+Row& operator<<(Row& row, const SignatureInformation& signature_info) {


While this is pretty awesome how you're making use of the << on Rows, I'm honestly not sure how to feel about it as it seems dangerous to overload an operator on a data structure that gets used in so many places in this high of namespace - /CC @theopolis to potentially put my worries to rest, but are there potentially other (arguably less awesome) ways to achieve the same behavior?

facebook-github-bot · 2017-09-29T08:52:05Z

@alessandrogario has updated the pull request. View: changes

facebook-github-bot · 2017-09-30T12:26:49Z

@alessandrogario has updated the pull request. View: changes

facebook-github-bot · 2017-09-30T13:45:09Z

@alessandrogario has updated the pull request. View: changes

facebook-github-bot · 2017-09-30T13:55:54Z

@alessandrogario has updated the pull request. View: changes

facebook-github-bot · 2017-09-30T14:13:23Z

@alessandrogario has updated the pull request. View: changes

facebook-github-bot · 2017-09-30T14:22:14Z

@alessandrogario has updated the pull request.

theopolis

I'll defer to @muffins but a quick note, some of the VLOGs seem a bit generic.

theopolis · 2017-10-04T15:25:50Z

osquery/tables/system/windows/signature.cpp

+
+namespace osquery {
+// These are defined in osquery/core/windows/wmi.cpp
+std::wstring stringToWstring(const std::string& src);


Are these not in a header somewhere?

@alessandrogario - Just in case you didn't see it they're here :)

theopolis · 2017-10-04T15:25:50Z

osquery/tables/system/windows/signature.cpp

+// clang-format on
+
+#include <osquery/core.h>
+#include <osquery/core/conversions.h>


This should be in it's own include block (double new line separated) and included using the local search:

#include "osquery/core/conversions.h"

theopolis · 2017-10-04T15:25:50Z

osquery/tables/system/windows/signature.cpp

+  return Status(0, "Ok");
+}
+
+QueryData generateQueryResults(QueryContext& context) {


Can you use a more specific symbol name here? genAuthenticode or something similar?

@theopolis

Dismissing as my review is dated. Will review again after the changes requested by @theopolis are addressed.

facebook-github-bot · 2017-10-05T17:25:37Z

@alessandrogario has updated the pull request.

facebook-github-bot · 2017-10-05T17:44:32Z

@alessandrogario has updated the pull request. View: changes

theopolis · 2017-10-10T15:18:37Z

osquery/tables/system/windows/signature.cpp

+      &certificate_context->pCertInfo->SerialNumber;
+
+  std::stringstream serial_number_string;
+  for (DWORD i = serial_number->cbData - 1; i < serial_number->cbData; i--) {


Can you explain this loop?

This is used to print the serial number backwards (as it is shown in the Authenticode dialog properties of Windows).

Since DWORD is unsigned, the loop uses the fact that according to the standard the value will wrap around once you decrement past zero.

theopolis · 2017-10-10T15:18:37Z

osquery/tables/system/windows/signature.cpp

+      return false;
+    }
+
+    std::string buffer;


You can create the buffer with a requested container size in the ctor

theopolis · 2017-10-10T15:34:55Z

ok to test

osqueryer · 2017-10-10T15:39:54Z

🙅 The commit 50f6de6 (Job results: 2631) failed the code audit or documentation test.

osqueryer · 2017-10-10T15:40:51Z

👎 The commit 50f6de6 (Job results: 5988) failed one or more tests (Linux).

osqueryer · 2017-10-10T15:42:26Z

👎 The commit 50f6de6 (Job results: 5989) failed one or more tests (Linux).

osqueryer · 2017-10-10T18:04:32Z

👎 The commit 2e99777 (Job results: 5991) failed one or more tests (Linux).

theopolis · 2017-10-12T14:54:53Z

specs/windows/signature.table

+    Column("issuer_name", TEXT, "The certificate issuer name"),
+    Column("subject_name", TEXT, "The certificate subject name"),
+    Column("result", TEXT, "The signature check result")
+])


Can we morph fields from darwin or windows such that we can have a unified signature table?

schema([ Column("path", TEXT, "Must provide a path or directory", required=True), Column("signed", INTEGER, "1 If the file is signed else 0"), Column("identifier", TEXT, "The signing identifier sealed into the signature"), Column("cdhash", TEXT, "SHA1 hash of the application Code Directory"), Column("team_identifier", TEXT, "The team signing identifier sealed into the signature"), Column("authority", TEXT, "Certificate Common Name"), ])

path cool!

original_program_name can this be called identifier on both?

serial_number, issuer_name, subject_name can be extended schema on Windows for now.

cdhash does this checksum have an authenticode equivalent?

result can stay on both, maybe on darwin we fill this in with "signed" or "unknown"

signed being a boolean should be included here, it should indicate if the binary is properly signed.

The reason that the build is breaking in the CI is because there are two specfiles named signature. This is not allowed.

original_program_name: I have renamed this to "identifier".
cdhash: There is something but to grab it you need to parse the PE header and locate the PKCS#7 structure that contains the hash. There is really no easy way to calculate it without a custom tool (and it's hard to use this value in a meaningful way) so I skipped it.
result: I'll add this column to the darwin schema, and set it to "signed" when "signed" is 1 and to "unknown" when it is 0.
signed: Should this value be 1 only when the signature is both valid and trusted?

What should I do with the remaining fields? Should both specfiles (darwin and Windows) contain the exact same columns?

Leave the fields blank if there are no meaningful compromises/merging options.

You can add "extended" schema like:

extended_schema(WINDOWS, [ Column("friendly_name", TEXT, "The friendly display name of the interface."), ])

The first option is the requirement. These columns will be set to HIDDEN on platforms that do not match. Such that a SELECT * does not return them.

Your options for evaluators right now are: WINDOWS, LINUX, POSIX, DARWIN, FREEBSD. They are small functions defined in gentable.py.

…tion

facebook-github-bot · 2017-10-13T22:54:44Z

@alessandrogario has updated the pull request.

osqueryer · 2017-10-13T22:55:13Z

🙅 The commit 6e5b348 (Job results: 2647) failed the code audit or documentation test.

osqueryer · 2017-10-13T22:55:49Z

👎 The commit 6e5b348 (Job results: 6006) failed one or more tests (Linux).

osqueryer · 2017-10-13T22:55:50Z

👎 The commit 6e5b348 (Job results: 923) failed one or more tests (FreeBSD).

osqueryer · 2017-10-13T22:57:19Z

👎 The commit 6e5b348 (Job results: 6007) failed one or more tests (Linux).

osqueryer · 2017-10-13T23:01:53Z

👎 The commit 6e5b348 (Job results: 924) failed one or more tests (FreeBSD).

facebook-github-bot · 2017-10-13T23:42:09Z

@alessandrogario has updated the pull request.

osqueryer · 2017-10-13T23:43:13Z

👎 The commit 087f1fb (Job results: 925) failed one or more tests (FreeBSD).

osqueryer · 2017-10-13T23:49:30Z

👎 The commit 087f1fb (Job results: 926) failed one or more tests (FreeBSD).

theopolis · 2017-10-14T03:16:57Z

LGTM, @muffins I'll let you have the final look and merge when you're ready.

muffins · 2017-10-14T07:08:52Z

LGTM

facebook-github-bot added the cla signed label Sep 21, 2017

muffins added virtual tables Windows labels Sep 22, 2017

theopolis requested changes Sep 23, 2017

View changes

muffins reviewed Sep 29, 2017

View changes

alessandrogario changed the title ~~[WIP] Authenticode verification support for the 'processes' table on Windows~~ [WIP] Authenticode verification support for Windows (similar to the 'signature' table on macOS) Sep 29, 2017

alessandrogario changed the title ~~[WIP] Authenticode verification support for Windows (similar to the 'signature' table on macOS)~~ Authenticode verification support for Windows (similar to the 'signature' table on macOS) Sep 30, 2017

theopolis reviewed Oct 4, 2017

View changes

muffins added the ready for review label Oct 5, 2017

theopolis reviewed Oct 10, 2017

View changes

theopolis requested changes Oct 12, 2017

View changes

alessandrogario added 8 commits Sep 29, 2017

signature table (Windows): Port of the old code.

fcb09e6

Code review changes.

924f231

signature table (Win): Get the original program name from the signature.

4cda0ab

signature table (Win): Added support for the signature serial number.

1c4bcd1

signature table (Win): Added support for the issuer name.

030d980

signature table (Win): Added subject name.

16d9954

Code review changes

4f9ae95

signature table (Win): Put a size limit on cert fields, fix an alloca…

03c575b

…tion

Renamed the signature (Windows) table to authenticode

Loading status checks…

087f1fb

theopolis approved these changes Oct 14, 2017

View changes

theopolis added approved and removed ready for review labels Oct 14, 2017

muffins approved these changes Oct 14, 2017

View changes

muffins merged commit e888f3e into osquery:master Oct 14, 2017
4 of 5 checks passed

4 of 5 checks passed

FreeBSD Build finished.
Details

Code Audit Build finished.
Details

Linux Build finished.
Details

Windows Build finished.
Details

macOS/OS X Build finished.
Details

trizt added a commit to trizt/osquery that referenced this pull request May 24, 2019

tables: Authenticode verification support for Windows (osquery#3716)

af2061b

osquery / osquery

Join GitHub today

Authenticode verification support for Windows (similar to the 'signature' table on macOS) #3716

Authenticode verification support for Windows (similar to the 'signature' table on macOS) #3716

Conversation

alessandrogario commented Sep 21, 2017 • edited

Goal

Changes

Examples

Verifying the authenticode signature for running executables

Listing unsigned/untrusted processes listening for connections

facebook-github-bot commented Sep 21, 2017

facebook-github-bot commented Sep 21, 2017

theopolis left a comment

alessandrogario commented Sep 24, 2017

theopolis commented Sep 25, 2017

facebook-github-bot commented Sep 28, 2017

facebook-github-bot commented Sep 28, 2017

alessandrogario commented Sep 28, 2017 • edited

muffins left a comment

This comment has been minimized.

muffins Sep 29, 2017 Contributor

This comment has been minimized.

alessandrogario Sep 29, 2017 Author Contributor

This comment has been minimized.

muffins Sep 29, 2017 Contributor

This comment has been minimized.

alessandrogario Sep 29, 2017 Author Contributor

This comment has been minimized.

muffins Sep 29, 2017 Contributor

This comment has been minimized.

muffins Sep 29, 2017 Contributor

This comment has been minimized.

alessandrogario Sep 29, 2017 Author Contributor

This comment has been minimized.

muffins Sep 29, 2017 Contributor

facebook-github-bot commented Sep 29, 2017

facebook-github-bot commented Sep 30, 2017

facebook-github-bot commented Sep 30, 2017

facebook-github-bot commented Sep 30, 2017

facebook-github-bot commented Sep 30, 2017

facebook-github-bot commented Sep 30, 2017

theopolis left a comment

This comment has been minimized.

theopolis Oct 4, 2017 Member

This comment has been minimized.

muffins Oct 5, 2017 • edited Contributor

This comment has been minimized.

theopolis Oct 4, 2017 Member

This comment has been minimized.

theopolis Oct 4, 2017 Member

facebook-github-bot commented Oct 5, 2017

facebook-github-bot commented Oct 5, 2017

This comment has been minimized.

theopolis Oct 10, 2017 Member

This comment has been minimized.

alessandrogario Oct 10, 2017 Author Contributor

This comment has been minimized.

theopolis Oct 10, 2017 Member

theopolis commented Oct 10, 2017

osqueryer commented Oct 10, 2017

osqueryer commented Oct 10, 2017

osqueryer commented Oct 10, 2017

osqueryer commented Oct 10, 2017

This comment has been minimized.

theopolis Oct 12, 2017 Member

This comment has been minimized.

theopolis Oct 12, 2017 Member

This comment has been minimized.

alessandrogario Oct 12, 2017 Author Contributor

This comment has been minimized.

theopolis Oct 13, 2017 Member

facebook-github-bot commented Oct 13, 2017

osqueryer commented Oct 13, 2017

osqueryer commented Oct 13, 2017

osqueryer commented Oct 13, 2017

osqueryer commented Oct 13, 2017

osqueryer commented Oct 13, 2017

facebook-github-bot commented Oct 13, 2017

osqueryer commented Oct 13, 2017

alessandrogario commented Sep 21, 2017 •

edited

alessandrogario commented Sep 28, 2017 •

edited

muffins Sep 29, 2017
Contributor

alessandrogario Sep 29, 2017
Author Contributor

muffins Sep 29, 2017
Contributor

alessandrogario Sep 29, 2017
Author Contributor

muffins Sep 29, 2017
Contributor

muffins Sep 29, 2017
Contributor

alessandrogario Sep 29, 2017
Author Contributor

muffins Sep 29, 2017
Contributor

theopolis Oct 4, 2017
Member

muffins Oct 5, 2017 •

edited

Contributor

theopolis Oct 4, 2017
Member

theopolis Oct 4, 2017
Member

theopolis Oct 10, 2017
Member

alessandrogario Oct 10, 2017
Author Contributor

theopolis Oct 10, 2017
Member

theopolis Oct 12, 2017
Member

theopolis Oct 12, 2017
Member

alessandrogario Oct 12, 2017
Author Contributor

theopolis Oct 13, 2017
Member