Augment recursor protobuf data output

[nickjacques]

• Program: Recursor
• Issue type: Feature request/enhancement

Short description

It appears that the Recursor protobuf implementation only includes a DNSRR message if the RR type is A, AAAA, or CNAME. For those of us wanting to log the answers received for TXT, NS, PTR, and SOA (and other non-A/AAAA/CNAME RRs), we are unable to get this information.

Environment

• Operating system: CentOS 7.4.1708
• Software version: Recursor 4.0.7
• Software source: PowerDNS CentOS repo

Steps to reproduce

1. Configure pdns_recursor to output protobuf (we use protobufServer("127.0.0.1:5454", 2, 100, 1, 32, 128) to send the data to a sidecar Go binary that parses protobuf.
2. Parse PBDNSMessage, DNSResponse inside of that, and attempt to extract data from DNSRR for non-A/AAAA/CNAME types.
3. Observe NULL data for non-A/AAAA/CNAME types.

Expected behaviour

The nested data in PBDNSMessage{ DNSResponse{ DNSRR {...} } } should be accessible/non-null, as A/AAAA/CNAME are. For A/AAAA/CNAMEs, our sidecar protobuf parser emits the following JSON:

{
  "class": "IN",
  "client": "10.49.x.y",
  "level": "info",
  "message_id": "3tT4bFtXSGaQTY1MavSN9w==",
  "message_timestamp": "2018-02-22T06:34:42.116518Z",
  "msg": "query",
  "question": "www.apple.com.",
  "server": "10.49.x.x",
  "time": "2018-02-22T06:34:42Z",
  "type": "A"
}
{
  "answer": "www.apple.com.edgekey.net.,www.apple.com.edgekey.net.globalredir.akadns.net.,e6858.dsce9.akamaiedge.net.,23.197.35.121",
  "client": "10.49.x.y",
  "level": "info",
  "message_id": "3tT4bFtXSGaQTY1MavSN9w==",
  "message_timestamp": "2018-02-22T06:34:42.116579Z",
  "msg": "response",
  "query_timestamp": "2018-02-22T06:34:42.116509Z",
  "question": "www.apple.com.",
  "response code": "NoError",
  "server": "10.49.x.x",
  "time": "2018-02-22T06:34:42Z",
  "type": "CNAME,CNAME,CNAME,A"
}

Actual behaviour

The data in the DNSRR message does not exist, likely due to

if (record.d_place != DNSResourceRecord::ANSWER ||
      record.d_class != QClass::IN ||
      (record.d_type != QType::A &&
       record.d_type != QType::AAAA &&
       record.d_type != QType::CNAME)) {
    return;
  }

in rec-protobuf.cc here. As such, our sidecar protobuf parser emits the following:

{
  "class": "IN",
  "client": "10.49.x.y",
  "level": "info",
  "message_id": "qNXmyzJ+QZW6+XcknEOHCQ==",
  "message_timestamp": "2018-02-22T06:33:34.511079Z",
  "msg": "query",
  "question": "apple.com.",
  "server": "10.49.x.x",
  "time": "2018-02-22T06:33:34Z",
  "type": "MX"
}
{
  "answer": "",
  "client": "10.49.x.y",
  "level": "info",
  "message_id": "qNXmyzJ+QZW6+XcknEOHCQ==",
  "message_timestamp": "2018-02-22T06:33:34.511182Z",
  "msg": "response",
  "query_timestamp": "2018-02-22T06:33:34.511066Z",
  "question": "apple.com.",
  "response code": "NoError",
  "server": "10.49.x.x",
  "time": "2018-02-22T06:33:34Z",
  "type": ""
}

As you can see, the answer and type keys are empty/null/nil in the response because there is no data in the DNSRR message (where these elements come from: PBDNSMessage{ DNSResponse{ DNSRR {type and Rdata} } }. The type and class in the query is non-empty/null/nil because we get that from PBDNSMessage{ DNSQuestion { qType and qClass }}.

Other information

Being able to access string data for records like TXT, NS, PTR, SOA, MX, SPF, and SRV would be beneficial to trace the majority of query responses.

Usecase

We are recording query/answer information at the recursor for analysis/forensics in case of malware using DNS for C&C. As such, the answers for RR TYPEs like TXT are extremely advantageous to have.

Description

The software should provide a DNSRR message for all valid responses. Popular and lower-hanging fruit include types mentioned above (TXT, NS, PTR, SOA, MX, SPF, and SRV). Some may have use cases for DNSSEC TYPEs.

[rgacogne]

Implemented in #6708, closing.