differential osquery query output to base_topic

[alexwoolford]

Bug report

versions

CentOS 7.7
osquery 4.0.2

What steps did you take to reproduce the issue?

The osquery.conf has the following properties:

{
  "options": {
    "logger_kafka_brokers": "cp01.woolford.io:9092,cp02.woolford.io:9092,cp03.woolford.io:9092",
    "logger_kafka_topic": "base_topic",
    "logger_kafka_acks": "1"
  },
  "packs": {
    "system-snapshot": {
      "queries": {
        "processes_by_port": {
          "query": "select u.username, p.pid, p.name, pos.local_address, pos.local_port, pos.remote_address, pos.remote_port from processes p join users u on u.uid = p.uid join process_open_sockets pos on pos.pid=p.pid where pos.remote_port != '0';",
          "interval": 10,
          "snapshot": false
        }
      }
    }
  },
  "kafka_topics": {
    "process-port": [
      "pack_system-snapshot_processes_by_port"
    ]
  }
}

The output is written to the process-port topic. If I toggle the snapshot property, the output is written to the base_topic. That is, in my opinion, a bug.

links:

• SO question: https://stackoverflow.com/questions/58384470/differential-osquery-query-output-to-catchall-topic
• video walk through of issue: https://youtu.be/sPdlBBKgJmY
• possible duplicate of: KafkaProducerPlugin can't get the right topic name if msg's payload contains data 'name' #5559

What did you expect to see?

The differential records should be written to the process-port.

What did you see instead?

The records are written to base_topic.

[directionless]

I know @zwass and @alexwoolford were talking in slack, but I'm not sure if there was a conclusion

[alexwoolford]

Hey @directionless, @zwass suggested that I file a bug in Github: https://osquery.slack.com/archives/C08V7KTJB/p1571159030065500?thread_ts=1571098086.052800&cid=C08V7KTJB, and so I did.

I'll gladly jump on a Zoom if you'd like to poke around in my environment to see what's going on.

[directionless]

I don't know this part of the code base. I'm recording that the conversation happened. The bug is great.