Implement MongoDB

[zakird]

No description provided.

[jordan-wright]

I'd be happy to take a pass at this. I'm excited to see zgrab2 and I like the modular approach. This seems a lot easier to write modules for than the previous version of zgrab. 😄

I'd be happy to take on MongoDB and other nosql databases/document stores (e.g. elasticsearch) as well as key-value stores (memcached/redis). I'll make issues for them before working on them for tracking.

I do have a quick question though: What kind of information is important to Censys? MongoDB without authentication can offer information about the host, such as database size compared to available space. It also contains info about the various collections and more. Shodan collects this information but I wasn't sure if it'd be important to Censys to collect it. It would result in more HTTP round trips, which might add overhead for mass ipv4 scans.

Once I know what information is important to y'all, I'll get to work. My plan is to work through an initial module, taking notes on how it's implemented. I'll put those notes in a blog post to encourage other participation, and we can consider if the notes might be useful in the wiki here.

[dadrian]

I'd be happy to take a pass at this. I'm excited to see zgrab2 and I like the modular approach. This seems a lot easier to write modules for than the previous version of zgrab. 😄

Awesome!

I do have a quick question though: What kind of information is important to Censys? MongoDB without authentication can offer information about the host, such as database size compared to available space. It also contains info about the various collections and more. Shodan collects this information but I wasn't sure if it'd be important to Censys to collect it. It would result in more HTTP round trips, which might add overhead for mass ipv4 scans.

We generally adopt the approach of "collect as much information as possible without needing to open a second TCP socket or logging in". I expect performance won't be that large of an issue here, since there shouldn't be nearly as many Mongo hosts as there are HTTPS hosts. We can always tone down some collection if needed. We'll have a better sense of this once we start seeing some initial output, etc as the module is written.

Tagging @justinbastress, who I think may have also been looking into this scanner.

[jordan-wright]

"collect as much information as possible without needing to open a second TCP socket or logging in"

This is a perfectly reasonable approach. And I suppose nothing says we can't add more information later. Really, just knowing the number of mongodb hosts is a huge lift. Collection, system, etc. information can be made available later if the cycles permit it 🤷‍♂️ Getting enriched information at the app layer is great, but alluding to the "one TCP socket" approach it definitely comes with a performance cost.

And @justinbastress please let me know if I'm stepping over work that's already done. The last thing I want to do is repeat work, so I can always start somewhere else, like Redis or Elasticsearch.

[justinbastress]

@jordan-wright I had not started on MongoDB yet, so no worries about duplicated labor.

[jordan-wright]

Out of curiosity, what's the current stance on leveraging external libraries to gather some service specific data? For example, MongoDB has a weird wire protocol that might be a bit tricky to implement adhoc if we wanted to do it ourselves.

The option exists to use mgo to handle setting up the connection and pulling data, but it adds a dependency.

The only reason I ask is because it looks like the option for MySQL scanning was implemented in a custom library in zgrab.

[parkanzky]

MongoDB scanner implemented.