Architecture

Here are some sample deployments of Moloch for different network architectures. Most folks will probably run a hybrid of the following since no one solution fits all. When planning please take into account that currently a single moloch-capture can only process around 2Gbps with no packet drops. The ability to scale either multiple capture machines or multiple-capture process per machine is needed.

Legend/Info:

• A box represents a physical machine.
• It is possible to run multiple capture processes per machine
• Run multiple Elasticsearch processes per machine since each ES node should be configured at most to 30G
• Except for single host, it is recommended/useful that all operator access flows through a single apache/viewer combination that can provide better authentication, logging, and a single choke point

Security

• All ES instances should have iptables for port 9200-920N and 9300-930N, where N is the number of ES instances per machine, and only allow the other elasticsearch, capture and viewer machines to connect
• All viewer hosts, except the apache/viewer box, should have iptables for port 8005 and only allow other viewer machines to connect. The viewer must listen on OS interface if using multiple machines
• The shared viewer instances can listen on localhost since only apache talks to it

Single Host

Multiple Hosts Monitoring Multiple Network Segments

Multiple Hosts Monitoring High Traffic Networks

Notes:

• Using a Network Packet Broker allows traffic to be load balanced. This is especially useful in HA or asymmetric routing
• By using a NPB other security devices can see the same traffic moloch sees
• When running multiple moloch-captures on the same host make sure the IO doesn't over whelm the disk and other subsystems.
• Use a TAP with High Traffic Networks since many mirror ports drop traffic under heavy load
• Operator uses an apache fronted viewer and don't hit the other viewers directly. The apache provides authentication.

Multiple Clusters

Notes:

• It is possible to use a single ES cluster using the prefix= ini configuration
• Operator uses apache fronted viewers and doesn't hit the other viewers directly. The apache provides authentication. Can use virtual paths to route to different clusters.