Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time
November 3, 2022 15:02
August 4, 2021 18:34
October 3, 2023 14:57
May 16, 2023 11:38
September 1, 2021 09:27
May 26, 2023 10:35
October 3, 2023 14:57
October 3, 2023 14:57
February 19, 2021 16:11
September 11, 2023 11:28

KTranslate - Kentik data to the world

Listen for a feed of data to or from Kentik and pass on in a common form. Supports rollups and filtering as well.

See the Wiki for more details. Come visit the Discord if you have any questions, need any assistance, or want to talk about the development of ktranslate.


make && make test

Build Docker Image:

To build and use a Docker image, you must specify MAXMIND_LICENSE_KEY as a build arg:

docker build --build-arg MAXMIND_LICENSE_KEY=xxxxx -t ktranslate:v2 .

To get your own MaxMind key, visit MaxMind.


  -api_device_file string
    	File to sideload devices without hitting API
  -api_devices string
    	json file containing dumy devices to use for the stub Kentik API
  -api_root string
    	API url prefix. If not set, defaults to (default "")
  -application_map string
    	File containing custom application mappings
  -asn string
    	Asn mapping file
    	Run as a AWS Lambda function
  -aws_local_file string
    	If set, process this local file and exit
  -aws_regions string
    	CSV list of region to run in. Will look for metadata in all regions, run SQS in first region. (default "us-east-1")
  -bootstrap.servers string
  -compression string
    	compression algo to use (none|gzip|snappy|deflate|null) (default "none")
  -dns string
    	Resolve IPs at this ip:port
  -enricher string
    	Send data to this http url for enrichment.
  -file_flush_sec int
    	Create a new output file every this many seconds (default 60)
    	If true, start writting to file sink right away. Otherwise, wait for a USR1 signal
  -file_out string
    	Write flows seen to log to this directory if set (default "./")
  -filters value
    	Any filters to use. Format: type dimension operator value
    	If true, don't poll snmp devices.
  -format string
    	Format to convert kflow to: (json|flat_json|avro|netflow|influx|prometheus|new_relic|new_relic_metric|elasticsearch|kflow) (default "flat_json")
  -format_rollup string
    	Format to convert rollups to: (json|avro|netflow|influx|prometheus|new_relic|new_relic_metric|elasticsearch|kflow)
  -gcloud_bucket string
    	GCloud Storage Bucket to write flows to
  -gcloud_content_type string
    	GCloud Storage Content Type (default "application/json")
  -gcloud_prefix string
    	GCloud Storage object prefix (default "/kentik")
  -gcp.project string
    	Google ProjectID to listen for flows on
  -gcp.sub string
    	Google Sub to listen for flows on
  -gcp_pubsub_project_id string
    	GCP PubSub Project ID to use
  -gcp_pubsub_topic string
    	GCP PubSub Topic to publish to
  -geo string
    	Geo mapping file
    	Listen for content sent via http.
  -http_header value
    	Any custom http headers to set on outbound requests
  -http_url string
    	URL to post to (default "http://localhost:8086/write?db=kentik")
  -iam_role string
    	IAM Role to use for processing flow
    	Also send stats about this collector
  -input_threads int
    	Number of threads to run for input processing
  -kafka_topic string
    	kafka topic to produce on
  -kentik_email string
    	Kentik email to use for API calls
  -kentik_plan int
    	Kentik plan id to use for creating devices
  -kentik_relay_url string
    	If set, override the kentik api url to send flow over here.
  -listen string
    	IP:Port to listen on (default "")
  -log_level string
    	Logging Level (default "info")
  -mapping string
    	Mapping file to use for enums
  -max_before_sample int
    	Only sample when a set of inputs is at least this many (default 1)
  -max_flows_per_message int
    	Max number of flows to put in each emitted message (default 10000)
  -max_threads int
    	Dynamically grow threads up to this number
  -metalisten string
    	HTTP interface and port to bind on
  -metrics string
    	Metrics Configuration. none|syslog|stderr|graphite: (default "none")
  -net_protocol string
    	Use this protocol for writing data (udp|tcp|unix) (default "udp")
  -net_server string
    	Write flows seen to this address (host and port)
  -netflow_version string
    	Version of netflow to produce: (netflow9|ipfix) (default "ipfix")
  -nf.addr string
    	Sflow/NetFlow/IPFIX listening address (default "")
  -nf.mapping string
    	Configuration file for custom netflow mappings
  -nf.message.fields string
    	The list of fields to include in flow messages. Can be any of Type,TimeReceived,SequenceNum,SamplingRate,SamplerAddress,TimeFlowStart,TimeFlowEnd,Bytes,Packets,SrcAddr,DstAddr,Etype,Proto,SrcPort,DstPort,InIf,OutIf,SrcMac,DstMac,SrcVlan,DstVlan,VlanId,IngressVrfID,EgressVrfID,IPTos,ForwardingStatus,IPTTL,TCPFlags,IcmpType,IcmpCode,IPv6FlowLabel,FragmentId,FragmentOffset,BiFlowDirection,SrcAS,DstAS,NextHop,NextHopAS,SrcNet,DstNet,HasMPLS,MPLSCount,MPLS1TTL,MPLS1Label,MPLS2TTL,MPLS2Label,MPLS3TTL,MPLS3Label,MPLSLastTTL,MPLSLastLabel,CustomInteger1,CustomInteger2,CustomBytes1,CustomBytes2 (default "TimeReceived,SamplingRate,Bytes,Packets,SrcAddr,DstAddr,Proto,SrcPort,DstPort,InIf,OutIf,SrcVlan,DstVlan,TCPFlags,SrcAS,DstAS,Type,SamplerAddress")
  -nf.port int
    	Sflow/NetFlow/IPFIX listening port (default 9995)
  -nf.prom.listen string
    	Run a promethues metrics collector here
    	Enable so_reuseport for Sflow/NetFlow/IPFIX
  -nf.source string
    	Run NetFlow Ingest Directly. Valid values here are netflow5|netflow9|ipfix|sflow
  -nf.workers int
    	Number of workers per flow collector (default 1)
  -nr_account_id string
    	If set, sends flow to New Relic
    	Verify body is valid json before sending on
    	If true, record size of inputs to NR but don't actually send anything
  -nr_region string
       NR Region to use. US|EU|GOV
  -olly_dataset string
    	Olly dataset name
  -olly_write_key string
    	Olly dataset name
  -prom_listen string
    	Bind to listen for prometheus requests on. (default ":8082")
  -prom_seen int
    	Number of flows needed inbound before we start writting to the collector (default 10)
    	Send both rollups and alpha inputs to sinks
  -rollup_interval int
    	Export timer for rollups in seconds
  -rollup_key_join string
    	Token to use to join dimension keys together (default "^")
  -rollup_top_k int
    	Export only these top values (default 10)
  -rollups value
    	Any rollups to use. Format: type, name, metric, dimension 1, dimension 2, ..., dimension n: sum,bytes,in_bytes,dst_addr
  -s3_bucket string
    	AWS S3 Bucket to write flows to
      AWS assume role ARN which has permissions to write to S3 bucket
      If to use EC2 Instance Profile of the machine (default false)
      S3 Bucket region where S3 bucket is created (default us-east-1)
  -s3_flush_sec int
    	Create a new output file every this many seconds (default 60)
        Refresh credentials of Assume Role or Instance Profile (whichever is earliest) after this many seconds (default 900)
  -s3_prefix string
    	AWS S3 Object prefix (default "/kentik")
  -sample_rate int
    	Sampling rate to use. 1 -> 1:1 sampling, 2 -> 1:2 sampling and so on.
  -service_name string
    	Service identifier (default "ktranslate")
  -sinks string
    	List of sinks to send data to. Options: (kafka|stdout|new_relic|kentik|net|http|prometheus|file|s3|gcloud) (default "stdout")
  -snmp string
    	yaml file containing snmp config to use
    	If true, try to discover snmp devices on this network as configured.
  -snmp_do_walk string
    	If set, try to perform a snmp walk against the targeted device.
    	If true, dump the list of possible mibs on start.
  -snmp_json2yaml string
    	If set, convert the passed in json file to a yaml profile.
  -snmp_out_file string
    	If set, write updated snmp file here.
  -snmp_poll_now string
    	If set, run one snmp poll for the specified device and then exit.
  -snmp_walk_format string
    	use this format for walked values if -snmp_do_walk is set.
  -snmp_walk_oid string
    	Walk this oid if -snmp_do_walk is set. (default ".")
  -sqs_name string
    	Listen for events from this queue for new objects to look at.
  -ssl_cert_file string
    	SSL Cert file to use for serving HTTPS traffic
  -ssl_key_file string
    	SSL Key file to use for serving HTTPS traffic
    	Log to stdout (default true)
  -syslog.format string
    	Format to parse syslog messages with. Options are: Automatic|RFC3164|RFC5424|RFC6587. (default "Automatic")
  -syslog.source string
    	Run Syslog Server at this IP:Port or unix socket.
    	Listen on TCP for syslog messages. (default true)
  -syslog.threads int
    	Number of threads to use to process messages. (default 1)
    	Listen on UDP for syslog messages. (default true)
    	Listen on a Unix socket for syslog messages.
  -tag_map string
    	CSV file mapping tag ids to strings
  -tag_map_type string
    	type of mapping to use for tag values. file|null
    	Tee log messages to sink
  -threads int
    	Number of threads to run for processing
  -udrs string
    	UDR mapping file
  -v	Show version and build information
  -vpc string
    	Run VPC Flow Ingest


To expose profiling endpoints, use the -metalisten flag. This can be used with tools such as go tool pprof to capture and view the data. For example, if ktranslate was started with -metalisten :6060:

go tool pprof -http :8080

To view all available profiles, open http://localhost:6060/debug/pprof/ in your browser.

This product includes GeoLite2 data created by MaxMind, available from