Skip to content

RedELK server installation

Jump to bottom
Marc Smeets edited this page Dec 9, 2022 · 15 revisions

In short

  1. extract elkserver.tgz on your RedELK server
  2. edit mounts/redelk-config/etc/redelk/config.json
  3. run install-elkserver.sh (optional with parameters)
  4. post installation config of mounts/redelk-config/etc/cron.d/redelk
  5. post installation config of mounts/redelk-config/etc/redelk/iplist_* files

In detail

Installation

  1. Copy and extract elkserver.tgz on your RedELK server as part of your red team infra deployment procedures.

  2. Modify mounts/redelk-config/etc/redelk/config.json to your liking. Explanation of the fields:

  • loglevel defines the logging level for the background daemons. Normally no need to change. Possible values: CRITICAL, ERROR, WARNING, INFO, DEBUG - Default: WARNING.
  • redelkserver_letsencrypt (optional) if you want to use certbot certificates for your Kibana interface.
    • external_domain: the domain name of RedELK server
    • le_email: the email used for Lets Encrypt registration
    • staging: staging
  • notifications settings for notification delivery. You will need to enable the module you want to use. By default none is enabled.
    • email: notifications via email. All fields are required.
    • msteams: notifications via Microsoft teams. Insert the webhook url you want to use. More info on configuring Teams webhooks here.
    • slack: notifications via Slack. Insert the webhook url you want to use. More info on configuring Slack webhooks here
  • alarms settings for which alarms RedELK should perform.
    • alarm_dummy: only used for testing purposes, probably no need to enable.
    • alarm_filehash: alarms SHA/MD5 hashes of your uploaded files that are also found on VirusTotal, IBM X-Force and/or Hybrid Analyses. Requires API key per provider. If you leave the API key empty the check is not performed.
    • alarm_httptraffic: alarms IP's that aren't listed in any iplist* but access redirector backends named c2*.
    • alarm_useragent: alarms User-Agents that are listed in config file blacklist_useragents.conf but access redirector backends named c2*.
    • alarm_backendalarm: alarms any traffic hitting a redirector backend named *alarm*.
  • enrich settings for data enrichment. You can keep these enabled even if you don't use a specific item.
    • enrich_csbeacon: enriches rtops data from Cobalt Strike implants.
    • enrich_stage1: enriches rtops data from Outflank's custom C2 framework.
    • enrich_greynoise: enriches redirtraffic data with info from Greynoise. If an IP address is listed in Greynoise, this data is added. You can enter your own API key to prevent you from hitting rate limits from a public API key.
    • enrich_tor: enriches redirtraffic with Tor. If an IP address is a known Tor exit node, this info is added.
    • enrich_iplists: background RedELK process. Better keep ik enabled.
    • enrich_synciplists: background RedELK process. Better keep ik enabled.
  1. Run: install-elkserver.sh. Optional parameters:
  • dryrun: only do pre install checks and write config to .env file. Its good practice to run this parameter the first time.
  • fixedmemory: skips the auto memory adjustment and sets memory for elasticsearch and neo4j to 1GB
  • limited: do not install Neo4j and Jupyter notebooks
  • dev: only used for development (rebuilds all docker containers and inserts some test logs)

This script will install docker images of logstash, elasticsearch, kibana, neo4j, jupyter notebook and certbot. The local .env. file contains all docker specific parameters

Configuration

You are not done yet. You need to manually enter the details of your C2 servers in mounts/redelk-config/etc/cron.d/redelk, as well as tune the config files in mounts/redelk-config/etc/redelk/:

  • iplist_redteam.conf: public IP addresses of your red team, one per line. Convenient for identifying testing done by red team members. Including an address here will set a tag for applicable records in the redirhaproxy-* index.
  • iplist_customer.conf: public IP addresses of your target, one per line. Including an address here will set a tag for applicable records in the redirhaproxy-* index.
  • iplist_unknown.conf: public IP addresses of gateways that you are not sure about yet, but don't want to be warned about again. One per line. Including an address here will set a tag for applicable records in the redirhaproxy-* index.
  • rogue_useragents.conf: User agents that are known bad when they access your C2 backend. We have included a basic list of UAs like curl, python-urllib and some other tools blue teamers like to use. The list also contains a list of UAs of instant messaging tools such as WhatsApp, Skype and Slack. Very useful for when your C2 us shared amongst analysts using IM. Feel free to add UAs to this list.
  • known_testsystems.conf: beacon characteristics of known test systems. You probably want to add info regarding your own test systems. One per line. Including data here here will set a tag for applicable records in the rtops-* index.
  • iplist_alarmed.conf: one IP per line that you don't want to be alarmed about.

Other config files not recommended to edit:

  • abusebotnetc2ip.conf: auto-updated list of known C2 IP addresses as listed by abuse.ch
  • abusesslcert.conf: auto-updated list of known hashes of C2 TLS certs C2 as listed by abuse.ch
  • roguedomains.conf: auto-updated list of known bad domains, from multiple sources.
  • torexitnodes.conf: auto-updated list of known TOR exit node IP addresses.
  • known_sandboxes.conf: beacon characteristics of known AV sandbox systems. One per line. Including data here here will set a tag for applicable records in the rtops-* index. This is not really relevant anymore as more and more sandboxes have fully random names.
  • redteamdomains.conf : list of domains as used by your offensive infrastructure. As there is no alarm coded yet to check this, there is no need to edit this file.

API and rate limit info third party security providers:

provider Rate limit API supports bundling requests Links with info
OTX 10k/hour with api key No bundling https://otx.alienvault.com/api, https://cybersecurity.att.com/blogs/security-essentials/the-upgraded-alienvault-otx-api-ways-to-score-swag
IBM free tier 5k/month and 3000/minute No bundling https://exchange.xforce.ibmcloud.com/faq#tiers_of_usage, https://api.xforce.ibmcloud.com/doc/#/Malware/get_malware__filehash_
IBM paid goes in batches of 10k/month, prolly about $2k for $10k calls No Bundling
VT free 4/minute and 500/day No bundling https://developers.virustotal.com/reference/public-vs-premium-api
VT commercial no limit No bundling
Hybrid Analyses unclear: either 2000/hour with no daily/weekly/monthly limit, or 5/minute and 200/hour Bundling supported https://www.hybrid-analysis.com/docs/api/v2#/Search/post_search_hash
Facebook Unclear 200 per hour No bundling (probably) https://developers.facebook.com/docs/graph-api/overview/rate-limiting, https://developers.facebook.com/docs/threat-exchange/reference/apis/threat-indicators/v15.0
Clone this wiki locally