Default docker image fails on Centos7 docker 19.03.3

[rpn0709]

14:41:11.846Z INFO elastalert-server: Server: Server started
14:41:13.489Z ERROR elastalert-server:
ProcessController: INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
To send them but remain verbose, use --verbose instead.

14:41:13.490Z ERROR elastalert-server:
ProcessController: INFO:elastalert:Starting up

14:41:13.528Z ERROR elastalert-server:
ProcessController: Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main

14:41:13.528Z ERROR elastalert-server:
ProcessController: "main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/opt/elastalert/elastalert/elastalert.py", line 1929, in

14:41:13.531Z ERROR elastalert-server:
ProcessController: sys.exit(main(sys.argv[1:]))
File "/opt/elastalert/elastalert/elastalert.py", line 1925, in main

14:41:13.531Z ERROR elastalert-server:
ProcessController: client.start()
File "/opt/elastalert/elastalert/elastalert.py", line 1106, in start

14:41:13.534Z ERROR elastalert-server:
ProcessController: self.run_all_rules()
File "/opt/elastalert/elastalert/elastalert.py", line 1158, in run_all_rules
self.send_pending_alerts()
File "/opt/elastalert/elastalert/elastalert.py", line 1534, in send_pending_alerts

14:41:13.536Z ERROR elastalert-server:
ProcessController: pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit)
File "/opt/elastalert/elastalert/elastalert.py", line 1526, in find_recent_pending_alerts

14:41:13.536Z ERROR elastalert-server:
ProcessController: size=1000)
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
return func(*args, params=params, **kwargs)
TypeError: search() got an unexpected keyword argument 'doc_type'

14:41:13.595Z ERROR elastalert-server: ProcessController: ElastAlert exited with code 1

[SysEngDan]

Also experiencing the same error with bitsensor/elastalert:latest (2.0.1). This has been an issue for a few weeks now (Yelp/elastalert#2204).

Why isn't this fixed by now? How is a new user like myself supposed to use ElastAlert with ES 7.4 with this issue? Is there a workaround?

[SlavikCA]

@SysEngDan I got it working by replacingbitsensor/elastalert:latest with bitsensor/elastalert:3.0.0-beta.1

Also, you'll have to delete old indices. Something like that:

curl -XDELETE 'http://elk:9200/elastalert_status' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_silence' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_error' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_status' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_past' -u elastic:password

[SolomonShorser-OICR]

My solution was the same as @SlavikCA though now I get different errors when testing the rules:

elastalert_1       |     Traceback (most recent call last):
elastalert_1       |       File "/opt/elastalert/elastalert/elastalert.py", line 1578, in writeback
elastalert_1       |         res = self.writeback_es.index(index=index, body=body)
elastalert_1       |       File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
elastalert_1       |         return func(*args, params=params, **kwargs)
elastalert_1       |       File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/client/__init__.py", line 364, in index
elastalert_1       |         "POST", _make_path(index, doc_type, id), params=params, body=body
elastalert_1       |       File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/transport.py", line 353, in perform_request
elastalert_1       |         timeout=timeout,
elastalert_1       |       File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/connection/http_requests.py", line 143, in perform_request
elastalert_1       |         self._raise_error(response.status_code, raw_data)
elastalert_1       |       File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/connection/base.py", line 168, in _raise_error
elastalert_1       |         status_code, error_message, additional_info
elastalert_1       |     RequestError: RequestError(400, u'illegal_argument_exception', u'Rejecting mapping update to [elastalert_status_status] as the final mapping would have more than 1 type: [_doc, elastalert_status]')
elastalert_1       |     

To be fair, I was testing with the Kibana plugin. Haven't tried from the command line, not sure I'll have time.

[SysEngDan]

@SysEngDan I got it working by replacingbitsensor/elastalert:latest with bitsensor/elastalert:3.0.0-beta.1

Also, you'll have to delete old indices. Something like that:

curl -XDELETE 'http://elk:9200/elastalert_status' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_silence' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_error' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_status' -u elastic:password
curl -XDELETE 'http://elk:9200/elastalert_status_past' -u elastic:password

Thank you. This worked for me.

[FedirYatseta]

Hello ! I have problem with running elastalert server.
my configuration
centos 7
elasticsearch 7.4.2
kibana 7.4.2
nginx
nginx config
`server {
listen 80;

server_name _;

location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-NginX-Proxy true;
    proxy_pass http://192.168.1.15:5601;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
    proxy_redirect off;
}

}
`
elastalert-kibana-plugin

I started docker
docker run -d -p 3030:3030 \ -v pwd/config/elastalert.yaml:/opt/elastalert/config.yaml \ -v pwd/config/config.json:/opt/elastalert-server/config/config.json \ -v pwd/rules:/opt/elastalert/rules \ -v pwd/rule_templates:/opt/elastalert/rule_templates \ --net="host" \ --name elastalert bitsensor/elastalert:3.0.0-beta.1

and after that i got running container "bitsensor/elastalert:3.0.0-beta.1" I decided stopped it and run after that I got next result

`[root@elastic elastalert]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
bitsensor/elastalert 3.0.0-beta.1 e4cff875f215 3 months ago 271MB
[root@elastic elastalert]# docker run e4cff875f215

@bitsensor/elastalert@3.0.0-beta.0 start /opt/elastalert-server
sh ./scripts/start.sh

09:18:08.004Z INFO elastalert-server: Config: No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
09:18:08.005Z INFO elastalert-server: Config: Proceeding to look for normal config file.
09:18:08.005Z INFO elastalert-server: Config: A config file was found in /opt/elastalert-server/config/config.json. Using that config.
09:18:08.009Z INFO elastalert-server: Router: Listening for GET request on /.
09:18:08.009Z INFO elastalert-server: Router: Listening for GET request on /status.
09:18:08.009Z INFO elastalert-server: Router: Listening for GET request on /status/control/:action.
09:18:08.009Z INFO elastalert-server: Router: Listening for GET request on /status/errors.
09:18:08.009Z INFO elastalert-server: Router: Listening for GET request on /rules.
09:18:08.010Z INFO elastalert-server: Router: Listening for GET request on /rules/:id.
09:18:08.010Z INFO elastalert-server: Router: Listening for POST request on /rules/:id.
09:18:08.010Z INFO elastalert-server: Router: Listening for DELETE request on /rules/:id.
09:18:08.010Z INFO elastalert-server: Router: Listening for GET request on /templates.
09:18:08.010Z INFO elastalert-server: Router: Listening for GET request on /templates/:id.
09:18:08.010Z INFO elastalert-server: Router: Listening for POST request on /templates/:id.
09:18:08.010Z INFO elastalert-server: Router: Listening for DELETE request on /templates/:id.
09:18:08.010Z INFO elastalert-server: Router: Listening for POST request on /test.
09:18:08.010Z INFO elastalert-server: Router: Listening for GET request on /config.
09:18:08.010Z INFO elastalert-server: Router: Listening for POST request on /config.
09:18:08.010Z INFO elastalert-server: Router: Listening for POST request on /download.
09:18:08.010Z INFO elastalert-server: Router: Listening for GET request on /metadata/:type.
09:18:08.010Z INFO elastalert-server: Router: Listening for GET request on /mapping/:index.
09:18:08.010Z INFO elastalert-server: Router: Listening for POST request on /search/:index.
09:18:08.013Z INFO elastalert-server: ProcessController: Starting ElastAlert
09:18:08.013Z INFO elastalert-server: ProcessController: Creating index
09:18:19.727Z ERROR elastalert-server:
ProcessController: Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/opt/elastalert/elastalert/create_index.py", line 262, in
main()
File "/opt/elastalert/elastalert/create_index.py", line 258, in main
create_index_mappings(es_client=es, ea_index=index, recreate=args.recreate, old_ea_index=old_index)
File "/opt/elastalert/elastalert/create_index.py", line 24, in create_index_mappings
esversion = es_client.info()["version"]["number"]
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/init.py", line 259, in info
return self.transport.perform_request("GET", "/", params=params)
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/transport.py", line 353, in perform_request
timeout=timeout,
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 139, in perform_request
raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f93a5e9fbd0>: Failed to establish a new connection: [Errno 111] Connection refused',))) caused by: ConnectionError(HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f93a5e9fbd0>: Failed to establish a new connection: [Errno 111] Connection refused',)))

09:18:19.728Z ERROR elastalert-server: ProcessController: Index create exited with code 1
09:18:19.728Z WARN elastalert-server: ProcessController: ElastAlert will start but might not be able to save its data!
09:18:19.729Z INFO elastalert-server: ProcessController: Starting elastalert with arguments [none]
09:18:19.740Z INFO elastalert-server: ProcessController: Started Elastalert (PID: 36)
09:18:19.741Z INFO elastalert-server: Server: Server listening on port 3030
09:18:19.742Z INFO elastalert-server: Server: Websocket listening on port 3333
09:18:19.742Z INFO elastalert-server: Server: Server started
09:18:20.638Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:GET http://localhost:9200/ [status:N/A request:0.002s]
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 123, in perform_request
response = self.session.send(prepared_request, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e69a85f90>: Failed to establish a new connection: [Errno 111] Connection refused',))

09:18:21.642Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:GET http://localhost:9200/ [status:N/A request:0.004s]
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 123, in perform_request
response = self.session.send(prepared_request, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e69912390>: Failed to establish a new connection: [Errno 111] Connection refused',))

09:18:24.646Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:GET http://localhost:9200/ [status:N/A request:0.001s]
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 123, in perform_request
response = self.session.send(prepared_request, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e69912490>: Failed to establish a new connection: [Errno 111] Connection refused',))

09:18:31.658Z ERROR elastalert-server:
ProcessController: WARNING:elasticsearch:GET http://localhost:9200/ [status:N/A request:0.004s]
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 123, in perform_request
response = self.session.send(prepared_request, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests-2.22.0-py2.7.egg/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e699122d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

09:18:31.658Z ERROR elastalert-server:
ProcessController: Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/opt/elastalert/elastalert/elastalert.py", line 1942, in

09:18:31.660Z ERROR elastalert-server:
ProcessController: sys.exit(main(sys.argv[1:]))
File "/opt/elastalert/elastalert/elastalert.py", line 1938, in main

09:18:31.661Z ERROR elastalert-server:
ProcessController: client.start()
File "/opt/elastalert/elastalert/elastalert.py", line 1102, in start

09:18:31.661Z ERROR elastalert-server:
ProcessController: self.run_all_rules()
File "/opt/elastalert/elastalert/elastalert.py", line 1158, in run_all_rules

09:18:31.662Z ERROR elastalert-server:
ProcessController: self.send_pending_alerts()
File "/opt/elastalert/elastalert/elastalert.py", line 1539, in send_pending_alerts

09:18:31.662Z ERROR elastalert-server:
ProcessController: pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit)
File "/opt/elastalert/elastalert/elastalert.py", line 1521, in find_recent_pending_alerts

09:18:31.662Z ERROR elastalert-server:
ProcessController: if self.writeback_es.is_atleastfive():
File "elastalert/init.py", line 50, in is_atleastfive
return int(self.es_version.split(".")[0]) >= 5

09:18:31.662Z ERROR elastalert-server:
ProcessController: File "elastalert/init.py", line 43, in es_version
self._es_version = self.info()['version']['number']
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/init.py", line 259, in info

09:18:31.662Z ERROR elastalert-server:
ProcessController: return self.transport.perform_request("GET", "/", params=params)
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/transport.py", line 353, in perform_request

09:18:31.663Z ERROR elastalert-server:
ProcessController: timeout=timeout,
File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 139, in perform_request
raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e699122d0>: Failed to establish a new connection: [Errno 111] Connection refused',))) caused by: ConnectionError(HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e699122d0>: Failed to establish a new connection: [Errno 111] Connection refused',)))

09:18:31.688Z ERROR elastalert-server: ProcessController: ElastAlert exited with code 1
09:18:31.688Z INFO elastalert-server: Server: Stopping server
09:18:31.688Z INFO elastalert-server: ProcessController: ElastAlert is not running
09:18:31.689Z INFO elastalert-server: Server: Server stopped. Bye!`

[nsano-rururu]

Share how to get it working

Elasticsearch 7.4.2
Kibana 7.4.2
kibana-elastalert-plugin
bitsensor/elastalert:3.0.0-beta.0

/home/user/docker-wk
|--docker-compose.yml
|--Dockerfiles
|  |--Dockerfile-elastalert
|  |--Dockerfile-kibana
|
|--es
|  |--config
|  |  |--elasticsearch.yml
|  |--data
|
|--kibana
|  |--config
|  |  |--kibana.yml
|
|--elastalert
|  |--bin
|  |  |--elastalert-start.sh
|  |  |--elastic_search_status.sh
|  |--config
|  |  |--config.json
|  |  |--elastalert-test.yaml
|  |  |--elastalert.yaml
|  |--rule_templates
|  |--rules

docker-compose.yml

version: "3.7"
services:
  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:7.4.2
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - ES_JAVA_OPTS=-Xms128m -Xmx256m
      - discovery.type=single-node
    restart: always
    volumes:
      - ./es/data:/usr/share/elasticsearch/data
      - ./es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:9200 || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 180s

  kibana:
    container_name: kibana
    build:
      context: .
      dockerfile: Dockerfiles/Dockerfile-kibana
    image: kibana:7.4.2
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch
    restart: always
    volumes:
      - ./kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:5601/api/status || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 200s

  elastalert:
    container_name: elastalert
    build:
      context: .
      dockerfile: Dockerfiles/Dockerfile-elastalert
    image: elastalert:3.0.0-beta.0
    ports:
      - 3030:3030
      - 3333:3333
    depends_on:
      - elasticsearch
      - kibana
    restart: always
    volumes:
      - ./elastalert/config/elastalert.yaml:/opt/elastalert/config.yaml
      - ./elastalert/config/elastalert-test.yaml:/opt/elastalert/config-test.yaml
      - ./elastalert/config/config.json:/opt/elastalert-server/config/config.json
      - ./elastalert/rules:/opt/elastalert/rules
      - ./elastalert/rule_templates:/opt/elastalert/rule_templates
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:3030 || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 200s

es/config/elasticsearch.yml

cluster.name: "docker-cluster"
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1

kibana/config/kibana.yml

server.name: kibana
server.host: "0"
elasticsearch.hosts: http://elasticsearch:9200
xpack.monitoring.ui.container.elasticsearch.enabled: true

# elastalert-kibana-plugin
elastalert-kibana-plugin.serverHost: elastalert
elastalert-kibana-plugin.serverPort: 3030

Dockerfiles/Dockerfile-kibana

FROM docker.elastic.co/kibana/kibana:7.4.2

USER root


RUN /usr/share/kibana/bin/kibana-plugin install https://github.com/bitsensor/elastalert-kibana-plugin/releases/download/1.1.0/elastalert-kibana-plugin-1.1.0-7.4.2.zip --allow-root

USER kibana

Dockerfiles/Dockerfile-elastalert

FROM bitsensor/elastalert:3.0.0-beta.0

USER root

RUN apk update && \
    apk add bash curl && \
    rm -rf /var/cache/apk/*

ADD elastalert/bin/elastalert-start.sh /usr/local/bin/
ADD elastalert/bin/elastic_search_status.sh /usr/local/bin/

RUN chmod +x /usr/local/bin/elastalert-start.sh 
RUN chmod +x /usr/local/bin/elastic_search_status.sh

USER node

ENTRYPOINT ["/usr/local/bin/elastalert-start.sh"]

elastalert/bin/elastic_search_status.sh

#!/bin/bash

set -e

if [ $# -gt 0 ]; then
  ES_URL="$1"
elif [[ -n $ELASTICSEARCH_URL ]]; then
  ES_URL="$ELASTICSEARCH_URL"
elif [[ -n $ES_HOST ]] && [[ -n $ES_PORT ]]; then
  ES_URL="http://$ES_HOST:$ES_PORT"
else
  ES_URL="http://elasticsearch:9200"
fi

until [[ "$(curl -fsSL "$ES_URL/_cat/health?h=status" | sed -r 's/^[[:space:]]+|[[:space:]]+$//g')" =~ ^(yellow|green)$ ]]; do
  # printf '+' >&2
  sleep 1
done

echo "Elasticsearch is up and healthy at "$ES_URL"" >&2

elastalert/bin/elastalert-start.sh

#!/bin/bash

set -e

echo "Giving Elasticsearch at $ELASTICSEARCH_URL time to start..."

elastic_search_status.sh

echo "Starting ElastAlert!"
npm start

elastalert/config/config.json

{
  "appName": "elastalert-server",
  "port": 3030,
  "wsport": 3333,
  "elastalertPath": "/opt/elastalert",
  "verbose": false,
  "es_debug": false,
  "debug": false,
  "rulesPath": {
    "relative": true,
    "path": "/rules"
  },
  "templatesPath": {
    "relative": true,
    "path": "/rule_templates"
  },
  "es_host": "elasticsearch",
  "es_port": 9200,
  "writeback_index": "elastalert_status"
}

elastalert/config/elastalert-test.yml

# NOTE: This config is used when testing a rule

# The elasticsearch hostname for metadata writeback
# Note that every rule can have its own elasticsearch host
es_host: elasticsearch

# The elasticsearch port
es_port: 9200

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: rules

# How often ElastAlert will query elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  seconds: 5

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 1

# Optional URL prefix for elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to elasticsearch
#use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for elasticsearch
#es_username: someusername
#es_password: somepassword

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

elastalert/config/elastalert.yml

# The elasticsearch hostname for metadata writeback
# Note that every rule can have its own elasticsearch host
es_host: elasticsearch

# The elasticsearch port
es_port: 9200

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: rules

# How often ElastAlert will query elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  seconds: 5

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 1

# Optional URL prefix for elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to elasticsearch
#use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for elasticsearch
#es_username: someusername
#es_password: somepassword

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

chmod 777 es/data
chmod 777 elastalert/rules
chmod 777 elastalert/rule_templates
docker-compose up -d

[CORP\sano@a-ngft53r34ong docker-wk]$ docker logs elastalert -f
Giving Elasticsearch at  time to start...
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
Elasticsearch is up and healthy at http://elasticsearch:9200
Starting ElastAlert!

> @bitsensor/elastalert@3.0.0-beta.0 start /opt/elastalert-server
> sh ./scripts/start.sh

14:45:00.311Z  INFO elastalert-server: Config:  No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
14:45:00.320Z  INFO elastalert-server: Config:  Proceeding to look for normal config file.
14:45:00.370Z  INFO elastalert-server: Config:  A config file was found in /opt/elastalert-server/config/config.json. Using that config.
14:45:00.431Z  INFO elastalert-server: Router:  Listening for GET request on /.
14:45:00.432Z  INFO elastalert-server: Router:  Listening for GET request on /status.
14:45:00.433Z  INFO elastalert-server: Router:  Listening for GET request on /status/control/:action.
14:45:00.434Z  INFO elastalert-server: Router:  Listening for GET request on /status/errors.
14:45:00.434Z  INFO elastalert-server: Router:  Listening for GET request on /rules.
14:45:00.439Z  INFO elastalert-server: Router:  Listening for GET request on /rules/:id.
14:45:00.440Z  INFO elastalert-server: Router:  Listening for POST request on /rules/:id.
14:45:00.441Z  INFO elastalert-server: Router:  Listening for DELETE request on /rules/:id.
14:45:00.442Z  INFO elastalert-server: Router:  Listening for GET request on /templates.
14:45:00.442Z  INFO elastalert-server: Router:  Listening for GET request on /templates/:id.
14:45:00.443Z  INFO elastalert-server: Router:  Listening for POST request on /templates/:id.
14:45:00.444Z  INFO elastalert-server: Router:  Listening for DELETE request on /templates/:id.
14:45:00.445Z  INFO elastalert-server: Router:  Listening for POST request on /test.
14:45:00.446Z  INFO elastalert-server: Router:  Listening for GET request on /config.
14:45:00.446Z  INFO elastalert-server: Router:  Listening for POST request on /config.
14:45:00.447Z  INFO elastalert-server: Router:  Listening for POST request on /download.
14:45:00.448Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/:type.
14:45:00.449Z  INFO elastalert-server: Router:  Listening for GET request on /mapping/:index.
14:45:00.449Z  INFO elastalert-server: Router:  Listening for POST request on /search/:index.
14:45:00.454Z  INFO elastalert-server: ProcessController:  Starting ElastAlert
14:45:00.454Z  INFO elastalert-server: ProcessController:  Creating index
14:45:06.677Z  INFO elastalert-server:
    ProcessController:  Elastic Version: 7.4.2
    Reading Elastic 6 index mappings:
    Reading index mapping 'es_mappings/6/silence.json'
    Reading index mapping 'es_mappings/6/elastalert_status.json'
    Reading index mapping 'es_mappings/6/elastalert.json'
    Reading index mapping 'es_mappings/6/past_elastalert.json'
    Reading index mapping 'es_mappings/6/elastalert_error.json'
    New index elastalert_status created
    Done!
    
14:45:06.677Z  INFO elastalert-server: ProcessController:  Index create exited with code 0
14:45:06.677Z  INFO elastalert-server: ProcessController:  Starting elastalert with arguments [none]
14:45:06.689Z  INFO elastalert-server: ProcessController:  Started Elastalert (PID: 215)
14:45:06.694Z  INFO elastalert-server: Server:  Server listening on port 3030
14:45:06.695Z  INFO elastalert-server: Server:  Websocket listening on port 3333
14:45:06.696Z  INFO elastalert-server: Server:  Server started
14:45:08.640Z  INFO elastalert-server:
    ProcessController:  0 rules loaded
    
14:45:24.774Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:45:55.364Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:46:25.876Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:46:56.393Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:47:26.901Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:47:57.316Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:48:27.759Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
^C
[CORP\sano@a-ngft53r34ong docker-wk]$ docker ps

CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS                   PORTS                                            NAMES
f7b1f2624b4f        elastalert:3.0.0-beta.0                               "/usr/local/bin/elas…"   4 minutes ago       Up 4 minutes (healthy)   0.0.0.0:3030->3030/tcp, 0.0.0.0:3333->3333/tcp   elastalert
5959a6777f42        kibana:7.4.2                                          "/usr/local/bin/dumb…"   4 minutes ago       Up 4 minutes (healthy)   0.0.0.0:5601->5601/tcp                           kibana
112613b8f089        docker.elastic.co/elasticsearch/elasticsearch:7.4.2   "/usr/local/bin/dock…"   4 minutes ago       Up 4 minutes (healthy)   0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   elasticsearch

[RanaBhagathChand]

Hi @nsano-rururu,

I have followed the above folder structure and files for the elastalert server using docker.
It worked fine without ElasticSearch authentication.

But after setting up authentication for ElastiSearch and Kibana (username: elastic, password: SomePassword) when we run docker-compose up it is throwing unauthorized exceptions.

We have added the username and password to the ./config/config.json, ./config/elastalert-test.yml, ./config/elastalert.yml and mentioned in environment variables in docker-compose.yml file.

After doing so, if we run docker-compose up it is still throwing the same error( unauthorized exceptions).

Can you please help us in resolving this issue?

[nsano-rururu]

@RanaBhagathChand

bitsensor's elastalert-server does not support es_username and es_password on the api side of node.js. Maybe that's the cause of the error. The fork johnsusek / elastalert-server corresponds to es_username and es_password on the api side of node.js. See the URL below for the docker image.
https://hub.docker.com/r/praecoapp/elastalert-server

Refer to the following URL for settings
https://github.com/johnsusek/elastalert-server#configuration

[nsano-rururu]

@RanaBhagathChand

Another possible cause. This may be more likely

The following files may not work as they are.

elastalert/bin/elastic_search_status.sh

The following files may not work properly without modifying the following description.

docker-compose.yml

test: ["CMD-SHELL", "curl -f http://localhost:9200 || exit 1"]

[nsano-rururu]

@RanaBhagathChand

If you can give me information about the settings you want to change, such as elasticsearch, kibana, docker-compose.yml, I'll investigate, but if not, I won't do anything.