Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting up APM question #368

Closed
duongpc opened this issue May 11, 2022 · 4 comments
Closed

Setting up APM question #368

duongpc opened this issue May 11, 2022 · 4 comments

Comments

@duongpc
Copy link

duongpc commented May 11, 2022

Hello,

I followed the instruction (http://localhost:5601/app/home#/tutorial/apm) to install APM server

curl -L -O https://artifacts.elastic.co/downloads/apm-server/apm-server-8.1.0-amd64.deb
dpkg -i apm-server-8.1.0-amd64.deb

After that I run

service apm-server start
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.663Z","log.origin":{"file.name":"instance/beat.go","file.line":670},"message":"Home path: [/usr/share/apm-server] Config path: [/etc/apm-server] Data path: [/var/lib/apm-server] Logs path: [/var/log/apm-server]","service.name":"apm-server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.663Z","log.origin":{"file.name":"instance/beat.go","file.line":678},"message":"Beat ID: e43de2b9-5f3d-4fca-bb66-4b490b775333","service.name":"apm-server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.663Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1048},"message":"Beat info","service.name":"apm-server","system_info":{"beat":{"path":{"config":"/etc/apm-server","data":"/var/lib/apm-server","home":"/usr/share/apm-server","logs":"/var/log/apm-server"},"type":"apm-server","uuid":"e43de2b9-5f3d-4fca-bb66-4b490b775333"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.663Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1057},"message":"Build info","service.name":"apm-server","system_info":{"build":{"commit":"7d34f3e579c7fadc5e70df76ed9b39332a40d0de","libbeat":"8.1.0","time":"2022-03-03T14:34:24.000Z","version":"8.1.0"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.663Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1060},"message":"Go runtime info","service.name":"apm-server","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":12,"version":"go1.17.6"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.664Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1064},"message":"Host info","service.name":"apm-server","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-05-11T00:02:22Z","containerized":true,"name":"c15db73e9a4a","ip":["127.0.0.1/8","172.17.0.2/16"],"kernel_version":"5.10.16.3-microsoft-standard-WSL2","mac":["02:42:ac:11:00:02"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.3 LTS (Focal Fossa)","major":20,"minor":4,"patch":3,"codename":"focal"},"timezone":"UTC","timezone_offset_sec":0,"id":"e4aa44123ef6412c8a2ef09168dec44b"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.664Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1093},"message":"Process info","service.name":"apm-server","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null},"cwd":"/","exe":"/usr/share/apm-server/bin/apm-server","name":"apm-server","pid":1159,"ppid":1158,"seccomp":{"mode":"filter","no_new_privs":false},"start_time":"2022-05-11T01:58:52.190Z"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.664Z","log.origin":{"file.name":"instance/beat.go","file.line":323},"message":"Setup Beat: apm-server; Version: 8.1.0","service.name":"apm-server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.665Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: http://localhost:9200","service.name":"apm-server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-11T01:58:52.665Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: c15db73e9a4a","service.name":"apm-server","ecs.version":"1.6.0"}
Config OK

Checking the server status it showed server was not available
image

Checking /var/log/apm-server I saw this error

{"log.level":"error","@timestamp":"2022-05-11T01:59:21.384Z","log.logger":"beater","log.origin":{"file.name":"beater/waitready.go","file.line":62},"message":"precondition 'apm integration installed' failed: error querying Elasticsearch for integration index templates: unexpected HTTP status: 404 Not Found ({\"error\":{\"root_cause\":[{\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [logs-apm.error] not found\"}],\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [logs-apm.error] not found\"},\"status\":404}): to remediate, please install the apm integration: https://ela.st/apm-integration-quickstart","service.name":"apm-server","ecs.version":"1.6.0"}

I didn't update apm-server.yml since I didn't know the what is elasticsearch's username/password - Is this the issue? If so is there any instruction on how to add an user for elasticsearch? Thank you

@spujadas
Copy link
Owner

Generally speaking I won’t be able to help you on how to set up APM, and would recommend heading over to the Elastic forums for guidance.

Having said that, going to the URL that’s referenced in the error in your APM logs (https://www.elastic.co/guide/en/apm/guide/current/apm-quick-start.html), there are several prerequisites on the set-up of the ELK stack, including having a "Secure, encrypted connection between Kibana and Elasticsearch."

As the documentation of the image says, the services are not set up with security enabled, so you’ll have to update the configuration of the services to do so (which can be a bit fiddly, hence this not being the case by default in the image).

Going back to the APM documentation, in the Prerequisites section, Self-managed tab, there are instructions to quickly secure Elasticsearch and Kibana for testing purposes, so that would be my recommendation to move forward with your set-up.

@duongpc
Copy link
Author

duongpc commented May 16, 2022

Apparently ELK 8+ requires fleet setup to use APM, I was under assumption that I could get away from it by setting up APM legacy server, turns out fleet is mandatory...

I decided to downgrade to build 7.17.1

FROM sebp/elk:7.17.1

RUN curl -L -O https://artifacts.elastic.co/downloads/apm-server/apm-server-7.17.1-amd64.deb
RUN dpkg -i apm-server-7.17.1-amd64.deb

EXPOSE 8200

I was able to get kibana to recognize the APM server, however, from windows's command prompt, if I run:

curl localhost:8200
curl: (52) Empty reply from server

but inside the container, I got some result

# curl localhost:8200
{
  "build_date": "2022-02-23T22:31:58Z",
  "build_sha": "990c162bf1d5106be95d1430c6f3e98df865df7d",
  "publish_ready": true,
  "version": "7.17.1"
}

I can't quite figure out what missing, does the base image require any additional setup to allow access to port 8200?

@spujadas
Copy link
Owner

spujadas commented May 16, 2022

Binding the exposed port to a port on the host machine should do the trick (e.g. see https://github.com/spujadas/elk-docker/blob/master/docker-compose.yml for ports 5601, 9200, 5044).

@duongpc
Copy link
Author

duongpc commented May 19, 2022

So by default apm-server.yml is using localhost:8200, changing it to either 0.0.0.0:8200 or :8200 will fix the connection issue

@duongpc duongpc closed this as completed May 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants