running amundsen in ec2 results in error

[javamonkey79]

1. Create an ec2 instance (I am working with t2.xlarge)
2. switch to root (sudo su)
3. run the following commands

yum update -y && \
yum install -y docker && \
yum install -y git && \
service docker start && \
sudo usermod -aG docker ec2-user && \
mkdir /amundsen && \
cd /amundsen && \
git clone https://github.com/lyft/amundsenfrontendlibrary && \
git clone https://github.com/lyft/amundsenmetadatalibrary && \
git clone https://github.com/lyft/amundsensearchlibrary && \
cd /amundsen/amundsenmetadatalibrary/ && \
docker build -f public.Dockerfile -t amundsendev/amundsen-metadata:latest . && \
cd /amundsen/amundsenfrontendlibrary/ && \
docker build -f public.Dockerfile -t amundsendev/amundsen-frontend:latest . && \
cd /amundsen/amundsensearchlibrary/ && \
docker build -f public.Dockerfile -t amundsendev/amundsen-search:latest . && \
cd /amundsen && \
curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose && \
ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose && \
docker-compose -f amundsenfrontendlibrary/docker-amundsen.yml up

Note the following errors:

es_amundsen         | [1]: max file descriptors [40000] for elasticsearch process is too low, increase to at least [65535]
es_amundsen         | [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

[javamonkey79]

Updating the ulimit in the docker-compose file still gives this:

es_amundsen         | ERROR: [1] bootstrap checks failed
es_amundsen         | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
es_amundsen         | [2019-06-01T00:07:09,462][INFO ][o.e.n.Node               ] [IhZCZcZ] stopping ...
es_amundsen         | [2019-06-01T00:07:09,493][INFO ][o.e.n.Node               ] [IhZCZcZ] stopped
es_amundsen         | [2019-06-01T00:07:09,493][INFO ][o.e.n.Node               ] [IhZCZcZ] closing ...
es_amundsen         | [2019-06-01T00:07:09,520][INFO ][o.e.n.Node               ] [IhZCZcZ] closed
es_amundsen         | [2019-06-01T00:07:09,523][INFO ][o.e.x.m.p.NativeController] [IhZCZcZ] Native controller process has stopped - no new native processes can be started
es_amundsen exited with code 78

[javamonkey79]

This is related to docker/compose#4143

Basically, you need to run this on your host system (ec2) first:
sudo sysctl -w vm.max_map_count=262144

I fixed the file descriptor issue by modifying my docker compose file:

  elasticsearch:
      image: elasticsearch:6.7.0
      container_name: es_amundsen
      ports:
          - 9200:9200
      networks:
        - amundsennet
      ulimits:
        nofile:
          soft: 1000000
          hard: 1000000

[Jwan622]

Does the docker compose yaml file not exist anymore?

[feng-tao]

@Jwan622 it is at https://github.com/lyft/amundsen/blob/master/docker-amundsen.yml