CloudwatchAgent on executors

[kayman-mk]

Discussed in #347

^{Originally posted by npeyrussie July 21, 2021}
Hello,

I am looking for a way to install CloudwatchAgent on the executors.
I seems that userdata_pre_install and userdata_post_install only apply to the runner and not to the executors.

What type of detailed monitoring metrics are added thanks to runner_instance_enable_monitoring option ?

I need to retrieve memory metrics in cloudwatch and need to have cloudwatch agent installed on executors.
Is there a way to achieve this ?

[kayman-mk]

Seems to be a valid use case to me.

@npeyrussie Any chance to implement it here?

[npeyrussie]

Hello,

Using version 4.24.1 of npalm/gitlab-runner/aws, I succeeded in configuring cloudwatch agent using :

docker_machine_iam_policy_arns = ["arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"]

And a dirty post install script as is :

userdata_post_install = <<EOT
cat <<EOF > /root/runner_init.sh
#!/bin/sh
wget https://s3.eu-west-3.amazonaws.com/amazoncloudwatch-agent-eu-west-3/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
dpkg -i -E ./amazon-cloudwatch-agent.deb
mkdir -p /usr/share/collectd
touch /usr/share/collectd/types.db
cat <<'EOS' > /opt/aws/amazon-cloudwatch-agent/bin/config.json
{
        "agent": {
                "metrics_collection_interval": 60,
                "run_as_user": "root"
        },
        "metrics": {
                "append_dimensions": {
                        "AutoScalingGroupName": "DOLLAR{aws:AutoScalingGroupName}",
                        "ImageId": "DOLLAR{aws:ImageId}",
                        "InstanceId": "DOLLAR{aws:InstanceId}",
                        "InstanceType": "DOLLAR{aws:InstanceType}"
                },
                "metrics_collected": {
                        "collectd": {
                                "metrics_aggregation_interval": 60
                        },
                        "disk": {
                                "measurement": [
                                        "used_percent"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "mem": {
                                "measurement": [
                                        "mem_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        },
                        "statsd": {
                                "metrics_aggregation_interval": 60,
                                "metrics_collection_interval": 10,
                                "service_address": ":8125"
                        }
                }
        }
}
EOS
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a start -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
EOF
sed -i 's/DOLLAR/\$/g' /root/runner_init.sh
EOT

Regards.

[npalm]

Can we close the question?

[npeyrussie]

Hello @npalm , I think you can close it.

Regards.