AWS CloudWatch metrics for Sidekiq
- Autoscaling an Elastic Beanstalk Sidekiq environment based off of the queue size
.env.production and fill in
.env.production with the real configuration values
Testing in Development
bundle exec foreman run -e .env.production rake metrics:update
Check the AWS console to see your custom metric
You can monitor your Sidekiq instances on AWS for free using Heroku Scheduler
- Fork the repo
- Push your fork to Heroku
- Setup your config vars with
- Use Heroku Scheduler to trigger
rake metrics:schedule_updateevery 10 minutes
- Have a beer
To get finer granularity on Heroku you can run a job every minute with Heroku Scheduler by creating 10 jobs spaced 1 minute apart.
POST to scheduler using the following command:
curl 'https://scheduler.heroku.com/jobs' --data 'command=rake+metrics%3Aupdate&dyno_size=11&every=10&at=9'
at is the minute you want to start the job and takes the values
9. You'll also need to post your cookie. To get the full
curl command I used Chrome Develper Tools and captured a request.
Scheduling and update with
rake metrics:schedule_update also works but the worker process will sleep on a free Heroku dyno.
Autoscaling with Elastic Beanstalk
Elastic Beanstalk doesn't let you configure alarms for custom metrics for autoscaling out of the box. However you can manually change the Autoscaling group to add custom Scaling Policies.
- Configure an Elastic Beanstalk Environment with Autoscaling
- Select CPU Utilization as your Trigger measurement with sensible defaults e.g. Upper threshold:
90, Lower threshold
- After the environment has been created it will create a new Autoscaling Group. You can find it in the AWS Web Console under
EC2 -> Autoscaling -> Autoscaling Groups
- Note the name of the autoscaling group and configure it in the Heroku config. e.g.
heroku config:set AWS_CLOUDWATCH_DIMENSION_VALUE=<auto-scaling-group-name>
- Set up two new CloudWatch alarms for your Sidekiq metric in the AWS Web Consule under
CloudWatch -> Custom Metrics. One should be for
sidekiq-worker-overloadand the other should be for
- Back in the AWS Web Console under
EC2 -> Autoscaling -> Autoscaling Groups -> Scaling Policiesadd two new policies. One for
aws-eb-autoscale-upand the other for
aws-eb-autoscale-down. Select the appropriate alarm created in step 5 for each policy.
- Remove the autogenerated scaling policies created by Elastic Beanstalk.
- Optionally remove the autogenerated alarms created by Elastic Beanstalk or configure them to send a notification.
Note if you ever rebuild or terminate your environment, Elastic Beanstalk will create a new autoscaling group and CloudWatch put the CloudWatch alarms in it. You'll need to repeat steps. 3-8 above.