Amazon CloudWatch is a monitoring and observability service for AWS resources. Using CloudWatch you can collect monitoring and operational data in the form of logs, metrics and events. Also CloudWatch alarms can beset and used to send notification when a metric crosses a specific limit.
CloudWatch metrics :
CloudWatch metrics gives the users’ visibility into the resource utilisation, application performance and operational health. These can help you make sure that you can resolve technical issues and streamline processes and that the application runs smoothly.
The following concepts are important for your understanding on CloudWatch metrics:
- Namespaces is a container for CloudWatch metrics. Metrics in different namespaces are isolated from each other so that metrics from different applications are not accidentally aggregated for computing statistics.
- Metrics represents a time ordered set of data points that are published to CloudWatch. It can be thought of as a variable that we need to monitor and the data points are the values of the variable over time. Metrics exist only in the region they are created.
- Dimensions is a name or a value pair that uniquely identifies a metric. You can assign a maximum of 10 dimensions to a metric. Dimensions help you design a structure for your statistics plan.
- Statistics are metric data aggregation over time specified by the user. Aggregation are made using the namespace, metric name, dimensions and the data point unit of measure within the time period you specify.
- Percentiles as the name suggests, the percentile indicates the relative standing of a value in a dataset. It helps you get a better understanding of the distribution of your metric data. Percentiles are used to detect anomalies.
- Alarms are used to initiate actions on your behalf. An alarm monitors a metric over a specified interval of time, and performs the assigned actions based on the value of the metric relative to a threshold over time.
Why we need custom metrics?
The default metric collection includes CPU Utilisation, Network Traffic and Disk Read/Writes. It does not include metrics to monitor Disk Space and Memory Usage. Hence, we are in the need of custom metrics if we want to monitor the Memory (RAM) usage and Disk space, of our instances.
In order to implement the cloud watch custom metrics in your existing Beanstalk Environment, please follow the following steps,
Prerequisites:-
Elastic Beanstalk Environment with an application running on it.
You should have a IAM user with full Administrator Access.
Steps to implement custom metrics
1. In IAM console choose roles and select aws-elasticbeanstalk-ec2-role and click Attach policy, search for AWS managed policy named “CloudWatchAgentServerPolicy” and attach with this role.
2. Add the file “cloudwatch.config” in your codebase under “.ebextension” folder. So the file structure will be look like (/ .ebextension / cloudwatch.config). If you don’t have “.ebextension” folder go ahead and create it, in the root of your application source code.
3. Add the following contents in your folder
packages:
yum:
perl-DateTime: []
perl-Sys-Syslog: []
perl-LWP-Protocol-https: []
perl-Switch: []
perl-URI: []
perl-Bundle-LWP: []
sources:
/opt/cloudwatch: https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
container_commands:
01-setupcron:
command: |
echo '*/5 * * * * root perl /opt/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl `{"Fn::GetOptionSetting" : { "OptionName" : "CloudWatchMetrics", "DefaultValue" : “— mem- util --disk-space-util --disk-path=/" }}` >> /var/log/cwpump.log 2>&1' > /etc/ cron.d/cwpump
02-changeperm:
command: chmod 644 /etc/cron.d/cwpump
03-changeperm:
command: chmod u+x /opt/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl
option_settings:
"aws:autoscaling:launchconfiguration" :
IamInstanceProfile : "aws-elasticbeanstalk-ec2-role"
"aws:elasticbeanstalk:customoption" :
CloudWatchMetrics : "--mem-util --mem-used --mem-avail --disk-space-util --disk-space- used --disk-space-avail --disk-path=/ --auto-scaling"
4. Deploy the code to your environment.
Thats it, now you have configured memory and disk space metrics for your environment. This script will continuously put the custom metrics to cloud watch every five minutes.
The Custom Metrics can be viewed under
Cloudwatch -> Metrics -> Linux System
InstanceId section will give you the Memory utilisation metrics (RAM) for each individual instances in your environment.
AutoScalingGroupName section will give you the Average Memory utilisation metrics (RAM) for your environment.
Filesystem section will give you the Disk utilisation metrics for each individual instances in your environment.
AutoScalingGroupName Filesystem section will give you the Average Disk utilisation for your environment.
Creating Dashboard
You can create your own dashboards for the above metrics by following the steps in the below
- Open the CloudWatch console
- In the navigation pane, choose Dashboards and then Create dashboard.
- Select the widget type you want and click Configure.
- Under Custom Namespaces choose Linux System and click AutoScalingGroupName and select the memory used checkbox and click Create widget.
- Click save dashboard.
- Similarly we can create as many custom dashboards we want.
In the upcoming blogs we will see more about the CloudWatch events and rules to trigger some operation, when these metrics reaches a certain threshold value.
https://github.com/mounick-mallow/AWS-CloudWatch
-Mounick Raj T,
DevOps Team,
Mallow Technologies.
Reference
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html