Configure The CloudWatch Agent On a Running EC2 Linux Instance
1. Start An EC2 Instance
Step 1: Log in to your AWS account and start the Amazon Linux EC2 instance that you have already created.
2. Attaching an IAM Role
Step 1: Create IAM Role with permission for reading information from the instance and writing it to CloudWatch.
For that, we need to execute the following steps:
- Go to the navigation panel, choose Roles, and then choose Create Role.
- Under Select type of trusted entity, choose AWS service.
- Under Use case, choose EC2 and then choose Next.
- In the list of policies, select the check box next to CloudWatchAgentServerPolicy and then choose Next.
Role Name
Step 1: For Role name, enter a name for your new role, such as “CloudWatchRole” or another name that you prefer.Confirm that CloudWatchAgentAdminPolicy appears next to Policies.Choose to Create role.
Step 2: Attach IAM Role from the previous step to our Amazon Linux EC2 instance. For that, we need to go to the navigation panel, choose EC2 and then choose Instances. Then select our instance and choose to Modify IAM role.
On the next page, we need to choose our role and then choose the Update IAM role.
3. Installing The CloudWatch Agent
Step 1: Install the CloudWatch agent. First of all, we need to log in to our EC2 instance. (Select instance –> push Connect button). On the next page choose EC2 Instance Connect and click Connect button. A command prompt will pop up after connecting.
Note: If any error occurs, then make sure that port SSH is open on your instance.
For installation CloudWatch agent we need to execute the following command:
sudo yum install amazon-cloudwatch-agent -y
After running this command you will see that installation is complete.
Step 2: Configure the CloudWatch agent with the wizard and fill in data about our log file.To create a configuration file execute the following command:
sudo /opt/aws/amazon-cloudwatch-agent
/bin/amazon-cloudwatch-agent-config-wizard
After running this command we need to answer the following questions (I will provide answers for my configuration):
- On which OS are you planning to use the agent? (Linux)
- Are you using EC2 or On-Premises hosts? (EC2)
- Which user are you planning to run the agent? (root)
- Do you want to turn on the StatsD daemon? (yes)
StatsD is a popular open-source solution that can gather metrics from a wide variety of applications. - Which port do you want the StatsD daemon to listen to? (8125)
- What is the collection interval for the StatsD daemon? (10s)
- What is the aggregation interval for metrics collected by StatsD daemon?(the 60s)
- Do you want to monitor metrics from CollectD?(No)
- Do you want to monitor any host metrics? e.g. CPU, memory, etc. (yes)
- Do you want to monitor CPU metrics per core? (yes)
- Do you want to add ec2 dimensions (ImageId, InstanceId, InstanceType, AutoScalingGroupName) into all of your metrics if the info is available? (yes)
- Do you want to aggregate ec2 dimensions (InstanceId)? (yes)
- Would you like to collect your metrics at high resolution (sub-minute resolution)? This enables sub-minute resolution for all metrics, but you can customize for specific metrics in the output JSON file. (60s)
- Which default metrics config do you want? (Standard)
On this question, you can choose the answer between Basic, Standard, Advanced, and None. (Detailed description below)
Detail level | Metrics included |
---|---|
Basic |
Mem: mem_used_percent Disk: disk_used_percent The disk metrics such as disk_used_percent have a dimension for Partition, which means that the number of custom metrics generated is dependent on the number of partitions associated with your instance. The number of disk partitions you have depends on which AMI you are using and the number of Amazon EBS volumes you attach to the server. |
Standard |
CPU: cpu_usage_idle, cpu_usage_iowait, cpu_usage_user, cpu_usage_system Disk: disk_used_percent, disk_inodes_free Diskio: diskio_io_time Mem: mem_used_percent Swap: swap_used_percent |
Advanced |
CPU: cpu_usage_idle, cpu_usage_iowait, cpu_usage_user, cpu_usage_system Disk: disk_used_percent, disk_inodes_free Diskio: diskio_io_time, diskio_write_bytes, diskio_read_bytes, diskio_writes, diskio_reads Mem: mem_used_percent Netstat: netstat_tcp_established, netstat_tcp_time_wait Swap: swap_used_percent |
After that, you will see the current configuration (JSON format) and you will need to answer the questions:
- Are you satisfied with the above config? (yes)
- Do you have any existing CloudWatch Log Agent? (no)
- Do you want to monitor any log files? (yes)
Because in this tutorial we will monitor our Apache log file(e.g. error_log) - Log file path: (/var/log/httpd/error_log)
- default choice: (e.g. CloudWatchDemo)
- Log stream name: ({instance_id})
- Log Group Retention in days (2)
- Do you want to specify any additional log files to monitor? (no)
- Do you want to store the config in the SSM parameter store? (no)
After answering these questions, we see that program exits:
The configuration file will store in the “bin” folder:
ls /opt/aws/amazon-cloudwatch-agent/bin/
Step 3: Start the CloudWatch agent with our configuration file.To launch the CloudWatch agent we need to execute the following command:
sudo /opt/aws/amazon-cloudwatch-agent/bin/
amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c
file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s
In this command, -a fetch-config causes the agent to load the latest version of the CloudWatch agent configuration file, and
-s starts the agent.
After running this command we see that execution is successfully finished.
How To Install and Set Up an AWS CloudWatch Agent Using CLI?
We will see how to install and configure the new unified CloudWatch agent on a running EC2 Linux instance. Collect logs from the Apache HTTP log file and collect metrics from our EC2 instance. New unified CloudWatch can collect both logs and metrics from EC2 instances. The older CloudWatch Logs agent is on the path to deprecation.