Amazon CloudWatch Documentation

Amazon CloudWatch is a monitoring and management service designed to provide data and actionable insights for Amazon Web Services, hybrid, and on-premises applications and infrastructure resources. With CloudWatch, you can collect and access your performance and operational data in the form of logs and metrics from a single platform. This helps you to address the challenge of monitoring individual systems and applications in silos (server, network, database, etc.). CloudWatch helps you to monitor your stack (applications, infrastructure, and services) and leverage alarms, logs, and events data to take actions and reduce Mean Time to Resolution (MTTR). This frees up important resources and allows you to focus on building applications and business value.

CloudWatch is designed to give you actionable insights that help you optimize application performance, manage resource utilization, and understand system-wide operational health. CloudWatch provides visibility of metrics and logs data, data retention (metrics), and the ability to perform calculations on metrics. This allows you to perform historical analysis for cost optimization and derive real-time insights into optimizing applications and infrastructure resources.

You can use CloudWatch Container Insights to monitor, troubleshoot, and alarm on your containerized applications and microservices. You can collect, aggregate, and summarize compute utilization information like CPU, memory, disk, and network data, as well as diagnostic information like container restart failures, to help DevOps engineers isolate issues and resolve them. Container Insights gives you insights from container management services such as Amazon ECS for Kubernetes (Amazon EKS), Amazon’s Elastic Container Service (Amazon ECS), Amazon Fargate, and standalone Kubernetes (k8s).

The Amazon CloudWatch Logs service helps you to collect and store logs from your resources, applications, and services. There are three main categories of logs 1) Vended logs. These are natively published by services of Amazon Web Services on behalf of the customer. 2) Logs that are published by services of Amazon Web Services. Currently over 30 services of Amazon Web Services publish logs to CloudWatch. 3) Custom logs. These are logs from your own application and on-premises resources. 

Collecting metrics from distributed applications (such as those built using microservices architectures) is time consuming. Amazon CloudWatch allows you to collect default metrics from more than 70 services of Amazon Web Services, such as Amazon EC2, Amazon DynamoDB, Amazon S3, Amazon ECS, Amazon Lambda, and Amazon API Gateway, without any action on your part. 

Amazon CloudWatch helps you to collect custom metrics from your own applications to help you monitor operational performance, troubleshoot issues, and spot trends. User activity is an example of a custom metric you can collect and monitor over a period of time. You can use CloudWatch Agent or the PutMetricData API action to publish these metrics to CloudWatch.

Container Insights is designed to simplify the collection and aggregation of curated metrics and container ecosystem logs. With it, you can collect compute performance metrics such as CPU, memory, network, and disk information from each container as performance events and generate custom metrics used for monitoring and alarming. The performance events are ingested as CloudWatch Logs with metadata about the running environment such as the Amazon EC2 instance ID, Service, Amazon EBS volume mount and ID, etc., to assist in monitoring and troubleshooting. CloudWatch custom metrics can be extracted from these ingested logs and further analyzed using CloudWatch Logs Insights’ advanced query language. Container Insights also provides an option to collect application logs (stdout/stderr), custom logs, predefined Amazon EC2 instance logs, Amazon EKS/k8s data plane logs and Amazon EKS control plane logs. 

CloudWatch Lambda Insights helps with the collection and aggregation of curated metrics and logs from Amazon Lambda functions. You can collect compute performance metrics such as CPU, memory, and network from each Lambda function as performance events, while generating custom metrics used for monitoring and alarming. The performance events are ingested as CloudWatch logs to simplify monitoring and troubleshooting. CloudWatch custom metrics can be extracted from these ingested logs and further analyzed using CloudWatch Logs Insights’ advanced query language. 

Amazon CloudWatch Metric Streams helps you to create continuous streams of metrics to a destination of your choice. Metrics Streams makes it easier to send CloudWatch metrics to popular third-party service providers using an Amazon Kinesis Data Firehose HTTP endpoint. You can also direct your metrics to your data lake on Amazon Web Services. 

Amazon CloudWatch dashboards enable you to create re-usable graphs and visualize your cloud resources and applications in a unified view. You can graph metrics and logs data side by side in a single dashboard to quickly get the context and go from diagnosing the problem to understanding the root cause. For example, you can visualize key metrics, like CPU utilization and memory, and compare them to capacity. You can also correlate the log pattern of a specific metric and set alarms to be proactively alerted about performance and operational issues. This is designed to give you system-wide visibility into operational health and the ability to troubleshoot issues, reducing Mean Time to Resolution (MTTR).

Amazon CloudWatch composite alarms allow you to combine multiple alarms and reduce alarm noise. If an application issue affects several resources in an application, you can set up receipt of a single alarm notification for the entire application instead of one for each affected service component or resource. This helps you stay focused on finding the root cause of potential operational issues to reduce application downtime. You can also provide an overall state for a grouping of resources like an application, Amazon Web Services Region, or Availability Zone. 

Amazon CloudWatch alarms helps you to set a threshold on metrics and trigger an action. You can create high-resolution alarms, set a percentile as the statistic, and choose to specify an action or ignore one as you deem appropriate.

Applications and infrastructure resources generate lots of operational and monitoring data in form of logs and metrics. In addition to providing you the ability to access and visualize these data sets in a single platform, Amazon CloudWatch can also correlate metrics and logs. This helps you diagnose potential problems and assist in understanding the root cause. For example, you can correlate a log pattern, such as an error to a specific metric, and set alarms to be actively alerted of performance and operational issues.

Amazon CloudWatch Application Insights provides setup of observability for your enterprise applications, to assist you in gaining visibility into the health of such applications. It can help identify, and you can set up, key metrics and logs across your application resources and technology stack i.e. database, web (IIS) and application servers, Operating System, load balancers, queues, etc. You can continuously monitor these telemetry data to detect and correlate anomalies and errors, to notify you of any problems in your application. Designed to aid in troubleshooting, Application Insights creates dashboards for the detected problems with correlated metric anomalies and log errors, along with additional insights to assist you in determining their potential root-cause. This helps you to take quick remedial actions to manage the health of your applications and assess impact on end-users.

Container Insights provides dashboards in the CloudWatch console. These dashboards summarize the compute performance, errors, and alarms by cluster, pod/task, and service. Each dashboard summarizes the list of running pods/tasks or containers by CPU and memory for the selected time window, and allows you to contextually - based on time window and selected pod/task or container - dive deeper into application logs, Amazon X-Ray traces, and performance events. 

Lambda Insights provides dashboards in the CloudWatch console. These dashboards summarize the compute performance and errors. Each dashboard includes the list of metrics for the selected time window and allows you to contextually dive deeper — based on time window and selected function — into application logs, Amazon X-Ray traces, and performance events.

Amazon CloudWatch Anomaly Detection applies machine-learning algorithms designed to assist you to continuously analyze data of a metric and identify anomalous behavior. You can create alarms that auto-adjust thresholds based on natural metric patterns. You can also visualize metrics with anomaly detection bands on dashboards. This helps you to monitor, isolate, and troubleshoot unexpected changes in your metrics.

You can use Amazon CloudWatch ServiceLens to help you visualize and analyze the health, performance, and availability of your applications in a single place. CloudWatch ServiceLens ties together CloudWatch metrics and logs as well as traces from Amazon X-Ray to help you obtain a complete view of your applications and their dependencies. This can assist you in pinpointing performance bottlenecks, isolating root causes of application issues, and determining users potentially impacted. CloudWatch ServiceLens is designed to allow visibility into your applications in three main areas: Infrastructure monitoring (using metrics and logs to understand the resources supporting your applications), transaction monitoring (using traces to understand dependencies between your resources), and end user monitoring (using canaries to monitor your endpoints and notify you when your end user experience has degraded). CloudWatch ServiceLens provides a Service Map that visualizes the contextual linking of your resources, along with an intuitive interface to assist you to dive deep into correlated monitoring data.

Amazon CloudWatch Synthetics helps you to monitor application endpoints. You can run tests on your endpoints and receive alerts if your application endpoints don’t behave as expected. These tests can be customized to check for availability, latency, transactions, broken or dead links, step by step task completions, page load errors, load latencies for UI assets, complex wizard flows, or checkout flows in your applications. You can also use CloudWatch Synthetics to isolate alarming application endpoints and map them back to underlying infrastructure issues to reduce mean time to resolution. With this new feature, CloudWatch can now collect canary traffic, which can assist you in verifying your customer experience enabling you to discover issues. CloudWatch Synthetics supports monitoring of your REST APIs, URLs, and website content, checking for unauthorized changes from phishing, code injection and cross-site scripting.

Amazon CloudWatch RUM is designed to give you visibility into your applications’ client-side performance and reduce MTTR. It is designed to allow you to collect client-side data on web application performance in near real time to identify and debug issues. CloudWatch RUM complements the CloudWatch Synthetics data to give you more visibility into your end-user experience. You can visualize anomalies in performance and use the relevant debugging data (such as error messages, stack traces, and user sessions) to fix performance issues (such as JavaScript errors, crashes, and latencies). You can gain insight into the range of end-user impacts, including number of users, geolocations, and browsers. CloudWatch RUM is designed to aggregate data on your users' journey through your application, which can help you determine which features to launch and bug fixes to prioritize.

Auto Scaling helps you automate capacity and resource planning. You can set a threshold to alarm on a key metric and trigger an Auto Scaling action. For example, you could set up an Auto Scaling workflow to add or remove EC2 instances based on CPU utilization metrics and optimize resource costs.

CloudWatch Events is designed to provide a near real-time stream of system events that describe changes to your Amazon Web Services resources. It helps you to respond quickly to operational changes and take corrective action. You can write rules to indicate which events are of interest to your application and what actions to take when a rule matches an event. 

For Amazon EKS and k8s clusters, CloudWatch Container Insights helps you to alarm on compute metrics to trigger auto scaling policies on your Amazon EC2 Auto Scaling group and provides you the ability to stop, terminate, reboot, and recover any Amazon EC2 instance.

Amazon CloudWatch helps you to monitor trends and seasonality with months of metric data (storage and retention). This data allows you to perform historical analysis to fine-tune resource utilization. With CloudWatch, you can also collect health metrics including custom ones, such as those coming from your on-premises applications. Granular real-time data enables better visualization and ability to spot and monitor trends to optimize application performance and operational health.

Amazon CloudWatch Metric Math helps you to perform calculations across multiple metrics for real-time analysis so you can derive insights from your existing CloudWatch metrics. You can visualize these computed metrics in the Amazon Management Console, add them to CloudWatch dashboards, or retrieve them using the GetMetricData API action. Metric Math supports arithmetic operations such as +, -, /, *, and mathematical functions such as Sum, Average, Min, Max, and Standard Deviation.

Amazon CloudWatch Logs Insights allows you to drive actionable intelligence from your logs to address operational issues without needing to provision servers or manage software. You can instantly begin writing queries with aggregations, filters, and regular expressions. In addition, you can visualize timeseries data, drill down into individual log events, and export query results to CloudWatch Dashboards. This is designed to give you operational visibility. With a few clicks in the Amazon Management Console, you can start using Logs Insights to query logs sent to CloudWatch. 

Container Insights simplifies the analysis of observable data from metrics, logs, and traces by simplifying deep linking from automatic dashboards to granular performance events, application logs (stdout/stderr), custom logs, predefined Amazon EC2 instance logs, Amazon EKS/k8s data plane logs and Amazon EKS control plane logs using CloudWatch Logs Insights’ advance query language.

Lambda Insights simplifies the analysis of observable data from metrics, logs, and traces by simplifying deep linking from automatic dashboards to granular performance events, application logs, and custom logs, using CloudWatch Logs Insights’ advanced query language.

Amazon CloudWatch now includes Contributor Insights, which analyzes time-series data to provide a view of the top contributors influencing system performance. Once set up, Contributor Insights is designed to run continuously without needing additional user intervention. This is designed to help developers and operators more quickly isolate, diagnose, and remediate issues during an operational event. Contributor Insights helps you understand who or what is impacting your system and application performance, such as a specific resource, customer account, or API call. This enables you to pinpoint outliers, find the heaviest traffic patterns, and rank the most utilized system processes. You can create Contributor Insights rules to evaluate patterns in structured log events as they are sent to CloudWatch Logs, including logs from services of Amazon Web Services like Amazon CloudTrail, Amazon Virtual Private Cloud, Amazon API Gateway, and any custom logs sent by your service or on-premises servers, such as Apache access logs. Contributor Insights is designed to evaluate these log events in real-time and display reports that show the top contributors and number of unique contributors in a dataset. A contributor is an aggregate metric based on dimensions contained as log fields in CloudWatch Logs, such as account-id or interface-id in VPC Flow Logs, or any other custom set of dimensions. You can sort and filter contributor data based on your own custom criteria. Contributor Insights report data can be displayed on CloudWatch dashboards, graphed alongside CloudWatch metrics, and added to CloudWatch alarms.

Amazon CloudWatch Metrics Insights (Preview) is designed to bed a fast, flexible, SQL-based query engine that enables you to identify trends and patterns within millions of operational metrics in near real time. Metrics Insights allows you to gain better visibility on your infrastructure and large-scale application performance with flexible querying and on-the-fly metric aggregations. Metrics Insights queries can be used to create powerful visualizations, helping you proactively monitor and pinpoint issues quickly, and reduce MTTR.

Amazon CloudWatch Evidently lets application developers conduct experiments and identify unintended consequences of new features before rolling them out for general use. Evidently allows you to validate new features across the full application stack before release, which makes for a safer release. When launching new features, you can expose them to a small user base, monitor key metrics such as page load times or conversions, and then dial up traffic. Evidently also allows you to try different designs, collect user data, and release the most effective design in production.

Amazon CloudWatch is integrated with Amazon Identity and Access Management (IAM) designed to help you configure which users and resources have permission to access your data and how they can access it.

Data is encrypted at rest and during transfer. You can also use Amazon KMS encryption to encrypt your log groups for added compliance and security.

For additional information about service controls, security features and functionalities, including, as applicable, information about storing, retrieving, modifying, restricting, and deleting data, please see https://docs.amazonaws.cn/en_us. This additional information does not form part of the Documentation for purposes of the Sinnet Customer Agreement for Amazon Web Services (Beijing Region), Western Cloud Data Customer Agreement for Amazon Web Services (Ningxia Region) or other agreement between you and Sinnet or NWCD governing your use of services of Amazon Web Services China Regions.