Posted On: Sep 25, 2020
Amazon EMR now supports Managed Scaling, a new feature that automatically resizes your EMR cluster for best performance at the lowest possible cost, without the need to specify scaling policies. You can reduce up to 60% cost compared with fixed-size clusters by setting the minimum and maximum compute resource limits for a cluster.
Previously, you could manually scale cluster size or leverage EMR Automatic Scaling by customizing scaling rules based on CloudWatch metrics. However, these approaches require in-depth understanding of application frameworks and workloads patterns; EMR Automatic Scaling supports instance groups only. EMR Managed Scaling applies to both instance groups and instance fleets. You can seamlessly scale Spot Instances and On-Demand Instances within the same cluster.
Amazon EMR Managed Scaling is available on Apache Spark, Apache Hive, and YARN-based workloads on Amazon EMR version 5.30.1 and above. You can use this feature in Amazon Web Services China (Beijing) Region, operated by Sinnet, and Amazon Web Services China (Ningxia) Region, operated by NWCD.
To get started, see EMR Managed Scaling in the Amazon EMR Management Guide.