Posted On: Jul 20, 2022
We are excited to launch two new features that can help you enforce access controls with Amazon EMR on EC2 clusters (EMR Clusters). These features are supported with jobs that are submitted to the EMR Cluster as EMR Steps. First is Runtime Role with EMR Steps. A Runtime Role is an Amazon Identity and Access Management (IAM) role that you associate with an EMR Step, and jobs use this role to access Amazon Web Services resources. The second is integration with Amazon Lake Formation to apply Table and Column-level access controls for Apache Spark and Apache Hive jobs with EMR Steps.
Previously, all jobs running on an EMR cluster used the IAM role associated with the EMR cluster’s EC2 Instances, Instance Profile, to access Amazon Web Services resources. For e.g. If a Spark job and Hive job running on the same cluster needed to access different S3 buckets then Instance profile must allow access to both the buckets. With Runtime Role for EMR Steps, you can now specify a different IAM role for the Spark and the Hive job, thus scoping down access at a job level. This allows you to simplify access controls on a single EMR cluster that is shared between multiple tenants, wherein each tenant can be easily isolated using IAM roles.
In addition, you can use Amazon Lake Formation to apply Table and Column-level permissions with Apache Spark and Apache Hive jobs submitted as EMR Steps. Amazon Lake Formation is a fully managed service that makes it easy to build, secure, and manage data lakes. Amazon Lake Formation enables you to apply fine-grained access control to data stored in data lakes, through a simple grant or revoke mechanism, much like a relational database management system (RDBMS). With this feature, table and column-level permissions defined in Amazon Lake Formation for an IAM Role are seamlessly enforced with Apache Hive and Apache Spark jobs submitted as EMR Steps. This allows you to further simplify access controls, and provide each jobs with access to specific Databases, Tables, and Columns.
Both are these features are available with Amazon EMR release 6.7, and are available in all regions where Amazon EMR is available. See Regional Availability of Amazon EMR, and our release notes for more details.