Posted On: Sep 10, 2021
Amazon ParallelCluster is a fully supported and maintained open source cluster management tool that makes it easier for scientists, researchers, and IT administrators to deploy and manage high performance computing (HPC) clusters on Amazon Web Services. HPC clusters are traditionally collections of tightly coupled compute, storage, and networking resources that enable customers to run large scale scientific and engineering workloads.
With its latest release, we have introduced a major update to Amazon ParallelCluster. Significant feature enhancements in this version include:
- Support for cluster management via Amazon API Gateway: Customers can now manage and deploy clusters through HTTP endpoints with Amazon API Gateway. This opens up new possibilities for scripted or event-driven workflows, such as creating new clusters when datasets are ingested or when existing infrastructure is insufficient to meet dynamic compute needs. These APIs also make it easier for builders to use Amazon ParallelCluster as a building block in their HPC workflows and create their own extensible solutions, services, and customized front-end interfaces. Amazon ParallelCluster’s command-line interface (CLI) has also been redesigned for compatibility with this API and includes a new JSON output option. This new functionality makes it possible for customers to implement similar building block capabilities using the CLI as well.
- Improved custom AMI creation: Customers now have access to a more robust process for creating and managing custom AMIs using EC2 Image Builder. Customers can specify build components to install software, apply security hardening steps, or modify operating system configuration in a replicable way. Custom AMIs can now be managed through a separate Amazon ParallelCluster configuration file, and can be created using the pcluster build-image command in the Amazon ParallelCluster command-line interface.
- Robust cluster configuration: Cluster configuration files have been retooled to make them more robust and easier to maintain.
Amazon ParallelCluster is available at no additional cost, and you pay only for the resources needed to run your applications. Learn how to launch an HPC cluster using Amazon ParallelCluster here. Amazon ParallelCluster is released via the Python Package Index (PyPI). Amazon ParallelCluster’s source code is hosted under the Amazon Web Services repository on GitHub at https://github.com/aws/aws-parallelcluster, where you can learn how to launch your own HPC cluster in Amazon Web Services.
For more detail you can find the complete release notes for the latest version of Amazon ParallelCluster here.