Amazon Step Functions Use Cases

You can use Amazon S3 to trigger Amazon Lambda to process data immediately after an upload. For example, you can use Lambda to thumbnail images, transcode videos, index files, process logs, validate content, and aggregate and filter data in real-time. 

You can perform secondary analysis on genomic data to identify meaningful information that clinicians and researchers can act on in a timely fashion. Step Functions can coordinate multiple Amazon Batch jobs that takes raw reads generated from sequencers and then processes them in a genomics pipeline to identify the variation in a biological sample compared to a standard genome reference. Step Functions will wait for each job to complete before moving to the next step in the pipeline.

You can identify valuable data that fits specific criteria related to litigation cases by using Step Functions to automate processing of the datasets, which can easily contain millions of records. In this example, various internet sites and data repositories are monitored, and the Step Functions workflow manages a manual approval from an administrator before continuing on to ingest the data. Data is then sent to Amazon SQS. SQS expands the data, extracts the hashes and metadata about the hashes, performs any necessary de-duplication, and publishes it to Amazon S3.

You can integrate Amazon SNS into your Step Functions workflows in order to trigger notifications regarding the success or failure of the workflow. By taking advantage of SNS message filtering, you can trigger another microservice if your workflow succeeds, or notify developers with a mobile notification if it fails, including the error type and exactly at what point in the execution the failure happened. At the same time, you can log the status of each workflow execution in Amazon SQS for later analytics.

You can use Step Functions to make decisions about how best to process data, for example, to do post processing of groups of satellite images to determine the amount of trees per acre of land. Depending on the size and resolution of the image, this Step Functions workflow will determine whether to use Amazon Lambda or Amazon Fargate to complete post-processing of each file, in order to optimize runtime and costs.

Step Functions is ideal for coordinating session-based applications. You can use Step Functions to coordinate all of the steps of a checkout process on an ecommerce site, for example. Step Functions can read and write from Amazon DynamoDB as needed to manage inventory records.

You can use Step Functions to accelerate the delivery of secure, resilient machine learning applications, all while reducing the amount of code that you have to write and maintain. Using Step Functions, you can automate the pre-processing of your data with Amazon Glue, create an Amazon SageMaker job to train your ML model on the data, and then trigger another SageMaker job to deploy your model into production for online prediction.  

You can use Step Functions to orchestrate multiple ETL jobs involving a diverse set of technologies in an arbitrarily complex ETL workflow. For example, you may want to explore the correlations between online user engagement and forecasted sales revenue and opportunities. You can use Step Functions to coordinate multiple Amazon Glue jobs to blend and prepare the data for analysis.