Posted On: Dec 20, 2019
We now support Parallelization Factor on Amazon Lambda, a feature that allows you to process one shard of a Kinesis or DynamoDB data stream with more than one Lambda invocation simultaneously. This new feature allows you to build more agile stream processing applications on volatile data traffic.
By default, Lambda invokes a function with one batch of data records from one shard at a time. For a single event source mapping, the maximum number of concurrent Lambda invocations is equal to the number of Kinesis or DynamoDB shards.
Now you can specify the number of concurrent batches that Lambda polls from a shard via a Parallelization Factor from 1 (default) to 10. For example, when Parallelization Factor is set to 2, you can have 200 concurrent Lambda invocations at maximum to process 100 Kinesis data shards. This helps scale up the processing throughput when the data volume is volatile and the IteratorAge is high.
You can get started with this feature with the Amazon Web Services Management Console, Amazon CLI, Amazon CloudFormation, or Amazon SDK for Lambda. It can be used at no additional cost in all Amazon Web Services China Regions where Amazon Lambda is available, including Amazon Web Services China(Beijing) region, operated by Sinnet and Amazon Web Services China(Ningxia) region, operated by NWCD. For more information on where Amazon Lambda is available, see the Amazon China Region table. To learn more, see Using Amazon Lambda with Amazon Kinesis and Using Amazon Lambda with Amazon DynamoDB in the Amazon Lambda Developer Guide.