Amazon Lambda auto scaling improvements for Amazon MSK and self-managed Apache Kafka as event sources

Amazon Lambda has launched improved auto scaling for Amazon MSK and self-managed Kafka as event sources to improve performance and help lower costs for customers. Lambda starts with one consumer and checks the OffsetLag metric (measure of backlog at source) and processing rate every minute and scales up or down every 3 minutes. Lambda allows up to one consumer per partition per topic for a Kafka cluster. Previously, Lambda used to start connection with Kafka clusters at a single consumer, then every 15 minutes would check for OffsetLag and scale up (or down) consumers.

For more details, please see Auto scaling of the Amazon MSK event source and Auto scaling of the Kafka event source. This feature is available in Amazon Web Services China (Beijing) Region, operated by Sinnet, and Amazon Web Services China (Ningxia) Region, operated by NWCD.