Amazon Kinesis Data Streams announces Service-Managed Partition Keys for simplified data ingestion
Amazon Kinesis Data Streams now supports service-managed partition keys for On-Demand Standard and On-Demand Advantage streams, automatically distributing records across shards without requiring customers to specify partition keys to publish data. This capability simplifies data ingestion for workloads where record ordering is not required, eliminating rate-limiting caused by uneven data distribution and reducing time-to-production for streaming workloads.
Amazon Kinesis Data Streams is a serverless streaming data service that makes it easy to capture, process, and store data streams at any scale. Many streaming use cases such as log aggregation, metrics collection, and IoT telemetry do not require ordering guarantees and benefit from pre-warmed capacity for instant scaling. Previously, customers generated random partition keys (such as UUIDs) to distribute data, but random partitioning can still produce uneven throughput across shards, causing rate-limiting for some partition keys even when the stream has sufficient aggregate capacity. By opting into service-managed partition keys, customers no longer need to specify partition keys when publishing data to streams in On-Demand mode. The service automatically distributes records based on available warm capacity, allowing customers to scale to gigabytes per second without maintaining any distribution logic.
Customers who want to send records without specifying a partition key can simply upgrade to the latest Kinesis Producer Library version to benefit from this capability. For all other customers, no KPL or KCL changes are required. KPL aggregation is not supported with service-managed partition keys.
Service-managed partition keys for Amazon Kinesis Data Streams are available today in the Amazon Web Services China (Beijing) Region, operated by Sinnet, and the Amazon Web Services China (Ningxia) Region, operated by NWCD. at no additional cost. To get started, visit the Amazon Kinesis Data Streams documentation.