Amazon MSK Replicator now supports log forwarding for replication visibility
Amazon MSK Replicator now delivers replicator logs to give you end-to-end visibility into replication health. Replicator logs surface critical replication events and errors along with guidance on how to resolve each issue, enabling you to troubleshoot faster without requiring Amazon Web Services Support.
MSK Replicator is a feature of Amazon MSK that automates data replication between Kafka clusters, eliminating the need to manage custom replication infrastructure or configure open-source tools. Until now, you could use Amazon CloudWatch metrics to track replication progress and get visibility into replication health. With this launch, MSK Replicator further simplifies diagnosing issues during replication with actionable log entries that surface the most common replication errors including insufficient permissions on source topics, partition quota exhaustion on target clusters, and records exceeding size limits, along with prescriptive guidance on how to resolve each issue. MSK Replicator also logs steady-state replication activity including offset commits, topic discovery events, and any errors or warnings from Kafka clients used internally by the replicator, giving you end-to-end visibility into replication health. You can enable log delivery when creating or updating a Replicator using the Amazon MSK console, Amazon CLI, or Amazon CloudFormation and forward logs to Amazon CloudWatch, Amazon S3, or Amazon Data Firehose.
This capability is supported in all Amazon Web Services Regions where MSK Replicator is available including Amazon Web Services China (Beijing) Region, operated by Sinnet and Amazon Web Services China (Ningxia) Region, operated by NWCD. Log delivery costs depend on the destination service you choose — refer to the pricing pages for Amazon CloudWatch, Amazon S3, and Amazon Data Firehose.
To learn more, visit the MSK Replicator documentation, and product page.