Posted On: Oct 12, 2021
Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) now supports cross-cluster replication, enabling you to automate copying and synchronizing of indices from one domain to another at low latency in same or different Amazon Web Services accounts or Regions. With cross-cluster replication, you can achieve high availability for your mission critical applications with sequential data consistency.
Amazon OpenSearch Service makes it easy for you to perform interactive log analytics, real-time application monitoring, website search, and more. To ensure redundancy and availability customers configure replicas and deploy their domains across multiple availability zones, protecting them against instance failures and availability zone outages. However, the domain itself can still be a single point of failure. To protect against such failures, customers previously had to create a second domain, fork their input data streams to the two clusters, and place a load balancer in front of the two domains to balance incoming search requests. This set-up adds complexity and cost as it requires you to use additional technologies like Apache Kafka or Amazon Web Services Lambda to monitor and correct data inconsistencies between the domains.
With cross-cluster replication for Amazon OpenSearch Service, you can replicate indices at low latency from one domain to another in same or different Amazon Web Services Regions without needing additional technologies. Cross-cluster replication provides sequential consistency while continuously copying data from the leader index to the follower index. Sequential consistency ensures the leader and the follower index return the same result set after operations are applied on the indices in the same order. Cross-cluster replication is designed to minimize delivery lag between the leader and the follower index. You can continuously monitor the replication status via APIs. Additionally, if you have indices that follow an index pattern, you can create automatic follow rules and they will be automatically replicated.
Cross-cluster replication is available on the service today for domains running Elasticsearch 7.10. Cross cluster replication is also available as an open-source feature in OpenSearch 1.1 which is planned to be available on the service soon.
Cross-cluster replication is now available for Amazon OpenSearch Service domains in the Amazon Web Services China (Beijing) region, operated by Sinnet, and the Amazon Web Services China (Ningxia) region, operated by NWCD. To learn more about cross-cluster replication, please see the documentation. To learn more about Amazon OpenSearch Service, please visit the product page