Centralized Logging

What does this Amazon Web Services Solution do?

The Centralized Logging solution enables organizations to collect, analyze, and display logs on Amazon Web Services across multiple accounts and Amazon Web Services Regions. The solution uses Amazon Elasticsearch Service (Amazon ES), a managed service that simplifies the deployment, operation, and scaling of Elasticsearch clusters in the Amazon Web Services Cloud, as well as Kibana, an analytics and visualization platform that is integrated with Amazon ES. In combination with other Amazon Web Services managed services, this solution offers customers a customizable, multi-account environment to begin logging and analyzing their Amazon Web Services environment and applications.

Amazon Web Services Solution overview

The following diagram demonstrates the architecture on Amazon Web Services for centralized logging across multiple accounts and Amazon Web Services Regions. You can automatically deploy using the solutions’ implementation guide accompanying Amazon CloudFormation template.

Click to enlarge

Centralized Logging architecture

The primary template deploys an Amazon ES domain, which is the hardware, software, and data exposed by Amazon ES endpoints. A custom Amazon Lambda function is deployed to load log data from Amazon CloudWatch to an Amazon ES domain, configured with a set of default Kibana dashboards as a starting point for data visualization.

A secondary template enables customers to index logs from secondary accounts and regions on the Amazon ES domain in the primary account or region.

A demo template deploys sample logs that customers can use for testing purposes.

Show less

Centralized Logging

Version 1.1
Last updated: 06/2020
Author: Amazon Web Services

Estimated deployment time: 30 min

Source code

View deployment guide

Launch primary stack in the Amazon Web Services Console

Launch secondary stack in the Amazon Web Services Console

Features

Centralized logging reference implementation

Deploy a centralized logging solution using Amazon CloudFormation. The CloudFormation template will automatically launch and configure the components necessary to upload log files from multiple accounts and Amazon Web Services Regions to Amazon ES for analysis and visualization in a customizable, user-friendly dashboard.

Optimized Resources for China Customers

Installation scripts and process is optimized for Chinese customers. This increases the deployment success rate and reduces the deployment time when you provision the solution in Amazon Web Services (Ningxia) Region operated by NWCD and Amazon Web Services (Beijing) Region operated by Sinnet.

Logging capabilities beyond default Amazon Web Services service logs

Extend your logging capabilities beyond default Amazon Web Services service logs. This flexible solution includes examples for capturing host-level log files and VPC flow logs, and is designed to scale with your growing business.

Data visualization using built-in Amazon ES support

Simplify data visualization using built-in Amazon ES support for Kibana, including a default set of preconfigured dashboards that give you a first glimpse into the customization capabilities of Kibana.

Explore all Amazon Web Services Solutions

Browse our portfolio of Amazon Web Services -built solutions to common architectural problems.

Learn more

Find a Partner

Find Amazon Web Services certified consulting and technology partners to help you get started.

Learn more

Start building in the console

Sign-up and start exploring our services.

Get started