As you grow to storing millions or billions of objects and terabytes to petabytes of data on Amazon S3, it becomes increasingly valuable to gain visibility into your storage usage. Amazon S3 offers a number of features to help you better understand, analyze, and optimize your storage at scale.

S3 Storage Lens

S3 Storage Lens delivers organization-wide visibility into object storage usage, activity trends, and makes actionable recommendations to improve cost-efficiency and apply data protection best practices. S3 Storage Lens is the first cloud storage analytics solution to provide a single view of object storage usage and activity across hundreds, or even thousands, of accounts in an organization, with drill-downs to generate insights at the account, bucket, or even prefix level. Drawing from more than 14 years of experience helping customers optimize their storage, S3 Storage Lens analyzes organization-wide metrics to deliver contextual recommendations to find ways to reduce storage costs and apply best practices on data protection.

Amazon S3 is used to store large shared data sets across tens to hundreds of accounts and buckets, multiple regions, and thousands of prefixes. With S3 Storage Lens, you can easily understand, analyze, and optimize storage with interactive dashboards to aggregate data for your entire organization, specific accounts, regions, or buckets. S3 Storage Lens delivers more than 30 individual metrics on S3 storage usage and activity for all accounts in your organization. These metrics are available in the S3 console to visualize storage usage and activity trends in a dashboard, with contextual recommendations that make it easy to take immediate action.

How S3 Storage Lens works

How S3 Storage Lens works

After you activate S3 Storage Lens in the S3 Console, you will receive an interactive dashboard containing pre-configured views to visualize storage usage and activity trends, with contextual recommendations that make it easy to take action. You can also export metrics in CSV or Parquet format to an S3 bucket. You can use the summary view, cost efficiency view, or the data protection view to see metrics related to your intended use case. In addition to the dashboard in the S3 console, you can export metrics in CSV or Parquet format to an S3 bucket of their choice for further use. You can also enable access of Storage Lens metrics via Amazon CloudWatch, or use the CloudWatch API to send metrics to integrated partners. To learn more about S3 Storage Lens, read the documentation.

Use cases:

  • Summary insights
  • Outlier detection
  • Data protection
  • Cost optimization

Amazon S3 Storage Class Analysis

S3 Storage Class Analysis enables you to monitor access patterns across objects to help you decide when to transition data to the right storage class to optimize costs. You can then use this information to configure an S3 Lifecycle policy that makes the data transfer. S3 Lifecycle policies can also be used to expire objects at the end of their lifecycles. If you observe that your access patterns are frequently changing, you can use S3 Intelligent-Tiering which automatically moves your data based on changing access patterns between 4 access tiers that include a frequent access tier, a lower-cost infrequent access tier, an archive access tier and a deep archive access tier for cost savings. Storage Class Analysis also provides daily visualizations of your storage usage on the Amazon Web Services Management Console that you can export to an S3 bucket to analyze using business intelligence tools of your choice such as Amazon QuickSight. To learn more about Storage Class Analysis, visit the Storage Class Analysis documentation guide.

Use cases:

  • Cost optimization

Amazon S3 Inventory

Amazon S3 Inventory is a feature that helps you manage your storage. S3 Inventory provides a report of your objects and their corresponding metadata on a daily or weekly basis for an S3 bucket or prefix. This report can be used to help meet business, compliance, and regulatory needs by verifying the encryption, and replication status of your objects. You can also use S3 Inventory reports to speed up business workflows and big data jobs.

S3 Inventory reports include corresponding metadata such as bucket names, key names, last modification dates, object size, storage class, replication or encryption status among other properties. S3 inventory reports can be delivered daily or weekly and can be encrypted to protect sensitive data. It can also be configured to deliver multiple reports that deliver different types of metadata that are relevant to your specific needs. It provides a list of your objects and their corresponding metadata for an S3 bucket or a shared prefix, which can be used to perform object-level analysis of your storage.

Use cases:

  • Audit encryption status of objects for security and compliance
  • Track the replication status of objects to another Amazon Web Services Region
  • Speed up business workflows and big data jobs
  • Identify target objects for S3 Batch Operation