What does this Amazon Web Services Solution do?

This solution deploys a platform that uses machine learning (ML) technology to help you analyze data from smart utility meters.

This solution is for utility companies and other organizations that are looking to gain insights from smart-meter data. The data comes from meter data management (MDM) or similar systems. Insights include unusual energy usage, energy-usage forecasts, and meter-outage details.

Amazon Web Services Solution overview

Utility Meter Analytics Platform enables you to:

1. Store, clean, aggregate, query data retreived from Meter Data Management System (MDMS) with a Data Lake built with Amazon S3, Amazon Redshift, Amazon Glue, etc.

2. Construct and automatically deploy machine learning(ML) models with Amazon Step Functions and Amazon SageMaker.

3. Query historical abnormal meter usage and malfunctioning devices, aggregate meter data and predict future meter usages, etc via RESTFul API calls, with an serverless archiecture built with Amazon API Gateway and Lambda.

Except for Amazon Redshift, this is a serverless architecture, which helps you save cost.

Architecture of Serverless Image Handler

Architecture Description

The solution sets up the following resources:

1. A virtual private cloud (Amazon VPC) configured with a private subnet, according to Amazon Web Services best practices, or your can bring your existing virtual network.

2. An Amazon Redshift cluster in private subnet, that stores business data for analysis, visualization, and dashboards.

3. An extract, transform, load (ETL) pipeline, including:

  • Amazon Simple Storage Service (Amazon S3) buckets to store data from an MDM or similar system. Raw meter data, clean data, and partitioned business data are stored in separate S3 buckets.
  • An Amazon Glue workflow including: 
    • crawlers, jobs, and triggers to crawl, transform, and convert incoming raw meter data into clean data in the desired format and partitioned business data.
    • Amazon Glue Data Catalog to store metadata and source information about the meter data.

4. An ML pipeline, including:

  • Two Amazon Step Functions workflows:
    • Model training, which uses the partitioned business data to build a Machine Learning model.
    • Batch processing, which uses the partitioned business data and the data from the model as a basis for forecasting.
  • Amazon S3 for storing the processed data.
  • Amazon SageMaker for real-time forecasting of energy usage.
  • A Jupyter notebook with sample code to perform data science and data visualization.

5. Amazon Lambda to query the partitioned business data through Amazon Athena or invoke SageMaker to provide application programming interface (API) query results.

6. Amazon API Gateway to deliver API query for historical abnormal energy surge, malfunctioning device, aggregated device data and prediction of future energy usage.

Utility Meter Data Analytics Platform

Version 1.0.0
Last updated: 01/2021
Author: Amazon Web Services

Estimated deployment time: 30 min

Source code 

Features

Data Lake

Data lake included in this solution provides 1. safe, reliable, industry-compliant, nearly unlimited storage services; 2. highly automated process of data cleaning, transforming and partitioning; 3. data warehouse for energy analysis report creation and user historical data aggregation.

Model Prediction

Energy prediction model workflows built with Amazon Step Functions, Amazon SageMaker provides a fully automated visualized pipeline for data ingestion, model training, model storage, inference endpoint creation, model hyperparameter tuning, and model retraining.

Serverless Architecture

Serverless architecture allows you to use highly scalable RESTful API. Users can perform future energy usage forecasts, historical energy usage queries, and abnormal electricity meter status queries by calling automatically generated API service endpoints. Serverless architecture can be easily integrated with your existing systems.
Standard Product Icons (Features) Squid Ink
Explore all Amazon Web Services Solutions

Browse our portfolio of Amazon Web Services -built solutions to common architectural problems.

Learn more 
Next-Steps-Icon_Find-a-Partner-B
Find a Partner

Find Amazon Web Services certified consulting and technology partners to help you get started.

Learn more 
Standard Product Icons (Start Building) Squid Ink
Start building in the console

Sign-up and start exploring our services.

Get started