What does this Amazon Web Services Solution do?
This solution deploys a platform that uses machine learning (ML) technology to help you analyze data from smart utility meters.
This solution is for utility companies and other organizations that are looking to gain insights from smart-meter data. The data comes from meter data management (MDM) or similar systems. Insights include unusual energy usage, energy-usage forecasts, and meter-outage details.
Amazon Web Services Solution overview
Utility Meter Analytics Platform enables you to:
1. Store, clean, aggregate, query data retreived from Meter Data Management System (MDMS) with a Data Lake built with Amazon S3, Amazon Redshift, Amazon Glue, etc.
2. Construct and automatically deploy machine learning(ML) models with Amazon Step Functions and Amazon SageMaker.
3. Query historical abnormal meter usage and malfunctioning devices, aggregate meter data and predict future meter usages, etc via RESTFul API calls, with an serverless archiecture built with Amazon API Gateway and Lambda.
Except for Amazon Redshift, this is a serverless architecture, which helps you save cost.
Architecture Description
The solution sets up the following resources:
1. A virtual private cloud (Amazon VPC) configured with a private subnet, according to Amazon Web Services best practices, or your can bring your existing virtual network.
2. An Amazon Redshift cluster in private subnet, that stores business data for analysis, visualization, and dashboards.
3. An extract, transform, load (ETL) pipeline, including:
- Amazon Simple Storage Service (Amazon S3) buckets to store data from an MDM or similar system. Raw meter data, clean data, and partitioned business data are stored in separate S3 buckets.
- An Amazon Glue workflow including:
- crawlers, jobs, and triggers to crawl, transform, and convert incoming raw meter data into clean data in the desired format and partitioned business data.
- Amazon Glue Data Catalog to store metadata and source information about the meter data.
4. An ML pipeline, including:
- Two Amazon Step Functions workflows:
- Model training, which uses the partitioned business data to build a Machine Learning model.
- Batch processing, which uses the partitioned business data and the data from the model as a basis for forecasting.
- Amazon S3 for storing the processed data.
- Amazon SageMaker for real-time forecasting of energy usage.
- A Jupyter notebook with sample code to perform data science and data visualization.
5. Amazon Lambda to query the partitioned business data through Amazon Athena or invoke SageMaker to provide application programming interface (API) query results.
6. Amazon API Gateway to deliver API query for historical abnormal energy surge, malfunctioning device, aggregated device data and prediction of future energy usage.
Utility Meter Data Analytics Platform
Version 1.0.0
Last updated: 01/2021
Author: Amazon Web Services
Estimated deployment time: 30 min
Features
Data Lake
Model Prediction
Serverless Architecture
Browse our portfolio of Amazon Web Services -built solutions to common architectural problems.
Find Amazon Web Services certified consulting and technology partners to help you get started.
Start to Build for Free with Amazon Web Services
Start to Build for Free with Amazon Web Services
