What does this Amazon Web Services Solution do?

Data Transfer Hub is a secure, reliable, scalable, and trackable solution offers a unified user experience that allows you to easily create and manage different types of transfer tasks from different sources to Amazon Web Service cloud-native services. You can launch this solution and start to transfer data within a few minutes.

With Data Transfer Hub, you can perform any of the following tasks:

1. Transfer Amazon S3 objects between Amazon Web Services regions.

2. Transfer data from other cloud providers' object storage services (including Alibaba Cloud OSS, Tencent COS, Qiniu Kodo, and other Amazon S3 Compatible Cloud Storage) to Amazon S3.

3. Transfer Amazon ECR Images between Amazon Web Services regions.

4. Transfer container images from public container registry (e.g., Docker Hub, Google gcr.io, Red Hat Quay.io) to Amazon ECR.

Amazon Web Services Solution overview

The diagram below presents the architecture of Data Transfer Hub, you can deploy in minutes using the solution's implementation guide and the associated Amazon CloudFormation template.

Architecture of Serverless Image Handler

Architecture Description

The Amazon CloudFormation template automatically deploys and configures a serverless architecture with services like Amazon AppSync, Amazon DynamoDB, Amazon ECS Fargate, Amazon Lambda, Amazon Step Functions and etc. The solution comes with a web portal hosted on Amazon Simple Storage Service (Amazon S3), and made available through Amazon CloudFront. You can authenticate against Amazon Cognito User Pool or OpenID Connect (OIDC) provider. If you deploy this solution in Amazon Web Services (Beijing) Region operated by Sinnet or Amazon Web Services (Ningxia) Region operated by NWCD, you are required to file for ICP Recordal or ICP License before you can access the web frontend, please refer to deployment guide for more details.

The web portal serves as a centralized place to create and manage the data transfer jobs. Each data type (e.g., Amazon S3) is a plugin for Data Transfer Hub, and packaged as an Amazon CloudFormation template hosted in a centralized S3 Bucket. When the you create a transfer task, a Lambda will trigger CloudFormation to start rolling, and all the state of tasks is stored in DynamoDB tables. According to this design, you can run each plugin independently.  Moreover, Developer can easily contribute CloudFormation packaged transfer plugin to Data Transfer Hub. 

Data Transfer Hub

Version 2.0.0
Last updated: 07/2021
Author: Amazon Web Services 

Estimated deployment time: 15 min

Source code 

Features

Friendly user interface

The solution offers intuitive UI allows you to easily create and manage different types of tasks, transfer data from different sources to Amazon Web Services Cloud.

Support different sources

Support data transfer from Alibaba Cloud OSS, Tencent COS, Qiniu Kodo and Amazon S3 Compatible Cloud Storage to Amazon S3. Transfer from Docker Hub, Google gcr.io, Red Hat Quay.io to Amazon ECR.

Serverless architecture

Serverless is entirely on-demand and pay-as-you-go, leading to lower costs for customers.
Standard Product Icons (Features) Squid Ink
Explore all Amazon Web Services Solutions

Browse our portfolio of Amazon Web Services -built solutions to common architectural problems.

Learn more 
Next-Steps-Icon_Find-a-Partner-B
Find a Partner

Find Amazon Web Services certified consulting and technology partners to help you get started.

Learn more 
Standard Product Icons (Start Building) Squid Ink
Start building in the console

Sign-up and start exploring our services.

Get started