Amazon ECS now supports GPU sharing with NVIDIA Multi-Process Service (MPS)
Amazon Elastic Container Service now supports GPU sharing with NVIDIA Multi-Process Service (MPS), enabling you to run multiple ECS tasks concurrently on a single NVIDIA GPU. GPU sharing helps you increase GPU utilization and reduce infrastructure costs for workloads that use only part of a GPU's capacity, such as small-model inference, embedding generation, and multi-model serving.
With GPU sharing, you can pack more workloads onto each GPU instance without changing your application code. You define the GPU memory each container needs and can optionally set a maximum share of GPU compute, and Amazon ECS places tasks on GPUs with available capacity as your services scale and deploy. GPU sharing is supported on both Amazon ECS Managed Instances and Amazon ECS on EC2. On ECS Managed Instances, Amazon ECS also monitors the MPS daemon that enables sharing on each instance. If the daemon stops or becomes unhealthy, Amazon ECS automatically replaces the instance and moves your tasks to healthy capacity, so your GPU workloads keep running without manual intervention.
To get started, use the Console, CLI, SDKs, or infrastructure-as-code tools to create a capacity provider with NVIDIA GPU instances, set the MPS sharing strategy and GPU memory in the container definition of your task definition, and create a service on that capacity provider. This capability is available in the Amazon Web Services China (Beijing) Region, operated by Sinnet and the Amazon Web Services China (Ningxia) Region, operated by NWCD. To learn more, visit the Amazon ECS GPU sharing documentation.