Automate application deployment to IoT devices using Amazon Web Services IoT Device Management

Introduction

The Internet of Things (IoT) industry is trending towards devices that are compatible with the latest standards, interfaces, and protocols. To remain competitive, device manufacturers have to release new features, perform system updates, and deploy security patches in a timely manner. Software applications have long been using automated continuous integration and delivery (CI/CD) pipelines to manage the delivery of these updates. However, automating deployments for IoT devices is challenging for a number of reasons, such as their remote location, intermittent connectivity, network bandwidth, and scale.

In this blog post, we present a CI/ID pipeline to continuously integrate and deploy an application to an IoT device. The pipeline automates application deployment and reduces the release time for IoT applications. The pipeline can also be used by IoT device operators to perform management tasks at scale, such as firmware updates, command execution, and security patch deployment.

Architecture overview

The architecture uses Amazon Web Services IoT Core to handle the connectivity and authentication of the IoT device. The CI/CD pipeline is setup using Amazon Web Services CodePipeline . The pipeline fetches source code from an Amazon Web Services CodeCommit repository and uses Amazon Web Services CodeBuild for the build and deployment steps. The Amazon Web Services IoT Jobs feature of Amazon Web Services IoT Device Management is then used to manage application deployment to the IoT device. The Jobs feature enables the execution of remote operations on one or more devices connected to Amazon Web Services IoT Core. Amazon Web Services IoT Device Management takes care of scheduling, retrying, and reporting the status of remote operations. The IoT device is responsible for subscribing to job notifications from Amazon Web Services IoT.

Figure 1.0 CI/CD pipeline for IoT devices

Prerequisites

• Install Amazon Web Services CLI to the local machine, follow the installation instructions .
• Setup the git and git-remote-code-commit utility , on the local machine.
• An Amazon Simple Storage Service (Amazon S3) artifacts bucket with versioning enabled to store CI/CD artifacts.
• An IoT device connected to Amazon Web Services IoT Core as a thing . For this post, we will use a virtual IoT device, by installing the Amazon Web Services IoT Device Client software on an Amazon Elastic Compute Cloud (Amazon EC2) instance . The getting started with Amazon Web Services IoT workshop , provides detailed guidance on setting up a virtual IoT device.

Walkthrough

In this walkthrough, we’ll setup a CI/CD pipeline that will create an IoT Job, which deploys an application to an IoT device. The application is a simple bash script, which upon execution, creates a file with the current timestamp on the IoT device e.g., 2023-05-0410-48.log .

Create an Amazon Web Services CodeCommit repository (Amazon Web Services Console)

• Create a code repository e.g., IoTApplicationRepo, where the application source code will be stored (For detailed instructions, see Create an Amazon Web Services CodeCommit repository ).
• Fetch the repository to the local machine, using the following command: git clone codecommit://IoTApplicationRepo IoTApplicationRepo

Create a CI/CD pipeline (Amazon Web Services Console)

Create a CI/CD pipeline using Amazon Web Services CodePipeline with three stages i.e., source, build, and deploy.

Step 1: Create and name the pipeline

• Go to the CodePipeline console , and choose Create pipeline .

Fig 1.1 – Creating pipeline

• In Step 1 : enter a Pipeline name . Under Advanced settings choose Custom location and select the artifacts Amazon S3 bucket (from prerequisites).

Fig 1.2 – Select Amazon Web Services Managed Key

• Choose Next .

Step 2: Create the source stage

• In Step 2: Add source stage , under Source provider , choose Amazon Web Services CodeCommit .
• In Repository name , choose the Amazon Web Services CodeCommit repository created earlier e.g., IoTApplicationRepo . In Branch name , enter master . Choose Next .

Step 3: Create the build stage

• In Step 3: Add build stage . From Build provider , choose Amazon Web Services CodeBuild and select Create Project . This will open a new window.

Fig 1.3 – Build stage

• Under Create build project , enter the Project name .
• Under Environment in Operating system choose Amazon Linux 2 .
• For Runtime(s) , choose standard and for Image , select aws/codebuild/amazonlinux2-x86_64-standard:4.0 (or use latest image of Amazon Linux 2) .
• In Additional configuration , under Environment variables . Create the variable ‘ bucket ’ and under value enter the artifacts bucket name.
• In Buildspec name – optional , enter build.yaml .
• Then select Continue to CodePipeline .
• Choose Next .
• Amazon Web Services CodeBuild automatically creates an IAM service role named codebuild-<project name>-service-role . This role needs additional permissions to upload build artifacts to the artifacts bucket.
• Go to the Roles hub of the IAM console and select this role.
• Choose Add permissions and select Create inline policy

Fig 1.4 – Create inline policy

• In Create policy , choose JSON and replace the contents with the following policy and replace your_S3_bucket , with the artifacts bucket name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::your_S3_bucket/*"
        }
    ]
}

• Choose Review Policy .
• In Create policy , enter a Name and select Create policy
• Navigate back to the Create new pipeline .

Step 4: Skip the deployment stage

• In Step 4: Add deploy stage , choose Skip deploy stage . We will use Amazon Web Services CodeBuild to deploy the application to the IoT device, in Step 8.

Fig 1.5 – Skip deploy stage

Step 5: Review Pipeline

• In Step 5: Review page, review the pipeline configuration, and then choose Create pipeline to create the pipeline.
• The CI/CD pipeline will start executing automatically, choose Stop execution -> Stop , to stop the pipeline. As the pipeline is not complete yet.

Step 6: Create the presigned Amazon Identity and Access Management (IAM) role

The CI/CD pipeline will upload the application file to the artifact’s Amazon S3 bucket. The Amazon S3 bucket objects are private, so Amazon Web Services IoT will automatically generate presigned URLs , which will grant the IoT device time-limited permission to download the application files (see Amazon Web Services IoT Device Management docs for more details).

To generate presigned URLs, Amazon Web Services IoT requires an IAM role that has permissions to download the objects from the artifacts bucket.

• Go to the Roles hub of the IAM console and choose Create role .
• In Select trusted entity , under Use cases for other Amazon Web Services services choose IoT , and select Next .

Fig 1.6 – Select Use case (IoT)

• In Add permissions , select Next .
• In Role details , enter a Role name and choose Create role .
• Select the newly created role and choose Add permissions -> Create inline policy
• In Create policy , choose JSON and replace the contents with the following policy and replace your_S3_bucket , with the CI/CD artifacts bucket name. Choose Review Policy .

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::your_S3_bucket/*"
        }
    ]
}

• In Create policy , enter a Name and choose Create policy

Step 7: Create the deploy stage

• Go to the CodePipeline console , select the created pipeline and choose Edit .
• Choose Add Stage , which appears after the Edit: Build stage .

Fig 1.7 – Select Add stage

• In S tage name , enter Deploy and choose Add stage .
• Choose Add Action Group , in Action Name enter Deploy and in Action Provider choose Amazon Web Services CodeBuild .
• Under Input Artifact choose BuildArtifact .
• In Project name, select Create Project .
  • Under Create build project , enter the Project name .
  • Under Environment in Operating system choose Amazon Linux 2 .
  • For Runtime(s) , choose standard and for Image , select aws/codebuild/amazonlinux2-x86_64-standard:4.0 (or latest version of it) .
  • In Additional configuration , under Environment variables .
    • Create the variable IOT_ARN and under value put the IoT thing’s ARN (from prerequisites).
    • Create the variable ROLE and enter the ARN of the presign IAM role, from Step 6.
    • Finally, create the variable BUCKET and enter the bucket name of the CI/CD artifacts bucket.
• In Buildspec name – optional , enter deploy.yaml.
• Select Continue to CodePipeline and choose Done .
• Select Save .

Step 8: Add permissions to the deploy stage IAM role

• Amazon Web Services CodeBuild deploy stage will automatically create an IAM service role named codebuild-<project name>-service-role. This role needs additional permissions to interact with Amazon Web Services IoT.
• Go to the Roles hub of the IAM console and choose this role.
• Choose Add permissions -> Create inline policy
• In Create policy , choose JSON and replace the contents with the following policy and replace:
  • Replace presign_role_arn with the URL of the presign IAM Role ARN from step 6.
  • Replace region and account , with the Amazon Web Services region and Amazon Web Services account ID.
  • Replace IoT_device_name , with the IoT device name
  • Replace your_S3_bucket , with the CI/CD artifacts bucket name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PassRole",
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:PassRole"
            ],
            "Resource": "presign_role_arn"
        },
        {
            "Sid": "IoTJobPermissions",
            "Effect": "Allow",
            "Action": [
                "iot:CreateJob"
            ],
            "Resource": [
                "arn:aws:iot:region:account_id:job/iot-device-job*",
                "arn:aws:iot:region:account_id:thing/IoT_device_name"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3::: your_S3_bucket/*"
        }
    ]
}

• Choose Review Policy . In Create policy , enter a Name and choose Create policy

Deploy the application to the IoT device

The CI/CD pipeline is complete. To trigger the pipeline, we can upload files to the repository.

Upload files to the repository (Amazon Web Services CLI)

Add four files to the repository, the application code file application.sh , the Amazon Web Services CodeBuild buildspec files build.yaml & deploy.yaml , and the IoT Job document .

• Navigate to the local repository directory and create the application script file application.sh with the following content:

#!/bin/bash
set -x
time_stamp=$(date "+%Y-%m-%d%H-%M")
filename="$time_stamp.log"
touch /home/ubuntu/$filename
sudo chmod a+rw /home/ubuntu/$filename
echo "Installing Firmware Version 1 at $time_stamp" >> /home/ubuntu/$filename

The script creates a new file with the current timestamp as file name.

• Create an Amazon Web Services CodeBuild build.yaml file, with the following content:

version: 0.2

phases:
  build:
    commands:
      - echo IoT Application Build started on `date`
      - sed -i "s/bucket-name/$bucket/g" deploy_instructions.json
artifacts:
  files:
    - '*.sh'
    - 'deploy.yaml'
    - 'deploy_instructions.json'

The build step inserts the Amazon S3 bucket name to the job document.

• Create an Amazon Web Services CodeBuild deploy.yaml file, with the following content:

version: 0.2

phases:
  build:
    commands:
      - echo "Deploying application to the IoT device"
      - time_stamp=$(date +%Y%m%d_%H%M%S)
      - aws s3 cp deploy_instructions.json s3://$BUCKET/job/deploy_instructions.json
      - aws s3 cp application.sh s3://$BUCKET/binaries/application.sh
      - aws iot create-job --job-id iot-device-job$time_stamp --targets $IOT_ARN --document-source  s3://$BUCKET/job/deploy_instructions.json --presigned-url-config roleArn=$ROLE,expiresInSec=60

The deploy step creates an Amazon Web Services IoT Job .

• Create a Amazon Web Services IoT job document deploy_instructions.json , with the following content:

{
  "version": "1.0",
  "steps": [
    {
      "action": {
        "name": "Deploy Application",
        "type": "runHandler",
        "input": {
          "handler": "bash",
          "args": [
            "wget -O /home/ubuntu/application.sh '${aws:iot:s3-presigned-url:https://s3.amazonaws.com/bucket-name/binaries/application.sh}' && sudo chmod u+x /home/ubuntu/application.sh && sudo /home/ubuntu/application.sh" 
          ],
          "path": "/bin"
        },
        "runAsUser": "-c"
      }
    }
  ]
}

The document has commands to download and execute the application script.

• Upload the files to the repository

git add .
git commit -m “adding files for the IoT application”
git push

The file upload will trigger the CI/CD pipeline. The deployment status can be seen from the Amazon Web Services IoT console under Remote actions , in Jobs .

Fig 1.8 – Jobs screen

Cleaning up

Remember to delete all resources that have been created in this walkthrough, to avoid incurring future costs.

Conclusion

In this post, we have presented automated application deployment guidance for IoT devices. This guidance allows you to focus on developing features, while Amazon Web Services IoT services take care of the deployment challenges like intermittent connectivity, authentication, and scalability. The automation saves development and testing time, which helps with releasing new features, updates, and security patches in a timely manner.

The next step would be to expand the pipeline with more stages like testing and monitoring. For further reading on how Amazon Web Services IoT Device Management handles deployments, refer to the Jobs documentation .

About the Authors

Asad Syed is a DevOps Architect with Amazon Web Services ProServe and is based out of Berlin. He works with global customers to modernize their applications & processes. Asad has a keen interest in helping customers utilize Amazon Web Services IoT services to achieve their business outcomes. Outside of work, he enjoys playing volleyball, bouldering and hiking.

Syed Rehan is a Sr. Global IoT Cybersecurity Specialist at Amazon Web Services (Amazon Web Services) working within Amazon Web Services IoT Service team and is based out of London. He is covering global span of customers working with security specialists, developers and decision makers to drive the adoption of Amazon Web Services IoT services. Syed has in-depth knowledge of cybersecurity, IoT and cloud and works in this role with global customers ranging from start-up to enterprises to enable them to build IoT solutions with the Amazon Web Services Eco system.