AWS IoT SiteWise with raspberry pi Temperature Sensor DS18B20 Part 0

These are the steps to get a raspberry pi setup with IoT greengrass

This is mainly a copy of this post  https://devopstar.com/2019/10/07/aws-iot-greengrass-cloudformation-raspberry-pi/ with a few comments

AWS IoT Greengrass is a managed service that allows you to easily manage and communicate with edge devices. It sells itself as a central way to deploy to huge fleets of devices in a repeatable way.

The irony of this is that deploying the foundation of AWS Greengrass can be quite complex and messy. In the official guide for Greengrass Core, AWS recommends using the Easy Creation button. This is perfectly fine for testing purposes, however if you are deploying production workloads then repeatability is really key.

CloudFormation resources are available for AWS Greengrass however the setup process can be rather complex when your device isn’t sitting in AWS. This blog post is a continuation of an existing post on the AWS Blog called Automating AWS IoT Greengrass Setup With AWS CloudFormation. This post works fine for setups where EC2 is used, however there’s a gap for people who deploy AWS IoT Greengrass Core to devices in the field.

Prerequisites

Setting up a Raspberry Pi

For our deployment we’ll be using a Raspberry Pi Zero W, however feel free to use any device you see fit. If you’re working on a Raspberry Pi as well you can follow the next steps to setup a new image.

Start of by downloading the Rasbian lite image. This can be done using wget if you are that way inclined.

wget https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2019-09-30/2019-09-26-raspbian-buster-lite.zip

Burn Image

Next we need to burn the image with Etcher or a similar imaging software.

Etcher Burn Raspbian

Once the MicroSD card is flashed with Rasbian continue to the next step.

Setup WiFi

Setting up WiFi and SSH access in a headless fashion is really useful if you haven’t got a monitor around. It’s already very convenient for bulk deployments.

This might be different based on your OS, the general idea is that you will need to create:

  • An empty file called ssh in the /boot/ partition
  • A file called wpa_supplicant.conf in the same partition
touch /Volumes/boot/ssh
touch /Volumes/boot/wpa_supplicant.conf
nano /Volumes/boot/wpa_supplicant.conf

# Add the following config
country=AU
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
    ssid="NETWORK-NAME"
    psk="NETWORK-PASSWORD"
}

NOTEBe sure to update your WiFi networking information. If you are using Ethernet you can skip the wpa_supplicant.conf step.

Connect to Pi

Plugin the Raspberry Pi and give it a couple minutes to first boot up. You can then SSH into your raspberry pi by either using Putty on Windows or ssh on MacOS & Unix.

ssh-keygen -R raspberrypi.local
ssh pi@raspberrypi.local
# password: raspberry

Once connected, we update the system with the latest packages and install Python Pip for later use with the Greengrass SDK.

sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install python-pip -y

Prepare for Greengrass

Now we need to perform a set of steps that are best described in the official AWS setup guide for the Raspberry Pi These steps are highly recommended if you want to ensure you don’t run into problems later down the track when running Greengrass Core.

Start off by adding a new system user and group that Greengrass will use when executing. This helps separate the permissions away from our default user.

# Create user and group
sudo adduser --system ggc_user
sudo addgroup --system ggc_group

Next we can enable hardlink and softlink (symlink) protection by editing 98-rpi.conf.

sudo nano /etc/sysctl.d/98-rpi.conf

Add the following lines to the file:

fs.protected_hardlinks = 1
fs.protected_symlinks = 1

Reboot the pi, and when logged back in confirm the settings were set.

sudo reboot
sudo sysctl -a 2> /dev/null | grep fs.protected

Edit your command line /boot/cmdline.txt file to enable and mount memory cgroups. Append the following to the single line.

cgroup_enable=memory cgroup_memory=1

The file should look like the following (with some changes to partition for example).

console=serial0,115200 console=tty1 root=PARTUUID=6c586e13-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_enable=memory cgroup_memory=1

Finally reboot the Pi again to confirm everything comes up okay.

sudo reboot

Greengrass Dependency Checker

To confirm that all prerequisites are fulfilled we’ll use the Greengrass Dependency Checker. Run the following commands from the home directory of the Raspberry Pi we SSH’d into.

mkdir /home/pi/Downloads && cd /home/pi/Downloads
mkdir greengrass-dependency-checker-GGCv1.9.x && cd greengrass-dependency-checker-GGCv1.9.x
wget https://github.com/aws-samples/aws-greengrass-samples/raw/master/greengrass-dependency-checker-GGCv1.9.x.zip
unzip greengrass-dependency-checker-GGCv1.9.x.zip && cd greengrass-dependency-checker-GGCv1.9.x
sudo modprobe configs
sudo ./check_ggc_dependencies | more

There will likely be a couple warnings when running:

  • It looks like the kernel uses ‘systemd’ as the init process.
  • Could not find the binary ‘nodejs8.10’.
  • Could not find the binary ‘java8’.

These are safe to ignore for now, the first warning should be kept in mind however as we’ll resolve it later.

Greengrass Setup

The deployment of the Greengrass group can be done through the UI or CloudFormation. If you have an interest in how to do it by the UI, please check the AWS guide on this, however I would advise against it for this tutorial.

CloudFormation is a much better option as it gives us flexibility for re-configuration later on through code changes.

CloudFormation Greengrass Deploy

To deploy using CloudFormation, ensure AWS CLI is configured on the system and proceed to run the command below:

( cd into cd aws-greengrass-cfn )

aws cloudformation create-stack \
    --stack-name "devopstar-rpi-gg-core" \
    --template-body file://aws/greengrass.yaml \
    --region "us-east-1" \
    --capabilities CAPABILITY_IAM

The deployment will take a couple minutes and can be monitored via the CloudFormation portal. A number of resources will be created and managed for you:

Greengrass CloudFormation Deployment

Next we’re going to build the tar.gz bundle with our certificates and Greengrass configuration. I’ve written a helper script for this in the aws folder of the GitHub repository.

The general idea is that the certificate details need to be zipped up and deployed to the Raspberry Pi, so we export this information from the CloudFormation stack using the following:

certificateId=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
    --query 'Stacks[0].Outputs[?OutputKey==`CertificateId`].OutputValue' \
    --region ${AWS_REGION} \
    --output text)

certificatePem=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
    --query 'Stacks[0].Outputs[?OutputKey==`CertificatePem`].OutputValue' \
    --region ${AWS_REGION} \
    --output text)

certificatePrivateKey=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
    --query 'Stacks[0].Outputs[?OutputKey==`CertificatePrivateKey`].OutputValue' \
    --region ${AWS_REGION} \
    --output text)

iotEndpoint=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
    --query 'Stacks[0].Outputs[?OutputKey==`IoTEndpoint`].OutputValue' \
    --region ${AWS_REGION} \
    --output text)

This information is used to save the various certificates.

echo -n "${certificatePem}" > certs/${certificateId}.pem
echo -n "${certificatePrivateKey}" > certs/${certificateId}.key

Then a config.json file is generated using all this information that will be used shortly on the Pi

{
    "coreThing" : {
        "caPath" : "root.ca.pem",
        "certPath" : "${certificateId}.pem",
        "keyPath" : "${certificateId}.key",
        "thingArn" : "arn:aws:iot:${AWS_REGION}:${AWS_ACCOUNT_ID}:thing/${CORE_NAME}_Core",
        "iotHost" : "${iotEndpoint}",
        "ggHost" : "greengrass-ats.iot.${AWS_REGION}.amazonaws.com"
    },
    "runtime" : {
        "cgroup" : {
        "useSystemd" : "yes"
        }
    },
    "managedRespawn" : false,
    "crypto" : {
        "principals" : {
        "SecretsManager" : {
            "privateKeyPath" : "file:///greengrass/certs/${certificateId}.key"
        },
        "IoTCertificate" : {
            "privateKeyPath" : "file:///greengrass/certs/${certificateId}.key",
            "certificatePath" : "file:///greengrass/certs/${certificateId}.pem"
        }
        },
        "caPath" : "file:///greengrass/certs/root.ca.pem"
    }
}

Finally all the files are zipped up and saved. To perform all these steps in one easy hit, run the following:

NOTEThis script assumes that you deployed to us-east-1 and left the stack name as devopstar-rpi-gg-core. If you changed that portion, replace references to devopstar-rpi-gg-core with your own stack name.

The script uses jq so you need to install it!

sudo apt-get install jq

cd aws
./greengrass.sh

Deployment Files to Raspberry Pi

In this step we’ll download AWS IoT Greengrass Core then copy it to our Raspberry Pi. We will also copy our previously packages certificates and configuration at the same time.

Download Greengrass Core

Depending on what version your raspberry pi is will determine what version we’ll need. Run the following command to get your architecture.

uname -a
# Linux raspberrypi 4.19.75+ #1270 Tue Sep 24 18:38:54 BST 2019 armv6l GNU/Linux

In this case my architecture is armv6l so from the Greengrass download page download the distribution for Rasbian that matches this.

# Download the `armv6l` tar
wget https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.9.3/greengrass-linux-armv6l-1.9.3.tar.gz
My case was 
wget https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.10.0/greengrass-linux-armv7l-1.10.0.tar.gz

Now that we have the files we need, copy them all to the Raspberry Pi.

scp greengrass-linux-armv6l-1.9.3.tar.gz pi@raspberrypi.local:/home/pi
scp aws/*-setup.tar.gz pi@raspberrypi.local:/home/pi
scp aws/greengrass-service.sh pi@raspberrypi.local:/home/pi
Extract Greengrass Files

SSH back onto the Raspberry Pi and extract the two file bundles we just downloaded by running the following.

sudo tar -xzvf greengrass-{ARCHITECTURE}-1.9.3.tar.gz -C /
sudo tar -xzvf {CERTIFICATE_ID}-setup.tar.gz -C /greengrass

Confirm everything copied across correctly by running the following

ls -al /greengrass
# drwxr-xr-x  6 root root 4096 Aug 31 04:40 .
# drwxr-xr-x 22 root root 4096 Aug 31 04:40 ..
# drwxrwxr-x  2 pi   pi   4096 Oct  6 18:01 certs
# drwxrwxr-x  2 pi   pi   4096 Oct  6 18:01 config
# drwxr-xr-x  3 root root 4096 Aug 31 04:40 ggc
# drwxr-xr-x  3 root root 4096 Aug 31 04:40 ota

Next we also need to download the ATS root CA certificate into /greengrass/certs

cd /greengrass/certs/
sudo wget -O root.ca.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem

Greengrass Start

With all the files in the right places on the device, it’s now time to try to start the service

cd /greengrass/ggc/core/
sudo ./greengrassd start

You should see a Greengrass successfully started message. Take note of the PID of the process so we can view the status of the daemon.

ps aux | grep PID_NUMBER
# root       761  2.4  3.0 854472 13508 pts/0    Sl   18:18   0:01 /greengrass/ggc/packages/1.9.3/bin/daemon -core-dir /greengrass/ggc/packages/1.9.3 -greengrassdPid 757

To setup Greengrass on startup, run the following (not this will reboot your Raspberry Pi). This assumes that you are using systemd.

sudo ./greengrass-service.sh
Greengrass SDK

If you don’t want to bundle the greengrasssdk in with your application, you can install it globally.

sudo pip install greengrasssdk

If you choose not to go down this route you can manually bundle greengrasssdk in with your lambda deployments in the future.

Greengrass Deploy [CLI]

To create our first deployment we first need to retrieve our Greengrass Group ID

aws greengrass list-groups
# {
#     "Groups": [
#         {
#             "Arn": "arn:aws:greengrass:us-east-1:123456789012:/greengrass/groups/..........",
#             "CreationTimestamp": "2019-10-06T10:31:53.950Z",
#             "Id": "41752ff2-54e5-49e9-8751-d489e3e6fa1f",
#             "LastUpdatedTimestamp": "2019-10-06T10:31:53.950Z",
#             "LatestVersion": "1c578332-44f7-411b-9ebb-1139fa1e453a",
#             "LatestVersionArn": "arn:aws:greengrass:us-east-1:123456789012:/greengrass/groups/..........",
#             "Name": "gg_cfn"
#         }
#     ]
# }

In my case the group ID and version can be seen above, simply substitute it into the following command to kick off your first deployment.

aws greengrass create-deployment \
    --deployment-type NewDeployment \
    --group-id "41752ff2-54e5-49e9-8751-d489e3e6fa1f" \
    --group-version-id "1c578332-44f7-411b-9ebb-1139fa1e453a"
Greengrass Deployment [GUI]

To deploy through the GUI, navigate to the AWS IoT Greengrass portal and kick off a new deployment under the Greengrass group we just created.

Greengrass Deployment

Greengrass Function

So what is Greengrass deploying? If you have a look at the contents of aws/greengrass.yaml under GGSampleFunction you will see the following YAML.

  GGSampleFunction:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: !Join ["_", [!Ref CoreName, "sample"] ]
      Description: Long running lambda that provides telemetry and pub/sub echo
      Handler: index.function_handler
      Runtime: python2.7
      Role: !GetAtt LambdaExecutionRole.Arn
      Timeout: 60
      Code:
        ZipFile: |
          import os
          from threading import Timer
          import greengrasssdk

          counter = 0
          client = greengrasssdk.client('iot-data')

          def telemetry():
              '''Publish incrementing value to telemetry topic every 2 seconds'''
              global counter
              counter += 1
              client.publish(
                  topic='{}/telem'.format(os.environ['CORE_NAME']),
                  payload='Example telemetry counter, value: {}'.format(counter)
              )
              Timer(5, telemetry).start()
          # Call telemetry() to start telemetry publish
          telemetry()

          def function_handler(event, context):
              '''Echo message on /in topic to /out topic'''
              client.publish(
                  topic='{}/out'.format(os.environ['CORE_NAME']),
                  payload=event
              )

This YAML handles the three topics inout and telem and processes them accordingly.

Greengrass Test

In order to test Greengrass, navigate to the Test portal under AWS IoT and subscribe to the gg_cfn/telem topic.

Greengrass Test

Greengrass Cleanup

Once you’re finished working with Greengrass it’s really easy to cleanup the AWS resources we used. Run the following command to destroy the CloudFormation stack.

aws cloudformation delete-stack \
    --stack-name "devopstar-rpi-gg-core" \
    --region "us-east-1"

Attribution

Extra things I learned

Greengrass deployment stuck on ‘In Progress’ ?

`ps -aux | grep greengrass`, and inspect the logs under /greengrass/ggc/var/logs/system

Or there is a problem with MQTT connection check by

openssl s_client -connect \
YOUR_ENDPOINT.iot.eu-west-1.amazonaws.com:8883 \
-CAfile root.ca.pem \
-cert CERTIFICATE.pem \
-key CERTIFICATE.key

You need to use your cert names of course, they will live in /greengrass/certs (you can also find them listed in /greengrass/config/config.json

On to part I -> https://michaelellerbeck.com/2020/01/08/aws-iot-sitewise-with-raspberry-pi-temperature-sensor-ds18b20-part-i/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s