These are the steps to get a raspberry pi setup with IoT greengrass
This is mainly a copy of this post https://devopstar.com/2019/10/07/aws-iot-greengrass-cloudformation-raspberry-pi/ with a few comments
AWS IoT Greengrass is a managed service that allows you to easily manage and communicate with edge devices. It sells itself as a central way to deploy to huge fleets of devices in a repeatable way.
The irony of this is that deploying the foundation of AWS Greengrass can be quite complex and messy. In the official guide for Greengrass Core, AWS recommends using the Easy Creation button. This is perfectly fine for testing purposes, however if you are deploying production workloads then repeatability is really key.
CloudFormation resources are available for AWS Greengrass however the setup process can be rather complex when your device isn’t sitting in AWS. This blog post is a continuation of an existing post on the AWS Blog called Automating AWS IoT Greengrass Setup With AWS CloudFormation. This post works fine for setups where EC2 is used, however there’s a gap for people who deploy AWS IoT Greengrass Core to devices in the field.
Prerequisites
- AWS Account with AWS CLI Setup (setup on the pi)
- Edge Device, in the case of this post I’m using a Raspberry Pi Zero W
- Clone t04glovern/aws-greengrass-cfn locally (git clone https://github.com/t04glovern/aws-greengrass-cfn)
Setting up a Raspberry Pi
For our deployment we’ll be using a Raspberry Pi Zero W, however feel free to use any device you see fit. If you’re working on a Raspberry Pi as well you can follow the next steps to setup a new image.
Start of by downloading the Rasbian lite image. This can be done using wget if you are that way inclined.
wget https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2019-09-30/2019-09-26-raspbian-buster-lite.zip
Burn Image
Next we need to burn the image with Etcher or a similar imaging software.
Once the MicroSD card is flashed with Rasbian continue to the next step.
Setup WiFi
Setting up WiFi and SSH access in a headless fashion is really useful if you haven’t got a monitor around. It’s already very convenient for bulk deployments.
This might be different based on your OS, the general idea is that you will need to create:
- An empty file called
sshin the/boot/partition - A file called
wpa_supplicant.confin the same partition
touch /Volumes/boot/ssh
touch /Volumes/boot/wpa_supplicant.conf
nano /Volumes/boot/wpa_supplicant.conf
# Add the following config
country=AU
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
ssid="NETWORK-NAME"
psk="NETWORK-PASSWORD"
}
NOTE: Be sure to update your WiFi networking information. If you are using Ethernet you can skip the wpa_supplicant.conf step.
Connect to Pi
Plugin the Raspberry Pi and give it a couple minutes to first boot up. You can then SSH into your raspberry pi by either using Putty on Windows or ssh on MacOS & Unix.
ssh-keygen -R raspberrypi.local
ssh pi@raspberrypi.local
# password: raspberry
Once connected, we update the system with the latest packages and install Python Pip for later use with the Greengrass SDK.
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install python-pip -y
Prepare for Greengrass
Now we need to perform a set of steps that are best described in the official AWS setup guide for the Raspberry Pi These steps are highly recommended if you want to ensure you don’t run into problems later down the track when running Greengrass Core.
Start off by adding a new system user and group that Greengrass will use when executing. This helps separate the permissions away from our default user.
# Create user and group
sudo adduser --system ggc_user
sudo addgroup --system ggc_group
Next we can enable hardlink and softlink (symlink) protection by editing 98-rpi.conf.
sudo nano /etc/sysctl.d/98-rpi.conf
Add the following lines to the file:
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
Reboot the pi, and when logged back in confirm the settings were set.
sudo reboot
sudo sysctl -a 2> /dev/null | grep fs.protected
Edit your command line /boot/cmdline.txt file to enable and mount memory cgroups. Append the following to the single line.
cgroup_enable=memory cgroup_memory=1
The file should look like the following (with some changes to partition for example).
console=serial0,115200 console=tty1 root=PARTUUID=6c586e13-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_enable=memory cgroup_memory=1
Finally reboot the Pi again to confirm everything comes up okay.
sudo reboot
Greengrass Dependency Checker
To confirm that all prerequisites are fulfilled we’ll use the Greengrass Dependency Checker. Run the following commands from the home directory of the Raspberry Pi we SSH’d into.
mkdir /home/pi/Downloads && cd /home/pi/Downloads
mkdir greengrass-dependency-checker-GGCv1.9.x && cd greengrass-dependency-checker-GGCv1.9.x
wget https://github.com/aws-samples/aws-greengrass-samples/raw/master/greengrass-dependency-checker-GGCv1.9.x.zip
unzip greengrass-dependency-checker-GGCv1.9.x.zip && cd greengrass-dependency-checker-GGCv1.9.x
sudo modprobe configs
sudo ./check_ggc_dependencies | more
There will likely be a couple warnings when running:
- It looks like the kernel uses ‘systemd’ as the init process.
- Could not find the binary ‘nodejs8.10’.
- Could not find the binary ‘java8’.
These are safe to ignore for now, the first warning should be kept in mind however as we’ll resolve it later.
Greengrass Setup
The deployment of the Greengrass group can be done through the UI or CloudFormation. If you have an interest in how to do it by the UI, please check the AWS guide on this, however I would advise against it for this tutorial.
CloudFormation is a much better option as it gives us flexibility for re-configuration later on through code changes.
CloudFormation Greengrass Deploy
To deploy using CloudFormation, ensure AWS CLI is configured on the system and proceed to run the command below:
( cd into cd aws-greengrass-cfn )
aws cloudformation create-stack \
--stack-name "devopstar-rpi-gg-core" \
--template-body file://aws/greengrass.yaml \
--region "us-east-1" \
--capabilities CAPABILITY_IAM
The deployment will take a couple minutes and can be monitored via the CloudFormation portal. A number of resources will be created and managed for you:
Next we’re going to build the tar.gz bundle with our certificates and Greengrass configuration. I’ve written a helper script for this in the aws folder of the GitHub repository.
The general idea is that the certificate details need to be zipped up and deployed to the Raspberry Pi, so we export this information from the CloudFormation stack using the following:
certificateId=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
--query 'Stacks[0].Outputs[?OutputKey==`CertificateId`].OutputValue' \
--region ${AWS_REGION} \
--output text)
certificatePem=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
--query 'Stacks[0].Outputs[?OutputKey==`CertificatePem`].OutputValue' \
--region ${AWS_REGION} \
--output text)
certificatePrivateKey=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
--query 'Stacks[0].Outputs[?OutputKey==`CertificatePrivateKey`].OutputValue' \
--region ${AWS_REGION} \
--output text)
iotEndpoint=$(aws cloudformation describe-stacks --stack-name ${CFN_STACK_NAME} \
--query 'Stacks[0].Outputs[?OutputKey==`IoTEndpoint`].OutputValue' \
--region ${AWS_REGION} \
--output text)
This information is used to save the various certificates.
echo -n "${certificatePem}" > certs/${certificateId}.pem
echo -n "${certificatePrivateKey}" > certs/${certificateId}.key
Then a config.json file is generated using all this information that will be used shortly on the Pi
{
"coreThing" : {
"caPath" : "root.ca.pem",
"certPath" : "${certificateId}.pem",
"keyPath" : "${certificateId}.key",
"thingArn" : "arn:aws:iot:${AWS_REGION}:${AWS_ACCOUNT_ID}:thing/${CORE_NAME}_Core",
"iotHost" : "${iotEndpoint}",
"ggHost" : "greengrass-ats.iot.${AWS_REGION}.amazonaws.com"
},
"runtime" : {
"cgroup" : {
"useSystemd" : "yes"
}
},
"managedRespawn" : false,
"crypto" : {
"principals" : {
"SecretsManager" : {
"privateKeyPath" : "file:///greengrass/certs/${certificateId}.key"
},
"IoTCertificate" : {
"privateKeyPath" : "file:///greengrass/certs/${certificateId}.key",
"certificatePath" : "file:///greengrass/certs/${certificateId}.pem"
}
},
"caPath" : "file:///greengrass/certs/root.ca.pem"
}
}
Finally all the files are zipped up and saved. To perform all these steps in one easy hit, run the following:
NOTE: This script assumes that you deployed to us-east-1 and left the stack name as devopstar-rpi-gg-core. If you changed that portion, replace references to devopstar-rpi-gg-core with your own stack name.
The script uses jq so you need to install it!
sudo apt-get install jq
cd aws
./greengrass.sh
Deployment Files to Raspberry Pi
In this step we’ll download AWS IoT Greengrass Core then copy it to our Raspberry Pi. We will also copy our previously packages certificates and configuration at the same time.
Download Greengrass Core
Depending on what version your raspberry pi is will determine what version we’ll need. Run the following command to get your architecture.
uname -a
# Linux raspberrypi 4.19.75+ #1270 Tue Sep 24 18:38:54 BST 2019 armv6l GNU/Linux
In this case my architecture is armv6l so from the Greengrass download page download the distribution for Rasbian that matches this.
# Download the `armv6l` tar
wget https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.9.3/greengrass-linux-armv6l-1.9.3.tar.gz
My case was
wget https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.10.0/greengrass-linux-armv7l-1.10.0.tar.gzNow that we have the files we need, copy them all to the Raspberry Pi.
scp greengrass-linux-armv6l-1.9.3.tar.gz pi@raspberrypi.local:/home/pi
scp aws/*-setup.tar.gz pi@raspberrypi.local:/home/pi
scp aws/greengrass-service.sh pi@raspberrypi.local:/home/pi
Extract Greengrass Files
SSH back onto the Raspberry Pi and extract the two file bundles we just downloaded by running the following.
sudo tar -xzvf greengrass-{ARCHITECTURE}-1.9.3.tar.gz -C /
sudo tar -xzvf {CERTIFICATE_ID}-setup.tar.gz -C /greengrass
Confirm everything copied across correctly by running the following
ls -al /greengrass
# drwxr-xr-x 6 root root 4096 Aug 31 04:40 .
# drwxr-xr-x 22 root root 4096 Aug 31 04:40 ..
# drwxrwxr-x 2 pi pi 4096 Oct 6 18:01 certs
# drwxrwxr-x 2 pi pi 4096 Oct 6 18:01 config
# drwxr-xr-x 3 root root 4096 Aug 31 04:40 ggc
# drwxr-xr-x 3 root root 4096 Aug 31 04:40 ota
Next we also need to download the ATS root CA certificate into /greengrass/certs
cd /greengrass/certs/
sudo wget -O root.ca.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem
Greengrass Start
With all the files in the right places on the device, it’s now time to try to start the service
cd /greengrass/ggc/core/
sudo ./greengrassd start
You should see a Greengrass successfully started message. Take note of the PID of the process so we can view the status of the daemon.
ps aux | grep PID_NUMBER
# root 761 2.4 3.0 854472 13508 pts/0 Sl 18:18 0:01 /greengrass/ggc/packages/1.9.3/bin/daemon -core-dir /greengrass/ggc/packages/1.9.3 -greengrassdPid 757
To setup Greengrass on startup, run the following (not this will reboot your Raspberry Pi). This assumes that you are using systemd.
sudo ./greengrass-service.sh
Greengrass SDK
If you don’t want to bundle the greengrasssdk in with your application, you can install it globally.
sudo pip install greengrasssdk
If you choose not to go down this route you can manually bundle greengrasssdk in with your lambda deployments in the future.
Greengrass Deploy [CLI]
To create our first deployment we first need to retrieve our Greengrass Group ID
aws greengrass list-groups
# {
# "Groups": [
# {
# "Arn": "arn:aws:greengrass:us-east-1:123456789012:/greengrass/groups/..........",
# "CreationTimestamp": "2019-10-06T10:31:53.950Z",
# "Id": "41752ff2-54e5-49e9-8751-d489e3e6fa1f",
# "LastUpdatedTimestamp": "2019-10-06T10:31:53.950Z",
# "LatestVersion": "1c578332-44f7-411b-9ebb-1139fa1e453a",
# "LatestVersionArn": "arn:aws:greengrass:us-east-1:123456789012:/greengrass/groups/..........",
# "Name": "gg_cfn"
# }
# ]
# }
In my case the group ID and version can be seen above, simply substitute it into the following command to kick off your first deployment.
aws greengrass create-deployment \
--deployment-type NewDeployment \
--group-id "41752ff2-54e5-49e9-8751-d489e3e6fa1f" \
--group-version-id "1c578332-44f7-411b-9ebb-1139fa1e453a"
Greengrass Deployment [GUI]
To deploy through the GUI, navigate to the AWS IoT Greengrass portal and kick off a new deployment under the Greengrass group we just created.
Greengrass Function
So what is Greengrass deploying? If you have a look at the contents of aws/greengrass.yaml under GGSampleFunction you will see the following YAML.
GGSampleFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Join ["_", [!Ref CoreName, "sample"] ]
Description: Long running lambda that provides telemetry and pub/sub echo
Handler: index.function_handler
Runtime: python2.7
Role: !GetAtt LambdaExecutionRole.Arn
Timeout: 60
Code:
ZipFile: |
import os
from threading import Timer
import greengrasssdk
counter = 0
client = greengrasssdk.client('iot-data')
def telemetry():
'''Publish incrementing value to telemetry topic every 2 seconds'''
global counter
counter += 1
client.publish(
topic='{}/telem'.format(os.environ['CORE_NAME']),
payload='Example telemetry counter, value: {}'.format(counter)
)
Timer(5, telemetry).start()
# Call telemetry() to start telemetry publish
telemetry()
def function_handler(event, context):
'''Echo message on /in topic to /out topic'''
client.publish(
topic='{}/out'.format(os.environ['CORE_NAME']),
payload=event
)
This YAML handles the three topics in, out and telem and processes them accordingly.
Greengrass Test
In order to test Greengrass, navigate to the Test portal under AWS IoT and subscribe to the gg_cfn/telem topic.
Greengrass Cleanup
Once you’re finished working with Greengrass it’s really easy to cleanup the AWS resources we used. Run the following command to destroy the CloudFormation stack.
aws cloudformation delete-stack \
--stack-name "devopstar-rpi-gg-core" \
--region "us-east-1"
Attribution
Extra things I learned
Greengrass deployment stuck on ‘In Progress’ ?
`ps -aux | grep greengrass`, and inspect the logs under /greengrass/ggc/var/logs/system
Or there is a problem with MQTT connection check by
openssl s_client -connect \
YOUR_ENDPOINT.iot.eu-west-1.amazonaws.com:8883 \
-CAfile root.ca.pem \
-cert CERTIFICATE.pem \
-key CERTIFICATE.key
You need to use your cert names of course, they will live in /greengrass/certs (you can also find them listed in /greengrass/config/config.json
On to part I -> https://michaelellerbeck.com/2020/01/08/aws-iot-sitewise-with-raspberry-pi-temperature-sensor-ds18b20-part-i/
Michael Ellerbeck 