Bulletproof Escape Fridge continuation…

/Bulletproof Escape Fridge continuation…

Building a Rube Goldberg Machine in AWS or How to Over complicate your architecture

the architecture

  • Run through the big diagram
  • Mini deep dive on
    • IoT Button
    • IoT Pub/sub
    • Shadow Device
    • Rekognition
    • Lambda Triggers
    • C9

In this post in our series we’ll be running through our Rube Goldberg machine and the components we use in AWS as well as on the Raspberry Pi.

The Admin API

First, we’ll get some admin pieces out of the way that aren’t directly related to the challenge. To keep track of how participants are doing in the challenge we needed a leaderboard.

We decided to store the leaderboard in DynamoDB. DynamoDB is a non-relational database service and is perfect for this type of simple data structure. Most importantly it is very cost effective for this type of exercise.

We created a single DynamoDB table called escape_fridge-production-leaderboard with the following attributes for each item:

  • name – John Ferlito
  • company – Bulletproof
  • email – john.ferlito@bulletproof.net
  • challengeId – The particular challenge this user attempted
  • startTime – When they started the challenge
  • endTime – When the successfully completed the challenge

Rather than have the application talk directly to DynamoDB, we created an HTTPS API using API Gateway in front of Lambda. We created a number of REST endpoints.

PATH
ACTION
Notes
/leaderboard POST
  • Expects name, company and email
  • Creates the entry in the DynamoDB table
/leaderboard PUT
  • Sets the start time and the challenge Id
/leaderboard DELETE
  • Set the stop time
/leaderboard GET
  • Retrieves the whole leaderboard
  • Retrieves one entry when query parameter, for example email=joe@example.com, is passed

The admin code can be found in the admin directory at /admin.

You will notice that we used the Serverless Framework to create and deploy the API. This framework provides an expedient way to create very simple APIs like the one above. It contains a single config file, serverless.yml, which contains

  • Lambda function configuration – language, memory size etc
  • Permissions for the lambda functions
  • The list of functions, where their code lives and the events that trigger them – HTTP API gateway, S3 event, IoT event, SQS queue etc
  • Cloudformation for any other AWS resource you need e.g. DynamoDB tables

Using serverless you can then deploy the whole stack with a couple of basic commands. It also supports being able to test locally as well as using next gen tools like babel to transpile JavaScript code.

Next we’ll move on to the user experience.

The Web Application

The Raspberry Pi runs a web application which allows the user to interact with the system. We chose to write the backend using Express, a “Fast, unopinionated, minimalist web framework for Node.js“. Express allows us to quite quickly prototype this type of application with minimal fuss.

For the frontend, we chose to keep the application nice and simple. We avoided using any modern frameworks like Angular or React and stuck to old school forms and posts back to the server. We wanted participants in the challenge to be faced with simple code they could come to grasps with quickly.

The Login

This first page the user is presented with is a login screen.

login

We collect a couple of details, so we can display something on the leader board and get in contact with the winners.

When the user hits create, this sends a POST request to /users route on the Express app. This first performs a number of validations on the data and then performs a POST to the /leaderboard API as described above.

The Start

 

start

The user is then presented with a set of instructions and clicks Start. This sends a POST to the /users/start route on the Express app, recording the time they started the challenge.

The App

the app

The user is now presented with the challenge screen. As you can see there is a displayyourarchitecture-TheIoTButton”>The IoT Button

of the video from the camera and a timer. This counts from zero to 10 minute at which stage the challenge ends.

iot button

The challenge has begun! The first thing the users does is click the AWS IoT Button.

The IoT button has already been set up to connect to the local wireless network. It has also been configured to connect with the AWS IoT service in our AWS account.

When the button is clicked it does the following:

  • Turns on and boots (LED blinks white)
  • Connects to the configured wireless (LED blinks white)
  • Publishes a message to an IoT topic called iotbutton/${DeviceSerialNumber} (LED is green on success, red on failure)

The message published to the topic looks like

{
  "serialNumber""G030.....",
  "batteryVoltage""1780mV",
  "clickType""SINGLE",
}

The clickType can be either SINGLE, DOUBLE or LONG.. Allowing you to have a button perform at least three different actions.

It is possible to have a lambda function run, whenever a message is published to an IoT topic. You can see how we have configured this using the Serverless Framework in api/serverless.yaml. In this particular case we used button.click lambda function.

The code for the function can be found in api/src/button.js. You can see that the code is fairly simple. We don’t actually care what is in the message, when we receive one we simply publish a message onto a topic called pi, which the Raspberry Pi is listening to via the Express Framework. The message we publish is

{
  cmd: 'take-photo',
}

which instructs the Pi it should take a photo.

Taking the Photo

When the express app starts, it registers itself as a device with AWS IoT. The code for this can be found in /pi/lib/iot.js. It listens on the pi topic for any commands from other parts of the system.

In this particular case it receives the take-photo command. The app also maintains a web socket connection to the web browser. It publishes the take-photo command down this socket.

On the web side, there is some javascript connected to the web socket, when it receives the take-photo command it then uses the browserAPIs to take the photo, the code can be found in /pi/public/javascripts/camera.js.

Upload to S3

After taking the photo we need to upload it to an S3 bucket, code is in /pi/public/javascripts/s3.js. Since we don’t want just anyone being able to access the bucket, we need to make a signed request to the S3 bucket.

To accomplish this, the Express app has a POST route listening on /s3 (/pi/routes/s3.js). This uses the S3 API and the secret AWS API keys to generate a signed URL. The browser then uses the signed URL to perform a POST straight into the S3 bucket, in this case named escape_fridge-production-faces.

Since we don’t want to keep the images around forever, we have configured the bucket with a life cycle policy to remove the images after 24 hours api/serverless.yml#L105-L10/9

Facial recognition

The S3 bucket has been configured to send any events to Lambda, /api/serverless.yml#L42-L43. In this case we call the /api/src/s3.js function.

This function is probably the most complicated piece of code that those particpiating in the challenge are exposed to. The first thing it does is to make a call to the AWS Rekognition DetectFace API. It passes in the path to the S3 object and asks for ALL attributes to be returned.

It then takes the results from the API call and publishes them on the pi IoT topic. The message looks like

{
  "cmd""face-data",
  "data": {
    "Smile": {
      "Value"false,
      "Cnfidence"99.99485643856237864,
    },
    "LOTS OF OTHER DATA HERE..."
  }
}

Once this is sent, the code then checks the data to see if the person is smiling. For our purposes anything over a confidence level of 60% is considered to be a smile. If we determine that the subject is smiling then we update the shadowDevice to set the fridge to unlocked.

shadowDevice is an AWS IoT construct that is designed to keep track of the state of a remote IoT device out in the field. For example it might track the state of a light bulb to be on or off. The device itself can update the state which is then sent to AWS IoT, or if another IoT client updates the state, like our code for example then the shadowDevice is informed of the state change and it should try to change the actual state of the device, in this case unlocking the firdge and then confirming the change via another update. Under the hood this is all achieved by publishing lots of messages back and forth on pre-determined IoT topics.

Unlocking the Fridge

When our code receives the change to the shadowDevice, code in /pi/lib/iot.js#L71-L90 state it then sends a signal to the GPIO pins as described in our previous post to unlock the fridge and flash the blue light, /pi/lib/gpio.js.

It also sends a DELETE request to the /leaderboard API to record the stop time.

Displaying the Results

the results

Finally the Express app receives the face-data command on the pi topic. As before it sends this to the web browser over the web socket. The browser then uses this data to render the results to the user.

The Leaderboard

On the day we wanted to be able to show a leaderboard of how people were doing on the various challenges.

This is probably the somplest part of the application and simply does a GET request on /leaderboard and then renders the result in the browser.

Wrap Up

That covers off our Rube Goldberg machine and the full software stack we used for the challenge. For the curious you can also find some of the admin scripts we used on the day to run the challenge in /admin.

In our next post we’ll cover off the actual challenges that our participants faced and the challenges we faced on the day in making it all work seamlessly in production.

By | 2018-07-11T04:43:01+00:00 July 11th, 2018|AWS, Digital Transformation, Technical Blog|