Creating a Global Live Streaming Website with Amazon Interactive Video Service and AWS CloudFormation within minutes
Learn how to build a serverless pipeline (of sorts?!!) to deploy a website which hosts a global video stream using Amazon IVS and Amazon S3.
Get yourself broadcasted to the world…. in minutes!
TL;DR
If you want to skip the read and hit the Github repository with quick instructions:
https://github.com/moosakhalid/amazon-ivs-proj
Introduction
When you think about how human interaction has changed over the last year due to COVID-19, it’s unmistakable how a technology as omnipresent as video has helped us all stay sane, communicate and most of all work remotely. The internet was not built with video in mind and hence protocols such as HLS (HTTP Live Streaming) and the likes, take great pain, at a higher application level, to ensure fluidity of video playback.
As the demand for streaming content increases and users become more aware of how to leverage them for various use cases, they want more control over the quality, presentation and interaction with their audiences while not caring about the backend streaming servers or intricacies of how to make it all happen.
Enter Amazon Interactive Video Service (IVS), a managed live streaming solution that is quick and easy to set up, and ideal for creating interactive video experiences. The term “managed”, in this case really means that you only have to care about generating content and pointing it at Amazon IVS. This leaves you time to focus on making your front-end website or application look beautiful and interactive. Talking about interactive what sets Amazon IVS apart, is its timed metadata feature which allows using API endpoints to present data to users to interact with on the same media stream going out globally, which means that all the viewers of that stream get the same data, which could be a quiz, or feedback, during the same point in the content being streamed.
Pre-requisites
With this blog post I wanted to showcase the quick off the shelf deployment of Amazon IVS and its ease of setup.
This blog is targeted towards fellow developers and infrastructure as code (IaC) enthusiasts who want a minimalist approach to creating an end-to-end streaming platform, and hence I’ll be providing a CloudFormation template which will deploy all the required resources. The user will only need to carry out very few steps to see the end results.
We will be building a live video streaming website which leverages Amazon IVS to stream video globally and also allows for sending users questions during the live stream automatically.
We’ll mainly be focusing on infrastructure as code (IaC) through CloudFormation to set it all up. This will be mostly a guided demonstration for which you will need the following tools:
- An AWS Account
- Streaming software like OBS, ffmpeg or StreamLabs
Resources needed:
In the backend, the deployment will pull down HTML and JavaScript code for our sample website which you will not need to modify for the deployment to work, however if you wanted to peek at it, the HTML file(index.html) can be found here and the JavaScript code (ivs.js) can be found here.
Now before you roll up your sleeves and dig in, let’s take a look at what we will be working with.
Solution Overview:
Dissecting the Solution Overview and CloudFormation template
Everything really starts with the provided CloudFormation stack, as indicated by Step 1. This stack deploys the following resources, the definitions below map the description to the logical resource ID for the respective resource that you will see in the CloudFormation template, which can be downloaded here:
1. An IVS Channel which holds the configuration for streaming i.e. latency, quality etc. => IVSChannel
2. An IVS Stream Key, which acts as a secret for your IVS channel(note that with a normal IVS channel create API call, the stream key is returned with IVS channel creation response, however in CloudFormation it needs to be created separately) => IVSStreamKey
3. An S3 Bucket with static website hosting enabled that will host our front-end code => S3IVSWebsite
4. An S3 Bucket resource policy for allowing public read permissions => S3IVSBucketPolicy
5. Lambda function for injecting metadata into live stream via PutMetadata API => CustomIVSLambda
6. A Lambda execution role for allowing ivs:PutMetadata API call to Lambda at #5. => LambdaIVSRole
7. A Lambda resource policy for allowing Lambda at # 5 to be invoked by EventBridge rule => LambdaResourcePolicy
8. An Amazon EventBridge rule for invoking Lambda at #5 when Stream state for our channel changes. For e.g. goes from Idle to Playing => IVSEventBridgeRule
9. Lambda function for syncing website HTML and JavaScript code to S3 Website Bucket at # 3, from GitHub, and also delete bucket on CloudFormation stack deletion. => UploaderLambda
10. A Lambda execution role for allowing access to S3 Website Bucket to Lambda at # 9. => UploaderLambdaRoleS3
11. A CloudFormation Custom Resource for feeding S3 bucket name so that Lambda at # 9 can upload files to it and also feeding in the Playback HTTPS URL into the code. => LambdaCustomUploader
Of course, CloudFormation handles the complexity of dependency resolution and determining what order to create resources in.
When you look at the overall picture, you will notice how small the footprint of Amazon IVS is in the overall design, which goes to highlight its ease of setup. Really, we are only creating 2 resources which pertain to Amazon IVS:
1) A channel which defines configuration for streams
2) A Stream Key for the channel which acts as a secret unique to the channel.
Both these values are required to have IVS ingest your streamed content to it.
Notice how the ease of setup for Amazon IVS has allowed me to focus more on building automation and features into the project.
The CloudFormation template spins up 2 Lambda function each with appropriate roles attached so that they may carry out the tasks as required by their code. Code for both Lambdas is written in Python3. Only one of the Lambda functions (at # 5 above) interacts with Amazon IVS for injecting metadata into a stream, let’s take a look at it:
In the Lambda code above, we are using Python’s boto3 AWS SDK for communicating with the IVS endpoint.
On line 1 we import the boto3 module for use in our code.
On line 4 we create an IVS client instance in preparation for making API calls later.
On line 6 we have the JSON formatted string which we will be posting with the IVS PutMetadata API call.
On line 12 we use the event metadata with which the Lambda function was invoked, this event variable is populated with information regarding the state change, Amazon EventBridge fires off the Lambda function with the details about the state change of the IVS channel and feeds it into the Lambda function as the event variable.
It looks something like this:
The snippet above is an example of event sent to Lambda by EventBridge. It’s in JSON format, as you can see that we are using the “event[‘detail’][‘channel_name’]” field to compare against the channel that we want to send our ivs:PutMetadata API call towards. Also note that in the Python code above we have compared the channel name in the event variable with the value “MyIVSChannel-CloudFormation”, which is the name we are setting for our IVS channel through the CloudFormation template.
The other Lambda function (not shown here) helps in uploading code to the S3 Website bucket that the CloudFormation template creates. It gets two inputs from a custom resource in the CloudFormation template:
1) The name of the bucket to upload code to, from GitHub.
2) The Playback URL (HLS) that the IVS Channel has returned on creation, it modifies the JavaScript code for the website and replaces a dummy placeholder value with the actual playback URL for the stream.
We also create an Amazon EventBridge rule for Amazon IVS. This rule basically matches a pattern to invoke our Lambda function. It matches all events where the state of an IVS stream changes, for example from Started to Ended or vice versa. On catching a stream state change, the rule triggers an invocation of the Lambda function configured in its properties which we discussed earlier.
For the S3 Bucket with static website hosting enabled, we ensure that its contents are publicly readable, as that is a requirement for serving content through it. We attach an S3 Bucket resource policy to it:
The S3 Resource ARN will be different for you as the name of the bucket is created by CloudFormation on the fly. Notice how we are allowing * for Principal and action s3:GetObject to make all objects within the bucket publicly accessible.
Amazon IVS SDK for Web
Now let us take a quick look at the front-end of our website to see how we are utilizing the IVS Player SDK.
Our deployment uploads code for displaying the website where the stream will be hosted. There are 2 components to this website:
1) An HTML file.
2) A JavaScript code file which invokes the IVS SDK and manipulates the player.
I have kept the HTML and JS code quite simple and minimal, so you won’t see a very flashy front-end.
The HTML file is simple enough and only includes jQuery, IVS SDK library and our own IVS JavaScript code included via <script> tags and in-line styling of HTML content.
The file ivs.js contains code that I’ve written to initialize the IVS player instance using the IVS SDK. It’s described below:
The “STREAM_PLAYBACK_URL” dummy placeholder is replaced with the real Playback URL returned upon IVS channel creation by a Lambda function during the CloudFormation deployment.
The “timed_metadata_action” function is invoked by the player.EventListener method and basically displays the metadata received by the event listener onto the HTML page.
You will also notice that I have used a switch statement to update the status of the live stream on the HTML page with the HTML element with id “status” just to make it a little more interactive. Here, I’ve used the IVS Player state map variable to do so.
The IVS SDK for web offers rich functionality in getting various properties of your live streams and manipulating them on the go.
Up until this point we had been dissecting the solution and understanding how the bits and pieces would come together to give us a global live streaming website. Now let’s dive into the steps that you will actually need to execute in order to make it all happen.
Deploying CloudFormation Stack
To deploy the CloudFormation stack:
1. Log into the AWS Console and from the Services drop-down select CloudFormation.
2. Click on “Create Stack” (if you see a drop-down with this button select “With new resources(standard)”
3. Under “Prepare Template” ensure the “Template is Ready” radio button is selected.
4. Under “Template Resource” click on the “Upload a template file” radio button.
5. Download and save the CloudFormation Stack file from here. If prompted, save the file as “ivs.json”.(“right-click and click on ‘Save link as’ to save it locally).
6. Click on “Choose File” and upload the CloudFormation stack that you just downloaded. Hit the “Next” button after the file has uploaded successfully.
7. Enter a name for your CloudFormation Stack, for example “my-stack”. Hit the “Next” button.
8. On the “Configure Stack Options” page leave everything as default and hit the “Next” button.
9. You’ll be taken for a review page for your CloudFormation stack, scroll down to the bottom and ensure that you check “I acknowledge that AWS CloudFormation might create IAM resources with custom names.”, so that your CloudFormation stack can deploy the required IAM roles, failing to do so will cause your deployment to fail.
10. Finally hit “Create Stack” button.
You will be taken to the properties of the CloudFormation template that you just deployed, and you can track its progress under the “Events” or “Resources” tab.
Click on the “Outputs” column, once the template has finished executing to copy over information required to stream to IVS and see the results.
Copy over values for “IngestEndpointIVS” and “StreamKeyIVS” for using later.
Open up a separate browser tab with the URL returned for “IVSWebsiteURL” output, this is where you live stream will be hosted, globally for viewers. The initial view of the website will look like this:
STREAMING TO IVS
Now comes the fun part, let’s plug the Stream Key and Ingest Endpoint values that we noted down from the CloudFormation stack execution output into OBS studio streaming software and see our streaming website in action!
OBS can be downloaded here. It is available for a range of OSes, in my case I downloaded it for MacOS, however the steps I have shown below should be pretty similar across different platforms.
Start the OBS application and from the drop-down menu at the top click on “OBS” and then on “Preferences”
Once the “Preferences” dialog box opens up, from the options available on the left pane, select “Stream”.
For the “Service” field drop-down, select “Custom”, you’ll be presented with two input values: Server and Stream Key.
Paste the values that you copied over from CloudFormation stack outputs into the respective fields on the OBS Streams config page. Press “OK” to save changes.
On the OBS main prompt, click on the “Start Streaming” button, this should be in the bottom right corner.
It should take a few seconds for OBS to connect and start streaming to IVS (it should also display a green marker at the bottom right corner to indicate that stream is live).
Testing the Live Streaming Website and Interactive Metadata
Now head over to the S3 Website URL that you were provided as part of CloudFormation output and refresh the page (it may take a couple of refreshes) and now you should be able to see your live stream.
Amazon IVS has a latency of about 3 to 5 seconds between sending the stream to IVS for ingestion and IVS playing it back globally. Once you see the stream play out live on the webpage, wait for about 10 to 15 seconds and you will notice the question injected as metadata shows up right above the video. For this blog post I did not incorporate storing and analyzing user interaction with injected metadata, this is just to demonstrate the quick deployment time and integration of Amazon IVS within the AWS ecosystem.
Tearing it all down and Summary
When you’re done testing, its best to delete all the resources created to save yourself from unnecessary cost. Head back to the CloudFormation page on AWS Console and look for the stack name that you deployed earlier, select it and click on the “Delete” button to delete it all.
On the pop-up dialog box click on “Delete Stack”, this should kick off the deletion of all resources deployed through CloudFormation. It should take a few minutes. You can track its progress over in the “Events” tab and wait for the deletion to complete.
Note: CloudWatch Logs from execution of Lambda functions created with this CloudFormation stack will be left over and you’ll need to go and delete them manually.
In this blog post we looked at using Amazon Interactive Video Service to deploy a streaming website and we witnessed how the well-integrated services within AWS allow for spinning up an end-to-end streaming platform using Infrastructure as Code.
We emphasized how Amazon IVS takes away the pain of standing up a complex video streaming infrastructure and lets us focus more on the components around it, thereby enabling us to be more creative and deliver interactive experiences to our customers and users on the push of a button.
