TransformingTheChurch's Adaptive Streaming Technology

Producing high-quality video in high definition is the first of a two-part video integration process. The next part of the process is to deliver our videos in the highest possible quality to each of our users. Of course, the ability to provide our video is directly related and dependent upon the bandwidth and performance of the broadband connection of each of our users.

The delivery phase of our video integration is made complex because of the broad scope of the broadband connections in use today.

The good news is that average end-user bandwidth speeds are increasing in the USA as cable operators move to the newest DOCSIS standard for cable modems and fiber-to-the-home connections become more plentiful. For users blessed with such cable modem connections, download speeds of 15-megabits per second and higher are becoming common and 50- to 100-megabits-per-second speeds are starting to appear in significant numbers. Users with fiber-to-the-home connections have up to gigabit-per-second connections. Users who have connections this fast, mostly in urban cities, will be able to view our full-HD 1080p files without problems.

However, at the same time, we also see mobile broadband services take root; some through 3G systems and some through the faster 4G systems. Wireless bandwidths, even 4G systems, have slower speeds than the newest generation of cable modems and fiber-to-the-home services. Also, the performance of wireless connections can vary a great deal from moment to moment simply due to the way RF technology performs, local conditions and busy times with lots of user contention for the bandwidth.

Add in the fact that we will also have users from small rural towns where bandwidth is still dear, we are now in a very diverse environment where the range of available connection bandwidths in our user pool is very broad; some will be on very high-speed connections, some in the middle-range connections and some on the low end of broadband connections.

The simple fact is that while we produce all video media in full HD (1080p), not all of our users will have sufficient bandwidth to view these high-quality files. High-action video at 1920 x 1080 produces an encoded video file, even when using the best multi-pass encoders, which requires 5,000-kbps to 9,000-kbps to show well. Only a small portion of our audience (but a growing portion) will be able to view videos of this quality.

We must have an intelligent method to accommodate the diverse bandwidth capabilities of our user base and yet still show the highest-possible quality video that their connection (and current performance) allow. This method must be adaptive to each user and dynamic to accommodate the quickly-changing states of each user connection.

Overall, we approach this problem with the use of two technologies:

• The use of efficient encoding and compression techniques using MP4 formats for video using H.264 codecs that produce a high-quality video with the smallest file sizes.
• The use of adaptive streaming technology in our media platform that can detect the capacity and error rates of users’ connection and stream the most appropriate video file for the current state of each user’s connection.

We first make the highest possible quality video files with advanced professional video production techniques (click here for details); the better the original masters are, the better and smaller compressions we can make. Using the high-quality "take" from our digital video cameras, we produce high-quality edit masters using one of our two video editing suites, Avid's Media Composer or Sony's Vegas Pro. Which editing suite we select for any particular project is a function of how the video was produced and the equipment used; both are excellent professional-grade editing suites. Our edit suites are used to render very-high-bit-rate intermediate-master files in DNxHD format in 1920 x 1080 full-HD from the edit masters. These lightly-compressed master files are used to encode the compressed video files that our servers use to stream.

To perform this important encoding step, we use state-of-the-art cloud-based encoder services to produce MPEG-4 files with H.264 codecs using the MPEG transport stream. Each master file is encoded at various spatial sizes and streaming speeds. Encoding is both a science (lots of math and processing algorithms involved) and art (lots of creative judgment and hand-crafting configurations are involved). The goal is to produce a set of distribution files with the highest possible quality video in the smallest files size.

With the best possible source video files made, we next use our adaptive streaming technology on our streaming media servers to intelligently stream the highest possible quality video file that each user’s connection can accommodate. Our media servers dynamically select the most appropriate file to stream to each user at any given instant in time. Those with higher-capacity connections will see higher-quality videos while those with lesser capacity connections will see the highest possible quality that their connection will allow.

Rather than relying on the user to select the correct version and speed of our media that they want to play, or, worst of all,  catering to one extreme or the other, merely using the lowest common denominator, our adaptive streaming technology allows us to have one player and have the adaptive streaming technology automatically select the highest quality file that is suitable for each end user's available bandwidth and stream that file to the player. Adaptive streaming is enabled by intelligence built into our media players that we embed in our web pages. The players evaluate various indicators like their buffers status and transmission error rates, in real time, to determine the actual bandwidth being used (after the impacts of speeds, errors, etc.) in each video play and seamlessly switches among the various speed video files accordingly to stream a slower or faster media files as needed.

There isn't anything the user has to select or do, the adaptive streaming technology simply works; all behind the scenes and totally transparent to the user.

Also, the adaptive streaming technology is dynamic; it will adjust automatically throughout the duration of the viewing session. If the end user's connection changes (degrades or improves) during a viewing session (common in mobile wireless connections), the adaptive streaming technology will automatically adjust up or down as needed. It also means that if a user's bandwidth improves during the session, the quality of the video file that is served to the user will likewise improve (and vice versa.)

Our adaptive-streaming technology throttles up and down so quickly that it also helps when our users experience transient conditions like congestion on local area networks (LANs) and wireless networks. To the adaptive streaming technology, local congestion (transient or sustained) is just another "brown-out" situation that limits available bandwidth or increases error rates and our adaptive-streaming technology will adjust accordingly. When congestion is high and bandwidth is very "bursty" or has high jitter rates, our adaptive streaming technology will throttle down. When congestion abates, the adaptive-streaming technology will sense the improved conditions and will throttle back up. This means that the quality of the video may degrade a bit (for a while) during the periods of congestion but the stream won't stop. For a stream to completely stop, repeated outages of several seconds each must occur or massive jitter over time must occur for the video to completely stop. We can't do anything about end user's network outages or their ultra-high-jitter situations (these are the responsibility of the end user and their ISP) but we can employ adaptive-streaming technology to minimize the effects of these connection issues and to use the available bandwidth as efficiently and effectively as possible to provide the best possible viewing experience.

The adaptive streaming software built into our embedded media players fulfill the role of a forward-deployed network probe resident in each end-user's location. The embedded player functioning as a cooperative intelligent network probe that can make informed judgments about streaming switching, selecting a larger or smaller file as need to match the dynamic transmission environment. The resulting intelligent approach operates continuously throughout the viewing session. The intelligence in the embedded player adjusts to real-world user conditions in seconds.

Dynamic stream switching will automatically switch the quality of the file streamed to match the available bandwidth of each end user. The player will first try to play the 8,856-kilobits-per-second 1080p version (1,920 x 1080) of the file as our default. The feature monitors filling the buffer of the video player for speed and errors. Within milliseconds, we can accurately determine the performance of each user's connection and dynamically change the speed and size of the file served. If the end user doesn't have sufficient bandwidth to play the media at the highest-quality, the streaming server will automatically switch to the next slower version of the file and so on down the list of available speeds and sizes. Switching among the streams is seamless, happens very quickly (in milliseconds); it will not be seen as a frame-skip or jump in the video. Switching from one size and speed file to another is completely smooth and happens within the natural pace and flow of the video out of the player's video buffer. In our technique, there will NOT be any skips in the audio track either; audio will play without interruption or change.

If the end-user's bandwidth improves during the viewing session, the streaming server will automatically throttle up to a better-quality version. Stream switching is totally automatic and transparent in both directions. It simply works unseen by the end user.

The server will select the best of the following files to match the available bandwidth:

• 8,884-kbps - spatial size 1,920 x 1,080 (29.97 fps)
• 6,884-kbps - spatial size 1,280 x 720 (29.97 fps)
• 5,384-kbps - spatial size 1,280 x 720 (29.97 fps)
• 3,756-kbps - spatial size 960 x 540 (29.97 fps)
• 2,256-kbps - spatial size 896 x 504 (29.97 fps)
• 1,756-kbps - spatial size 768 x 432 (29.97 fps)
• 1,392-kbps - spatial size 640 x 360 (29.97 fps)
• 1,192-kbps - spatial size 640 x 360 (29.97 fps)
•    792-kbps - spatial size 640 x 360 (29.97 fps)
•    528-kbps - spatial size 480 x 270 (15 fps)
•    328-kbps - spatial size 416 x 234 (12 fps)

Note that this is approximately a 27x spread of supported speeds; a very wide range that will serve the widest possible number of user-connections!

Yes, this means to use adaptive streaming, we must encode the same video master file eleven times. While it is additional work, this is a small price to pay for the resulting quality in real-world network environments. The abilities that adaptive streaming provides the ability to support the largest possible population of broadband users with the best possible media experience.

In the technique we are using, all streams, regardless of their native spatial size, will expand or shrink to fill the player window embedded in our web pages. In other words, as the stream is switched, the video will not appear to change in any way except its quality. If the changes are within a reasonable band, the change in quality will be equally subtle. This means that the end users might not even notice that the adaptive streaming technology is throttling their media during a viewing session unless there are big swings in the users' bandwidth.

If the fluctuation of users' connection is extreme, browning-out to almost nothing for seconds at a time, we will do the best we can while these poor conditions exist. This means we may switch the stream all the way down to the lowest quality video file during these periods. However, we will restore the streams to higher quality streams as soon as the quality of users' connections improve.

Also, the video player that we use and is embedded in our web pages has a "full screen" button that will allow the video to be shown at the full dimension of each end user's monitor screen (good for 720p and 1080p viewing.)

Our streaming server will dynamically cycle through the list of available speeds and size of video files (both up and down) to match the end users' bandwidth. The adaptive streaming feature likely will go un-noticed by our users; all they will see is that the quality of the video may degrade a bit as their connectivity degrades (if they even notice). However, they will see an uninterrupted video stream unless their average connectivity falls below 328-kbps (which is a pretty poor broadband connection).

While there isn't anything we can do to improve users' connections (other than suggest they consider using a better connection product or a better carrier/ISP), we can use adaptive streaming technology us to provide the best possible video quality that users' connections allow.  Adaptive streaming allows us to provide a wide range of video files and automatically adapt among them to provide streams that can be viewed by all end users; even those with less than optimal broadband connections.

Try It Out!

So you can try out our Adaptive Streaming technology, we have provided a sample for you to use.

Click here to try our Adaptive Streaming sample.

The video file used in this sample is the animated title sequence of the up-coming "Vision for Distance Learning" course by Bill Anderton. This title sequence is composed of a custom 3-d graphic animation with a custom music score (produced and directed by Bill Anderton and scored by Nick Seeley.) The length of the sample is 1:12 (m:ss). The original master file was a 1080p video rendering of the 3-d animation and, as a result, very "clean."  The master rendering was then encoded in the speeds and sizes shown above from 2,000-kbps to 256-kbps (the 1080p version is not a part of this sample).

Because of the high-resolution details in the leaves of the tree and the grass and plants on the ground, their motion throughout the sequence and the color sweep of the sky and clouds in the background, this master file is  a very good encoding test. This is a tough master video file to encode well; it is an encoding torture test, actually, ideal for testing the entire technique from encoding to adaptive streaming.

The encoding for this sample was done with our low-cost experimental open-source encoder using our customized "high-profile" settings for H.264 MPEG-4 files. While, we continue to use Sorenson Squeeze has our primary encoder in the production workflow of our course development, we are also testing this open-source encoder as an experiment. We are always looking to improve our techniques and tool set.

Adaptive streaming will first try to stream the 720p high-definition version of the sample video. If you do not have sufficient bandwidth (at least 2,000-kbps) or you are currently experiencing transient congestion in your network connection, the adaptive-streaming technology will throttle back to the next best file that your actual broadband connection will support at the time. If your connection improves during your viewing session, the adaptive-streaming technology will automatically throttle up.

The quality of the resulting files that is served to the embedded player on the page will be directly related to the amount of bandwidth actually available to you at any instant in time during you viewing session and any errors or network congestion you are experiencing in real time. The more bandwidth you have and the better your network is performing, the higher the quality of your viewing experience will be, all the way up to 720p high-definition video.