We receive, process, and store A TON of video files, and we have different options when it comes to encoding. This post discusses some of the differences between hardware and software encoding, and some selected technology options for achieving the best results.

Hardware vs. Software

Video encoding can be performed by specialized encoding hardware like Amazon’s Elemental encoders, or by software on a regular computer.

Hardware encoders are fast, efficient, and the simplest ones require only one video input to provide one video output. These devices can be super fast, and are commonly used for streaming video and, with few maintenance requirements, you get a reliable solution for encoding video. So, what’s the catch?

The weak point is actually in the hardware itself. You are about to build a one-man media streaming company and platform, you bought yourself a shiny hardware encoder, and you are basically bankrupt and left with an already-obsolete piece of hardware. The initial price for hardware encoders can be really high, as they are typically specialized and hard to upgrade - especially compared to a software solution. Introducing a new video codec or bitrate ladder, for example, often means that you have to buy a new piece of hardware. This can be easily overcome by using an encoder-as-a-service like AWS Elastic Transcoder or Zencoder. They provide many of the benefits of hardware encoders with few downsides. The cost can add up, but it’s often very reasonable when compared to buying hardware.

With a software-based solution, everything seems to be easier. A new codec can be introduced in a matter of minutes, and upgrading existing hardware could not be easier. Parts are cheap in comparison to dedicated hardware encoders, and you can reuse it afterwards for different purposes.

Prices for hardware - computers, servers, etc - are (almost) continually declining, and parts are widely available. The main downside of a software solution is performance, which is highly-dependent on the hardware you are using. In most cases, all of the processing is done by the CPU. You have total control of a machine and you can fine-tune parameters of your encoding workflow to fit your needs and experiment a little bit.

Core meltdown

Over the past few years, security researchers have focused on the security of processors, discovering multiple vulnerabilities linked to various implementations of speculative execution. Multiple security issues have been discovered in many CPU models and designs, but Intel’s CPUs were affected the most. The most well-known vulnerabilities are Meltdown and Spectre, both covering multiple design flaws. Intel has responded with updated microcode with fixes, but, unfortunately, many of them have a heavy effect on CPU performance. More information can be found here.

Even though those mitigations should be enabled for security reasons, Linux offers command line arguments to disable the mitigations, and allows us to use full CPU power again. However, such machines should be hardened by rejecting inbound communication outside of trusted zone (e.g. isolated network).

Rethinking hardware

We’ve used Intel-powered machines hosted at Hetzner’s as the powerhouse of our encoding farm for many years. We changed our configuration several times, but never left Intel. However, AMD has made a huge technological leap in recent years and it is currently ahead of Intel in the field of used lithography, where Intel is still struggling with transition to 7nm manufacturing and even returning back to 22nm to meet demand. Intel also refuses to lower their prices, which gives AMD an advantage in offering cheaper CPUs with higher performance. So, decided to give it a try.

You can read how we process and deliver video files in a two-part blog post Divide & Encode: How to Encode Videos Blazingly Fast - part 1 and Divide & Encode: How to Encode Videos Blazingly Fast - part 2. In short, we are using FFmpeg to encode delivered video files to 3 video codecs H.264, HEVC, and VP9. We utilize the multithreading that FFmpeg offers to encode one chunk of video into multiple qualities at once to save a lot of time. In production, this process is spread to all encoders evenly, but, for the purposes of this test we use only one machine.

Hardware used for test

prices and labels are based on Hetzner’s pricelist

Files used for test

As a performance metric, we are using “speed ratio” - calculated like this:

We calculated this value for the whole processing time and separated for each video codec that we currently support: AVC1, HEVC, and VP9, with several resolutions and bitrates encoded for each codec by one FFmpeg instance in multiprocess mode. The results were quite interesting, and we were surprised how well the new machines performed. Let’s take a look.

Results

all charts are interactive

From the chart above, it’s easy to spot the winner of this test. Now let’s move on to some alterations before we conclude the results.

Let’s take a look at the impact of Spectre and Meltdown mitigations on our use case. Note that I performed this test on PX61 machine as Intel should be the most-affected platform:

It turns out that disabling security mitigations on the Intel platform shows very little difference and is probably caused by other factors. The impact of kernel upgrade on AMD-based servers is also negligible and in the tolerance range:

Conclusion

From the results above, we see that AMD’s Ryzen 7 provides better performance than our current solution - and at a reduced cost. In fact, a Ryzen 5-based system provides almost the same performance at half the price, which means that we can scale our encoding farm to nearly double its current size for the price of our current setup.

Based on the results of our test, we decided that switching our encoding farm over to AMD’s Ryzen 5-based AX41 machines would be the more cost effective solution for us. It’s a nice bonus that the machines proved to be a little bit faster for our purposes. Intel is currently working on SVT video encoders and putting some effort into optimizing those encoders for AMD CPUs, so we may want to take a look at the performance differences between Intel and AMD platforms. But that’s another story.