Showmax CMS team attended DjangoCon 2022 in Porto where they learnt about Django and the things you can do with it, about the developer community around Django and how Django can be used in a wide variety of ways, from giving opportunities to get into coding with Django Girls to how to use hidden features of the framework to do things you didn’t even know you wanted to do! We share a list of recordings of very interesting and insightful talks from Django 2022. conference that we liked.

Experiences from DjangoCon 2022 in Porto

After a year of just virtual events taking place due to the COVID-19 pandemic outbreak, the Django community finally gathered in person for the DjangoCon 2022 conference and chose a very beautiful site – the city of Porto. Located in northwest Portugal, Porto is known for its rich history, stunning architecture, and thriving startup scene. Needless to say, we were excited to attend the conference and learn from the experts behind the Django framework, which has been a crucial tool in building Showmax's Content Management System (CMS).

During our visit to Porto, we were able to explore the city's famous landmarks and attractions, such as the iconic Clérigos Tower and the Cais da Ribeira. We were also treated to the city's famous golden sunset views and sampled its delicious seafood and world-renowned port wine. It was a wonderful learning opportunity and a chance to network with our peers, but also a moment of exploring the rich culture and history of this vibrant and dynamic city.

The event was held in The Super Bock Arena (Pavilhão Rosa Mota) which was definitely a good choice. The venue provided ample space for attendees to network and enjoy the talks and workshops. One of the unique features that stood out was the roof tour, which was a great opportunity for attendees to take a break from the conference and experience a different perspective of the city. The tour finished with a tasting of traditional port wine poured into a cup made out of chocolate!

The conference was full of amazing speakers, which were not only talking about Django and the magic things you can do with it but also about our developer community and how Django can be used in a wide variety of ways, from giving opportunities to get into coding with Django Girls to how to use hidden features of the framework to do things you didn’t even know you wanted to do!

One of the most interesting talks was done by Daniel Roy Greenfeld (@pydanny), the legend behind the series of books called Two Scoops of Django, which every Django developer has used, directly or indirectly, to learn how to build things with the framework at some point in their career. During his talk on Scaling from one to Billions, Daniel delved into the fundamental aspects of being a good developer and how to translate these skills into projects that can impact billions of people while keeping in mind the ethical considerations that arise from such influential work. In the end, Daniel’s talk had little to do with Django itself, but left us with a lot to think about regarding our daily job and how it affects people around the world.

Another speaker, Katie Mclaughlin, made us think "What should you have to worry about?” She presented her view about the multiple trends we have popping up all the time on our daily job as developers and how to identify the ones that are actually worth our attention or even our time. In the talk she described in detail three main topics: Stable and Scalable Infrastructure, Reliable and Resilient Code, and Thoughtful and Thriving Community. We really recommend you watch her talk as it was not only very informative but also presented in a very playful and fun way.

On a more technical side, we had a lot of amazing speakers talking about different topics, but the ones who stand out the most for us were:

Add Multi-Factor Authentication (MFA) to Django in Mere Minutes: we had the opportunity to learn from Justin Mayer (@Jmayer) about the importance of Multi-factor Authentication (MFA), what are the types available and, of course, how to add it to Django in mere minutes!

Introducing Django Ninja: presented by Vitaliy Kucheryaviy, who walked us through a new way of building REST APIs in Django that uses python type hints in a clever way to validate, convert and serialize the endpoints’ inputs and outputs, and also automate API documentation!

“Observe!”: Honza Král (@honzakral) covered many useful tips and tricks to help improve monitoring, observability, and debugging of the running Django applications, such as logging, APM (application performance monitoring), distributed tracing, error tracking, and other things which are essential for any production Django app.

Handling Django in highly concurrent & scale environment: Tarun Garg told us about his journey toward trying to improve the performance of such things as Django admin pagination, text search, and Django models caching in a highly concurrent environment, which was a very educational experience for us!

	By the end of every conference day there was a chance for people to participate in the Lightning Talks, giving five minutes to each speaker to talk about a topic of their choice. This was a really nice opportunity for those who wanted to have their first talk in a less strict format. That doesn’t mean that the talks were not serious or anything like that. We listened to multiple speakers talking about various interesting topics. For example, we listened to a talk about the maintainer of django-upgrade, which quickly presented a tool to us which will make the upgrade of Django major versions easier. It was a major announcement which the Django community has been waiting quite a while for.

In conclusion, the Django Conference is a testament to the thriving and dedicated community of developers using the Django framework. From all around the world, individuals and teams are utilizing Django to build impressive and innovative projects. The community of Django users is a major factor in the continued success and popularity of the framework within the Python ecosystem. The conference was very well organized – the great venue, interesting talks, and inspirational networking were wrapped up with an excellent final party and the extra addition of a free boat trip around the city of Porto. It was a very nice addition to switch the focus from the conference and get to know and enjoy this wonderful city from an unusual angle.

We are already looking forward to DjangoCon 2023, which this time will be in yet another amazing city: Edinburgh! Hopefully this time we might even be one of the speakers telling the world about the crazy things we do with this amazing technology in order to bring moments of joy to our users!

Thanks Bogdan Bodnar for co-writing this post!

DjangoCon 2022 in Porto: Insights from the World’s Biggest Django Conference

Learn about the most popular JSON libraries for Python such as orjson, ujson, simplejson, rapidjson, and nujson. This article provides an overview of each library's purpose and history, and then compares their performance, features, and use cases to help you choose the best option for your needs. Also, includes a benchmark comparison for encoding and decoding operations of different JSON libraries.

JSON Python Libraries Overview: A Comparison of Performance and Features

JSON has become one of the most popular formats for encoding structured data, particularly when building RESTful APIs that exchange data with clients or between services. It can also be used in other contexts, such as data processing or machine learning, where structured data needs to be stored or transmitted in a standardized format. 

While Python's built-in json module provides a simple and reliable way to work with JSON data, it is useful to know that there are several other libraries available that offer better performance and additional features. 

In this article, let’s compare some of the most popular JSON libraries for Python such as `orjson`, `ujson`, `simplejson`, rapidjson, and `nujson`. We will do a brief overview of each library's purpose and history, and then compare their performance, features, and use cases to help you choose the best option for your needs.

First of all let’s take a look at JSON itself. JSON (JavaScript Object Notation) is a lightweight data interchange format. The main purpose of JSON is to be easily writable and readable for humans and easily parsable for machines at the same time.

JSON is suitable for a variety of purposes, including transferring data between a client and a server in web applications and APIs and storing data in databases or files (MongoDB, Elasticsearch, etc.).

It’s not necessary to have a root object by specification, although it’s widely used. A JSON object is represented as a collection of key-value pairs, enclosed in curly braces. 

Each key-value pair consists of a key (enclosed in double quotes) followed by a colon and a value. Multiple key-value pairs are separated by commas.

Values can be of several types, including strings (enclosed in double quotes), numbers (integers or floating-point), booleans (true or false), null value (null), arrays (ordered collections of values enclosed in square brackets), and objects (collections of key-value pairs enclosed in curly braces).

Here is an example of a JSON object for the movie Shrek (we will use it later for our benchmark part):

If you aren't familiar with the syntax of JSON or want to investigate how it is defined in detail, there are two main resources worth checking: json.org or ECMA-404.

Serialization of JSON is the process of converting structured data, such as Python objects or database records, into a JSON-formatted string that can be transmitted or stored. This serialized JSON data can then be deserialized, or converted back into its original format, when needed.

Serialization of JSON is a common task in web development, particularly when building RESTful APIs that exchange data with clients. It can also be used in other contexts, such as data processing or machine learning, where structured data needs to be stored or transmitted in a standardized format.

JSON serialization has several potential disadvantages, including limited support for data types, lack of schema validation (which can be solved by using JSON Schema), large file sizes, and performance overhead.

As we embark on this delightful journey through the world of Python JSON libraries, let's take a closer look at their unique flavors and characteristics:

What was originally called simplejson was added as json to the Python standard library starting from version 2.6. It was written by Bob Ippolito. The built-in json library is included with Python. While it may not be the most performance-oriented choice, it's perfect for simple use cases and quick JSON encoding and decoding tasks. For many developers, this library is the go-to choice for everyday JSON handling.

simplejson follows its own development path, separate from the standard json module, while still maintaining compatibility. This allows simplejson to cater to the needs of users who require support for older Python versions.

ujson is a high-performance JSON library, written in pure C, designed for speed without sacrificing compatibility with the built-in json library's API. For applications that require rapid JSON encoding and decoding, particularly when dealing with large data sets, ujson is an excellent choice to elevate your JSON processing capabilities. It is compatible with Python 3.7 and above.

`nujson`: A fork of the high-performance `ujson` library, nujson specializes in handling NumPy objects, particularly arrays, by providing native serialization support. It's an excellent choice for projects that involve scientific or numerical data and require efficient processing of NumPy objects alongside standard JSON data types.

RapidJSON is a high-performance library, written in C++. It’s fast, doesn’t have additional dependencies, is memory-friendly, and it also has validation support based on JSON Schema. While RapidJSON itself is a C++ library, there are Python bindings available, such as python-rapidjson, that allow you to use its functionality within Python projects. It’s compatible with Python 3.

orjson is a high-performance JSON library implemented in Rust, offering Python bindings and focusing on speed and JSON specification compliance. Its unique features include serialization of non-string keys in dictionaries and improved handling of datetime objects. It’s suitable for projects where performance is critical and non-traditional data types are frequently used.

The usage of these JSON libraries in Python is quite similar due to their compatibility with the built-in json library's API. The primary functions for encoding and decoding JSON data, dumps() and loads(), are consistent across these libraries. It allows you to simply replace the built-in json library with an alternative by importing it as json.

# Built-in json
import json

# simplejson
import simplejson as json

# ujson
import ujson as json

# python-rapidjson
import rapidjson as json

# orjson
import orjson as json

# nujson
import nujson as json


Then, the usage of each of the mentioned libraries will look the same:

# Serialization
encoded_data = json.dumps(data)

# Deserialization
decoded_data = json.loads(encoded_data)


Thus, since the usage of these libraries is quite similar, let's proceed to the pinnacle of our exploration journey and evaluate the performance of these JSON libraries in terms of speed.

First of all we need to install all those libraries:

pip install numpy simplejson ujson python-rapidjson orjson nujson

We will use a previously mentioned sample JSON dataset derived from the movie Shrek. This dataset will be stored in a file named shrek.json for our testing purposes.

The code for our benchmarking function will appear as follows:

def benchmark(library):
    encoded_data = library.dumps(data)
    decoding = lambda: library.loads(encoded_data)
    encoding = lambda: library.dumps(data)

    encode_time = timeit.timeit(encoding, number=10000)
    decode_time = timeit.timeit(decoding, number=10000)

    return encode_time, decode_time

Whereas data is already loaded `shrek.json` into the python dictionary:

with open('shrek.json', 'r') as f:
data = json.load(f)


Here are the results of the benchmark tests for the JSON libraries, displaying the time taken for encoding and decoding operations:

Based on these results, we can observe the following:

Built-in json: This library shows moderate performance in both encoding and decoding operations.

simplejson: The results indicate that this library is slower in encoding compared to the built-in json, and has approximately the same decoding times.

ujson: This library demonstrates better performance in both encoding and decoding operations than the built-in json and simplejson.

nujson: Tailored for handling NumPy objects, nujson demonstrates competitive performance in both encoding and decoding operations compared to ujson. While its primary focus is on NumPy serialization, it remains a viable option for standard JSON data types.

python-rapidjson: The performance of this library is quite similar to that of ujson and nujson, with slightly faster encoding times but marginally slower decoding times.

orjson: This library shows the best performance in both encoding and decoding operations among all the tested libraries.

As mentioned earlier, these results provide a general idea of the performance differences between the JSON libraries. Although, some JSON libraries are more suitable for handling NumPy objects, such as arrays, thanks to their specialized support for these data types. One such library is `nujson`. Let’s perform a separate benchmark for these libraries and check how they hold NumPy data.

Instead of Shrek our data will be generated with the NumPy library:

data = np.random.rand(1000, 1000).tolist()

Built-in `json`: This library shows moderate performance in both encoding and decoding operations. It is suitable for simple use cases and general-purpose JSON handling, even when dealing with NumPy arrays.

`simplejson`: The results indicate that this library is slower in both encoding and decoding operations compared to other libraries.

`ujson`: This library demonstrates better performance in both encoding and decoding operations than the built-in json and simplejson. It is a great choice for applications that require faster JSON processing, including handling NumPy arrays.

`nujson`: Surprisingly, its encoding performance is slightly slower than ujson, but it is still twice as fast at decoding.

`python-rapidjson`: The performance of this library is quite similar to that of `ujson` and `nujson`. It can be considered an alternative for handling JSON data, including NumPy arrays.

`orjson`: This library shows the best performance in both encoding and decoding operations among all the tested libraries. Seems to be an excellent choice for projects that require high-performance JSON handling, especially when dealing with NumPy arrays.

To summarize, `orjson` stands out as the top performer, but the built-in json library remains a practical and convenient option for many projects. When choosing a JSON library, consider your specific requirements and weigh the benefits of each library against your needs.

If you seek further optimization, non-JSON serialization alternatives can offer enhanced performance and storage efficiency. Understanding their benefits and use cases allows you to make informed decisions for your project's data handling needs. Let’s briefly take a look at some of them:

Protocol Buffers (protobuf): Developed by Google, Protocol Buffers are a language-agnostic binary serialization format. They are designed to be compact, efficient, and fast. The advantages of protobuf include smaller message sizes, faster encoding and decoding, and a strongly-typed schema. They are especially useful for communication between microservices or distributed systems.

MessagePack: This binary serialization format is an efficient alternative to JSON. MessagePack is designed to be compact and fast, with support for many programming languages. It offers smaller payload sizes compared to JSON and better performance in encoding and decoding. However, it might not be as human-readable as JSON.

Apache Avro: Avro is a binary serialization format developed by the Apache Software Foundation. It is especially suitable for big data and distributed systems like Hadoop. Avro is schema-based, which ensures compatibility between different versions of data structures. It offers compact data representation, fast encoding and decoding, and supports many programming languages.

Apache Thrift: Originally developed by Facebook, Apache Thrift is a binary serialization format that supports a wide range of programming languages. It provides a compact and efficient binary format, a strongly-typed schema, and enables cross-language communication. Thrift is suitable for microservices and distributed systems.

These alternatives offer various advantages, such as improved performance, smaller message sizes, and support for type safety and schema evolution. However, they may not be as human-readable or as widely supported as JSON. It's important to consider the specific requirements of your project when choosing a serialization format.

In this article, we have explored various Python JSON libraries, comparing their features and performance. While the built-in json library is suitable for many use cases, several alternative libraries offer unique characteristics that may cater to specific requirements, such as nujson can be a good choice for working with NumPy data. We have also discussed non-JSON serialization alternatives, which can provide further performance improvements and storage efficiency. The choice of a serialization library or format will depend on your project's needs, and conducting thorough benchmark tests will help you determine the most suitable solution.

More than two years ago, in July’s high season, we launched a new top-notch product, Showmax PRO. In Forbes, Showmax Pro was called a Streaming Soccer Service. But there is much more to Showmax Pro than just live soccer games (find out more). This brand new subscription plan bundles the existing Showmax library together with various live sports from SuperSport and adds music channels and news on top.

Have a look at Showmax Pro’s launching video

The Business and Product teams did a great job coming up with an idea of live sports on Showmax. In Africa, live sport streaming brought us an important competitive edge. Rugby and football World Cups are the most popular sporting events in both South Africa and the rest of Africa. We’ve also covered the majority of the 2020 Summer Olympics and are getting ready for the 2022 FIFA World Cup. Needless to say, the new plan and functionalities brought our engineering team a bunch of interesting problems and challenges to tackle.

We’ve been in the video on demand (VOD) business for a few years already and came up with a solution to (almost) every issue, so it was time to open another door and take on another challenge – live streaming.

Now that the new product has been launched and running for some time, I can say this: with a bit of exaggeration, streaming movies on demand is easy when compared to streaming live sports. Simply put, all you have to do in the VOD business is cut and encode the videos, load them into your internal CMS, set up the metadata and upload the images, run it through QC, publish, and after a few minutes the movie appears in your apps. The rest is about streaming the content using the CDNs. Sounds simple, right? There’s nothing to stop you from making the content available for months or years. If it works once, it should keep working.

But when it comes to live sports streaming, there’s a lot more to handle on top of what’s needed for VOD. The data is always outdated. There are always last minute changes to the tournament roster, delays or overtimes, or even cancellations due to bad weather. These are just some of the scenarios that we’ve faced and that led us to re-think how our platform works. With live sports, you simply cannot plan for the future. You cannot use the processes fine-tuned for VOD; they are neither fast nor flexible enough and you need constant supervision and full control over the content. If a live stream goes down, fixing it in 10 minutes is not good enough as the match goes on – users would have already missed Ronaldo’s awesome goal.

On the other hand, sports were not an entirely new challenge for us. Our sister company SuperSport has decades of experience in broadcasting sporting events to all of Africa. Additionally, DStv, another sister of Showmax providing catch-up service with live channels, helped us with the live streaming experience and infrastructure. So along with the content we gained a lot of useful know-how. It’s great to be a part of a big family, right? Since we had the content and know-how, it may seem like we were all set, but there was still a lot to learn.

Showmax PRO was launched as a new type of Showmax subscription, with both a normal and a mobile-only variant. Once the users upgrade to the new PRO subscription, they can immediately start watching the sport content. But making that content available is not such a straightforward process.

Manual curation and scheduling is not possible for live streaming, because, as explained above, it would mean a tremendous and neverending load of work. To solve that, we integrated Showmax with the SuperSport systems to synchronise our systems with the latest schedules available. We rely on full automation with rare manual interventions – used in cases of, for example, unexpected overtime.

The content database that we’re using for VOD has long update times, and to tackle this problem, we have developed a parallel metadata pipeline. It is used solely for live content and enables the immediate propagation of any change to production. Immediate in this case means that the process of updating the schedules across our applications is finished within one minute, compared to ~10 minutes for our VOD content.

To create the actual live stream, we are using the same video feeds (channels) as we do for SuperSport’s satellite TV broadcast. We run it through on-premise AWS Elementals and control when or for how long the feed is available for the users to stream the match or event they want to watch. From the user’s perspective, this is quite different from what most of the competitors offer and we believe it makes perfect sense. The concept of 24/7 live channels works for TV, but not necessarily for an online VOD service.

Focusing on the customer experience, we want to stream the match from start to finish, nothing more and nothing less. It may seem easy at a first glance, but it proved to be very challenging. Different countries need to receive different video feeds, sometimes at different times. The requirements for serving ads differ as well. For example in Mauritius, alcohol advertising is illegal. We needed to come up with a solution for that – in the first phase, the solution was to dynamically replace the ad with a still image. Later on, we plan to replace the ads within the video feed with dynamic ads.

The live feed can also be influenced by other events and the stream can actually change before the event is finished. And that’s where the need for full real-time control originates. Still, we do believe that our approach of “events” rather than “channels” makes sense from the user’s perspective and we plan to keep it in the future.

Now, when the live stream is set up and everything is in order, the users can start tuning in for the kick off. There’s one small problem though – a majority of the users will launch the app and the live stream at almost the same time, which results in a huge spike of traffic and requests that need processing. Surprisingly, very similar spikes can also be seen during half-times or at the end of the match, when people start leaving the stream and potentially looking for another movie to watch.

We took multiple steps to avoid overloading the platform. Apart from several small optimizations, two features play the key role.

Firstly, we have improved our “overload mitigation” functionality. Based on the current need, we simply set the mitigation value (0 - no mitigation, 1 - maximum mitigation) in our HAProxies, which pass it in a header as part of all requests to our services. Each service can then work with the header as necessary. In general, the higher the value, the less personalised and more cached the platform is.

The second feature is much simpler – we allow the users to start the stream 10 minutes before the actual start of the match. During these 10 minutes, push notifications are also sent out, staggered across the duration. This spreads the user requests out over the time period and significantly helps with the load distribution. It’s also convenient for the users who do not need to wait for a specific time to start the stream.

In the near future, after the transition to AWS is finished, the platform will become much more scalable within a short time, which will also help significantly.

Everyone hates when there’s an outage and you have to learn about your home team’s goal from the ecstatic screaming of your neighbour…we’ve all experienced that, and our customers deserve better service. It’s crucial to catch such interruptions as soon as possible. In Connected Video, we manage more than five hundred live channels and use several levels of monitoring.

We leverage Prometheus and Alertmanager heavily as the primary monitoring tools of our services. For the live channels we use Blackbox exporter, allowing us to probe channel playlists and manifests in regular intervals. We make a simple assumption, that if a probing HTTP request returns 200 OK and the last-modified header is not older than segment length, then the live channel is alive. It’s a relatively cheap approach to probe availability from different destinations and CDN paths.

On top of that, we also monitor statistics and events from devices during playback, which provide effective insight into the state of live streaming (and VOD as well). This helps us catch various issues across a variety of devices.

Premium and most popular SuperSport channels or events are additionally monitored by a commercial monitoring tool called Stream Probe and Witbe. It’s another level of active monitoring to ensure the top quality.

We have also extended our CMS with a page allowing the sports team to monitor all the running live streams simultaneously, and to easily adjust the schedule if necessary.

In general, you can say that quality of experience arises from the convenience of use. To help users easily find the matches they want to watch, we designed a brand new timeline-based sports page which looks nothing like a typical EPG (Electronic Programme Guide). Users can filter the content based on the sport genre or time and also set up notifications for selected future live events.

From the technical point of view, we did several upgrades compared to the first solution. At the beginning, we provisioned dedicated live streams for Showmax provided by the DStv infrastructure. It was not optimal, as the same content which was broadcast on both platforms was packaged and cached on the CDN twice. In the next iteration, we focused on sharing the content on the CDN better, as the events were often broadcast on both platforms (even though the platforms target different groups of customers). It helped lower cache fragmentation.

Another enhancement is active content prefetching. It means that the CDN tries to fetch the next segment of the stream before the first customer asks for it. It lowers the response time for the first customers, as well as on less popular channels in general or in sparse areas. This feature is provided by Akamai, which we use as the primary CDN provider for our live streaming and it’s also supported by our Unified Origin component via hinting the next segment in headers.

The final indicator of good quality of experience is “glass to glass” latency. It’s a delay between real live action in the stadium and what you see on your screen. Standard latency of HTTP based live streaming is between 20 to 40 seconds. By tuning parameters (like segment length or buffer size in player), you can get significantly below these values. But it’s always a matter of balance between picture quality, buffering issues, and other attributes of good experience. We took inspiration from Zatoo’s posts 1, 2, and 3 to tweak it on our side.

Content piracy is a big issue all over the world, and Africa is no exception. Live sports streaming is more targeted and at higher risk of being pirated than VOD. Content rights are sold for shocking amounts of money and streaming companies do their best not only to protect the content but also track the leaks and catch the violators.

Nowadays, all content is protected by DRM, and surprisingly even DRM can’t provide sufficient protection. We all remember Widevine being breached several times in the last year and it probably won’t be completely fixed ever again. Although it still works fine against casual attackers (understood as BFU).

Anti-piracy efforts are mainly focused on a different group of violators – highly specialised crackers who build their business on re-selling stolen content. They don’t just cause losses by re-selling but also re-using the CDN infrastructure of the content owner. For such cases, we incorporate geo-blocking on the CDN to mitigate that behaviour. We work closely with Irdeto (another sister company) who help us identify and track pirates. Together we look for suspicious behaviour in the streaming environment, like unexpectedly high traffic on specific IPs. Thanks to them, several piracy businesses were traced and closed down eventually.

Missed the live action? Highlights are short previews of matches that our users can watch to quickly catch up on the games they’ve missed, or to enjoy the best moments again. To ensure we receive sport highlights as soon as possible, we had to significantly speed up our video delivery workflow. We imposed a rule of releasing these highlights within one hour after the game ended, and in order to comply, we started using standardised “mezzanine” video input within our encoding pipeline and limited the number of resulting video profiles.

We briefly mentioned at the very beginning that enabling live streaming included not just sports but also 24/7 streams of multiple news and music channels, including the South African national TV channel SABC with regular COVID-19 updates by the country’s president, Cyril Ramaphosa. The Nigerian reality show “Big Brother Naija”, which we were able to stream 24/7, was a great success too. We can also stream live concerts, beauty contests, or other live shows. Live streams are a huge leap forward in the variety of Showmax content libraries and the effort definitely paid off.

We already have plenty of ideas of what to do next and what else to improve. One of the areas we plan to focus on is content discovery – making sure the right people find the right content. We have already done a great job in VOD, so now it’s time to use the know-how for sports. For that, we plan to utilise more of the league’s and teams’ data.

Would you like to understand some of the above even further? Let us know on Twitter.

Delivering content securely, with best possible user experience

On the Showmax Engineering blog, we try to explain solutions to a pretty specific problem: how to deliver a high quality video streaming in Africa. To make our stories more intelligible, we started the series The Showmax engineering story, with the goal of explaining the meanings of in-depth media engineering terms that we use so commonly.

Now, with the very last piece of vocabulary at hand, you have the tools to understand it all completely. If you missed the previous parts, you can read about encoding terms in The Showmax Engineering story, Part I, and streaming terms in The Showmax Engineering story, Part II. In this very last edition, we describe the terminology around content protection and quality evaluation.

Let’s have a look behind the curtain of content protection against piracy.

Digital rights management (DRM) is a technology-based control over copyright-protected content, and it’s a fundamental part of the content protection environment. Its rules are defined within the contracts with content owners — in our case, that means contracts with studios. Basically, for every piece of content we have device constraints (what device we can display the content on and in which quality/resolution) and time constraints (how long we’re allowed to host it).

There are three main DRM types used in the market.

Widevine by Google is mainly supported by web browsers (Chrome, Firefox, Edge), Android devices, and SmartTV devices. Google provides a cloud solution for the license server, which is free to use, and includes no charges for license generation or device integration. It also provides an SDK to implement your own license server.

PlayReady by Microsoft is mainly supported by the Edge web browser, SmartTV devices and gaming consoles (Xbox, PlayStation). Microsoft provides the SDK to implement your own license server, or you can use any third party DRM provider that supports PlayReady.

FairPlay by Apple is mainly supported by the Safari web browser and iOS/tvOS devices, and was originally developed to protect music in the iTunes Store. Apple itself doesn’t provide any license server. You can either follow the guide on how to implement your own server, or use a third party DRM provider. Further details and specifications that go beyond the scope of our vocabulary can be found in the OTT Verse glossary.

All three variants use a family of AES encryption algorithms, the difference is only in supported mode - full sample encryption in CTR mode (cenc), or pattern encryption in CBC mode (cbcs). This is defined as part of Common Encryption Scheme (CENC), which defines the standards for encryption and key mapping.

If DRM systems support CENC, content can be decrypted by any of these systems as long as the same key IDs and content keys are used. This allows content providers to serve the same content encrypted once to a broader range of devices using different DRM systems. It’s also called multiDRM.

To close this section, we need to mention two MPEG standards, MPEG Common Encryption, which defines encryption and container formats; and MPEG-A Part 19: Common media application format (CMAF) which handles conversion between them without re-encryption and re-encoding. Unfortunately, it does not define which AES mode should be used. While Apple decided to support “cbcs”, Widevine and PlayReady started with “cenc” and added “cbcs” mode later. Therefore, implementation of CMAF in practice is slow as you still need to have both variants to support older devices.

Technically the content is encrypted by a key, which is shared as part of the license that defines the conditions on which a device can use the key(s) to decrypt the content. Limited validity, requested security levels of the device, or the minimal supported HDCP are all examples of such conditions (a decryption key is also referred to as a content key). To find the right key, the device must provide a content ID (ID identifying a piece of content in the register of the DRM provider), and/or key ID (ID of the key used for content encryption) to the license server. Those values are usually baked into playlists/manifests, or directly in media segments. The License server then issues the actual license with the key to the device. The license is processed in a trusted execution environment (TEE) of the device — a secure area of the processor that ensures that the code and loaded data are protected, and provides hardware security for trusted applications.

Content decryption module (CDM) can be a software (compiled binary library) stored in a browser or hardware solution. The CDM decrypts the DRM-protected content using the key issued by the DRM license server.
Encrypted Media Extensions (EME) is specified as a communication layer between CDM and web browsers to play HTML5 video files protected by DRM. This obsoletes third party plugins like Adobe Flash or Microsoft Silverlight provided this functionality in the past.

Security level defines the minimal level of DRM security requested, and specifies where the decryption can be processed. Levels of security can differ across DRMs, but in general, it can be defined as software DRM (less secure, usually up to SD quality) and hardware DRM (required for HD and higher).

Security LevelOutput protectionNoteSD (up to 576p)L3 / SL2000 (Software)-Less strict DRM protectionHD (up to 720p/1080p)L1 / SL3000 (Hardware)Any HDCPStricter DRM protectionUHD (4K/8K)L1 / SL3000 (Hardware)HDCP v2.2+Highest DRM protection (may require additional security like forensic watermarking)

High-bandwidth content protection (HDCP) is a security measure used to transfer content to a display using compatible digital outputs including DisplayPort, DVI or HDMI. HDCP prevents unauthorized devices from reading high-quality content. Both the display and the digital outputs must be certified HW, and must support HDCP, otherwise the content is downgraded to a lower quality. If no HDCP is found on any side, it leads to a failure. HDCP can be restricted based on the content type (0 – any version, 1 – 2.0 and higher), or by version directly.

Fairplay and PlayReady use content type, and Widevine uses versioning.

Be aware that DRM encryption is not the same as what’s used for the HDCP encryption when sending the signal over the cable.

Forensic watermarking (or digital watermarking) is the practice of adding invisible markers into video tracks to uniquely identify authorized users. It can be understood as something like a signature. A simplified explanation is that, at the beginning of the packaging process we have two variants of the stream, one signed with A char, and the other with B char. The playlist is then composed as a unique sequence of segments from both streams. This signature can’t be removed by re-encoding the content, so it can be easily used for tracing leaked streams.

Content delivery network is a set of servers distributed across the country or continent to increase the speed of content distribution. It usually consists of many powerful cache servers connected into layered/tree topology.

Origin servers contain all of the catalog content, and are therefore a kind of source of truth. They have massive disk capacity and don’t serve the content directly to customers — they are protected behind edge servers, or mid tier servers that sit between origins and edges. Mid tier/edge servers hold the most popular content in local cache with less capacity but more speed (RAM or quick disks like SSDs), in order to serve higher demand from popular content very quickly. The effectiveness of a given CDN architecture can be evaluated by cache-hit ratio. The higher the ratio, the more requests are served from the local caches (faster response) with fewer requests passed to lower layers (and, eventually, to origins). Such architectures, just at a larger scale, are also used by big third party CDN providers like Akamai, Limelight and Cloudfront.

You can find out more about the Showmax content delivery network in the CDN series.

All of the terms mentioned within this series describe factors that influence the quality of our customers’ experience. Quality of experience (QoE), our very own metric, describes how our customers are satisfied with the product we provide them. It’s an attempt to objectively measure subjective opinion.

Some general QoE metrics worth mentioning:

Buffering time is the time when the player waits for data/content while playback is paused. A related value is re-buffering, which is calculated as the proportion of buffering time to consumed/played time. It can be influenced by an overloaded CDN, network issues between CDN and the device, and several other things. In general, lower video bitrates are less prone for buffering.

Another one is the number of failures during or before real playback that cause unrecoverable interruption. They can be caused by bugs on the backend or frontend, issues with the network between CDN and the device, damaged encoded video titles, and more.

Last but not least is average bitrate, which tells us the average video quality the user watched - which is often tightly-connected to buffering. Lower average bitrate usually means that viewers watched lower quality videos, which can be a precursor of buffering issues. It can be caused by overloaded CDN or device, network capacity, etc.

Mean opinion score (MOS) is an arithmetic combination of individual values/metrics that expresses a user’s QoE in one value. It’s very useful to define the relative importance of the metrics, and it helps us set priorities for improvements — as an improvement in one metric can cause degradation in others and vice versa. For example, we can decrease the buffering time and get a smoother stream by lowering the average bitrate. On the other hand, with the lower bitrate we also decrease the picture quality. So we need to clearly identify the higher priority, whether it is lower buffering or better picture quality.

In the edition about encoding parameters, we omitted to mention how you can evaluate the outputs produced by different encoder configurations. Popular mathematical approaches are Peak Signal-to-Noise Ratio (PSNR), or Structural Similarity Index Measure (SSIM). They evaluate picture quality from their exact perspective, and do not necessarily take into account the subjective perception of humans. Despite this, they are still heavily used as base evaluation methods, where anything under 35 PSNR score can be considered as ugly (e.g., with artifacts), while scores beyond 45 are considered to be unrecognizable from the original source (therefore wasting bits).

Subjective video quality can be predicted by Video Multimethod Assessment Fusion (VMAF), which takes into account the perception of quality by human eyes. A well-balanced score of quality and reasonable data consumption may impact the aforementioned QoE metrics and lead to their improvement. The trend is to keep bitrate low and quality high - which are in most cases two contradictory characteristics. More about using VMAF to find optimal bitrate ladder you can read in Jan Ozer’s posts 1 and 2.

After finishing this series introducing the common terms from the media engineering world — the core of our business — it’s easy to see why providing a quality SVOD service requires the joint effort of several teams. It’s challenging, and we love it.

Continue your discovery path by reading another series of blog posts, this one about delivering the content the right way. Our great CDN series starts here: Selecting the right URL. In case you’ve already read the beginning, you can continue with the second or third part.

Join us and build a streaming platform used by millions. 

We’re excited to witness the recently announced partnership between Showmax, Sky and NBCU coming to life here in Prague. Following the recent announcement of our partnership; that will bring some of the world’s best content and technology to streaming customers across sub-Saharan Africa, Sky are hiring developers to make it happen.

You might know Sky for TV. But we’re more than that. We’re a tech business. We’re broadband and broadcasting, streaming and sports, mobile and magic. And we’re proud to be part of the Comcast NBCUniversal family, creating incredible technology and entertainment that connect millions of people to the moments and experiences that matter most. 

We’ve already rolled out NBCUniversal’s streaming platform Peacock to our European audiences. Now you have the opportunity to work on an exciting new venture powered by Peacock’s leading, globally scaled technology. 

You will be part of a dynamic team developing client applications and making sure our technology delivers the best experience for our customers.

Craft high-quality software utilizing agile methodologies

Design clean, efficient, and reliable code

Deliver highly scalable and extremely reliable products (our customers demand 99.999% availability)

Collaborate with fellow developers, exchanging knowledge and insights - prepare to learn a lot!

Passionately introduce top-notch technology and software development practices into the team

Participate in the entire development cycle, from analysis and implementation to testing and operation

Embrace operational excellence and support as integral aspects of the development team's responsibilities

Proficiency in native iOS languages, specifically Swift

Familiarity with Design Patterns and Object-Oriented Design principles

Comprehensive knowledge of key iOS architectural components, including Model-View-Controller (MVC), Model-View-ViewModel (MVVM), and Coordinator patterns

Practical knowledge of network protocols, along with an understanding of latency and reliability challenges

Demonstrated experience with multithreading, request queuing, and caching in code

Familiarity with Continuous Integration/Deployment tools and processes

Excellent teamwork, self-motivation, and eagerness to learn

Creative thinking, strong analytical skills, and attention to detail

 Working in an international environment on an impactful globally scaled technology in cross-functional team with an agile approach 

Open and healthy working culture where we care about people as much as about stability of the code  

Work equipment as per your specifications 

Hybrid work, combining working from our office to be closer to the team and working from home 

Work life balance thanks to flexible working hours

Generous free day scheme providing 25 days of holidays + 5 days of on-demand leave (sick days)

Ergonomic office and kitchen full of tasty snacks 

Contribution to Multisport Card or Benefit Card 

Learning opportunities with access to Udemy, O’Reilly, and our internal library of physical books, as well as a budget for attending conferences, including strong support if you want to present at a conference yourself 

Events, meetups, hackathons in the office or elsewhere

 We’re excited to witness the recently announced partnership between Showmax, Sky and NBCU coming to life here in Prague. Following the recent announcement of our partnership; that will bring some of the world’s best content and technology to streaming customers across sub-Saharan Africa, Sky are hiring developers to make it happen.

We’ve already rolled out NBCUniversal’s streaming platform Peacock to our European audiences. Now you have the opportunity to work on an exciting new venture powered by Peacock’s leading, globally scaled technology.  

Our team develops and supports market-leading video streaming services, underpinned by state-of-the-art engineering principles. We do this at huge scale: for millions of customers globally. No matter the device, the time or the place, we make sure that our diverse audiences can easily find and enjoy whatever they want to watch, choosing from the world’s best entertainment, news and sport.

You will join the Online Video Platform team (OVP) within Global Video Engineering & Apps where you will be positioned within business-critical propositions that support all our global streaming initiatives such as NOW, Sky Go, Peacock and SkyShowtime. 

You will be responsible for automating and streamlining tooling for cloud-based services using a range of scripting languages, automation tools, and cloud-native services. You will work with a variety of development / DevOps teams and business partners to ensure that cloud automation activities are secure, reliable, and efficient. We are on the lookout for motivated and outstanding individuals who want to be involved in ground breaking technology that truly pushes the boundaries and growth of streaming! 

Exploring and implementing new ways to automate systems, across a domain that powers business-critical streaming propositions globally.

Design and implement automation strategies, solutions and policies across a range of web API, network and cloud-based technologies.

Automate aspects of deployment and management of cloud-native applications.

Track and fix automation issues and develop solutions

Develop and maintain scripts to automate tasks and processes.

Collaborating with other scrum teams to help support their technical roadmaps by offloading deployment activities into our team workflow and achieving cross-team and department goals through automation.

Provide technical support and training to other team members.

Create and maintain detailed technical documentation.

What skills do you need to play your role well?  

Experience working with REST APIs and multi-distributed systems e.g. Cassandra

Familiarity with configuration and build tools including Jenkins, Gradle, Maven

An understanding of message driven systems e.g. Kafka, Amazon Kinesis, ActiveMQ

Applied knowledge of agile methodologies and software engineering concepts e.g. XP, Scrum, Pair Programming, TDD, BDD

Ability to communicate well with other teams and consult with developers of all levels of experience

 

Working in an international environment on an impactful globally scaled technology in cross-functional team with an agile approach 

We’re one of Europe’s leading entertainment and communication business. We’re always looking to improve by developing pioneering products and services which give every one of our customers experiences they’ll love. 

Join us as an QA Automation Tester and you'll be part of a dynamic team that makes sure our technology delivers the best experience for our customers.

Help to define and manage our testing processes

Participate in the team's agile planning & delivery process

Responsible for ensuring features meet specified requirements

Definition of test plans, test scripts, test data requirements, etc.

Planning test environments and automation strategies.

Preparation of automation frameworks and test runs for API back ends

About the team

Read our blog posts #media

DjangoCon 2022 in Porto: Insights from the World’s Biggest Django Conference

JSON Python libraries overview

Sports streaming at Showmax

The Showmax Engineering story, Part III.

DjangoCon 2022 in Porto: Insights from the World’s Biggest Django Conference

JSON Python libraries overview

Sports streaming at Showmax

The Showmax Engineering story, Part III.

Check out our other teams