Learn how to systematically test all endpoints in Django using powerful testing tools like the Django test framework and pytest. Ensure a seamless user experience and robust security by identifying and fixing broken links and maintaining all endpoints. Dive into this step-by-step overview of the testing process.

Comprehensive Guide to Testing All Endpoints in Django

Testing is a crucial aspect of software development, particularly in web development. Even a single broken link or endpoint can result in a poor user experience or security vulnerabilities. Luckily, Django offers powerful testing tools, including the Django test framework, which can be enhanced with third-party packages like pytest to tackle these challenges.

When dealing with large, long-standing systems, it's common to have numerous endpoints. Many of these endpoints may be infrequently used but still necessary. Unfortunately, they can easily slip from our minds when implementing changes, leading to unforeseen impacts on users that go undetected.

In this developer blog post, we'll dive into an approach for testing all the endpoints of a Django server. We'll utilize the "show_urls" command from django-extensions and the "get" method of pytest's django_app_factory plugin. Together, these tools will enable us to systematically test each endpoint. Join us as we walk through the entire process, providing a detailed explanation of how it all works.

The show_urls command is a useful tool provided by django-extensions that offers a list of all endpoints in JSON format. Each entry returned has four fields: url, module, name, and decorators. By iterating over the returned list, we can test each endpoint with a GET request to identify broken links and ensure that all endpoints are functioning properly.

To implement this test, we can start by using Django's call_command method to run the show_urls command and get a list of all endpoints.

The first step after acquiring all endpoints is to sort them based on whether they are maintained or not. This can be achieved by checking the module field that contains “path” for the endpoint, because the first part of the path is the app name. 

Having the app name, we can now easily check if the app is maintained or not, and skip the ones that are not.

Next, let's focus on constructing the URL for our GET request. To ensure we have all the necessary data, we'll need to consider adding query parameters and keyword arguments to the URL field. In Django, we can achieve this using the url field and leverage the Django syntax.

For keyword arguments, there are many that are generated by django as a model. These models follow a pattern that we can easily use and replace for its intended value. But there are some that don’t follow this pattern which are mostly custom endpoints that need to be checked individually to add their value.

Unfortunately, there is no easy way to check query parameters as there is no denotation for them on the url, so they need to be checked individually by looking into the code, usually after a request fails.

In this step we need to create a high amount of pytest fixtures too as most of the values needed are model identifiers or are related to them.

Once we have built the URL and necessary data, we can perform the GET request using pytest's django_app_factory plugin's get() method. In this step, more fixtures may be required and there's also a possibility of internal errors occurring. I had some occurrences of implementation errors from old endpoints in production showing some good results from these tests already.

There is not much that can be done here other than waiting for a response and following to the next step or handling the errors that happen from time to time.

Finally, we need to check the status code returned by the GET request. If the code is outside the 2xx range and is not 405, indicating that the endpoint does not allow GET requests, the test will fail unless this endpoint is explicitly listed as expecting another code. The request follows redirects, so we should not receive any 3xx code either.

In conclusion, thoroughly testing all the endpoints of a Django server is a crucial step in guaranteeing the smooth functioning of a web application and delivering a positive user experience. Fortunately, with the powerful Django test framework and the aid of valuable third-party packages such as django-extensions and pytest, developers can effortlessly create comprehensive tests. By leveraging these tools, broken links and endpoints can be identified, and potential errors can be caught before they have a chance to impact users.

Testing All Endpoints in Django: An Overview

Learn about the most popular JSON libraries for Python such as orjson, ujson, simplejson, rapidjson, and nujson. This article provides an overview of each library's purpose and history, and then compares their performance, features, and use cases to help you choose the best option for your needs. Also, includes a benchmark comparison for encoding and decoding operations of different JSON libraries.

JSON Python Libraries Overview: A Comparison of Performance and Features

JSON has become one of the most popular formats for encoding structured data, particularly when building RESTful APIs that exchange data with clients or between services. It can also be used in other contexts, such as data processing or machine learning, where structured data needs to be stored or transmitted in a standardized format. 

While Python's built-in json module provides a simple and reliable way to work with JSON data, it is useful to know that there are several other libraries available that offer better performance and additional features. 

In this article, let’s compare some of the most popular JSON libraries for Python such as `orjson`, `ujson`, `simplejson`, rapidjson, and `nujson`. We will do a brief overview of each library's purpose and history, and then compare their performance, features, and use cases to help you choose the best option for your needs.

First of all let’s take a look at JSON itself. JSON (JavaScript Object Notation) is a lightweight data interchange format. The main purpose of JSON is to be easily writable and readable for humans and easily parsable for machines at the same time.

JSON is suitable for a variety of purposes, including transferring data between a client and a server in web applications and APIs and storing data in databases or files (MongoDB, Elasticsearch, etc.).

It’s not necessary to have a root object by specification, although it’s widely used. A JSON object is represented as a collection of key-value pairs, enclosed in curly braces. 

Each key-value pair consists of a key (enclosed in double quotes) followed by a colon and a value. Multiple key-value pairs are separated by commas.

Values can be of several types, including strings (enclosed in double quotes), numbers (integers or floating-point), booleans (true or false), null value (null), arrays (ordered collections of values enclosed in square brackets), and objects (collections of key-value pairs enclosed in curly braces).

Here is an example of a JSON object for the movie Shrek (we will use it later for our benchmark part):

If you aren't familiar with the syntax of JSON or want to investigate how it is defined in detail, there are two main resources worth checking: json.org or ECMA-404.

Serialization of JSON is the process of converting structured data, such as Python objects or database records, into a JSON-formatted string that can be transmitted or stored. This serialized JSON data can then be deserialized, or converted back into its original format, when needed.

Serialization of JSON is a common task in web development, particularly when building RESTful APIs that exchange data with clients. It can also be used in other contexts, such as data processing or machine learning, where structured data needs to be stored or transmitted in a standardized format.

JSON serialization has several potential disadvantages, including limited support for data types, lack of schema validation (which can be solved by using JSON Schema), large file sizes, and performance overhead.

As we embark on this delightful journey through the world of Python JSON libraries, let's take a closer look at their unique flavors and characteristics:

What was originally called simplejson was added as json to the Python standard library starting from version 2.6. It was written by Bob Ippolito. The built-in json library is included with Python. While it may not be the most performance-oriented choice, it's perfect for simple use cases and quick JSON encoding and decoding tasks. For many developers, this library is the go-to choice for everyday JSON handling.

simplejson follows its own development path, separate from the standard json module, while still maintaining compatibility. This allows simplejson to cater to the needs of users who require support for older Python versions.

ujson is a high-performance JSON library, written in pure C, designed for speed without sacrificing compatibility with the built-in json library's API. For applications that require rapid JSON encoding and decoding, particularly when dealing with large data sets, ujson is an excellent choice to elevate your JSON processing capabilities. It is compatible with Python 3.7 and above.

`nujson`: A fork of the high-performance `ujson` library, nujson specializes in handling NumPy objects, particularly arrays, by providing native serialization support. It's an excellent choice for projects that involve scientific or numerical data and require efficient processing of NumPy objects alongside standard JSON data types.

RapidJSON is a high-performance library, written in C++. It’s fast, doesn’t have additional dependencies, is memory-friendly, and it also has validation support based on JSON Schema. While RapidJSON itself is a C++ library, there are Python bindings available, such as python-rapidjson, that allow you to use its functionality within Python projects. It’s compatible with Python 3.

orjson is a high-performance JSON library implemented in Rust, offering Python bindings and focusing on speed and JSON specification compliance. Its unique features include serialization of non-string keys in dictionaries and improved handling of datetime objects. It’s suitable for projects where performance is critical and non-traditional data types are frequently used.

The usage of these JSON libraries in Python is quite similar due to their compatibility with the built-in json library's API. The primary functions for encoding and decoding JSON data, dumps() and loads(), are consistent across these libraries. It allows you to simply replace the built-in json library with an alternative by importing it as json.

# Built-in json
import json

# simplejson
import simplejson as json

# ujson
import ujson as json

# python-rapidjson
import rapidjson as json

# orjson
import orjson as json

# nujson
import nujson as json


Then, the usage of each of the mentioned libraries will look the same:

# Serialization
encoded_data = json.dumps(data)

# Deserialization
decoded_data = json.loads(encoded_data)


Thus, since the usage of these libraries is quite similar, let's proceed to the pinnacle of our exploration journey and evaluate the performance of these JSON libraries in terms of speed.

First of all we need to install all those libraries:

pip install numpy simplejson ujson python-rapidjson orjson nujson

We will use a previously mentioned sample JSON dataset derived from the movie Shrek. This dataset will be stored in a file named shrek.json for our testing purposes.

The code for our benchmarking function will appear as follows:

def benchmark(library):
    encoded_data = library.dumps(data)
    decoding = lambda: library.loads(encoded_data)
    encoding = lambda: library.dumps(data)

    encode_time = timeit.timeit(encoding, number=10000)
    decode_time = timeit.timeit(decoding, number=10000)

    return encode_time, decode_time

Whereas data is already loaded `shrek.json` into the python dictionary:

with open('shrek.json', 'r') as f:
data = json.load(f)


Here are the results of the benchmark tests for the JSON libraries, displaying the time taken for encoding and decoding operations:

Based on these results, we can observe the following:

Built-in json: This library shows moderate performance in both encoding and decoding operations.

simplejson: The results indicate that this library is slower in encoding compared to the built-in json, and has approximately the same decoding times.

ujson: This library demonstrates better performance in both encoding and decoding operations than the built-in json and simplejson.

nujson: Tailored for handling NumPy objects, nujson demonstrates competitive performance in both encoding and decoding operations compared to ujson. While its primary focus is on NumPy serialization, it remains a viable option for standard JSON data types.

python-rapidjson: The performance of this library is quite similar to that of ujson and nujson, with slightly faster encoding times but marginally slower decoding times.

orjson: This library shows the best performance in both encoding and decoding operations among all the tested libraries.

As mentioned earlier, these results provide a general idea of the performance differences between the JSON libraries. Although, some JSON libraries are more suitable for handling NumPy objects, such as arrays, thanks to their specialized support for these data types. One such library is `nujson`. Let’s perform a separate benchmark for these libraries and check how they hold NumPy data.

Instead of Shrek our data will be generated with the NumPy library:

data = np.random.rand(1000, 1000).tolist()

Built-in `json`: This library shows moderate performance in both encoding and decoding operations. It is suitable for simple use cases and general-purpose JSON handling, even when dealing with NumPy arrays.

`simplejson`: The results indicate that this library is slower in both encoding and decoding operations compared to other libraries.

`ujson`: This library demonstrates better performance in both encoding and decoding operations than the built-in json and simplejson. It is a great choice for applications that require faster JSON processing, including handling NumPy arrays.

`nujson`: Surprisingly, its encoding performance is slightly slower than ujson, but it is still twice as fast at decoding.

`python-rapidjson`: The performance of this library is quite similar to that of `ujson` and `nujson`. It can be considered an alternative for handling JSON data, including NumPy arrays.

`orjson`: This library shows the best performance in both encoding and decoding operations among all the tested libraries. Seems to be an excellent choice for projects that require high-performance JSON handling, especially when dealing with NumPy arrays.

To summarize, `orjson` stands out as the top performer, but the built-in json library remains a practical and convenient option for many projects. When choosing a JSON library, consider your specific requirements and weigh the benefits of each library against your needs.

If you seek further optimization, non-JSON serialization alternatives can offer enhanced performance and storage efficiency. Understanding their benefits and use cases allows you to make informed decisions for your project's data handling needs. Let’s briefly take a look at some of them:

Protocol Buffers (protobuf): Developed by Google, Protocol Buffers are a language-agnostic binary serialization format. They are designed to be compact, efficient, and fast. The advantages of protobuf include smaller message sizes, faster encoding and decoding, and a strongly-typed schema. They are especially useful for communication between microservices or distributed systems.

MessagePack: This binary serialization format is an efficient alternative to JSON. MessagePack is designed to be compact and fast, with support for many programming languages. It offers smaller payload sizes compared to JSON and better performance in encoding and decoding. However, it might not be as human-readable as JSON.

Apache Avro: Avro is a binary serialization format developed by the Apache Software Foundation. It is especially suitable for big data and distributed systems like Hadoop. Avro is schema-based, which ensures compatibility between different versions of data structures. It offers compact data representation, fast encoding and decoding, and supports many programming languages.

Apache Thrift: Originally developed by Facebook, Apache Thrift is a binary serialization format that supports a wide range of programming languages. It provides a compact and efficient binary format, a strongly-typed schema, and enables cross-language communication. Thrift is suitable for microservices and distributed systems.

These alternatives offer various advantages, such as improved performance, smaller message sizes, and support for type safety and schema evolution. However, they may not be as human-readable or as widely supported as JSON. It's important to consider the specific requirements of your project when choosing a serialization format.

In this article, we have explored various Python JSON libraries, comparing their features and performance. While the built-in json library is suitable for many use cases, several alternative libraries offer unique characteristics that may cater to specific requirements, such as nujson can be a good choice for working with NumPy data. We have also discussed non-JSON serialization alternatives, which can provide further performance improvements and storage efficiency. The choice of a serialization library or format will depend on your project's needs, and conducting thorough benchmark tests will help you determine the most suitable solution.

Showmax CMS team attended DjangoCon 2022 in Porto where they learnt about Django and the things you can do with it, about the developer community around Django and how Django can be used in a wide variety of ways, from giving opportunities to get into coding with Django Girls to how to use hidden features of the framework to do things you didn’t even know you wanted to do! We share a list of recordings of very interesting and insightful talks from Django 2022. conference that we liked.

Experiences from DjangoCon 2022 in Porto

After a year of just virtual events taking place due to the COVID-19 pandemic outbreak, the Django community finally gathered in person for the DjangoCon 2022 conference and chose a very beautiful site – the city of Porto. Located in northwest Portugal, Porto is known for its rich history, stunning architecture, and thriving startup scene. Needless to say, we were excited to attend the conference and learn from the experts behind the Django framework, which has been a crucial tool in building Showmax's Content Management System (CMS).

During our visit to Porto, we were able to explore the city's famous landmarks and attractions, such as the iconic Clérigos Tower and the Cais da Ribeira. We were also treated to the city's famous golden sunset views and sampled its delicious seafood and world-renowned port wine. It was a wonderful learning opportunity and a chance to network with our peers, but also a moment of exploring the rich culture and history of this vibrant and dynamic city.

The event was held in The Super Bock Arena (Pavilhão Rosa Mota) which was definitely a good choice. The venue provided ample space for attendees to network and enjoy the talks and workshops. One of the unique features that stood out was the roof tour, which was a great opportunity for attendees to take a break from the conference and experience a different perspective of the city. The tour finished with a tasting of traditional port wine poured into a cup made out of chocolate!

The conference was full of amazing speakers, which were not only talking about Django and the magic things you can do with it but also about our developer community and how Django can be used in a wide variety of ways, from giving opportunities to get into coding with Django Girls to how to use hidden features of the framework to do things you didn’t even know you wanted to do!

One of the most interesting talks was done by Daniel Roy Greenfeld (@pydanny), the legend behind the series of books called Two Scoops of Django, which every Django developer has used, directly or indirectly, to learn how to build things with the framework at some point in their career. During his talk on Scaling from one to Billions, Daniel delved into the fundamental aspects of being a good developer and how to translate these skills into projects that can impact billions of people while keeping in mind the ethical considerations that arise from such influential work. In the end, Daniel’s talk had little to do with Django itself, but left us with a lot to think about regarding our daily job and how it affects people around the world.

Another speaker, Katie Mclaughlin, made us think "What should you have to worry about?” She presented her view about the multiple trends we have popping up all the time on our daily job as developers and how to identify the ones that are actually worth our attention or even our time. In the talk she described in detail three main topics: Stable and Scalable Infrastructure, Reliable and Resilient Code, and Thoughtful and Thriving Community. We really recommend you watch her talk as it was not only very informative but also presented in a very playful and fun way.

On a more technical side, we had a lot of amazing speakers talking about different topics, but the ones who stand out the most for us were:

Add Multi-Factor Authentication (MFA) to Django in Mere Minutes: we had the opportunity to learn from Justin Mayer (@Jmayer) about the importance of Multi-factor Authentication (MFA), what are the types available and, of course, how to add it to Django in mere minutes!

Introducing Django Ninja: presented by Vitaliy Kucheryaviy, who walked us through a new way of building REST APIs in Django that uses python type hints in a clever way to validate, convert and serialize the endpoints’ inputs and outputs, and also automate API documentation!

“Observe!”: Honza Král (@honzakral) covered many useful tips and tricks to help improve monitoring, observability, and debugging of the running Django applications, such as logging, APM (application performance monitoring), distributed tracing, error tracking, and other things which are essential for any production Django app.

Handling Django in highly concurrent & scale environment: Tarun Garg told us about his journey toward trying to improve the performance of such things as Django admin pagination, text search, and Django models caching in a highly concurrent environment, which was a very educational experience for us!

	By the end of every conference day there was a chance for people to participate in the Lightning Talks, giving five minutes to each speaker to talk about a topic of their choice. This was a really nice opportunity for those who wanted to have their first talk in a less strict format. That doesn’t mean that the talks were not serious or anything like that. We listened to multiple speakers talking about various interesting topics. For example, we listened to a talk about the maintainer of django-upgrade, which quickly presented a tool to us which will make the upgrade of Django major versions easier. It was a major announcement which the Django community has been waiting quite a while for.

In conclusion, the Django Conference is a testament to the thriving and dedicated community of developers using the Django framework. From all around the world, individuals and teams are utilizing Django to build impressive and innovative projects. The community of Django users is a major factor in the continued success and popularity of the framework within the Python ecosystem. The conference was very well organized – the great venue, interesting talks, and inspirational networking were wrapped up with an excellent final party and the extra addition of a free boat trip around the city of Porto. It was a very nice addition to switch the focus from the conference and get to know and enjoy this wonderful city from an unusual angle.

We are already looking forward to DjangoCon 2023, which this time will be in yet another amazing city: Edinburgh! Hopefully this time we might even be one of the speakers telling the world about the crazy things we do with this amazing technology in order to bring moments of joy to our users!

Thanks Bogdan Bodnar for co-writing this post!

DjangoCon 2022 in Porto: Insights from the World’s Biggest Django Conference

Discover effective strategies for optimizing VMAF performance in video quality assessment. Explore techniques to reduce computational time, lower resource consumption, and improve cost-efficiency in your encoding pipelines. Gain insights into sampling frames, leveraging threading, managing video resolution and bitrate, and evaluating VMAF for multiple distorted files simultaneously. Enhance your video quality assessment workflows with practical tips and data-backed recommendations. Read now!

Optimizing VMAF Performance: Reducing Costs and Enhancing Efficiency

VMAF, short for Video Multimethod Assessment Fusion, was developed by Netflix in 2016 and swiftly established itself as the industry standard tool for video quality assessment. It offers an open-source implementation on GitHub and seamless integration with FFmpeg, making it easily accessible to developers and video encoding professionals. However, one aspect that cannot be overlooked is its demanding resource requirements. VMAF can be highly resource intensive, especially when it needs to be calculated multiple times per profile within a per-title encoding pipeline. In such cases, the cost of quality assessment can potentially exceed the cost of the actual encoding process itself. Therefore, it becomes crucial to explore strategies and optimizations to make VMAF more cost-effective. Let's delve into effective techniques to mitigate these costs and enhance the efficiency of VMAF in video quality assessment.

First, this is one way to run VMAF using FFmpeg:

ffmpeg -i reference_480_1050.mp4 -i distorted_288_375.mp4 
-lavfi 
"[0:v]setpts=PTS-STARTPTS[reference];[1:v]scale=720:480:flags=bicubic,setpts=PTS-STARTPTS[distorted];[distorted][reference]libvmaf=n_threads=12:n_subsample=1" -f null -

You will use a reference (mezzanine, source) file and a distorted (encoded) version of that reference. The reference and distorted duration, frame rate, and resolution must match, otherwise you will need to apply some extra filters to make it so, like the one in the command above that scales up the distorted file to match the reference.

As the reference file on the experiments below we used the trailer for the acclaimed HBO TV show The Last of Us, a 126s video encoded with h264 at 1080p and 9580 kb/s (144MiB in total). For the distorted version we also used a h264 encoded file at 720p 2800 kb/s. The computer is equipped with an AMD Ryzen™ 5 4600H processor with six CPU Cores and 12 threads running at 3.0GHz, and as for the software FFmpeg 5.1.2 with libvmaf 2.3.1. The charts below were made using plotly.

Maybe you noticed the n_subsample parameter in the command above. It limits the number of frames for which VMAF will be calculated, reducing the computational time in exchange for accuracy. VMAF outputs one value for each n_subsample frames, and the most common way of analyzing the overall quality of the title is by further calculating the mean, median, or the 1st percentile (which gives you a notion of the worst frames) across all frames. To make an informed decision, let’s first see how much accuracy we lose by increasing this parameter value, then the increased speed we get.

We can see that the error grows slowly for mean and median and is below 0.5 VMAF points even when sampling once every ten frames, which is pretty good considering that the Just Noticeable Difference (JND) for VMAF is of about six points, so as long as you remember to add error bands in your analysis to account for it you should be fine. On the other hand, the 1st percentile is a different story, as the goal of the metric is to identify the worst frames, cutting 1/10th of them implies a higher and more unpredictable error.

How sampling frames affect computational time

Sampling every two frames already cuts the processing time by about 45%. Gain is slower for higher values but with n_subsample=5 we are at about one-third of the time it takes to sample every frame. Additionally, the red line shows the peak memory consumption for each case, and for n_subsample=1 that would be 1322 Mb – quite a lot for a simple 144 Mb reference file – so don’t forget to add some extra gigs to your 4K catalog!

The time relative to the video duration is important because for an ABR ladder with multiple renditions checking the encoding quality of a two-hour movie might take longer than the encoding pipeline itself. Also, it is not as straightforward to parallelize it on a cluster by segmenting the video because VMAF always considers the previous frames to calculate the quality of the current one, so it would be necessary to discard them and account for the offset of the frames in the beginning of the segments.

By default VMAF will use all available CPU threads, but let’s see how much speed up we get from extra cores. The CPU used in the experiments has 6 real cores and 12 threads, and we can see in the chart below that speed up is minimal after filling up all real cores. The gray dotted line indicates the theoretically perfect scenario where doubling the number of threads would cut in half the computational time, and you may also notice that Speed Up is pretty much the same regardless of the distorted file resolution (360p, 720p and 1080p), more on that in the next section. 

Another thing that might be counterintuitive at first is that calculating VMAF for a 1080p encoded file doesn’t take much more time than for a 144p. See the chart below, which considers a range of resolutions and bitrates for H264 and HEVC.

To understand that, remember that the encoded files will be upscaled to the reference resolution beforehand, so the resolution and bitrate of the reference is what matters the most. To illustrate that, we created HD (720p) and UHD (2160p) versions of our original FHD “The Last of Us” trailer and calculated VMAF over it using n_subsample 1, 2 and 5. In all three cases, FHD was approximately 2.1x slower than HD, and UHD 3.8x slower than FHD.

VMAF over multiple distorted files at once

We can leverage FFmpeg complex filters to run VMAF over multiple distorted files simultaneously. When encoding for multiple outputs there is a gain as reference is decoded only once. Let’s see if libvmaf can benefit from it in the same way. The command line for running FFmpeg over three encoded files will look something like this:

ffmpeg -i reference.mp4 -i video_h264_360p_400k.mp4 -i video_h264_720p_2800k.mp4 -i video_h264_1080p_5000k.mp4 -lavfi 
'[0:v]split=3[r0][r1][r2]; [1:v]scale=1920:1080:flags=bicubic[d0]; [2:v]scale=1920:1080:flags=bicubic[d1]; 
[d0][r0]libvmaf=log_fmt=csv:log_path=video_h264_360p_400k.csv:n_threads=1:n_subsample=5; 
[d1][r1]libvmaf=log_fmt=csv:log_path=video_h264_720p_2800k.csv:n_threads=1:n_subsample=5; 
[2:v][r2]libvmaf=log_fmt=csv:log_path=video_h264_1080p_5000k.csv:n_threads=1:n_subsample=5' -an -sn -f null -

Input reference is split into three streams, one for each output, producing three CSV files with the results. Distorted files are upscaled if necessary, and as we have n_threads=1 for each distorted file, we are in fact using three cores. Thus, to evaluate the difference we can only compare multiple and single output solutions when the ladder is executed with n_threads 3, 6, 9 or 12. To make it more interesting we tried increasing the n_subsample variable as well, and here is the result, with the percentage gain of the multiple output solution in the y-axis, and total number of threads in the x-axis.

We can only see some meaningful gain above 6% with higher n_subsample values and all threads in use (4 per output in this case, 12 in total). In this scenario parallelizing execution in multiple servers would produce better results than generating multiple outputs in a single machine from the FFMpeg one-liner.

If you use VMAF sporadically to assess the quality of your encoded videos the computational time might not be that relevant, but adding VMAF into more sophisticated pipelines might get very expensive, so we hope the colorful charts above might help to calculate and optimize that. 

Here are some conclusions that we have drawn from them:

Increasing n_subsample will reduce computational time significantly with low impact on accuracy for mean and median, but the 1st percentile will get more unpredictable.

The distorted bitrate or resolution have no significant impact on computational time.

The reference resolution and bitrate is what matters, and going from FHD to UHD will make VMAF almost four times more expensive.

Calculating VMAF for multiple distorted files at once doesn’t provide a meaningful speedup unless you use at least n_subsample=10 and have dozens of cores to spare.

Join us and build a streaming platform used by millions. 

We’re excited to witness the recently announced partnership between Showmax, Sky and NBCU coming to life here in Prague. Following the recent announcement of our partnership; that will bring some of the world’s best content and technology to streaming customers across sub-Saharan Africa, Sky are hiring developers to make it happen.

You might know Sky for TV. But we’re more than that. We’re a tech business. We’re broadband and broadcasting, streaming and sports, mobile and magic. And we’re proud to be part of the Comcast NBCUniversal family, creating incredible technology and entertainment that connect millions of people to the moments and experiences that matter most. 

We’ve already rolled out NBCUniversal’s streaming platform Peacock to our European audiences. Now you have the opportunity to work on an exciting new venture powered by Peacock’s leading, globally scaled technology. 

Our team develops and supports market-leading video streaming services, underpinned by state-of-the-art engineering principles. We do this at huge scale: for millions of customers globally. No matter the device, the time or the place, we make sure that our diverse audiences can easily find and enjoy whatever they want to watch, choosing from the world’s best entertainment, news and sport.

We are looking for a highly skilled and experienced Senior Backend Developer to join our development team with a great hands-on attitude. As a Senior Backend Developer, he/she should embrace the full lifecycle of software development in the integration layer between client services and the backend, participating in all phases: Solution Design (HLD and LLD), Implementation, Testing (functional and non-functional), Operational Readiness and Support on Production, embracing a build-it own-it mind-set. His/her primary focus will be on developing efficient server-side logic, collaborating with cross-functional teams, and ensuring the overall performance and reliability of our systems. Additionally, he/she should have good communication skills and be able to mentor and support Junior Developers. While proficiency in Node.js is preferable, we welcome candidates with expertise in other backend technologies as well.

Take the Tech Owner role in several features, participating and contributing to the Solution Design, Implementation, Code Reviews, Testing, Operational Readiness, deploy to the delivered environments and support on Production.

Collaborate with cross-functional teams, including Architects, Project Managers, QA to define and implement robust backend solutions.

Lead and participate in architectural discussions and decisions, providing technical expertise and guidance to the team.

Write clean, maintainable, and efficient code that adheres to best practices and coding standards.

Optimize application performance by identifying bottlenecks and implementing appropriate solutions.

Troubleshoot and debug complex issues, providing timely resolutions and recommendations.

Stay up to date with the latest industry trends and technologies, continuously enhancing your skills and knowledge.

Mentor and support junior developers, contributing to a collaborative and growth-oriented team environment.

Contribute to the improvement of development processes, tools, and methodologies

What skills do you need to play your role well? 

Experience working as a Backend Developer, with a proven track record of delivering high-quality software.

Strong proficiency in at least one backend programming language (preferably in Node.js, but also Java or Go).

Solid understanding of backend development principles, including asynchronous programming, RESTful APIs and caching mechanisms.

Experience with modern backend frameworks and libraries (e.g., Express.js, GraphQL) is highly desirable.

Familiarity with NoSQL databases (e.g., MongoDB, Redis) is a plus.

Basic knowledge of cloud platforms and technologies (e.g., AWS, Azure, GCP), containerization (e.g., Docker, Kubernetes) and CI/CD is mandatory.

Strong problem-solving skills and the ability to analyze and debug complex issues.

Excellent collaboration and communication skills, with the ability to effectively convey technical concepts to both technical and non-technical stakeholders.

Self-motivated, with a passion for continuous learning and keeping up with emerging technologies and best practices.

Experience working in an Agile development environment is preferred.

Working in an international environment on an impactful globally scaled technology in cross-functional team with an agile approach 

Open and healthy working culture where we care about people as much as about stability of the code  

Hybrid work, combining working from our office to be closer to the team and working from home 

Work life balance thanks to flexible working hours

Generous free day scheme providing 25 days of holidays + 5 days of on-demand leave (sick days)

Ergonomic office and kitchen full of tasty snacks 

Contribution to Multisport Card or Benefit Card 

Events, meetups, hackathons in the office or elsewhere

Millions of customers use our apps and we have always been looking to improve by pushing the boundaries in our products and services providing every one of our customers experiences they love.

We are looking for a strong Java Developer who would like to help develop our latest large scale CMS project for Sky OTT International.  Working in an Agile environment, this role will suit an experienced Java Developer who wants to broaden their skill set in an international multilingual CMS application.

Deliver excellent quality software using agile techniques

Assist in developing application deployment, monitoring and automated control artifacts

Strive for continuous improvement – be able to improve existing practices and techniques from within the team with enthusiasm

Assist in the development the CMS application using Java and Spring

Unit, Functional and Performance test code for robustness including edge cases, usability and general reliability. 

Assist in the maintenance and support of the AWS infrastructure, Postgresql Database and Object Caching layers

Actively help the team improve our automated test systems

Extensive experience creating quality Java based applications

Having worked on live high-volume websites

Proficiency in at least one Continuous Integration tool (ideally Jenkins)

Experience in automated testing frameworks

Ability to use a wide variety of open source technologies and cloud services such as AWS

Strong understanding of Spring and Spring Boot

Extensive experience with Agile Principles

Exposure to Content Management and Content Distribution Networks

Knowledge of Unix/Linux and scripting languages (Bash, etc.)

The role of the Content Delivery team is to build the infrastructure and logic to deliver the best quality video based on a deep working knowledge of HTTP video streaming, quality monitoring, IPv4/IPv6 networking, and other delivery-related technologies. Our team focuses on maintaining and improving the origin server infrastructure for media storage and content delivery networks. This also includes configuring, performance testing and tuning of Linux-based systems, specifying and purchasing HW, deploying it in remote locations, and (mainly) defining and building decision algorithms to pick the best possible server and/or CDN for each particular customer.

showmax10 (1080p) (1).mp4

We work with existing technologies, but sometimes we have to build something new from scratch. We mostly use Nginx, Varnish, Apache, and Debian-based Linux distributions on bare metal servers. For the servers, we use Supermicro hardware with in-house defined specifications. For the network infrastructure, we use equipment from Arista Networks. Most of our code is written in Python and Go, occasionally we also script using Bash.

We build and manage the video storage infrastructure where all the media files are ingested and kept. This infrastructure is accessed by the CDNs if the content is not cached yet. Some of our tools are better suited for a containerized environment, so we therefore utilize a shared Kubernetes infrastructure running on AWS to which we deploy our internal services.

About the team

How we work

Read our blog posts #media

Testing All Endpoints in Django: An Overview

JSON Python libraries overview

DjangoCon 2022 in Porto: Insights from the World’s Biggest Django Conference

VMAF - Performance vs. Accuracy

Testing All Endpoints in Django: An Overview

JSON Python libraries overview

DjangoCon 2022 in Porto: Insights from the World’s Biggest Django Conference

VMAF - Performance vs. Accuracy

Check out our other teams