One of the most important sub-metrics of our Quality of Experience metric is buffering time. Nobody wants to see the infamous loading circle icon! Here at Showmax, we invest a lot of time and effort helping our customers avoid it.

We created several dashboards and charts to monitor user sessions and visualize and analyze the dynamics of buffering time metrics. Our goal is to pinpoint the possible root cause(s) of problems we detect. We monitor and search for correlation with the country of origin, the CDN the customer was routed to, the platform used (Android, iOS, web, etc), type of event (live or VOD), and more.

These dashboards need to be responsive — specifically, the querying and grouping performed over several millions of sessions that feed each chart should be completed in less than 15 seconds. If that doesn’t happen, we get a timeout. To make this possible, we pre-aggregate some of our data into smaller tables containing only the metrics and dimensions that we need. The aggregated data can be easily represented with percentiles.

If you aren’t familiar with the term, here’s an example.

A situation in which 95% of the website users are served within 1 second can be described using percentiles like this: the 95th percentile of the website response time is 1s. The most commonly-used percentiles are 99th, 95th, 75th (also called third quartile), and the 50th percentile, which is also the median. The applications are vast, and it has a few advantages over the average, which only behaves well under normal distributions and is easily skewed by outliers. Specifically, we use percentiles to remove outliers from our dataset, and to define goals for all metrics.

We want to use percentiles to describe aggregated data, but there is a small issue with percentiles — they need the full dataset to be calculated precisely. As noted earlier, we need pre-aggregations to have a responsive dashboard. To see why percentiles don’t work with pre-aggregated data check out the table below. It displays a dataset of hypothetical buffering times, followed by another table with data grouped by country, with the 95th percentile as metric.

Now we want to further aggregate the country table to have a general overview of all the sessions. To do so, we calculate the percentile again, getting the value of 17.34. Compared to the 95th percentile calculated from the original table, 15.49, we are off by more than 10%. The error rate is not only high, but its magnitude is unpredictable.

The same problem happens with average, but the solution is much simpler. You store the count of sessions and the sum of buffering time for each level of grouping, with the actual average calculated only at the last step when displaying the data in the dashboard.

One possible solution to this problem is to drop the percentiles altogether and define buckets with fixed ranges. For instance, you could have a field named buffering_time_zero, counting the sessions with no buffering, buffering_time_two, counting sessions with buffering time up to two seconds, and so on. When displaying the result, it’s possible to divide the value in each bucket by the number of all sessions to get a percentage of sessions in each range.

This solution is easy to implement, but it’s less flexible. Any change in your goals requires changing the database schema and recreating all the aggregated tables. The alternative we found was the PostgreSQL extension called TDigest, based on a paper by Ted Dunning from 2013, that implements t-digest histograms and provides approximations for percentiles. Tdigest is fast and simple to use, and the extension adds a new data type and a few functions. To create a Tdigest structure you need to specify only the number of buckets. More buckets mean more accuracy, but also more storage space and lower performance. We did a few experiments to decide which value would be the most suitable for us.

First, we prepared a dataset containing buffering time and timestamp, that included roughly 1 million sessions. As most of the sessions don’t present buffering, the data distribution is very unbalanced, as you can see in the figure below. The same experiment could have different results on more uniform datasets.

To analyse the error, we grouped the sessions by hour to compare the estimated percentiles (as calculated by Tdigest) to the real percentile values (as calculated by the percentile_cont 

function from PostgreSQL). The errors were averaged out for the 10 hours in the dataset, and the following table displays how off target the percentiles were:

Precision / # of bucketsPctl 99thPctl 95thPctl 90thPctl 75th104.53%41.41%84.79%366.52%500.90%8.21%10.43%39.08%1000.76%3.13%4.79%12.61%2500.35%0.87%1.16%2.85%5000.21%0.31%0.32%1.27%10000.19%0.17%0.20%1.06%

As described in the paper by Ted Dunning, and as evident in the above table, Tdigest is more precise closer to the extremes (the 99th percentile shows minor error compared to the 95th).To get an error below 1% on 95th percentile, you need at least 250 buckets. To check the 75th percentile of your data with high precision, you need at least 1000 buckets.

Next comes performance and storage. For storage we used the same data as before, but buckets ranging from 100 to 1000. The figure below shows the growth in disk space needed per record as we increased the number of buckets. Fortunately, the growth is not linear because of compression, and 500 buckets uses only 3.3x more space than 100 buckets.

Last but not least, the performance comparison with PostgreSQL percentile_cont function shows a clear advantage for Tdigest — 3x faster with 100 buckets, and 2.5x faster with 10000 buckets.

There are a few drawbacks to using Tdigest. The limited precision when calculating percentiles that are far from the extremes, were already mentioned. In fact, anything between the 10th and 90th percentile will yield poor results with fewer than 1000 buckets.

Although the extension can be easily installed from source, or via package manager (after setting up the PostgreSQL repository), it’s worth checking with your cloud provider before settling on the solution. Azure currently supports it on their Citus database, but Amazon Redshift doesn’t.

The extension has a function named tdigest_percentile_of, which should get a float number as input and return the percentile where the value first appears as output. So, if 70% of the users experience zero buffering time, tdigest_percentile_of(0) should return 0.70. The issue is that we don’t know beforehand in which percentile a given value will be, and the limitation that percentiles far from extremes are not reliable also applies here. That all means that we can’t really trust the result.

In the end, our workaround was a hybrid solution using Tdigest to get the 90th and 95th percentiles, and another field counting the number of sessions with no buffering to know the percentage of our happiest customers.

How to Aggregate Percentiles - Tdigest to the rescue

The first part of our machine learning series was all about helping you understand some basic machine learning (ML) concepts and jargon (read it here). This post builds on it, and goes a bit deeper in its explanations of how a machine learning model is born.

ML is only growing in importance. As more and more models are trained, its applications are widening and are present in our every-day lives right now. Think about pedestrian detection systems in modern cars, Google completing your emails and correcting your grammar, and so on—ML is all around us.

So, how do we get from concept to real-world application? How does a data science team work? What are the steps within their workflow when they tackle analytical and machine learning problems? Let’s get into it…

Every data science workflow is like any other workflow. It starts with gathering as much information as possible about what we want to analyze. When we have a clear idea of what the problem is, and how we will solve it, we need to get the right data for it.

The data gets imported from different sources—it comes from our data warehouse (DWH), events that were performed by our users, external data, and other sources. There are often mistakes in the data, like missing data or problems with formatting. And, importing and cleaning the data can get time-consuming—the reality is that we spend a lot of time day-to-day on data cleaning.

Next, we get into a circle of transforming, visualising, and modelling the data to help us better understand the data. Within this step, we learn how to best represent the data and Visualize it to make our findings easier to communicate (that’s the next step). At Showmax we are firm believers in data-driven decision making, and we use these visualisations with our findings to help our key stakeholders make critical business decisions.

In most cases, simple data science is enough to solve the problems at hand, like importing, transforming, and visualising the data. But sometimes we have more challenging problems that ask for a ML-based approach. For example, we wanted to find out why users choose to continue using our service, what makes them happy, and what compels them to become long-term subscribers.

This is a complex task that requires many different data sources to be able to even begin explaining the phenomenon—there are simply way too many possibilities. So, we decided to use ML to see what we could find.

Putting together a ML workflow is a little bit different than a data science workflow. It starts with the same steps, “problem and data analysis,” which is similar to the import, clean, and visualize steps in a regular data science workflow. The next step is “preprocessing and feature extraction,” the process of generating more features from the existing data. Think of features as columns in the dataset, e.g. date, time, first subscription day, and so on. In this case, feature extraction means taking the “date” feature and requesting data like, “is_weekday”, “is_holiday”, and more.

After the dataset is available, it has to be split into multiple datasets. There is the training dataset, which is used to train the model itself. The validation set is used to evaluate the model and set the hyperparameters correctly. And last is the test set, which is used for reporting the final metrics you want to optimize, like accuracy, weighted accuracy, recall, MSE, or RMSE.

The next step is “model training and hyperparameter tuning,” which involves choosing the right model and setting the hyperparameters so that the performance is as good as it can possibly be. Hyperparameters are model-specific parameters that can be altered to change the behavior of the model. For example, in a decision tree, the number of different decisions is a hyperparameter.

The last part is the analysis of your model, where you compare different models (if you trained multiple models). With this feedback, you can run through the flow once again to improve upon the previous iteration. Don’t be afraid to come up with a new feature to add or another ML algorithm to try out.

Let’s have a look at an example to understand the steps within the workflow. It’s an example we often use when talking to fellow Showmaxers—a very simple model that clusters our colleagues based on movies and series they’ve watched. In fact, this exercise can bring you some valuable insights. You might use it to form groups in Slack to connect people with similar interests, for example.

There’s no data science without the data, right? All Showmaxers have access to the Showmax app and most of them watch Showmax content. So, to use the user behavior data, we first need to import the data.

Because we want to work only with data from our colleagues, we have to use some filters. We should filter the dataset for email addresses ending with “showmax.com”. To limit the size of our dataset and keep assets as similar as possible, we should also limit the dataset for time. Third, we only want to include people that really did watch something, excluding those who only tested some feature. So, we put a lower limit of 30 seconds on net_time, defined as the time the user spends watching.

SELECT email, master_party_id, rvds.asset_id, grand_parent_name, genre, tag
FROM cached_data.reports_viewing_durations_streaming rvds
JOIN reporting.dim_party dp ON dp.master_party_id::text = rvds.userr_id::text
JOIN reporting.dim_asset da ON da.asset_id::text = rvds.asset_id::text
WHERE dp.email LIKE ‘%%showmax.com’
AND rvds.start_time > ‘2020-01-01’
AND rvds.net_time >30


The data we have now is still not the data we want to train a ML algorithm on. It needs to be tidied and transformed. These two actions together are called pre-processing.
In the tidy phase, you drop data that is incorrect or would confuse the algorithm. Let’s have a look at some examples:

Remove users who are in the dataset multiple times

Remove users with unrealistic clicking behaviour (as they were most likely testers)

Remove users who don’t have a ‘.’ in their email address as that is not compliant with the Showmax email address structure

Filter out entries where the asset_id is missing

Here we can see some of the email addresses that are not compliant with the rules above.

In our original data frame, we have the users together with assets they watched. We call this a long form dataset. For ML purposes, we transfer the format from long to wide, as seen below. It basically means that, instead of listing the users with assets they watched, each entry in the dataset is just one user. Each column stands for a specific series, and the value for the user is 1 if they watched it, and 0 if they didn’t. By transforming the dataset like this, each row in the dataset represents one user and we can immediately see what they did and did not watch.

A crucial part in any data science or ML project is data visualization. By visualizing the data you make it more understandable to you, your stakeholders, and a wider audience.

This dataset is not very big as it includes only a few different features, but we can still find some interesting things in it. For example, we can see that Grey’s Anatomy is the most-watched series, almost 10% of the users in our dataset watched at least a bit of it. This is followed by The River, a Showmax Original series, and then comes Lockdown. Our live channels, CNN, SABC News, etc., also score quite well.

Another way to visualize the high-scoring assets is to see how long users watched on average. You should, however, take into account the different average length of the assets. You cannot compare a movie that lasts 75-120 minutes to a series in which every episode is roughly 20-30 minutes long. We can see that the top 15 looks quite different, and there is little overlap. Grey’s Anatomy, the most-watched series in the previous visualization, is now #11. Combining these two visualizations, we see that this series is the most-popular one within our dataset. It is watched by the most people, and is also in the top 15 in terms of mean seconds watched.

Now we get to the part that most people consider “the work of people in ML.” You can see that we spent quite a lot of time already on gathering the data, tidying, and transforming it, and only now we are actually training a machine learning algorithm.

For this simple use case, we went with a KMeans model with hyperparameter n_clusters = 50. The model is then able to classify the data and assign a cluster number to it. This process is called predicting and we save these predictions to a variable ‘y. Then we can add it to our dataframe as column name cluster. This allows us to see the original data frame and filter it on a certain cluster.

For example, we saw a cluster of users watching Westworld, an American sci-fi series created by Jonathan Nolan and Lisa Joy, and produced by HBO. Finding out that our colleagues watch it on Showmax, we could connect them with each other.

When we have the results, we need to communicate it to the different stakeholders. It is important at this stage to keep in mind that not all your stakeholders will speak the same language in terms of the technology and methods you are using. You will need different visualisations and explanations for each of the different audiences.

For all target audiences, we always use a couple of sections:

Having read this post, you can see that training the ML algorithm itself is only a tiny percentage of our workload. We don’t want to disappoint the ML enthusiasts, but being a machine learning practitioner means spending more than a half of your time on gathering, tidying and transforming data.

We hope we gave you a basic understanding of some ML terms, and a bit of guidance for how to classify and solve a machine learning problem. Enjoy!

Machine Learning basic principles, Part II

We are very enthusiastic about machine learning (ML) here at Showmax. We have a solid amount of internal knowledge on the topic, and we’re constantly learning more. Recently, we implemented our first ML model — it measures our own Happiness metric and tries to predict churn based on it (read more about it in this blog post. To help you understand that piece, and our upcoming ML projects, we put together a comprehensive summary of some need-to-know terms.

Let’s start with a simple exercise. Ask yourself: “What is the weather going to be tomorrow?”
Give it time, think about it, and guide your thoughts with the illustrative pictures below.

Now, investigate the genesis of your answer and ask yourself: “Why did I answer the question like this? What is my answer based on?” Most likely you will have used some information that you know already:

In ML, this data and extra information you used are referred to as features. All of these features can be fed into a ML algorithm. This will result in a weather prediction algorithm that can tell you what the weather will be like tomorrow based on the features you give it.

Now, let’s take a look at a Showmax-specific example. At Showmax we need a recommendation engine, a machine learning algorithm that will guide our users and show them relevant content to watch.

If I recommend something to watch to a friend, it’s likely that, in most cases, I’ll consider following:

My own preferences: “I really liked movie X and you should watch it”

Your interests: “Because you like sports, I recommend Y”

What you’ve seen before: “Because you watched A and B, I recommend Z”

Popularity/virality: Everyone’s watching ABC, and you should check it out”

Actually, the process is very similar, but the machines don’t use the information of 1 user (like yourself), or a couple of users (yourself and your friends). They use the viewing history of every single Showmax user. So, instead of a couple of data points, there are data points for hundreds of thousands of users.

The advantages that machines have over people are quite clear:

Much greater capabilities in terms of memory

Ability to collect viewing statistics for thousands of people

Ability to make correlations in a fraction of the time

When thinking about different problems that could be solved with ML, it’s good to have a framework. The first question you have to ask yourself is: “Do I have labels?”

With labels, we understand an outcome for at least a part of the dataset. For example; if you have 2000 pictures, 1000 of dogs and 1000 of cats, then “dog” and “cat” are the labels. But, if you just have a lot of pictures and you don’t know what’s on them, the dataset is unlabelled.

Unsupervised Learning - with unlabelled data

Supervised learning only works labelled data. Within this there are two types of supervised learning. The type depends on the kind of labels, as there are two distinct types of problems. If your data has labels like “income”, for example (60,000 dollars per year, 80,000 dollars per year, etc.), then we say it’s a regression problem. If you take the example of the pictures of cats and dogs, this is categorical data - a classification problem.

In unsupervised learning, you don’t have labels, but there is still value to be extracted from this kind of dataset. One option is clustering, which could be used in marketing to create different target groups for marketing campaigns, for example. The clustering could be based on their watching behaviour, and then we could send them notifications for related content — letting a “sports fans” cluster know about an upcoming sporting event or documentary, for example.

Another example is association, which is often used in recommendation engines. It is used for discovering interesting relationships between variables in large databases. The aim is to identify strong rules discovered in databases using some measures of similarity.

Dimensionality reduction: plotting assets to plane - film2vec

If you’ve ever taken even a basic ML course, it’s likely that you’ve heard of bias and variance. It’s easily explained by looking at it like a target in archery. In a perfect world, you have low variance and low bias. As you can see, this means that you hit the bull’s eye every time. In ML, it means that your accuracy, your Mean Squared Error (MSE), or whatever other metric you decide to use, is good.

When you train a model and have low bias but high variance, it means that you still hit the target reasonably often, but in some cases you can be a bit off. On the other hand, if the bias is high and the variance low, you would hit very consistent shots (low variance) but always be slightly off target. You should avoid high variance and high bias at all costs. This situation occurs when you are not consistent (high variance) and don’t really get near the target either (high bias).

In ML, we call it the Bias-Variance trade-off. It is the characteristic of a model that the variance of the parameter estimates across samples can be reduced by increasing the bias in the estimated parameters. In other words, the more complex the model you are training, the lower the bias and the higher the variance. As such, there is an optimal model complexity where the bias and variance are such that the total error in your estimations is kept at a minimum.

When discussing bias and variance, it’s impossible not to mention model complexity, overfitting, and underfitting. All of these terms are closely-related, and inform how you can achieve low bias and low variance.

If you make a very simple model, like a linear regression model which doesn’t fit the particular dataset, you will encounter underfitting. The model is too simple for the dataset at hand and you create a high bias, low variance model which will not do a great job.

On the other hand, if you would make a very deep decision tree (lots of different decisions), you can have a model that is overfitted to the current dataset. In this case, you get high variance and low bias. This model might perform excellently on your training dataset, but, when you feed new data to the model and you predict the outcome, it might give rather bad results.

A basic understanding of machine learning concepts and jargon is increasingly in-demand, as new models are trained every day. We hope that this post comes in handy. The next piece in our ML series will highlight one specific problem that we have already introduced in presentations and at hackathons. It will also show the sequence of various steps in the ML workflow, and how much of our time is actually spent on developing the model itself. Stay tuned!

Machine Learning basic principles, Part I

Machine Learning model focused on outcomes

When starting with machine learning (ML) in any organization, it’s good to begin with something tangible. Here at Showmax we really care about our customers, and we don’t want to lose a single one of them. Addressing the issue of churn struck us as a great way to introduce a ML model of real value.

We’ve been exploring the idea of linking the data we measure (proxy metrics) to business values (e.g. how long the customer stays, how active they are, etc) since mid-2019. In truth, it kept us awake at night. Most companies have some kind of churn prediction dream, with which they could see how likely a user is to leave the platform. We wanted this, but we also want to improve user experience overall, and to do that we need to see if and when our platform doesn’t resonate with our users and why.

The fact is that churn is measured quite easily. The problem is that when the user churns, it’s already too late. We want to know when a user is becoming somehow less happy with the service, predict churn, take action, and (if at all possible) prevent it.

source: https://towardsdatascience.com/how-do-you-measure-if-your-customer-churn-predictive-model-is-good-187a49a9eee3

Around October 2019, we (the Showmax Analytics team) performed a correlational analysis. The approach consisted of collecting numerous different metrics and running simple correlation. It didn’t yield very satisfactory results, and we decided to focus on a more complex solution. It was still a valuable idea, so we decided to apply machine learning on a far bigger dataset.

From there, we go to work on a metric that we called “happiness”, which combines multiple parameters into one metric.

Back in 2020, when we were asked to present something about churn or likelihood of users dissatisfaction with our platform, we needed to present dozens of charts. Every team has different charts, different priorities, and so on, and this is just not the way to get the results we want.

Our goal is to improve our services and tailor them to our customers. To do this, we are constantly running AB tests and other experiments to improve content discovery. But, we need to be able to measure the impact.

In a perfect world, we would have one golden metric that defines if our customers will stay or not. It would take into account a matrix of values with different behavior patterns across clusters of customers. That way we could simplify future AB-tests and analysis, and define this as Customer Happiness.

This combines a number of metrics and feeds them into a ML model that evaluates the probability of the customer staying with us.

The data we need is mostly present in our on premise data warehouse (PostgreSQL), or can be fetched from Elastic. This is automated with Airflow, with the heavy-lifting done with DASK. Once all the preprocessing is done, the final dataset is fed into a CatBoost classifier.

When the relevant data was fetched, we tried to come up with as many features as possible. We would implement all of them and then let the algorithm decide which ones are more or less relevant. These included things like the number of trailers watched, or the use of the search function.

CatBoost is the ML technique we used for training our model. It stands for Categorical Booster, this has three main advantages:

1) It handles categorical data (e.g. platform, genre of the asset) much more conveniently than other Boosters

2) Boosting is a very popular and smart technique in ML, as it places extra emphasis on getting every sample right. A booster typically trains hundreds of decision trees where each decision tree focuses on the data where the previous decision tree was wrong.

3) CatBoost needs less hyperparameter optimization.

The churn predictor tells us what to focus on next by showing which parameters are highly-correlated with happy users. We use the happiness metric in AB testing to check if our changes impacted the happiness of our users. One thing we noticed is that happiness is very hard to move because it takes so many things into account. In short, it’s hard to impact happiness with a single AB test.

But, we can also check the SHAP values of the happiness model and see which features are the most important to focus our efforts better and try to improve these metrics.

We managed to put a ML model into production for the first time. It outputs a probability (in %) for each user to be happy with the service, helps us to evaluate AB tests, and it eases other analysis. For now, this is a churn prediction model, but we are also thinking about extending this to other labels (instead of churn it could be “watching tomorrow”).

Join us and build a streaming platform used by millions. 

We’re excited to witness the recently announced partnership between Showmax, Sky and NBCU coming to life here in Prague. Following the recent announcement of our partnership; that will bring some of the world’s best content and technology to streaming customers across sub-Saharan Africa, Sky are hiring developers to make it happen.

You might know Sky for TV. But we’re more than that. We’re a tech business. We’re broadband and broadcasting, streaming and sports, mobile and magic. And we’re proud to be part of the Comcast NBCUniversal family, creating incredible technology and entertainment that connect millions of people to the moments and experiences that matter most. 

We’ve already rolled out NBCUniversal’s streaming platform Peacock to our European audiences. Now you have the opportunity to work on an exciting new venture powered by Peacock’s leading, globally scaled technology. 

You will be part of a dynamic team developing client applications and making sure our technology delivers the best experience for our customers.

Craft high-quality software utilizing agile methodologies

Design clean, efficient, and reliable code

Deliver highly scalable and extremely reliable products (our customers demand 99.999% availability)

Collaborate with fellow developers, exchanging knowledge and insights - prepare to learn a lot!

Passionately introduce top-notch technology and software development practices into the team

Participate in the entire development cycle, from analysis and implementation to testing and operation

Embrace operational excellence and support as integral aspects of the development team's responsibilities

Proficiency in native iOS languages, specifically Swift

Familiarity with Design Patterns and Object-Oriented Design principles

Comprehensive knowledge of key iOS architectural components, including Model-View-Controller (MVC), Model-View-ViewModel (MVVM), and Coordinator patterns

Practical knowledge of network protocols, along with an understanding of latency and reliability challenges

Demonstrated experience with multithreading, request queuing, and caching in code

Familiarity with Continuous Integration/Deployment tools and processes

Excellent teamwork, self-motivation, and eagerness to learn

Creative thinking, strong analytical skills, and attention to detail

 Working in an international environment on an impactful globally scaled technology in cross-functional team with an agile approach 

Open and healthy working culture where we care about people as much as about stability of the code  

Work equipment as per your specifications 

Hybrid work, combining working from our office to be closer to the team and working from home 

Work life balance thanks to flexible working hours

Generous free day scheme providing 25 days of holidays + 5 days of on-demand leave (sick days)

Ergonomic office and kitchen full of tasty snacks 

Contribution to Multisport Card or Benefit Card 

Learning opportunities with access to Udemy, O’Reilly, and our internal library of physical books, as well as a budget for attending conferences, including strong support if you want to present at a conference yourself 

Events, meetups, hackathons in the office or elsewhere

 We’re excited to witness the recently announced partnership between Showmax, Sky and NBCU coming to life here in Prague. Following the recent announcement of our partnership; that will bring some of the world’s best content and technology to streaming customers across sub-Saharan Africa, Sky are hiring developers to make it happen.

We’ve already rolled out NBCUniversal’s streaming platform Peacock to our European audiences. Now you have the opportunity to work on an exciting new venture powered by Peacock’s leading, globally scaled technology.  

Our team develops and supports market-leading video streaming services, underpinned by state-of-the-art engineering principles. We do this at huge scale: for millions of customers globally. No matter the device, the time or the place, we make sure that our diverse audiences can easily find and enjoy whatever they want to watch, choosing from the world’s best entertainment, news and sport.

You will join the Online Video Platform team (OVP) within Global Video Engineering & Apps where you will be positioned within business-critical propositions that support all our global streaming initiatives such as NOW, Sky Go, Peacock and SkyShowtime. 

You will be responsible for automating and streamlining tooling for cloud-based services using a range of scripting languages, automation tools, and cloud-native services. You will work with a variety of development / DevOps teams and business partners to ensure that cloud automation activities are secure, reliable, and efficient. We are on the lookout for motivated and outstanding individuals who want to be involved in ground breaking technology that truly pushes the boundaries and growth of streaming! 

Exploring and implementing new ways to automate systems, across a domain that powers business-critical streaming propositions globally.

Design and implement automation strategies, solutions and policies across a range of web API, network and cloud-based technologies.

Automate aspects of deployment and management of cloud-native applications.

Track and fix automation issues and develop solutions

Develop and maintain scripts to automate tasks and processes.

Collaborating with other scrum teams to help support their technical roadmaps by offloading deployment activities into our team workflow and achieving cross-team and department goals through automation.

Provide technical support and training to other team members.

Create and maintain detailed technical documentation.

What skills do you need to play your role well?  

Experience working with REST APIs and multi-distributed systems e.g. Cassandra

Familiarity with configuration and build tools including Jenkins, Gradle, Maven

An understanding of message driven systems e.g. Kafka, Amazon Kinesis, ActiveMQ

Applied knowledge of agile methodologies and software engineering concepts e.g. XP, Scrum, Pair Programming, TDD, BDD

Ability to communicate well with other teams and consult with developers of all levels of experience

 

Working in an international environment on an impactful globally scaled technology in cross-functional team with an agile approach 

We’re one of Europe’s leading entertainment and communication business. We’re always looking to improve by developing pioneering products and services which give every one of our customers experiences they’ll love. 

Join us as an QA Automation Tester and you'll be part of a dynamic team that makes sure our technology delivers the best experience for our customers.

Help to define and manage our testing processes

Participate in the team's agile planning & delivery process

Responsible for ensuring features meet specified requirements

Definition of test plans, test scripts, test data requirements, etc.

Planning test environments and automation strategies.

Preparation of automation frameworks and test runs for API back ends

Preparation of automation frameworks and test runs for frontend clients

What skills do you need to play your role well? 

Strong at programming in at least one of the following languages: Ruby, Python, Java, C# or JavaScript

Experience with CI process and tools (Jenkins, etc.)

Knowledge of networking and HTTP Protocol

Ability to track down the root cause of a defect using debugging tools

Ability to work well in a team, self-driven and eager to learn

Creative thinking, analytical skills, with attention to detail

Millions of customers use our apps and we have always been looking to improve by pushing the boundaries in our products and services providing every one of our customers experiences they love.

Develop excellent quality software using agile techniques

Deliver highly scalable and highly reliable products (yes, our customers expect 99.999% availability)

Collaborate with other developers, sharing and receiving knowledge

Promote best technology and software development practices into the team with passion and enthusiasm

Contribute to the whole development cycle, analysis, implementation, testing and operation  

Operation excellence and support are part of the development teams’ responsibility

Ensure that our applications are compatible with different versions of Android and different screen sizes.

Development in languages such as Java or Kotlin

Experience with Android SDK, Android Studio, Gradle, and Git.

A passion for Android development and a desire to stay up-to-date with the latest trends and technologies.

Strong problem-solving and debugging skills

We're the Online Video Platforms department. The team behind your favourite Sky products, and the platforms that power them. We make every moment magical, everywhere. Our team is made up of self-motivated, big thinkers who have an interest in solving problems and find new ways to captivate millions of customers by putting them at the heart of everything we do! From Sky Glass, Sky Q, Peacock and NOW to news and sports apps, we make entertainment even better and we can’t wait to get started on what’s next!

Lead the design, development and testing for new and existing Java microservice applications.

Promote the use of container technologies such as Docker and Kubernetes

Utilize pair-programming and help mentor and progress junior developers

Chip in or support training sessions to share knowledge with other members of your team/wider department e.g. training workshops, proof of concept overviews, lightning talks

Maintain software delivery principles and guidelines based on internal Sky standards and industry best-practice

Strive for continuous improvement of our systems and ways of working

Participate in research and technical initiatives to improve department technical capabilities.

Good system development knowledge in building scalable applications within Java

Experience working with REST APIs and noSQL Databases e.g. Cassandra

An understanding of systems e.g. Kafka, Amazon Kinesis, ActiveMQ

Familiarity with configuration and build tools such as Jenkins, Gradle, Maven

Familiarity with standard monitoring and logging tools including Prometheus and Grafana

A willingness to collaborate in a cross-team environment.

Knowledge of design patterns and SOLID principles

Practical experience building and working in a CI/CD environment

 Develop new user-facing features and get direct customer feedback on them.

About the team

How we work

Read our blog posts #data

How to Aggregate Percentiles - Tdigest to the rescue

Machine Learning basic principles, Part II

Machine Learning basic principles, Part I

Showmax customers' happiness matters

How to Aggregate Percentiles - Tdigest to the rescue

Machine Learning basic principles, Part II

Machine Learning basic principles, Part I

Showmax customers' happiness matters

Check out our other teams