Scaling YouTube with Machine Learning + Human Review

Inside the Minds of Zefr Technology

On this month’s episode of Technically Speaking, I got to sit down with Principal Data Scientist, Ryan Deak and Product Manager, Rob Knopfler. We discussed how Zefr is using both human review and machine learning to scale YouTube more efficiently and effectively, ultimately leading to the highly relevant targeting capabilities we are known for. 

John Merrifield: Tell us a little bit about the role you each play in building Zefr’s technology.

Ryan Deak: I work on the data science team, where our primary focus is understanding video content and relating that to the needs of our advertising partners. My job is to create statistical models to help curate our advertisers’ campaigns by finding the right video content to pair with their brand and help them steer clear of content that doesn’t fit with their desired message.

Rob Knopfler: I wear a lot different hats as a product manager, but what I really do is connect the dots for customers, commercial teams, and engineering in order to solve problems, and build great products. In this context, the problem we are solving is ensuring that our partners only run their campaigns in front of highly safe and highly relevant video content. This problem is especially challenging because YouTube is a rapidly growing ocean of video content that changes every single second.

JM: How do machine learning and human review work together to review all the videos on YouTube at scale?

RK: So, human review and machine learning are kind of like peanut butter and jelly. Machine learning is an algorithmic approach to scanning a ton of video and finding patterns between these videos. However, you cannot build these models without human review. Our team of experts reviews this content, finds details in it and makes nuanced decisions that models aren’t capable of making on their own. They are both really important, but you can’t have one without the other.

RD: Ya, I think Rob’s right—we really do need both. We use the human review to supply the behavior that we want the models to have for the specific videos that we review. But for a model to be useful, it has to be able to generalize to videos it hasn’t seen before. The interesting thing is that we get a confidence associated with each prediction it makes. So, if a model isn’t confident about a particular video, we can tell the review team to go back and review that video and thereby help that model to learn that type of content.

JM: Ok, Rob—question for you. YouTube is an ever-changing platform, so how do you not only keep up with it but take things to the next level?

RK: The beauty and the challenge of working on YouTube is Zefr’s commitment to building great products and move quickly with it. Today, we evaluate a ton of different data within a video: titles, descriptions, duration, imagery, amongst other things. We use all of these data points to evaluate videos holistically which allows our models to become even stronger and learn much faster.

JM: What’s the key for Zefr in bringing this all together to best help brands navigate YouTube?

RD: I think efficiency is key. Because YouTube is already so big, with more content being added all the time, it is crucial that we are smart about how we review videos. We need to spend our monetary, time, and computational budgets efficiently. By continuing to search out the best videos to make our models even better, we can scale this to brands and their specific campaigns needs.