The VideoVerse
The VideoVerse
TVV EP 08 - Delivering 8K 50 Hz video to today's limited networks and devices.
In today's episode of the video verse, we have Jan De Cock, Director of Coding Development at Synamedia, joining us to discuss the new encoder HEVC and the complexity of delivering 8K 50 Hz video to today's limited networks and devices. While quality has always been a top priority for Synamedia, they continue to make significant steps forward in optimizing their encoders and improving quality control. With a brand new statistical multiplexing algorithm ready to go, they are poised to stay ahead of the curve in the ever-changing landscape of video delivery.
Watch the full video version.
Learn more about Visionular and get more information on AV1.
Jan De: So, my name is Jan De Cock I work at Synamedia as director for Coding Development.
Mark: Hey, Jan. It's great to have you on inside The Video Verse. You and I have crossed paths many, many times throughout the years, and I've always wanted to get you on a podcast. So here we are. You get to reveal all your secrets.
Jan De: Thanks so much, Mark, for giving me the opportunity to be here.
Mark: It's great to have you on. You know, I think both of us, we had an opportunity to talk just, I guess, what, a month ago now since NAB? Time flies, right?
Jan De: That's right. It was actually an awesome opportunity to finally be back among the people. First time back in the US actually in two years. And also it was an awesome moment to meet everyone, and yeah, also show the progress that we made.
Mark: That's right, that's right. Well, that's what was most exciting to me about the show was it did feel a bit like two years has passed, for some of us, even three years, really, and all of a sudden, it's like there's this great unveiling that happened for many companies, and that was really exciting. And I know for Synamedia, you showed something super cool, and I want you, rather than me tell the listeners, why don't you tell the listeners what you guys showed. What you built and were demonstrating.
Jan De: Yeah, absolutely. So I did get the opportunity to demonstrate it a lot. It was, yeah, Synamedia is not, I mean, it just has been there for a couple of years, about four years now. And for me, it was my first NAB being part of Synamedia. And it felt a bit like we'd been working in stealth mode for the past couple years. And for a lot of companies must have felt like that almost. So actually, yeah, last year, we started talking with BTsport about, well, the possibility of doing a trial event for 8K. And they kind of came to us asking, okay, are you able to handle live?
Mark: Can you even do it? They threw down the gauntlet, right? The challenge.
Jan De: Yes, absolutely. I was kind of like challenge accepted. But yeah, actually, we kind of knew that we could because we'd done some successful experiments already. At that point, it was not productized yet. And we knew there were still a couple of challenges to be done like on the input side and so on. But still, we were in a good position, and our HEVC encoder that we have, so it's a software based encoder. We've seen before that also offline, it does a great job. We offer it also as part of a very stable and robust platform, our vDCM platform.
And then for this project, we had to bring it all together, together with BT. And they've been a great partner in this. And then over the past year, I mean, we've been working on, yeah, the ingest side, the input side, the processing, and then for me and the team, the focus is on the actual encoding, the 8K encoding, and for what they wanted, 50 hertz, 8K all in software on a single server. It all turned out really great and, yeah, we got it working, and I think the test set up worked really well last month.
Mark: That's amazing. And this supports 10 bit HDR as well, is it?
Jan De: Yep. So it's 10 bit, HDR 10. So that supported in the encoder, and actually in the same server, we also got the packaging working, so it's dash packaged. And then our server was also used as origin and it's transmitted. And BT developed their apps, Tizen apps for Samsung TVs. So they were able to handle the signal. I think one of the challenges was also to stay within the limit of the network card, one hundred MBPS.
But we stayed well below that at 60 MBPS, and that turned out to be a good sweet spot for the quality that we were looking for. And actually, so the trial event was a premiership rugby game in the UK. So lots of action, lots of motion also. The details in the background with the audience. But yeah, I think it looked awesome. And that's also, I mean, we were very proud, of course, that we could show the content and the captures at an AB level.
Mark: That's right. Yeah, that's amazing. And I believe this was built on AMD, is that correct?
(05:39 Single socket)
Jan De: Yeah, that's right. So we used the third generation AMD mounts for this. We'd also been working with AMD, and been a great partnership also. And yeah, we leveraged their CPUs. They obviously offer CPUs with very high core counts, and that was a natural match also for our encoder. So we were able to leverage that, and I mean, that definitely gave, moved I think the needle also in what we could do. And made 8K on a single socket actually 8K encoding possible.
Mark: So this is a single socket, not a dual proc?
Jan De: The actual encoding is running on less than a single socket.
Mark: Wow, and then you're able to do packaging, so you're doing decode, encode, and packaging on 64 physical cores?
Jan De: So for this one, we had, so the encoding coming in through SDI cards. Yeah, 48 gigabits per second coming in. We can also do transcoding, but for this one, it is encoding coming in from 48 gigabits per second. And then compressed down to 60 MBPS.
Mark: Yeah, amazing.
Jan De: That was kind of the challenge. It sounds like a huge ratio, but I mean it is, of course, but.
Mark: Yeah, yeah. Wow, wow. And so was the test, was this like a contribution encoding type application or, I mean, 60 megabits is still a fairly high bit rate to streams, so.
Jan De: Yeah, 60 MBPS, this is targeting distribution. So it's of course testing the feasibility of what you can do at this moment. I mean, in the future, I can imagine that 8K will be going more hand in hand with VVC, for example, as a format. But at the moment, we wanted to show that the ecosystem is pretty much ready.
So with off the shelf servers, off the shelf CPUs, SDI input cards, packaging and TVs. So actually, I mean, the TV that we showed it on at the NAB was a 2019 model Samsung TV, and it was able to decode this 60 MBPS, I mean, up too close to a hundred MBPS. It is able to handle that. Now, 60 MBPS is not low, obviously. But on a lot of broadband connections, you will be able to hit this.
Mark: Yeah. Yeah, it's true. It's amazing to me how, you know, we used to think like, you know, 20 megabits or 50 megabits was like kind of high speed, you know? And if you're really lucky or if you really had the big budget you paid for like 100 or 200. Now I think everything's a hundred megabits. I don't even think you can get an internet package. I mean, at least, I guess it depends what part of the world you're in, of course, but yeah.
You know, let's talk a little bit about, as much as you can tell us, about your encoder. And presumably, this was predicated on an existing solution, or did you develop, I mean, did you develop this HEVC encoder from the ground up? Is it something Synamedia's had that you extended, you optimized? Can you tell us some more about it?
Jan De: Yeah. So actually in Synamedia, we have our own set of in-house developed encoders. Synamedia also has, as a background, some legacy even dating to Cisco. We were spun out of Cisco. We have Scientific Atlanta Heritage and so on. But we started from encoders, and over the past couple of years, we've really put so much emphasis on compression and improving and optimizing our encoders. And I mean, that was also one of the fun things for me too. And I was coming back to Europe to find that. Yeah, Synamedia is really doing this. But yeah, fast forward to HEVC that we've been using here.
So it's an encoder that's really mature. I mean, we also have a great team of coding developers in different locations working almost around the clock to improve it. And yeah, we've been pushing the limits in terms of video quality, compression, performance, also pushing the latency down, improving what we call the density, how many channels can you process on a single server on the CPUs that we're offering. And in this case, it was kind of a natural evolution also. But of course, there were some additional challenges to get 8K running on our platform.
But also here, actually, we started from our vDCM platform at Synamedia, which is very robust, like 24/7 stable video processing platform. And that's also where, as part of that, we are offering our encoders.
Mark: Got it, got it. Now, do you always run on AMD, or do you have any special optimizations for AMD? Or is it just purely core count that gives you the advantage on AMD over Intel or, well, I guess that'd be the only other obvious choice.
Jan De: Yeah, numbers definitely matter. That's the short answer, but we offer both Intel and AMD. So one of the, I think, fun tasks that I've been working on, especially in the first year that I joined is really seeing, okay, I mean, are we leveraging all that we can from these CPUs? And we went to several transitions, Intel generations also. We evaluate a lot of CPUs. And at the moment, we offer CPUs of both.
But what we've seen is that really on the high end applications, and when we're looking at 8K, for example, but also if we wanna get the most out of 4K ABR ladders and so on a single CPU, that AMD has been working really well for us. And then of course, we can go up to 64 cores on a single socket. And and it does pay off.
Mark: Yeah. Yeah, I know the companies I work with, we're seeing the same things. So really, really remarkable. AMD's just doing a fabulous job, and cores matter. More cores matter. There's no other way to put it. So, yeah.
Now, are you able to talk about things like, are you doing AVX 512 optimization or has that been a benefit for you? Because I have seen in some cases where it didn't really help. I'm just curious, you know, what you have seen.
(12:24 AVX512 optimization)
Jan De: Yeah. AVX 512, it has its benefits, but it's a difficult balance. I mean, it's a fine line that you have to walk. And also it's, I mean, one of the things that, I mean, one of the reasons that we are also investing so much in software is that we want to keep maximum flexibility. Flexibility in deployment, flexibility in adding features, and doing quick iterations and improving our encoders. And one of the reasons or one of the arguments there is also, I mean, we want to stay flexible also in instruction sets. And I mean, up to now, AVX 512 is kind of making it difficult to move there because it kind of locks you in to certain CPUs.
And also, I mean, if you optimize your code well, you will see benefits in certain parts of the code, but not everywhere. I mean, it also affects your clock speed, for example. It'll push it down a bit, so that's why it's not always a net positive, so.
Mark: When you think about hardware and all the way the continuum from purpose-built, you know, ASICs, purpose-built silicon, to GPU, FPGA, so maybe more the accelerated CPU type solution, and then purely CPU, what have you seen? You know, are there tradeoffs? Are you guys looking at some of those hardware architectures?
Jan De: Yeah, I mean, absolutely. And it's a question that keeps popping up, right? Indeed they are pros and cons, and they evolve over time as well. We also come from, I mentioned our VDCM. It has a legacy going back to the digital content manager with which app, hardware, implementations, or hardware, firmware, FPG type of--
Mark: Well, back then, that was the only way to do it, right? Largely?
Jan De: Right. And the densities you could hit, but I mean, they were amazing also at the time. And it's still, DCM is so widely deployed still. But for our virtualized DCM now, we made a very conscious decision to focus on software. So again, for the same flexibility in deployment that I mentioned. Also to be able to integrate it into our cloud workflows on, I mean, pretty much any type of CPU. And, but there's more arguments actually why this, it's almost a strategic bet, of course, but why it has paid off. And I must say that knowing what I know now, I mean, it was a good bet. Also looking at the evolution, for example, of the CPUs. I mean, if the CPUs would be stagnating, then maybe at some point, I mean, it brings hardware back into the picture for premium applications.
But also with the growing number of cores, it really helps software a lot and software implementations. And it makes our lives easier as well to integrate high-end applications, like full 4K ABR ladders on a cloud instance, or 8K on a single socket. And that's really helping.
Mark: You know, what are you interested in? What do you think about some of the new standards? You even mentioned previously VVC, and of course, you know, you and I have had conversations around AV1, and yeah. What's your team focused on? What are you excited about? What do you see coming, you know?
(16.25 Team focus on)
Jan De: Yeah. Well, there's a lot of exciting stuff. New standards.
Mark: We have all the time in the world.
Jan De: New standards in the, I mean, AV1. It's still a format that I see growing. It's a very powerful standard as well. So that will be part of our VDCM platform.
Also, we are working on our own VVC encoder. So it--
Mark: Do tell.
Jan De: I mean, it makes a lot of sense for us as well. And if I compare it to the days where, I mean, I was implementing an AVC encoder and transcoder. If you look at VVC now, it's a massive standard. It will give us so much opportunities for improvements for optimization and so on, so that's definitely something to look forward to. Lots of innovation there.
But other things that also we're working on and also that I believe in, it's a bit based on my background and video quality measurement and optimizing video streams. And I mean, in my previous job, I had the opportunity to work with also a great team introducing per title encoding.
Mark: And what was your previous job, Jan?
Jan De: Yeah, so before I went back to Europe, I was working in Los Gatos, California with Netflix. It was great to be part of the team also and push the limits there and see what you can do also, I mean, optimize VOD streams. I mean, it's very different of course. I mean, it has a lot of similarities on one hand with what we are doing now for live distribution, but also it's different in the sense that back at Netflix, it's spend a whole lot more complexity optimizing a single shot than what we can do right now.
Now it's really trying to push the limits in terms of, yeah, what can you get out of a single core or out of a single CPU for life? How many channels can you run, can you encode on a single CPU? So that introduces its own challenges.
And then over the past one or two years introducing, for example, machine learning algorithms. How far can you go with that? Obviously not the very deep CNNs or something, but you can do other things that are really exciting.
Mark: Sure. Have you had to change your view on certain, you know, how you viewed Kodak optimization and coder optimization when you were at Netflix compared to Synamedia? You know, is there anything that you're kinda like, yeah, I used to believe you couldn't do that, or you should always do this, and now it's like, you know, the tables are turned.
Jan De: Yeah, that's a good question, Mark. And indeed, I mean, there's quite a few differences in how we used to approach things and Netflix and also now how we do it in Synamedia. I mean, the focus on life, of course, it introduces new challenges. And I mean, what we constantly have to work with is this like rate distortion complexity trade off. So it's adding this other dimension on top of what we used to do. But then I'm also surprised at how many optimizations we can actually squeeze out of the CPUs.
Mark: I think it would be really helpful. I know that 8K is, on one hand, it's new and novel, and on the other hand, there have been a few trials and shall we say early attempts is maybe a good way to frame it. How did some of those early solutions, or how does your solution differ from some of the early ones? Were they all running on software, you know? Or have they been on hardware and now this is the first time that you're running all on CPU? Can you give us an overview?
(20:38 Overview of solutions)
Jan De: Yeah, that's indeed. It's a good question because also, I mean, showing this at NAB, I mean, we're not the first ones to do 8K, obviously. And I remember back in, was it 2008, 2009 visiting NHK Labs? Where I saw the first 8K demo on like a 250 inch projector screen. It looked amazing. It was very futuristic, I guess, at the time.
But then for a long time, also myself, I associated 8K encoding with very dedicated solutions, combining multiple encoders in huge like, I don't know how many rack units, all working together on different slices, tiles, or whatever, and then bringing it all back together into a single stream. And I can imagine that's still, I mean, a lot of people are still visualizing this as what you need for 8K.
But the downsides, of course, there was also that this comes at a cost if you have to split up everything, process in parallel, and bring it all together. Undoubtedly, you will lose efficiency and you will lose video quality.
So this is also one of the areas where, again, the evolution of the CPUs has enabled us to do things, I mean, one CPU that you couldn't do a couple of years ago. And also for us, I mean, we had to bring together a really efficient HEVC encoder, but also with the CPU cores. But that is definitely a significant step forward compared to even a couple of years ago, two, three years ago when you start seeing publications about these huge hardware, like, well, great engineering solutions, but maybe not as elegant as what you can get now.
Mark: Yeah. Yeah, exactly. Now, talking about quality, because you know, we did reference and you referenced that hardware is typically associated with not being as compression efficient, so you don't get the efficiency. And then usually the quality, you know, let's be kind, sometimes suffers a little bit, you know? Just the absolute video quality is just not there like it can be in software. But yet, even in an all software solution, you're always trying to push the bounds, improve quality while retaining bit rate efficiency and performance. So it's like these three levers, you know? And you can kind of move one or you can move two, but you can almost never move all three without something else being sacrificed. So what are you focused on on the quality side? Surely you're also looking to even improve the quality of what you have. What does that look like? What are you doing?
Jan De: Yeah, so there's a number of areas that we are trying to push the limits in quality. And if I compare also what we have right now with referring to a couple of years ago when I think back in the Cisco days, there was not too much emphasis on video quality. So I think we brought it back, and now over the past two years, something like that, we've made significant steps forward. And we do that by really optimizing our encoders. Making them smarter, both in objective terms, but also subjective quality. Because in a lot of cases, we are in shootouts as we call them, right?With competitors.
It depends on the test content. And we have to do good. I mean, the logos have to look good. The grass and the sports teams and so on. I mean, the details have to, like the stock tickers and everything.
So, I mean, we need to be able to compress those things really well. But also our rate control, our rate control is essential, of course. And improving our CBR, but also our quality control actually inside our VBR, these are things that we're working on. And also we've got a brand new statistical multiplexing algorithm ready now. And this is also still very important--
Mark: And that's new for you, right? Statistical multiplexing? We're talking about the internet. Like, what do we need to multiflex?
Jan De: I know. I know, and it's amazing what everybody's trying to do. I mean, with ATSC 3, but also, I mean, freeing up Spectrum and so on and trying to get the most out of ATSC 1, and then last week, I was at DVB World, We were talking about DVB and what's coming next and so on. is gonna be around for a while.
But I mean, these rate control algorithms, obviously, but we're also trying to make it even more exciting. And that's by being really smart about measuring video quality and also steering video quality. These are things that I'm also very passionate about. I know these are things that Netflix is doing and also in like shot based encoding and so on. Obviously we cannot spend all the compute power that, I mean, that you can do to optimize a single shot if you have to encode it only once, but distribute millions of times in a VOD scenario. But in a live scenario, there's still very exciting things that you can do. And if you can be really smart about the video quality and the way you steer it, there's a lot of exciting things that we're doing there.
Mark: Well, Jan, congratulations on all the great work that you and your team are doing, and it's great talking with you. And yeah, thank you for coming on Inside the Video Verse.
