We recently sat down with some of our leading technologists at NEP to discuss the state of IP Technology in the industry. See their thoughts below on where we are, what is working, what isn’t working and where we are headed.
Meet the Contributors:
Marc Segar – Director, Technology, NEP Australia
Joe Signorino - Vice President, Systems Integration & Design, NEP US Mobile Units
Casper Choffat - Vice President, Global Media Solutions, NEP Group
Scott Rothenberg – Senior Vice President, Technology & Asset Management, NEP Group
The current state of IP Technology/SMPTE2110
Give me a snapshot, from your perspective, of where IP technology is in the industry right now.
Casper: In my opinion IP technology, in the form of SMPTE2110, is in a maturing state, but IP technology in the broadcast industry as a whole is still very immature and has many challenges. NEP has been using IP uncompressed audio and video since 2015 and has always been at the forefront of this technology, helping the industry to move forward in its developments, and function as a test site for many manufacturers. In order to become mature in this space there is a need to invest in people and training. Learning how to work with IP within the broadcast industry is very challenging and has a very steep learning curve. We are investing in this here at NEP, investing in our people and training programs, to ensure we are where we need to be.
Joe: I agree. As Casper says, we have been using IP for a while at NEP. It has been 5 years since we started design on our first IP deployment here in the U.S., Supershooter CBS, and we are currently rolling out number seven with Supershooter 9. In the industry, IP is starting to settle in. We are just getting to the point that gear with IP I/O is being delivered. We are now in the stage of getting this gear fully functional, to pass interoperability tests, and sort out standard issues.
Marc: Yes. And we have to remember, IP technology comes in many flavours and is not one-size-fits-all-projects. Ultimately native and pure IP, whereby the baseband layer is completely removed, is still challenging. A more common approach is a hybrid, or blended, IP and baseband system. IP is still immature and requires a high skill level.
Scott: Yes, it is still in early stages, and everyone in the industry is figuring out interoperability. There is a lot going well out there, but generally it’s not as smooth/ easy a transition as the move to SDI from analog. Training of engineers - and having really good engineers - is key.
Can you talk a little bit about where we are with standards in the industry?
Casper: If we look at uncompressed IP, it is my belief we have gotten to a good basic standard with SMPTE2110. The advantage of SMPTE2110 over other uncompressed IP standards is that it is able to create separate flows for each media type, meaning audio, video, metadata, etc. all have their own flows.
Marc: Yes, SMPTE 2110 was a great first step in a suite of standards for transporting media over IP with discreet elements. Now we need to work on the overarching control standards like NMOS IS04 and IS05, which will simplify the approach for less skilled operators and IP will become more commonplace.
Joe: I agree, the basic standards have been adopted and are in revision stages. We are now finding a number of areas where the standard is loosely defined or missing critical areas of definition. This will be an ongoing process for the foreseeable future, but it is going well.
Scott: Absolutely. The basic standards are there, but they are general and leave room for interpretation, since not all manufacturers interpret them the same way, there have been some challenges. NMOS is not there and we really need something to make device discovery easier and more consistent.
How far have the manufacturers come on the equipment side of things?
Marc: More manufacturers are producing IP products, but at NEP we are still seeing issues in conforming to the standards. So, we continue to work closely with our vendors where they themselves are missing the skill to help move this in the right direction.
Casper: Yes. The manufacturers are the ones implementing the standards, and we see that most of them are now moving ahead with SMPTE2110 as a standard. Interoperability between manufacturers is in a maturing state, and we have a large role in pushing the manufacturers in this direction. We are letting the manufacturers know what they need to develop in order for us to be able to work with their equipment.
Joe: With the exception of the core routing systems the majority of equipment is in the first release stage. While some equipment has been available with IP I/O for some time it has, until recently, all come from one manufacturer – whose products were less common here in the US. Here, to make finding operators easier, our clients want the manufacturers they consider to be the “standard brands” to get the desired functionality up and running. This makes designing and implementing IP systems more difficult today, as standards develop and we run into interoperability issues. But it is following a logical course, standards are settling in, and the amount of available IP gear is increasing rapidly. We are moving in the right direction.
Scott: Absolutely, more equipment is becoming IP capable. However, there is still a need for SDI and I think all of the manufacturers are struggling with how much SDI to keep on equipment and how to make IP an option. This impacts cost... for them, for us, for our clients.
What capabilities are a reality today versus just hype?
Casper: NEP is at a stage where for new broadcast facilities we are building, IP is a given. All new deployments of NEP will be based on native IP platforms.
Marc: Yes. And now It is completely possible to build an entire IP system from camera to broadcaster; this isn’t hype but reality. Its why the Andrews Hubs have been so successful.
Joe: Absolutely. We have a lot of fully functional large-scale broadcast systems in operation performing well. That being said, at this point, it’s not always easier, it’s not cheaper, it doesn’t save a ton wire, it’s hot and really loud. But here is what isn’t hype: It does allow us to build larger, more scalable, flexible and format-agnostic systems that should perform well into the future and meet the needs of our clients, production folks and operators. Also, IP should allow us to replace or upgrade certain components in the system with less of a need to replace everything.
Scott: Yes, IP allows for greater scale for things like The Andrews HUB, it reduces some cabling, enables some things in centralized production to be easier, etc. But the notion that IP is cheaper is hype – at least at this point.
Casper: Also, we should remember that connectivity is a prerequisite for using IP at scale, especially when looking across borders and connecting different parts of the world to each other. NEP is doing this with our strategically positioned hubs in US, Norway, Netherlands and Australia. For this global connectivity, IP is an essential need. There is no way to do what NEP is doing today without the use of an IP native platform. That’s not hype.
There are a lot of buzz words out there in the industry surrounding IP right now. What are they? What do they mean? Which ones are actually meaningful, and which ones are just… buzz words?
Joe: "Future-proof" may still be the biggest buzz word, and it’s pretty much just a buzz word. It’s impossible. COTS- Common Off the Shelf. Most of the gear we purchase is not really "common off the shelf”, but some is used in a much larger scale in other industries. Which is unlike the very limited run SDI gear we are used to.
Marc: Buzz words aside, IP is the only technology that is infinitely scalable. It doesn’t care about frame rate or resolution and because its natively virtual, means we can truly put people anywhere on the planet with an appropriate connection.
Lessons learned
With what we have learned providing IP tech over the last few years, what does IP actually make easier? What does it make harder?
Joe: IP has made implementing large, more flexible and scalable systems easier. But, up to now, IP is harder to configure, monitor and trouble shoot. It requires training and really skilled engineers. Luckily, we have the best engineers at NEP and are dedicated to developing our talented team.
Marc: Also, managing your network and ensuring that the correct security is in place is a challenge. The rest is not hard if you, again, have properly trained staff. For us, it meant building a new department of network specialists around the world who all share one common goal – to make things easier.
Casper: What does it make easier? Doing what NEP does with our Centralized Production platform with our hubs around the world, and connecting these hubs to different venues or locations, would not be possible without using an IP-based platform. For us, IP results in faster deployments, generally, because cabling is a bit less compared to a baseband infrastructure.
I agree, the hard part about IP and IP deployments is the skillset required to build, run and maintain it. But like Joe said, NEP has a lot of trained engineers, and we constantly invest in talent. There is a steep learning curve with IP. But we are out there, and have been out there, doing it well.
What have we seen that has worked well and what hasn’t worked?
Marc: The early days were hard. Interpretation of the standards was not consistent, and many vendors didn’t really understand them.
Joe: Yes, since we are moving a bit ahead of the standards curve, it is still harder to interface systems together, add gear to systems and change configurations than it is in the SDI world. But we have been able to deliver all of the features our clients are accustomed to.
Casper: What really works is the flexibility of your sources and destinations. Basically, once you have your sources in IP you can virtually send them anywhere in your platform. At the same time that flexibility creates its own issues, since bandwidth management is a real challenge. We found that the current control systems in the market are a bit limiting in controlling our IP platform in a flexible, scalable way. Bandwidth management, instant configuration of equipment, instant network switch port deployments are all very labor intensive and complicated workflows that require highly skilled engineers. At NEP, we are spending time and energy figuring out how to solve this problem in a seamless, easy-to-use way. We are also dedicated to investing in our people and continuing to train our engineers so they have the skills they need.
What were the real “gotchas” with deploying the technology? What surprised each of you the most?
Scott: Some of the basics, like switching between 720p and 1080i, were harder. Also, 23.98 didn't work out of the box on some equipment.
Joe: Lack of GPS availability at venues. The general complexity of deploying PTP as reference. And since this is fairly new tech, and a huge change from analog or SDI, the setup, test and commissioning process is challenging. Any changes or software updates that occur can affect things that have already been tested, which then requires a large amount of re-testing. When you are up against time constraints, like we often are with live TV, that isn’t always practical.
Casper: The scale of the platform. I always thought that with IP we would be able to scale to infinite numbers, but the management of that scale, deployment and reconfiguration really surprised me. This is especially true as we see the need for bandwidth keep rising as people make the switch to HDR and 4k, want more graphics, need more replay channels, more localization… Again, we are working on ways to solve this scale management issue.
Where we are headed
Looking at the current landscape, what is coming up immediately with IP technology?
Joe: I think in the near-term, we should see a larger variety of IP connected gear becoming available, allowing us to pick from a greater variety of equipment.
Scott: Yes. It will continue to mature and become more pervasive. People will become more skilled in operational aspects, and the tools for troubleshooting and control will get better.
Casper: Also, one of the constraints today is quick access to affordable bandwidth – especially as the need for bandwidth more is increasing. I believe the standardization in the use of compression technology within IP will be a major focus over the next few years. Uncompressed IP requires so much bandwidth, and technology out there is already capable of limiting that bandwidth per signal in a 1:10 ratio, without even delaying the feed milliseconds. NEP is already using this technology in our Centralized Production platforms around the globe. But I think there will be more focus – and there should be more focus – from the manufacturers to standardize and get to even better ratios of bandwidth reduction per signal.
I also think we will, at some point, see a push towards public cloud and public internet infrastructure usage for transporting IP from A to B. But this will still take quite some time. We are already using this for some customers, but to use this for large-scale event productions is still a bridge too far for the quality level that NEP delivers.
What are you most excited to see as IP technology develops in the future?
Marc: Three years along in the Andrews Hub project and we see the incredible power of IP. In that short three years, network switches have doubled in processing power, port speeds have quadrupled from 100Gbs ports to 400Gbs ports – and all for less investment. In the next three years we will see 1Tbs ports. It’s this speed of change that is exciting. Europe is building a 400Gbs inter-capital network and we are already working with telcos in Australia on the next generation of the Hub WAN.
Joe: I am excited to see detailing and adherence to the SMPTE and AMWA standards. This will help us make great steps to interoperability so we can get to a more “plug-and-play” environment. This will really help to reduce most of the issues we have all discussed.
Casper: I, personally, am most excited to see the expansion of the NEP Centralized Production platform as IP matures - the sky is the limit I would say.
Any final or additional thoughts?
Joe: When you look back, analog was difficult when it came to remote TV and large event setups. There were a lot of challenges to contend with. Then, when you think of the change from SD to HD, it had its moments, but it was much simpler. It started out slower, with a few early adopters, and it was a, relatively speaking, long time before the switch was thrown and our industry changed. There was a lot of gear that was made before standards were adopted, which made it quickly obsolete. That is the price of progress, though, and the price of being an early adopter and a leader in the business. It was almost exactly 5 years ago that we threw out our SDI design on Supershooter CBS and started over in IP. Comparatively, in the big picture, IP is moving along pretty well, and it has helped us accomplish a lot.
Marc: Yes, and for NEP, we can clearly see the future. A world where we can work in a completely virtualized manner. A world where we share resources and equipment from anywhere - and you can do your job anywhere on the planet as long as you have the bandwidth available. Bandwidths are doubling every few years, and as we adapt and continue to innovate, we will harness its power to do amazing things.