Interdependence Day — taking testing out of the lab

Rise of 5G fuels network complexity and IT convergence

The charge towards 5G is crystallising the opportunities and challenges that a cloud-native network infrastructure present for the telecoms industry.

Virtualised infrastructure brings multiple suppliers providing cloud and network functions, with new vendors entering the supply chain at a rate that Spirent believes is unprecedented.

“It’s a pretty radical change in terms of features that we’re trying to coordinate as an industry to unleash the power of 5G”, says Stephen Douglas, Spirent’s Head of Market Strategy.

“ 5G’s use of cloud software is moving us away from isolated or siloed point solutions towards a more holistic, joined-up IT-oriented offering, and, with the complexity of new 5G systems, the complexity of the testing that goes with it increases ten times — a hundred times, even — compared to the past.”

Douglas.

Douglas recognises the need for 5G systems to be agile, but also notes the danger that the increased level of complexity can become an anchor on progress. “The last thing you want is to invest in 5G and discover that the speed at which you can innovate or make changes on the network is incredibly slow”, he says.

Managing this complexity may require operators to adopt practices already in place in the world of IT, to facilitate a surge in software releases and infrastructure updates that contrast with the traditional telco vendor cadence, particularly in areas such as testing.

“ Traditionally, a big vendor would give you a few releases a year. Now, you’re seeing releases every few weeks, and that changes how you think about testing. Historically, validation might take months, but with the new volume and velocity of changes, you can’t simply do that. That’s leading the industry towards the IT principles of continuous testing.”

Douglas.

Meeting the need for speed

Doug Roberts, Spirent’s Senior Vice-President and General Manager for Lifecycle Service Assurance, encapsulates the pressing challenges across network operation, maintenance, and testing within service providers as essentially reflecting the need to move faster.

A key challenge with more open, disaggregated infrastructure is that, while each supplier’s component may meet industry standard specifications, “the specs don’t cover how they all work when five, six, or more of them are put together and you try to drive an OTT delivery model across them”, he elucidates. Any expectation that “everything will work as advertised” is likely to be misplaced. Consequently, within a network environment that will be continuously evolving, Roberts considers it is essential to embed testing that is capable of anticipating problems.

Automating testing is also becoming crucial as the new infrastructure landscape emerges, with Roberts noting that “people simply do not have the time, the resources, the capability, or the expertise to manually deliver sufficient test cases with any viable time to market”.

He also anticipates operators becoming more fluid structurally to facilitate the cloud- and software-driven transformation of networks, blending lab-based teams, maintenance teams, and operations.

“ You can’t just continue with a manual operating procedure model in the lab, then transfer it over to maintenance engineering, and then operationalise it. This is now a DevOps environment: it’s continuously on, and so a lot of these traditional walls that we have seen and experienced historically are starting to be torn down.”

Roberts.

Maintenance engineering is said to be particularly affected by having been “thrust into this new world order of interdependent infrastructure”. After traditionally working with defined structures, virtualisation and cloudification have “dramatically slowed down maintenance engineering’s ability to deploy and manage new infrastructure rollouts”, according to Roberts.

Live operations, meanwhile, are dependent on minimising the time required for triage and troubleshooting, but, again, complexity is heightening the challenge where traditional methods such as passive probing monitoring can prove ineffective in supplying the data required to address or pre-empt network failure.

Reassessment of fundamentals has been reflected in Spirent’s own approach to the market. Historically, Spirent’s focus was on what Roberts described as “the feet on the street” — the engineers, technicians, and tacticians responsible for testing across network parameters. Nowadays, Spirent has a model built on its traditional relationships to deliver a series of verticalised product sets that reflects the strategic relevance of continuous monitoring and assurance in the new network landscape.

“We began building what we call ‘outcome-driven services’”, confided Roberts. These are intended to address service provider goals, such as deploying 5G standalone networks, by identifying the steps needed to certify disaggregated multi-vendor infrastructure and address the ongoing monitoring and maintenance. In doing this, Spirent has created subscription-based access to a library of the test use cases necessary to support operational key performance indicators in a live environment. “While that may seem a very simple change, it has transformed our commercial engagements”, he shares.

Continuous testing: completing the network virtualisation toolkit

Aloke Tusnial, Vice-President for Spirent’s Cloud Business & Solutions, sees continuous testing becoming part of the adoption of DevOps practices within networks, building on the shift towards a continuous implementation and continuous deployment (CI/CD) process. He believes this will help address “thebiggest hole in the virtualisation story” — the absence of a ‘closed-loop’ environment.

“ As you move into the cloud world, and a place where you can actually scale and heal, and achieve the promise of network cloudification or virtualisation, you have to have visibility. If you can’t monitor, if you can’t assure, if you can’t test, and if you can’t measure, then you can’t have that. It’s not a true closed-loop environment.”

Tusnial.

At present, Tusnial sees network faults and crashes being addressed by an approach that can be summarised as “kill it, clean it up, and restart it”. This fails to identify and address the root problems — “‘Restarting’ is not necessarily the answer to everything”. Instead, visibility should be critical: “You need some kind of monitoring, testing and measuring that can tell you in real-time where the challenge is, to help you in fixing it if possible, and then allowing you to close that loop”.

New (shared) infra needs new performance metrics

Another aspect of the network environment evolution is the greater use of shared infrastructure across fixed, mobile, physical and cloud. The Spirent team consider that new thinking on service level agreements (SLAs) and key performance indicators (KPIs) will be necessary for networks based on shared assets and partnerships.

For Aloke Tusnial, this hybrid environment is already here: “there is no service provider in the world today that owns all its own infrastructure”. Regardless of asset ownership, the service provider remains ultimately responsible for the SLA to the end-customer, “and they need to have visibility into places that are theirs, and places that aren’t”.

“ If service providers are missing SLAs, they need to know, ‘is it because I’m having issues or is it because one of my partners is?’. They need to be proactive and ensure the right level of redundancy at an optimal cost.”

Tusnial.

Reimagining test as a variable cost and driver of efficiency…

The increase in network elements necessitates a rethink on costs, including around testing and assurance, with Aloke Tusnial considering that “when you are working at that massive volume, you can’t afford to be at the same cost structure that you were before”.

He anticipates that, while cost structures are changing around 5G, testing will remain proportionally similar at around 10%–30% of budget although more aligned with deployment costs — as they decrease, so will the cost of testing in a continuous environment. “Testing becomes integrated with what you do, and part and parcel of rolling networks out”, he explains.

… and revenue generator

“ Continuous testing becomes a mechanism for service delivery, whatever the industry you’re working with, with autonomous test and measurement embedded in the infrastructure.”

Doug Roberts.

Beyond improving efficiency, Spirent asserts that assurance can be leveraged as a revenue generator for service providers, rather than just a cost centre.

“Nobody has figured out how to make money out of 5G yet”, notes Aloke Tusnial, but the enterprise segment is most promising. “This explains the push to the edge, and the push towards working with IoT vendors on new use cases. Helping industry automate its world — and that’s where the money is going to come from”.

Spirent foresees the emergence of embedded assurance as an enabler of added-value service delivery for industry-specific solutions evolving alongside 5G, IoT, and other next-generation propositions.

As operators seek to ensure they are not cordoned off as connectivity providers, and deliver services that go beyond simply facilitating new enablers such as IoT, they will need to adapt their way of thinking on assurance. According to Tusnial, this change is already happening: “We’re now hearing operators saying ‘I need to meet these SLAs’, or ‘I want to sell this new kind of service, can you help?’. It’s not just questions on testing a network environment; it’s a different conversation”.

As operator engagements move from engineering focus to operational, “the problem statements change”, notes Tusnial. These could revolve around generating revenue, or understanding the limits and potential of use cases. “‘If I want to invest in self-driving cars, how do I ensure that even when I have a black spot in the network, the cars are not going to crash?’, or, if operating thousands of drones, ‘How do you ensure they don’t go into a no-fly zone?’” he shares as examples. According to Tusnial, these use cases are already being explored by operators in markets around the world, with 5G network overlays helping provide control, and measurement and testing helping provide visibility.