The Role of KPIs in Network Management

How do network providers measure performance to ensure that they are providing the best possible service? They use key performance indicators (KPIs). These are metrics established to quantify specific aspects of a functioning network. Reliability, as defined by IEEE, is “the ability of a system or component to perform its required functions under stated conditions for a specified period of time.” Each provider develops KPIs specific to their environment to ensure reliability and maintain proper controls on their network. (For more on the use of KPIs and other metrics, see Web Analytics: Terms You Need To Know.)

The Nature and Purpose of KPIs

KPIs are used in many different industries to determine the effectiveness of an enterprise. The selection and organization of those measurements are the purview of management. Certain KPIs may be examined by operational personnel to make improvements. Others are delivered to business units to inform executive actions. KPIs are also used to verify that customer agreements are properly met. In any organization, these metrics may be adaptable and have different forms and uses. Let’s discuss their application to network management.

Trouble Management

Operating a network is a complex endeavor, and problems occur. Network managers will often set a baseline for operations. What is expected of the network when it is working properly? Network designers determine optimal values and performance thresholds, which are then integrated into network management tools. When a high threshold is reached, the icon for a network element may turn red, or an automated ticket may be generated. These same metrics can be channeled into a KPI management system. In addition, the speed and efficiency with which trouble tickets or customer calls are handled may be captured for KPI reporting.

Reporting

One advantage of running a network is that you will likely have plenty of computing resources on hand to help crunch the numbers. And once these numbers are crunched, creative people will then be able to translate them into visually attractive presentations. Busy managers and executives need clear and succinct summaries. KPI metrics offer the quantified data that will help them make well-informed decisions. KPI reports are also effective tools for engineering departments in the design, planning and capacity management of current and future networks.

Optimization

We know that it’s important to keep the network out of trouble. But how can we improve it? That’s where network optimization comes in. Long-term profitability depends on delivering the best possible service to the customer at agreed levels. Efficiency is the key. While quality control teams may not be stressed by constant network surveillance, they are often tasked with in-depth analysis to determine the best path to network improvement.

Service-Level Agreements (SLAs)

Network providers normally offer a multi-tiered service portfolio with specific deliverables for each level. Customers have a right to expect service delivery as promised. These agreements are elaborated in the service-level agreement. KPIs can be used to ensure that SLA requirements are met and to keep customers informed about their service.

Test Case Methodology

One common methodology for network optimization is the use of test cases. A white paper titled “Measuring and Improving Network Performance” explains how Ericsson was able to use KPIs effectively in assessing and addressing a real customer issue. The following description borrows concepts discussed in the paper.

Define KPIs, Test Cases and Test Tools

Each element in the network produces performance data locally. But in order to determine end-to-end quality, a variety of elements must be polled. Collecting such data across the network can provide mission-critical information to address possible transmission deficiencies. To define and gather KPIs, network providers may hire individual optimization experts, or assign the work to specialized teams. These engineers will often create test cases for simulation of data to coax necessary information from the network. Test agents, such as dedicated measurement devices or end user equipment, are employed in the execution of the tests.

Prepare for Measurements and Execute Test Cases

Test cases may be used in different phases of design or operations. In test labs, they are an integral part of product development. During roll-outs, they are routinely used to verify connectivity and quality of service (QoS). During troubleshooting or to address known network issues, test cases may be prepared and executed to analyze and localize problems. The test cases may be written by design engineers, or they may be prepared by experienced operations engineers in consultation with more qualified experts. In mobile networks, drive testing is one way that KPIs are gathered. Test cases are also performed in the network operations center (NOC) or the back office.

Analyze Data and Create Improvement Plan

Without adequate data, the improvement of the network is left to guesswork. Once all KPI data is available, however, the optimization team can go to work. Before attacking the problem, an improvement plan should be developed. (Tweaking a live network without going through a proper change control process can be hazardous to networks, not to mention the careers of overly ambitious engineers.) Once in place, systematic changes can be implemented during designated maintenance windows.

Repeat Test Cases, Analyze Data and Prepare Final Report

The first look at the network may not be enough. It may be necessary to revise test plans or repeat them to thoroughly address the issue. The final report may include the test cases along with an explanatory email, or it may require a more detailed analysis.

Examples of Network KPI Usage

Network Operations

In the NOC, technicians use network management software to keep a close eye on the network – while department managers keep a close eye on them. To gain oversight of both the network and departmental productivity, managers make use of pertinent KPIs. Daily, weekly or monthly reports may track mean time to restore (MTTR), availability, number of calls, dropped calls or network congestion. A given network may automatically track hundreds of measurements. Selection of KPIs for reports is at the discretion of managers and senior engineers.

Quality Control

Those responsible for quality control and optimization may look for more finely tuned metrics. Here is a small sampling of KPIs chosen for reports prepared by this writer in a previous 4G mobile project:

• Average active users
• Average cell throughput download
• Average cell throughput upload
• Cell availability
• Maximum cell throughput download
• Maximum cell throughput upload
• Upload traffic volume (MB)

The focus and granularity of reports in these cases will depend on the nature of the problem under analysis. How many users or network elements are involved? Which specific technologies are in play? What is the time frame? Such a game of twenty questions can help when setting up the investigation. (For another example of KPIs in use, see How Far is Your Data From Your Analytics? An Overview of the SVoD Analytics Landscape.)

Customer Service

Ticket management and customer satisfaction are the responsibilities of customer service personnel. Mean time to close (MTTC) is one example of a KPI that may be used to evaluate ticket handling. In this arena, it’s important to respond to customer needs and requests quickly. Incoming phone calls are tracked, and the evaluation of departmental performance may involve a whole array of KPIs unrelated to the network itself.

Conclusion

The overall KPI management model can be described as “Measure – Manage – Monetize.” Engineering, planning and operations departments regularly use KPIs to manage network effectiveness. Equally important, KPIs help financial, sales and marketing departments to generate income and improve the bottom line. Key performance indicators take advantage of the tremendous power of computing that has become available to us today. Those who wield that power wisely to improve their networks may find that they have also improved their own chances to succeed in a very competitive marketplace.