IIoT Performance Metrics: Friend or Foe?

This audio was created using Microsoft Azure Speech Services

There is a fascinating story about British Airways which was building an unwanted reputation as an underperforming airline and how it was turned around when the Chairman of the Board decided to focus on just one metric…on-time departure*. The story goes that whenever a plane was delayed more than 2 hours, the airport manager at the relevant airport would receive a call directly from the Chairman asking for an explanation. The importance of this metric soon became very clear to all at BA and it drove improvement across the entire airline for years to come.

I like telling this story, along with the following quote reportedly from a lecture by Lord Kelvin on ‘Electrical Units of Measurement’, delivered in May 1883:

“I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.”

It’s clearly a very engineering-oriented view of the world and you can find various extrapolations of it throughout the technical literature.  For example, Schneider Electric presents the  28 metrics every plant needs to monitor in our discussion of how metrics need to be aligned to larger goals and objectives using the SMART principle (Specific, Measurable, Actionable, Realistic, and Time-Based).

The flip side to the inherent value of metrics, however, can be illustrated in this quote attributed to the British Prime Minister, Benjamin Disraeli by Mark Twain:  “There are three kinds of lies: lies, damned lies, and statistics.”

Gaming: playing to win

In a new book called The Tyranny of Metrics by Jerry Z. Muller, the author talks about what happens when the use of metrics gets out of control and we lose focus on what we are ultimately trying to achieve. Consider a surgeon who only takes straight-forward cases to improve his success rate.  Or teachers who “teach to the test” where higher test scores, and not student learning, is the main measure of success. In both cases the “metrics” looks great, but do you really get the results you want when expertise and real understanding have given way to the pursuit of “a number”?  I think we can agree on the answer to that one.

There is also an excellent paper on this type of approach which takes a critical look at LTIFR (Lost Time Injury Frequency Rate) as the be all and end all of safety metrics. The paper cites the infamous example of how an oil rig catastrophically exploded the very day they were celebrating its 7 year milestone without any LTI.

The point is that, 1) any number can be “gamed” and 2) it is the overall system of prevention, inspection, culture etc. that truly counts.

So what is the solution?

As long as metrics exist there will always be the temptation to chase some numeric holy grail, but Muller gives some suggestions and guidelines in his book to keep us on track, including:

• Think about the marginal utility of adding MORE metrics
• Recognize that metrics WILL be gamed

Schneider Electric has been delivering a solution for many years now that is perfect for the IIoT age. It is embedded in a product and service we call EcoStruxure™ Profit Advisor. It combines a top down and bottom up view of operational performance in both real time and monetary terms, and it uses a clear and common language that provides a number of useful measures that can facilitate a collaborative effort in helping you improve performance.

So maybe metrics are a slippery slope, but that’s not a good enough reason to not make them your friend, especially if you go into that friendship with open eyes and a good understanding of what you hope to get out of it.

* “Of course, British Airways, or any airline for that matter, needs to make sure it doesn’t value on-time arrivals and departures at the expense of safety and customer satisfaction. Employees ranging from flight attendants, pilots, mechanics, and bag handlers, who are under pressure to achieve time- based metrics, may be tempted to cut corners in ways that could have serious consequences.” Performance dashboards: Measuring, Monitoring, and Managing Your Business, by Wayne Eckerson.