Thursday, January 27, 2011

A Statistician Explains Escalations and Best Practices

NH Musica in Amstelveen has a name that evokes melody, something soft that combines with its being a hotel to bring up images of peaceful slumber.

However, even though the corporate class being conducted in its spacious conference room had the most forbidding of names –‘Statistics for Six Sigma Management’ – sleep was not the prevalent occupation in spite of the lulling allure of the name, comfortable plush chairs and excellent air conditioning. The man who was conducting the class, Aniruddha Sensharma, proved to be an engaging speaker. Not least because he had the ability to engage some of the audience in open combat. And to make it easy on our tongues as our minds struggled with the concepts, he asked us to call him Sen.

A tall, athletic man in his late thirties, he assured me that his presentation skills had not always been this good. “Well, there was a period of a previous life when I could hardly make myself heard on the first attempt …,” he disclosed over post class coffee. “On my first onsite trip to New York, I was – well – tongue tied, especially when the audience included pretty women ... blonde ones...”

Well, he had not only managed to make himself heard, he had proved to be a voice that was sure to reverberate in everyone’s conscience. And what could a statistician possibly say that would have such far reaching echoes? Well, the man talked with numbers and explained some phenomenons of corpocracy.

The attendees were from different echelons of our organisation. Some introduced themselves as Black Belts and Green Belts following the weird expertise ranking system followed by Six Sigma borrowed from the martial arts. When I confessed that I was not anything like that – and tai chi, the only martial art I trained in, did not offer coloured belts, he patted me on the shoulder and whispered softly, “Simon, you are better off than most. Given your martial arts background, let me use a Zen cliché. Your cup is empty.”

He proceeded to pose a problem for the group.

“Suppose you have the data of the number of days it takes for the support team to respond to very similar problems reported,” he began. “From the data collected honestly over the last one year, you have certain information. You now have a new project in which the client wants to set a Service Level Agreement on 4 days. From the past, you know the solution takes 0 to 8 days. What would you do?”
 A Senior Manager from the side smirked. “We cannot say no to the client, can we? And I hear several of these issues solved within 4 days or less. So, certainly we are capable of doing this.”

A smile touched the instructor’s lips. “So, if we manage to solve one of the issues within 4 days, we are capable of doing that?”

“Obviously,” the senior manager answered. “We have demonstrated our capability. We just have to follow whatever we did to solve that issue. You did mention the problems were similar, didn’t you?”

“Well, if Brian Lara scores 400 in one match, does that mean he can go on scoring 400 in every game?”

The Manager looked confused. “He definitely proves that he is capable of doing so.”

Sen shook his head.

“We are very susceptible to this mistake of confusing ability with the statistical definition of capability.”

The Senior Manager was one who had risen to his position by making his point stick. He did not yield.

“Sen, I don’t know about statistics, but I do know about business. I see no such thing as a marathon 400 to be reached. I see a lot of those numbers to be well within 4 days, and it is more like scoring a 30 or a 40 for a decent batsman.”

Sen looked at me and smiled.

“Simon, I apologise for the number of references to cricket.”

I laughed and informed him that I had worked long enough for Indian companies to have developed an idea of the game.

“Now, I will make it easy for you. What you say is right. 70% of these figures actually are less than or equal to 4.”

The Senior Manager shrugged.

“If we can do it 70% of the times, I’d say we can do it all the time if we made sure that the guys knew that their asses would be on the line if they slipped. I mean 70% is a damn good probability. I understand all the issues may not be solved within 4 days and some may slip the deadline, but again, speaking from a business point of view, what is required is a good first impression. If there are 10 issues that come in the first month and the team solves everyone of them within the SLA, I’d say they could relax after that. With 70% chances of keeping to the SLAs, with some commitment and accountability, we can do it.”

He laughed, and a lot of people laughed with him. Sen joined in. A lady among the Six Sigma Black Belts, however, differed in opinion.

“Well, I would approach it differently,” said the expert. “I would compute the mean and standard deviation of the numbers ...”

Sen responded readily.

“We have a mean of 3 and a standard deviation of 1.2,” he said.

The Black Belt Lady computed furiously in her mind. “That means probability of less 4 is than is the corresponding probability value of 4 minus 3.2 by 12. The z-score is .67 and we can convert into the probability value from the normal table ...”

Sen smiled at us. “Let me tell you the probability value of the z-score the lady has so efficiently computed. It is nearly .75.”

The Senior Manager clapped his hands. “There. You see, The Statistics vindicates whatever I said in layman’s terms. The probability of us meeting the SLA is even more than what I had anticipated. It is three fourths. I would definitely not stand for any screw up there. As I said. For the first ten issues, I would ask the team to deliver. Go beyond themselves if necessary.”

“Or else?” Sen asked.

“Or else? I would escalate the pants off them. Business has to go on. And what I found and the lady verified with her Black Belt expertise, the probability is as good as certainty.”

I spoke now, probing a little, implying that there was something unsaid in Sen’s reasoning. Why did he give me the sneaking feeling that he had a surprise up his sleeve?

Sen laughed.

“Well, my calculations are somewhat different, but let me tell you, it is hardly surprising to me. I have been around. I have been in the industry for around 13 years now, minus a two year forced break following the slow down after 9/11. And going by my experience, this is how business indeed operates.”

The Senior Manager nodded. “We have no choice.”

Sen smiled and moved to the board.

“Okay. So your assumption is that it is within the capability of the team to deliver the resolutions of the first ten issues within SLA.  Otherwise it is Escalation ...”

“Look, Sen. There are so many instances of meeting the SLA. Surely if we go deep into them there are a whole lot of best practices we can unravel ... I mean, I understand they are figures and can be translated into statistics, but it is business too ...”

A change came over our soft spoken instructor. His complaint tone made way for brisk, confident explanations.

“Okay. We will come back to you. First let me talk about the lady’s comments about the probability through a Z-score. The way the mean and variance were computed to get to the z-score and thereby the probability of solving an issue within 4 days was indeed correct but for a very basic error. The data of the days we have out here are integers. All between 0 and 8. They are essentially a set of discrete data points whereas you have gone ahead assuming it to be normally distributed. Forget that normality implies continuous data. Before applying the z-score, it is essential to perform a normality test.”

He paused. There were confounded faces across the room as people struggled with the concepts. Not too many understood the terms, but the situation was one of confrontation and some very axiomatic ideas were being challenged.

“Now, let us forget the z-scores and the normal approximation. We will go ahead with the first information that I derived from this data. 70% of the issues are solved within the given SLA. Now, our Senior Manager would provide motivational and escalation driven carrot and stick method to ensure that the first ten issues are solved within the SLA time frame. And since the probability of meeting the SLA is a high 70%, it seems well within capability. Now let us find out how feasible this actually is.”

He looked at us.

“Every time an issue arrives, we define success as solving it within 4 days. Historically, the probability of success is 0.7. Our Senior Manager friend got that much correct with his considerable intelligence. Technically, each issue is a Bernoulli Trial with probability of success 0.7.

“Now comes the second part. Our friend here wants the first ten solved within SLA. What it means statistically is 10 successes in 10 independent Bernoulli trials. Or 10 successes in a Binomial distribution with 10 samples with probability 0.7.  Whichever way you choose to look, it amounts to the same thing. However we try to compute it, the formula being 10 Choose 10 multiplied by 0.7 to the power 10 for the Binomial distribution, the probability of 10 successes in 10 trials turns out to be 0.028 which is less than  3%. So there is just about 3% chance given the current capability that the first 10 issues will be solved within SLAs and no escalation results from the efforts. In fact, there is just a 60% probability that we will get more than 7 issues resolved within SLAs.”

Sen stopped, eyeing the audience. The Senior Manager glared at the board. The Black Belt Lady looked chastened.

“It just can’t be true,” the former said. “I know it can’t. We can’t run a business that way. I know we can lie with statistics ...”

Sen smiled.  “In the supplied booklets, you will find brief write ups about Bernoulli trials, Binomial Distributions and Normal Distributions.  If whatever I said still surprises you after you have gone through them, I can offer you some help. But, believe me ... the human mind is not really tuned to understand probability by intuition. And this is why we have an industry that more or less runs on escalations. Because people forget to realise that the law of nature says that some things will go wrong. Sometimes even a Brian Lara can get out for a duck, although he has the ability to score 400.”

Now, as I sat with him in the cafeteria after class, he confessed something.

“Six Sigma is a sham, Simon. Human beings have used numbers to study a situation and figure out answers ever since they started to count. It is only a fancy term and a lot of propaganda. Oldest wine in sparklingly polished bottles. And unless a black belt has proper grounding of statistics, he is like a first aid trained social worker who tries to cure cancer.”

When I pointed out that he was a Six Sigma black belt himself, he sighed.

“I had applied for a job which had advertised for a process champion with a preferred black belt in Six Sigma. Till then I had had a day’s training in Six Sigma concepts and a Black Belt in Shotokan Karate. Both were documented in my CV. A search algorithm found me as a suitable candidate by matching strings. And time pressure and immediate assignment ensured that I was hired. So, although I already had a post graduate in Statistics, when my employers found out that I did not have a Six Sigma certification, they coaxed me to get one – almost at gun point. It’s such a wonder. Chance plays such a big role in shaping destiny.”

After initial doubts, I realised that he was not kidding. I mentioned that as a probability expert he should know the working of chance in this world. Particularly his explanation of rampant escalations that left many a manager red faced.  He became more animated than he had allowed himself to be in the class.

“Trust me, Simon. That is how the world operates. You did see that this sort of SLAs, which an ambitious decision maker will not think twice before agreeing to, is prone to bring in failures by lots. And with these black belts growing like mushrooms with five day crash courses in statistics, they will advised by flamboyantly ignorant men. This extraordinary lack of true understanding of reality makes the industry so prone to escalations.”

He paused for a while and winked.

“The funny part is, the chance factor will be ignored totally, and the successes that resulted in meeting SLAs in the first place may be analysed to limits for nonexistent best practices. Since it was chance all the way, for success and failure, there will be best practice reports written up on the fly, which is more removed from reality than reading tea leaves.

“Besides Simon. Escalations themselves. They always seem to improve performance. But does anyone analyse to see whether it has anything to do with the heat turned on?”

I asked him what he meant.

“Simon, escalations take place when outliers occur. When SLA is breached or something worse happens. That is a rare enough event. That happening twice in succession is even rarer. So, irrespective of whether or not the senior manager decides to deliver his tongue lashing, by the law of probability, the tendency will be to move to the better zone. It has nothing to do with the dressing down. The senior managers would be far better served to busy themselves learning some basics of probability than thinking up biting remarks and delivering them. But, they get touted as great motivators. Unknowingly leveraging on probabilistic laws.”

I said that the statistical insights were eye openers.

“It goes beyond this industry, Simon. It is everywhere. You see, there was this 3% chance of all the 10 issues meeting SLAs. So 3 of every hundred stupid managers will be success stories by sheer chance. And they will be the celebrated model managers. Most of the business gambles are the same. There can be analogous ventures in the business world with a probability of success something like .01 per cent. By sheer law of probabilistic chance, 1 of 10000 imbecile CEOS who take this gamble will come out winners. And then they will be worshipped as world leading Innovators, as superstar CEOs, appearing on Forbes. There will be books analysing their backgrounds, parentage, links to holocaust, seven habits, movement of cheese and all such nonsense. They will author books too – telling the world how to do business, how to invest, how to take decisions straight from different parts of their anatomy ...”

I laughed. “Now you sound very much like someone I know ... who, in fact, ghosted one such book for a Superstar CEO ... ghosted blogs too ...”

“I would like to meet your friend very much,” Sen smiled. “You know, this entire statistically ignorant caper is like watching a 1000 monkeys typing away on type writers. Sooner or later, one of them is probabilistically bound to come up with a line by Shakespeare. And when that happens everyone starts analysing the type of banana he eats, the trees he swings from, the monkey friends who groom him and so on.”

I was laughing very loudly by now.  There were odd looks from the other tables. Two of the managers were staring at me with a frown.

“Oh well, since you have left a couple of managers more than a little embarrassed, I ought not be laughing out loud at your comments. It might become an attitude problem and my annual appraisal may go for a toss ...”

He winced.

“Ah... the dear old bell curve. Don’t even get me started on that.”

Aniruddha Sensharma, aka Sen, is the hero of  Big Apple 2 Bites by Arunabha Sengupta.

2 comments:

  1. Replies
    1. Hi,

      I also think that Sig sigma could help BI specially in the section of defining effective K Pis.
      It is a broad concept and contains lots of techniques with which we could improve business processes. However, using theses techniques is strictly depends on how much the stakeholders want to be adherent to Six Sigma.
      Six Sigma Certification

      Delete

Simon van der Wiel is a fictitious character who appears in the novel The Best Seller by Arunabha Sengupta.

These lines are both collected from the novel and extrapolated from it - additional musings of the author through his alter ego


About Simon

My photo
Amsterdam, Netherlands