Not statistically significant does NOT mean insignificant

Friday, July 22, 2016

AME Author and President, Fulcrum ConsultingWorks, Inc
http://www.fulcrumcwi.com/

My fully-loaded 2012 Audi A6 had an intermittent frustrating problem since the day I bought it.

No diagnostic codes indicated a problem. Escalation to German engineering had me ready to move back to Lexus. Their response was “it must not really be happening. Our codes would indicate if it were.”

That obnoxious response was based on the assumption they had thought of every cause of failure in developing the diagnostic codes. FMEA is not 100% and never will be.

Do you have customer data that you’re not actively using to improve your product?

Four years after I first reported the issue, Audi issued an urgent safety recall for the problem that I had been experiencing.

Why the delay?  It only happens on fully loaded vehicles with air conditioned seats. They say it took them four years because of the limited number of complaints. But you and I both know that even low level interest and basic statistical analysis could have identified correlation and subsequently led to root cause analysis.

Is your company only examining customer complaints when they reach a certain established quantity? Ten may not be a statistically valid sample size, but it may well be enough to identify potential correlations. Twenty may be enough to narrow in. Don’t let the statisticians (I have a PhD level Econometrics background myself) talk you out of recognizing and dealing with problems.

Excellent companies look at problems from the standpoint of their customer and with the assumption that their buyers are not crazy. While a reported problem may affect less than .01% of your sales, it’s 100% to the customer who has it.

Saying you care about quality is only believable if you demonstrate that to each customer. Data collection only has value if you use it.  Letting data age until a certain “critical mass” is reached is not the mark of a company committed to operational excellence. Neither is technical resources implementing “emissions cheating software” on a very statistically significant number of vehicles. Oh! Unethical behavior also precludes a commitment to excellence. Mistakes happen.  Fraud is not a mistake.

Data can tell a fascinating story, if you look at it.