Meaningless numbers

Courts in Pakistan, our government and our academia have discovered metrics. Every day, there is talk of data being our solution. Put numbers to performance reports, hiring processes, performance delivery, teacher attendance, student attainment and hospital management and we will have great governance.

There are those who argue that mobile phones and apps will control us. In the digital world, measurement is so easy we can be in a measurement utopia. All our problems will be solved.

Jerry Z Muller has written an important book to challenge this ‘metric-madness’. He claims a whole industry of evaluations has arisen that seeks to measure projects, performance and service delivery while human welfare and larger goals remain elusive. The Tyranny of Metrics asks the important questions: a) “Can we measure every aspect of human life and organisation?” and b) “Does the measurement culture distort individual behaviour to deliver perverse outcomes?”

The book talks about measurement mania that arose from the movement of scientific management of Fredrick Winslow Taylor who sought to measure every movement on the assembly line and shop floor for a distant manager to control. A class of managers with MBAs was created to manage by number. By the 1980s, the giant US manufacturing corporations that were managing by numbers had lost to Japan. Robert McNamara was a famous exponent of scientific management who obsessed over “body counts” during the Vietnam war. Yet despite technology and modern management, the Vietnam war was lost to human spirit.

McNamara went on to use his preoccupation with measurement in the World Bank which he ruled for a decade or more. Maybe that is one reason that the development world is now ruled with the new mantra of ‘evidence-based’. As Muller notes, distant generalists use metrics to control processes and systems without dirtying their hands. The pretension that evidence can easily be generated on human endeavours and organisations is what gives rise to the consulting business that has growth by leaps and bounds in the post war era along with the development business.

In Pakistan, we are now fully trained to focus on numbers whether they relate to our reality or not. Thus, the debate on whether GDP growth was 2.3 or 2.5 percent as if the difference means something carries no weight. We remain fixated on current account and fiscal numbers while we have lost governance and management to develop deep fissions in society, schooling and market development. Every few days donors and their consultants give us a new set of numbers to chase from their distant vantage points while our dispirited younger generation labours without real learning. Our values and trust and our organisations are eroding while we are on this number chase.

The measurement myth rests on three premises: we can measure all human goals in quantitative terms; once measured we can make the metrics open and transparent; and all we need is rewards clearly aligned to measurable goals. McNamara and the development advisor and their allies the consulting world now had a clear pecking order. They could set the system up in distant capitals and let the poor developing countries, so bereft of intelligence that this new metric-mania will solve the poverty problems. Thus, aid and conditionality were born to continue the white man’s burden of civilizing the world.

The Tyranny of Metrics cites medical studies to show that when doctors are incentivised for success in the operation theatre, they often refuse to take on difficult and risky cases even if it meant losing lives. Policemen who had bonuses tied to arrests and other time-bound success avoided complex investigations that were more critical metrics to larger security concerns such as lengthy drug and terrorist concerns. Standardised tests have been shown to focus students and teachers on gaming the test at the cost of larger learning objectives.

Closer to home, we have seen what happened to research when the HEC somewhat naively accepted the Taylor approach to research. Promotions were linked to the number of publications in so-called impact factor journals, which led to plagiarism, meaningless publications with no real link to Pakistan’s problems and citation mafias. The result is 200 universities with a dwindling demand for research and jobs for graduates.

Muller cites two important laws in trying to understand the passion for metrics that the world is consumed with. There is Campbell’s Law which says that “the more a quantitative metric is visible and used to make important decisions, the more it will be gamed”. Another similar law is Goodhart’s law which says that “anything that can be measured and rewarded will be gamed.” Simply put, all metrics when announced and visible will be gamed and the processes to measure will be distorted and corrupted.

Those enamoured of data cannot see that humanity is far too complex for accurate measurement. Not everything that matters is measurable. Nor does everything that’s measurable matter. Artificial intelligence still cannot replicate human judgement and intuition. Contrary to what Pakistani courts and bureaucracy say, the hiring, procurement and the performance management processes can never be fully devoid of human judgement. A lack of understanding of this simple point has contributed hugely to destroying our governance and our projects and public procurement.

Metric fixation seeks to replace judgment based on experience with standardised measurement. Since it is not possible to measure things like productivity, leadership and creativity, focus is shifted to inputs and experience. Our advertisements for hiring key personnel are full of meaningless metrics like long experience, age and related activity. Often the metrics are rigged to please the courts and the media but with a prearranged outcome.

Muller’s maxim, “The demand for measured accountability and transparency waxes as trust wanes”, needs careful consideration. Human systems and organisations like human consciousness are not fully measurable. Human judgement, professionalism and creativity are to be nurtured and not merely measured.

Society as well as the economy is most importantly built on trust, values and meaningful organisation, all of which are truly too complex to measure.

The writer is VC, Pakistan Institute of Development Economics.

Email: vc@pide.org.pk

Twitter: @nadeemhaque

Website: pide.org.pk