When the economy grew 6.4% from a year ago, does it really mean it grew exactly at that rate?
Those kinds of statistics are supposed to give us the hard figures that we all can fall back as the one and only truth. Like the physical ones where a meter ruler is a meter long. But those with statistical learning would understand these macro numbers, from the GDP to industrial production to prices are not free from errors (even the ruler has an error but I would think that error here would be considerably smaller compared to that suffered by macro numbers, unless, it is an astonishingly bad ruler). The GDP for instance is not exactly an account of a small company that has in it all of the company’s expenditure. That macro figure is at best an estimation of what is happening in the economy. The fact that we keep restating (not rebasing) the GDP figures every now and then tells you that just as much.
Yet, after working in the financial sector, I am quite surprised to learn at how standard error/standard deviation plays a minuscule role in most analyses. I think this is a problem because without reference to errors, data providers in various government agencies as well as analysts and economists in the financial market give the illusion that their data and their analyses (strictly non-normative commentary of the data) come with absolute confidence (academic economists have better record at this). When the economy grew at 6.4% in a period, it is 6.4%.
But that confidence is overblown. It is not really 6.4% exactly. The truth is that it is possible the GDP had grown around that figure. What exactly, nobody knows. Maybe that is a technological question that would be solved some time in the future. In the meantime, there are some errors in the data.
Before we go on further, for the benefits of those without basic statistical training, I want to emphasize that these errors are not mistakes. They are simply uncertainty that comes along with the data. Uncertainties are there because we cannot know everything about the world. But we can know enough to know about the general situation. Hence the usefulness of these inexact macro figures. I call it inexact, because the true figures fall within a range and it is a stretch to suggest a point figure is the true figure with certainty.
I have no doubt that these economists, analysts and statisticians understand the meaning of errors and its importance. I am not overly worried about this group who work revolves around data. They know there are revisions and they know the numbers can change. They know there are errors. They know these macro numbers provide a useful guide to the happenings in the economy and these figures are not exact numbers. It is more of a sample — a good sample to generalize the population — rather than the actual universe. Whenever they refer to a number, they have the statistical caveat at the back of their mind.
I worry about the non-expert consumers of these data and analyses. These users do not understand this and they take the figures put down as the truth. Consider for example the discontent against the inflation rate in Malaysia, where there are critics who claim it is too low. I think the publication of standard errors of consumer prices would partly help address their concerns by telling them that there is uncertainty in the recorded prices. Still, this will not address the criticism against the CPI too much because the critics also appear fail to understand that the weight of the final CPI number is in such a way that it measures the middle Malaysians. But we have enough microdata that those weights can be reconstructed to fit more than the middle Malaysian. But I think this is a different issue which I have addressed in the past.
I am bringing up the non-reporting of standard error/standard deviation issue because I am bit peeved when I see news reports that goes something like ”Malaysian industrial production growth grew slower at 4.3% from 4.4% last month.” Or the GDP grew faster at 6.4% in 2Q versus 6.2% in 1Q. I mean really, is it truly a deceleration/acceleration? Are we not just sensationalizing it? I am particularly annoyed when economic-illiterate politicians start to sensationalize these figures, spreading uninformed views to the wider public.
(Another example is the idea that China is the largest economy in the world in PPP terms. But how about including the standard error inside too before making that pronouncement? I bet Chinese GDP has an outrageously big interval.)
Is it not enough just to say, ”hey, the economy is doing okay”?
When I see that kind of changes, I am more inclined to say it is stable. In fact, we can get more scientific about it. Calculate the index’s standard deviation and do hypothesis testing to see if the change is significant or not. It is very easy to do such testing these days.
I admit, it is less sexy and mouthful to say ”there is X% probability that the economy grew faster compared to the rate in previous period”, than to say ”the economy grew faster today versus yesterday.” But are we sacrificing truth for sexy, short, punchy, headlines?
I think yes.
I am guilty of not providing the standard deviation too, but I think we (can I use the pronoun we?) need to change our ways. Yes, I think we mainly write for each other, but we have to realize, these writings go out to the public as well. Our statistical caveat might not exist in others’ mind. We need to put those caveats explicitly in the open.
By sharing the standard deviation, I also think it shows others that we are being humble about our data. It says, “these are my best bets” instead of “this is it and there is no other way about it.”
Ideologically, from libertarian point of view, the humbleness is important. Libertarians believe in the superiority of the market over state actions. My belief (before I get banged up for being a blind market apostle, there are instances of market failure where the government needs to come in) in the superiority comes partly from the fact that we do not know everything about the world. I think the idea of standard error is part of that philosophy: the idea that we do not know everything. Again, there might be a time when technology will solve that and bring about a libertarian nightmare, but right now, there are enough cases out there to tell us to be humble.
