Data spotlight: GDP revisions

Improved estimates suggest that the economy has grown by less than we thought. Is there anything more to the story?

When people talk about how well “the economy” is doing, generally the metric that they have in mind is gross domestic product, or GDP - a measure of the market value of everything produced within a country. How we came to this measure, and how well it serves that purpose, is a far bigger topic than I can tackle here - indeed whole books have been written on the matter. Here, I’m going to focus on just one of the issues that we have to be careful of when using this data: it can take a long time to settle on what the numbers actually are.

Like most other countries, New Zealand produces GDP figures every three months. For each release, Stats NZ don’t just compile it for the latest quarter; they also go back and recalculate the history of GDP, and all of its components, using the most complete information they have at the time. Sometimes those revisions can be large - enough to challenge our narratives about how fast or slow the economy is growing.

The latest release was one of those occasions. Much of the coverage on the day was around the fact that GDP had shrunk by 0.3% in the September quarter, against the small increase that economists had predicted (with a few exceptions). But there were also some sizeable - mostly downward - revisions to the level of GDP going back several years. You can see the extent of these revisions in the chart below. The upshot was that, since emerging from the 2020 Covid lockdown, the economy had grown by 1.4% less than previously reported. And for the past year in particular, the rate of growth was quite a bit weaker than thought.

I didn’t see much analysis of these revisions at the time - probably not helped by the fact that they were released in mid-December, when economists are usually looking to square things away before their Christmas break. The issue didn’t come up again until a speech by RBNZ Chief Economist Paul Conway last month. Obviously, their interest is in how these GDP revisions might affect our understanding of inflation pressures, and in that respect he was quick to downplay them:

It is capacity pressures – the net balance between supply and demand in the economy – that matters most for inflation. Yes, lower GDP indicates weaker demand, but also that the productive capacity of the economy was lower than previously assumed. That is, the recent GDP revisions do not necessarily mean that capacity pressures in the economy are much lower than previously assumed.

I think the “not necessarily” is doing a lot of heavy lifting in that statement. Whether GDP revisions tell us anything about inflation depends not only on the reasons for the changes, but also how recent they are. If it’s five years ago, then sure, we can probably ignore them - we already know what inflation was doing back then. But if it’s more like the last year or so, that may tell us something about whether inflation pressures are building or easing.

I’ll come back to this speech later. But first I want to give a brief explanation of why these revisions happen.

Why can’t we get it right the first time?

The heart of the problem is that the economy is big. Really big. Millions of households, hundreds of thousands of businesses, around $400bn of output in a year. It’s impossible to measure anywhere near all of this activity on a regular basis. Not only would it involve a massive amount of work, but the burden on respondents would quickly become an issue. (Just look at how hard it is to get people to fill out a household Census once in five years.)

As a result, quarterly GDP statistics are in many cases based on a more stripped-down data set. It may be a survey of a small proportion of businesses within an industry (hoping that they’re representative of the whole). It may use less detail, e.g. businesses’ sales, but not their expenses or margins. In a few cases there’s no quarterly indicator at all, so they have to extrapolate from recent history. Once Stats NZ has published its initial estimate, it will continue to refine it as more information becomes available.

There are three broad reasons why GDP might be revised:

1: Late data

In a few cases, the complete data set may not arrive until after the GDP publication date (in New Zealand, that’s about two and a half months after the end of the period). Where this happens, Stats NZ will use preliminary data for the first release, then will go back to update it as needed.

These revisions happen regularly in a few sectors, such as agriculture. These changes generally only go back a quarter or two, and the impact on overall GDP is usually small - a fraction of a percent.

2: More detailed but less frequent data

To reduce the administrative burden, Stats NZ does some of its more intensive questioning on an annual basis. A particularly important data source here is the Annual Enterprise Survey, which feeds into the national accounts in several ways.

Once a year, Stats NZ does a benchmarking exercise, to reconcile the information it’s getting from various sources. For instance, that might involve revising up the quarterly output estimates for an industry so that they add up to the (more detailed) annual data, while still retaining the pattern of changes from quarter to quarter. It could involve updating the weights for sectors, sub-sectors and so on the reflect the changing make-up of the economy. It could involve updating ‘deflators’ to better identify the split between prices and volumes. There are a lot of moving parts here, much of which happens below the level of detail that we get to see.

The annual revisions are published along with the September quarter GDP report - yes, the one that’s released just before Christmas. The revisions in this release tend to be larger, and go back further, than in other quarters.

Even after these revisions, the job isn’t done. It takes quite a while to prepare the data needed for the benchmarking exercise; for the production measure of GDP, the dataset that’s just been incorporated is for the year to March 2022. Anything beyond that date has been extrapolated to some degree. So those downward revisions to GDP over the last year or so could change substantially again once we get the next benchmarking exercise at the end of this year.

3: Changes in measurement

A less frequent source of revisions is when Stats NZ changes how it measures activity in a sector. That could be due to a new method, or a better source of data. Or it may be by necessity, if the existing data is no longer available or fit for purpose.

These sorts of changes are necessary for maintaining the usefulness of GDP over time. It’s relatively easy to measure things like the quantity of milk produced, or the number of cars sold. But we’re increasingly living in a dominated by services, where it can be conceptually difficult to measure what the ‘quantity’ of a service is. One example that’s cropped up over the years is broadband services. How should we measure the ‘quantity’ of service provided: the number of connections? The amount of data that people use? Or the amount that they’re paying for on their plans?

It’s important to remember that none of these three types of revisions imply that the previous figures were ‘wrong’. Each GDP report is based on the best information that Stats NZ had at the time; when better information arrives, they update accordingly. Sometimes there are genuine errors that are discovered and corrected later, but the GDP data is drawn from so many sources that a mistake in one of them probably isn’t going to make a meaningful difference to the total.

What drove the latest revisions?

Let’s go back to Conway’s speech:

We need to understand the reasons for these downward revisions. Digging into the expenditure GDP data reveals that weaker (inflation-adjusted) government expenditure accounts for much of the revisions. Some of this is due to methodological changes, such as Statistics New Zealand now accounting for school attendance when measuring value-added by the education sector… Private demand in the economy, which is more interest-rate sensitive, has mostly been revised up, with stronger consumption and business investment than first reported.

Here’s the chart that accompanies that statement:

I have a few issues with this interpretation. Firstly, I’m not sure that this government vs. private split is telling us anything useful. The public sector competes with the private sector for resources, so they both have a say in how close the economy is running to its non-inflationary ‘speed limit’.

Secondly, it glosses over the fact that the revisions were becoming larger in the last few quarters - which has the effect of lowering the rate of growth in GDP. And that downward revision in growth rates was happening in the private sector; growth in government spending was actually being revised up by that point.

And thirdly, it doesn’t really reflect where the revisions happened. The only example that Conway gives is the change in how output is measured for the education sector. It’s true that this doesn’t really tell us anything about capacity pressures - schools still used the same number of inputs (classrooms, teachers etc) regardless of how many students turned up.

But from the speech, you might think that this was the main source of the revisions. And that’s just not the case. Here I think it’s more useful to switch to the production measure of GDP, so we can see the contributions by sector. To keep things tidy, I’ve limited this chart to the 10 sectors with the largest revisions (in either direction), and I’ve used the cumulative change over the last four quarters.

Education is, let’s say, a curious choice to single out; it only just makes it into the top 10. The biggest revision by far was in construction - the heat has come out of that sector to a far greater degree than we knew. The details show that the revisions were in both housing and infrastructure, the latter of which contributes a lot to the downward revision in ‘government’ in Conway’s chart.

Looking at this chart though, I still don’t feel like I’m any closer to answering the question: what does this mean for inflation? Does a downward revision to professional services, for instance, tell us that the economy has more spare capacity than we thought? I dunno, maybe. Or maybe we should be looking at the whole rather than the individual parts.

If I’m picking on Conway’s speech it’s because, unlike the rest of us, how the RBNZ chooses to interpret this data has real consequences. I don’t really have a better answer, but I think we deserve more than the highly selective reading that we’ve been given so far.

---

To point out the obvious: yes, the blog is still going. The verdict seems to be that it’s OK to continue with it as long as it doesn’t clash with work in any way. That’s not terribly restrictive in terms of the content I have in mind, so the remaining question is how much of my personal time do I want to devote to it.

This post was something of a test of that. I ended up spending way more time than I intended on rearranging data, staring at charts, willing them to yield some insights… In the end, I don’t think I found anything beyond the surface-level story: the economy has shown less momentum than we thought.

I’d like to keep doing these ‘data spotlights’, but I’m going to try to pick topics where I already have an answer in mind. And I’d like to get into the habit of doing shorter, snappier pieces that need less crafting. Even with that, the frequency of posting going forward is probably going to be once a week at best.