4 additional things to know about Forecast Accuracy

How is your forecast accuracy measurement project going? I hope the last post convinced you to start measuring this. But there are still some open questions. Let’s take a look at some critical items that you should consider.

TIME SPAN

One of the things people often get confused about is the type of forecast accuracy that they should measure. We often create forecasts for many months out. Technically speaking, I could therefore calculate 1,2,3,4,5,etc month forecast accuracy (e.g. I take a forecast value from 6 months ago and compare it to the actuals from today or I take my forecast from last month and compare it with the actuals that just came in). That’s a lot of data! Based on my own experience and discussions with many controllers, I have come to believe that most businesses should focus on a short-term measure (say 1-3 months). The reason for that is simple: the further out we look, the higher the probability for random errors (who can forecast the eruption of an Icelandic volcanoe?). Short-term accuracy is usually more important (think: adjusting production volumes, etc.) and we should have way more control over it than over longer-term accuracy. So, pick a shorter-term accuracy and start measuring it.

FREQUENCY

How often should we measure forecast accuracy? Every time we forecast! Why wouldn’t we? Measuring once in a while won’t help us much. The most interesting aspect of this measure is the ability to detect issues such as cultural and model problems. Just make sure to setup the models correctly and the calculations will be automatic and easy to handle. You will soon have plenty of data that will provide you with excellent insights.

LEVEL

Where should we measure forecast accuracy? We simply calculate this for each and every line item, correct? Hmm…better now! We already have so much data. I would suggest to look at two key dimensions to consider (in addition to time): the organizational hierarchy and the measure. The first one is simple: Somebody is responsible for the forecast. Let’s measure there. We could probably look at higher level managers (say: measure accuracy at a sales district level as opposed to each sales rep). In terms of the specific measures, experience shows that we should not go too granular. Focus on the top 2-3 key metrics of your forecast. They could be Revenue, Units, Travel Expenses for a sales forecast. The higher up we go in the hierarchy we would obviously focus on things such as Margin, Profit etc.. The general advice is to balance thirst for knowledge with practical management aspects. Generating too much data is easy. But it is the balance that turns the data into a useful management instrument. So, you should measure this at a level where people can take accountability and where the finance department doesn’t have to do too much manual follow-up.

CAUTION

But before I finish here, just a quick word of caution. Inaccurate forecasts can have different causes. Don’t just look at the plain numbers and start blaming people. There are always things that are out of our control (think about that unexpected event). Also, there are timing differences that occur for various reasons (think about a deal that is pushed to next month).  We need to go after those differences that are due to sloppy forecasts.

What about analyzing and communicating forecast accuracy? More about that in the next post. Do you have any other experiences that are worth sharing?