The Six Forecasting Puzzles Experts Face

June 21, 2016 – Mike McCann


June 21 2016

Stories about the coming age of big data, data science, and predictive analytics are becoming more and more common.

Science fiction aside, while we all want better tools, it is hard to imagine anything will replace the seasoned finance professional’s role in bringing all the pieces together into realistic forecasts for strategic planning, long-range capital programs, and operating budgets.

Instead, finance professionals will use these tools to help build better forecasts. In this three-part series, we will explore the challenges experts face when making a robust forecast, the processes experts follow to forecast, and conclude with a discussion of how new technologies offer forecasters unprecedented benefits.

Every professional must consider many factors in building accurate forecasts:

1. Learning an annual process: Forecasts are hard to learn because they are not completed often enough to sustain the normal learning processes of repetition and feedback. A smart, well-educated accountant or analyst might move into a position with significant forecasting duties after a few years and continue to gain responsibility and scope for several more before assuming a title such as Finance Director or Budget Manager.

That means she may have seen predictions done five times before beginning to take ownership of the process. It is hard to learn any complex task with that few repetitions – and with a year between each pass, it is even more challenging.

2. Inflation and CPI adjustments: Both affect revenue and expense forecasts, and calculations are more complex than commonly understood. We find dense explanations like this on the web:

“Various indexes have been devised to measure different aspects of inflation. The CPI measures inflation as experienced by consumers in their day-to-day living expenses; the Producer Price Index (PPI) measures inflation at earlier stages of the production and marketing process; the Employment Cost Index (ECI) measures it in the labor market; the BLS International Price Program measures it for imports and exports; and the Gross Domestic Product Deflator (GDP Deflator) measures combine the experience with inflation of governments (Federal, State and local), businesses, and consumers. Finally, there are specialized measures, such as measures of interest rates and measures of consumers’ and business executives’ expectations of inflation.

The “best” measure of inflation for a given application depends on the intended use of the data. The CPI is generally the best measure for adjusting payments to consumers when the intent is to allow consumers to purchase, at today’s prices, a market basket of goods and services equivalent to one that they could purchase in an earlier period. The CPI also is the best measure to use to translate retail sales and hourly or weekly earnings into real or inflation-free dollars.”

It is left to each forecaster to choose the correct index (among many) and period for calculations, and to bend the curve to accurately reflect future rates.

3. Detailed information for personnel cost forecasts: There may be dozens of individual pay and benefit elements aggregated in a single employee’s compensation. In most governments, there are so many options that no two employees have identical components with matching elections. Costs come from several sources, all subject to change on cycles outside the government’s control:

  • Federal, state, and local tax rates and rules.
  • Labor unit negotiations change compensation or create new categories.
  • Group retirement and insurance costs are negotiated with third party vendors.

Summary forecasts based on prior year actuals with known changes applied as percentages are often close enough for annual updates, but less adequate for long-range planning, or use in changing circumstances.

4. External forces shape revenue: Forecasts depend on understanding external economic forces on national, regional and local levels. Interest rates, new and resale housing price trends, and sales volumes affect property taxes and development fees. Major shifts in local employers and transportation options further affect housing markets.

Sales taxes respond to broad economic patterns, major retailers entering or leaving, and trends in internet commerce. Utility taxes correlate to energy costs and may be impacted by innovations in how utility companies deliver services. Hotel taxes are highly sensitive to the economy, with conventions often the first victim of economic downturns. Finally, recreation fees are related to how flush local residents feel as well as changing age demographics, with kids and elders both using more services than working adults. 

5. Construction has its own cost planning methodology: The construction industry widely uses Published Construction Cost and Building Cost indices, and each has materials and labor components. Costs also fluctuate when slowing economies encourage competitive bids, while healthier times result in fewer bids from busy contractors and bidders expect higher profit margins.

6. Accounting for deferred maintenance: Deferred maintenance is a form of cost shifting, usually causing higher costs at some future point. Infrastructure maintenance cannot be deferred forever. Vehicles and equipment reach the end of their economic service lives, and even aggressive maintenance programs only go so far. Hardware and software become obsolete and more difficult to support over time, as the tech world continues to advance.

Next time we will look at bringing all these elements together into a final working forecast, and later, will look at some of the issues involved in automating this key process with data science, big data, and specialized models.


Mike McCann moved into government service in Ukiah, then Monterey CA, after beginning his career in corporate (ADP, Wells Fargo Bank, Blue Shield of CA), not-for-profit (Blue Shield of Ca, Mendocino Private Industry Council), and start-up accounting. For the last 20 years, Mike has been hands-on with budget, financial reporting and accounting operations, including City budgets and CAFRs. He holds a B.S.  in Accounting from SJSU and M.S. in Instructional Technology from  CSUMB.

Contact Mike with questions or comments at mmccann@opengov.com.

Category: Government Finance