Photo Credit: Lynne Albright/SHUTTERSTOCK
July 19, 2016
Over the last four weeks, we have looked at external and internal factors affecting financial forecasting and its professional practitioners (finance directors, budget managers, and analysts). We looked at developing the formal annual forecast and surveyed some of the relevant literature.
Today we conclude the series with a conversation with Dr. Gabor Melli, Director of Data Science for OpenGov, Inc. Gabor’s background includes 20 years of experience, numerous publications, and success in many diverse assignments. We explore how new data science techniques can enhance financial forecasting for governments.
FO: Welcome Dr. Melli. It is a pleasure to speak with you today. Let’s jump right in. What does “data science” mean?
GM: Data science is about applying basic scientific principles to data-rich environments. We typically use data to develop and test hypotheses in the form of predictive models that react to changes in the various provided inputs.
Modern technology helps us collect and analyze increasingly larger datasets. When applied to government finance, data science can gain new insights into trends, such as expenses and revenues, along with non-financial metrics.
FO: What early wins for governments do you see coming through data science?
GM: I think data science can help governments make better decisions by equipping leaders with actionable insights. For example, at OpenGov, we’re preparing our first generation of “projections”. This product examines historical trends to generate full year-end estimates at any point in the current year.
Finance and budget teams will be able to use these projected estimates to more quickly identify unexpected variances and to begin each year’s budget work. This work benefits from good estimates for the rest of the current year because they are part of the base to begin next year’s work.
Projections will also help department heads, project and grant managers, and other analysts with full-year forecasts for planning future work and funding needs.
FO: What are some of the major hurdles and roadblocks in this work?
GM: Like most data science initiatives, the main starting challenge of projections was to prepare the data to efficiently perform the task at hand. As I am sure your readers well know, governments can associate transactions to thousands of interrelated accounts for different departments, funds and object codes. Each time that new data is received or old data is changed, we need to effectively perform the predictive process.
The next, also typical, challenge was to define a measure of “accuracy” between each predicted value and what it actually turned out to be. Many of your readers will probably have encountered relevant concepts such as “mean squared error” and “absolute relative error”. I’ve been surprised how little prior discussion there has been about these critical predictive modeling concepts in the area of government financial analysis.
FO: Important aspects of forecasting include communication and education for many types of stakeholders, as well as providing operational insights to management for early corrective action. In light of this critical socialization need, how hard will it be to keep predictions transparent and credible?
GM: We understand that forecasts must be believable and credible so our customers can use them with confidence. This requires that the forecasting processes, its assumptions, and the actual algorithms used must be transparent to all stakeholders who reference the predictions. We think that this is possible by providing users with incremental gradations of model complexity. Users are able to select their own preferred method and introduce more and more automation to their selections.
FO: To follow up on forecasting practices, how do you react to the thought that credibility may actually be more important than accuracy?
GM: That goes to a bigger issue: a single predicted quantity is a very rough insight. For important decisions, a decision maker should also require a confidence score (for example low, medium, or high) whether it comes from some seasoned expert or from a trained predictive model.
Even better would be the availability of confidence bounds that suggest the highest or lowest value for a forecast with, say, 95 percent confidence. Forecasting algorithms are more suited to delivering this range of outputs, increasing the entire process’s credibility.
FO: How do processes for current-full year projections and next-year budget forecasts differ from what is involved in longer term forecasts, such as five and 30-year forecasts?
GM: We understand that most governments do five-year forecasts, and many need longer term (10-20-30 years) for strategic planning and or debt issues. Simple trend analysis may be adequate for short-term work in government where rapid short-cycle changes are not the norm.
But for longer horizons, this extrapolation of historical trends is less satisfactory. We have to better understand and model recurring vs nonrecurring activity, how fund balances can support forecasting alternatives, and the effects of national and regional economic trends.
This gets at the heart of the data science effort. We will use OpenGov’s large and growing governmental database, together with our unique aggregation of public data from many sources, to develop forecasting models and test them empirically across the data to find repeatable, reliable models.
FO: A recent article in Government Finance Review discussed three techniques: Expert judgment and analysis, deterministic forecasting, and econometric modeling. It sounds like you are hoping to help governments to rely less on the first two techniques, and move more towards the econometric model?
GM: Yes, exactly. That article gave a good overview of the current situation and pointed to some of the risks involved in the work. We believe that we will be able to help forecasters with modern technology and data science approaches. Many private sector organizations are benefitting from these techniques, and we think governments can too.
FO: Thank you for your time, and for the work your team is engaged in. We will stay in touch and look forward to hearing more from you over the next few years.
This concludes our Finance Officer’s Desk series on government forecasting. We hope you found this information useful and thought-provoking. As always, we welcome your comments, suggestions or other feedback. If you have experiences or stories you would like to share, we invite you to discuss possibly making a contribution to this space.
Want to read more content like this? Subscribe to our weekly digest!
Mike McCann moved into government service in Ukiah, then Monterey CA, after beginning his career in corporate (ADP, Wells Fargo Bank, Blue Shield of CA), not-for-profit (Blue Shield of Ca, Mendocino Private Industry Council), and start-up accounting. For the last 20 years, Mike has been hands-on with budget, financial reporting and accounting operations, including City budgets and CAFRs. He holds a B.S. in Accounting from SJSU and M.S. in Instructional Technology from CSUMB.
Contact Mike with questions or comments at email@example.com.