A statistical test for forecast evaluation under a discrete loss function .

  1. Eransus Armendáriz, Francisco Javier
  2. Novales Cinca, Alfonso
Journal:
Documentos de Trabajo (ICAE)

ISSN: 2341-2356

Year of publication: 2014

Issue: 24

Pages: 1-19

Type: Working paper

DIALNET GOOGLE SCHOLAR lock_openDocta Complutense editor

More publications in: Documentos de Trabajo (ICAE)

Abstract

We propose a new approach to evaluating the usefulness of a set of forecasts, based on the use of a discrete loss function de�ned on the space of data and forecasts. Existing procedures for such an evaluation either do not allow for formal testing, or use tests statistics based just on the frequency distribution of (data , forecasts)-pairs. They can easily lead to misleading conclusions in some reasonable situations, because of the way they formalize the underlying null hypothesis that �the set of forecasts is not useful.� Even though the ambiguity of the underlying null hypothesis precludes us from performing a standard analysis of the size and power of the tests, we get results suggesting that the proposed DISC test performs better than its competitors.

Bibliographic References

  • Ash, J.C.K., Smyth, D.J and Heravi, S.M. (1998). Are OECD Forecasts Rational and Useful?: a Directional Analysis, International Journal of Forecasting 14, 381-391.
  • Greer, M. (2003). Directional Accuracy Tests of Long-Term Interest Rate Forecasts, International Journal of Forecasting 19, 291-298.
  • Henriksson, R.D., Merton and R.C. (1981). On Market Timing and Investment Performance. II: statistical procedures for evaluating forecasting skills, Journal of Business 54, 513-533.
  • Joutz, F. and Stekler, H.O. (2000). An Evaluation of the Predictions of the Federal Reserve, International Journal of Forecasting 16, 17-38.
  • Kolb, R.A. and Stekler, H.O. (1996). How well do Analysts Forecast Interest Rates?, Journal of Forecasting 15, 385-394.
  • Leitch, G. and Tanner, J.E. (1995). Professional Economic Forecasts: Are they Worth their Costs?, Journal of Forecasting 14, 143-157.
  • Merton, R.C. (1981). On Market Timing and Investment Performance. I: an Equilibrium Theory of Value for Market Forecasts, Journal of Business 54, 363-406.
  • Mills, T.C. and Pepper, G.T. (1999). Assessing the Forecasters: an Analysis of the Forecasting Records of the Treasury, the London Business School and the National Institute, International Journal of Forecasting 15, 247-257.
  • Oller, L. and Bharat, B. (2000). The Accuracy of European Growth and Ináation Forecasts, International Journal of Forecasting 16, 293-315.
  • Pesaran, M.H. and Timmermann, A. (1992). A Simple Nonparametric Test of Predictive Performance, Journal of Business and Economic Statistics 10, 461-465.
  • Pons, J. (2000). The Accuracy of IMF and OECD Forecasts for G7 Countries, Journal of Forecasting 19, 53-63.
  • Schnader, M.H. and Stekler, H.O. (1990). Evaluating Predictions of Change, The Journal of Business 63, 1, 99-107.
  • Stekler, H.O. (1994). Are Economic Forecasts Valuable?, Journal of Forecasting 13, 495-505.
Data source: Dialnet