Istanbul Business Research

SUBMITINFORMATION
EnglishTürkçe

Research Article


DOI :10.5152/iujsb.2017.001   IUP :10.5152/iujsb.2017.001    Full Text (PDF)

Comparing and Combining MLP and NEAT for Time Series Forecasting

Serkan Aras, Anh Nguyen, Allan White, Shan He

Neural networks are one of the widely-used time series forecasting methods in time series applications. Among different neural network architectures and learning algorithms, the most popular choice is the feedforward Multilayer Perceptron (MLP). However, it suffers from some drawbacks such as getting trapped in local minima, human intervention during the stage of training, and limitations in architecture design. The aims of this study were twofold. The first was to employ NeuroEvolution of Augmenting Topologies (NEAT), which has many successful applications in numerous fields. In this paper, we applied it to time series forecasting for the first time and compared its performance with that of the MLP. The second aim was to analyse the performance resulting from the pairwise combination of these methods. In general, the results suggested that the forecasts from the NEAT algorithm were more accurate than those of the MLP. The results also showed that pairwise combined forecasts in general were better than single forecasts. The best forecasts of all were obtained by pairwise combination of MLP and NEAT.

Keywords: Forecasting, Time Series, NEAT, MLP, Combined Forecast, Neural Networks

PDF View

References

  • Angeline, P. J., Saunders, G. M., & Pollack, J. B. (1994). An evolutionary algorithm that constructs recurrent neural networks. IEEE transactions on Neural Networks, 5(1), 54-65. google scholar
  • Aras, S., & Kocakoç, İ. D. (2016). A new model selection strategy in time series forecasting with artificial neural networks: IHTS. Neurocomputing, 174, 974-987. google scholar
  • Armstrong, J. S. (1989). Combining forecasts: The end of the beginning or the beginning of the end? International Journal of Forecasting, 5(4), 585-588. google scholar
  • Armstrong, J. S. (Ed.). (2001). “Combining forecasts”, Chapter 13 in Principles of forecasting: a handbook for researchers and practitioners (Vol. 30). Springer Science & Business Media. google scholar
  • Armstrong, J. S., & Fildes, R. (1995). Correspondence on the selection of error measures for comparisons among forecasting methods. Journal of Forecasting, 14(1), 67-71. google scholar
  • Bates, J. M., & Granger, C. W. (1969). The combination of forecasts. Journal of the Operational Research Society, 20(4), 451-468. google scholar
  • Clemen, R. T. (1989). Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5(4), 559-583. google scholar
  • Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems, 2(4), 303-314. google scholar
  • De Gooijer, J. G., & Kumar, K. (1992). Some recent developments in non-linear time series modelling, testing, and forecasting. International Journal of Forecasting, 8(2), 135-156. google scholar
  • De Groot, C., & Würtz, D. (1991). Analysis of univariate time series with connectionist nets: A case study of two classical examples. Neurocomputing, 3(4), 177-192. google scholar
  • De Menezes, L. M., Bunn, D. W., & Taylor, J. W. (2000). Review of guidelines for the use of combined forecasts. European Journal of Operational Research, 120(1), 190-204. google scholar
  • Enders, W. (2008). Applied econometric time series. John Wiley & Sons. google scholar
  • Floreano, D., Dürr, P., & Mattiussi, C. (2008). Neuroevolution: from architectures to learning. Evolutionary Intelligence, 1(1), 47-62. google scholar
  • Ghiassi, M., & Saidane, H. (2005). A dynamic architecture for artificial neural networks. Neurocomputing, 63, 397-413. google scholar
  • Ginzburg, I., & Horn, D. (1994). Combined neural networks for time series analysis. Advances in Neural Information Processing Systems, 224-224. google scholar
  • Granger, C. W. (1989). Invited review combining forecasts—twenty years later. Journal of Forecasting, 8(3), 167-173. google scholar
  • Gruau, F., Whitley, D., & Pyeatt, L. (1996, July). A comparison between cellular encoding and direct encoding for genetic neural networks. In Proceedings of the 1st annual conference on genetic programming (pp. 81-89). MIT Press. google scholar
  • Hagan, M. T., Demuth, H. B., Beale, M. H., & De Jesús, O. (1996). Neural network design (Vol. 20). Boston: PWS publishing company. Haykin, S. S. (2001). Neural networks: a comprehensive foundation. Tsinghua University Press. google scholar
  • Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359-366. google scholar
  • Hyndman, R. J. (2012). Time series data library. www.robjhyndman.com/TSDL/. Accessed on 12 June 2012. google scholar
  • Islam, M. M., Yao, X., & Murase, K. (2003). A constructive algorithm for training cooperative neural network ensembles. IEEE Transactions on neural networks, 14(4), 820-834. google scholar
  • Kaastra, I., & Boyd, M. (1996). Designing a neural network for forecasting financial and economic time series. Neurocomputing, 10(3), 215-236. google scholar
  • Khashei, M., & Bijari, M. (2010). An artificial neural network (p, d, q) model for timeseries forecasting. Expert Systems with applications, 37(1), 479-489. google scholar
  • Keppel, G. (1973). Design and Analysis: A Researcher’s Handbook. Prentice-Hall, New Jersey. google scholar
  • Kodogiannis, V., & Lolis, A. (2002). Forecasting financial time series using neural network and fuzzy system-based techniques. Neural Computing & Applications, 11(2), 90-102. google scholar
  • Lachtermacher, G., & Fuller, J. D. (1995). Back propagation in time-series forecasting. Journal of Forecasting, 14(4), 381-393. google scholar
  • Leung, F. H. F., Lam, H. K., Ling, S. H., & Tam, P. K. S. (2003). Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Transactions on Neural Networks, 14(1), 79-88. google scholar
  • Luxhøj, J. T., Riis, J. O., & Stensballe, B. (1996). A hybrid econometric—neural network modeling approach for sales forecasting. International Journal of Production Economics, 43(2), 175-192. google scholar
  • Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., & Winkler, R. (1982). The accuracy of extrapolation (time series) methods: Results of a forecasting competition. Journal of Forecasting, 1(2), 111-153. google scholar
  • Maniezzo, V. (1994). Genetic evolution of the topology and weight distribution of neural networks. IEEE Transactions on Neural Networks, 5(1), 39-53. google scholar
  • Newbold, P., & Granger, C. W. (1974). Experience with forecasting univariate time series and the combination of forecasts. Journal of the Royal Statistical Society. Series A (General), 131-165. google scholar
  • Newbold, P., Carlson, W., & Thorne, B. (2009). Statistics for business and economics. Prentice Hall. google scholar
  • Nikolopoulos, K., Goodwin, P., Patelis, A., & Assimakopoulos, V. (2007). Forecasting with cue information: A comparison of multiple regression with alternative forecasting approaches. European Journal Of Operational Research, 180(1), 354-368. google scholar
  • Palm, F. C., & Zellner, A. (1992). To combine or not to combine? Issues of combining forecasts. Journal of Forecasting, 11(8), 687-701. google scholar
  • Palmes, P. P., Hayasaka, T., & Usui, S. (2005). Mutation-based genetic neural network. IEEE Transactions on Neural Networks, 16(3), 587-600. google scholar
  • García-Pedrajas, N., Hervás-Martínez, C., & Muñoz-Pérez, J. (2003). COVNET: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Transactions on Neural Networks, 14(3), 575-596. google scholar
  • Perrone, M.P. & Cooper, L.N. (1993). When networks disagree: ensemble method for hybrid neural networks. in: R.J. Mammone (Ed.), Neural Networks for Speechand Image Processing, Chapman & Hall, London, pp.126-142. google scholar
  • Priestly, M.B. (1988). Non-linear and Non-Stationary Time Series Analysis. Academic Press. google scholar
  • Reid, D. J. (1968). Combining three estimates of gross domestic product. Economica, 35(140), 431-444. google scholar
  • Reisinger, J., Bahceci, E., Karpov, I., & Miikkulainen, R. (2007, April). Coevolving strategies for general game playing. In 2007 IEEE Symposium on Computational Intelligence and Games (pp. 320-327). google scholar
  • Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2(6), 110-114. google scholar
  • Stanley, K. O., & Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation, 10(2), 99-127. google scholar
  • Stanley, K. O., & Miikkulainen, R. (2004). Competitive coevolution through evolutionary complexification. J Artif Intell Res (JAIR), 21, 63-100. google scholar
  • Stanley, K., Kohl, N., Sherony, R., & Miikkulainen, R. (2005, June). Neuroevolution of an automobile crash warning system. In Proceedings of the 7th annual conference on Genetic and evolutionary computation (pp. 1977-1984). google scholar
  • Stock, J.H. & Watson, M. (1999). A Comparison of Linear and Nonlinear Uni-variate Models for Forecasting Macroeconomic Time Series. in Robert F. Engle and Halbert White eds., Cointegration, Causality, and Forecasting: A Festschrift in Honor of Clive W. J. Granger, Oxford University Press, Oxford, U.K., pp.1-44. google scholar
  • Stone, L., & He, D. (2007). Chaotic oscillations and cycles in multi-trophic ecological systems. Journal of Theoretical Biology, 248(2), 382-390. google scholar
  • Tseng, F. M., Yu, H. C., & Tzeng, G. H. (2002). Combining neural network model with seasonal time series ARIMA model. Technological Forecasting and Social Change, 69(1), 71-87. google scholar
  • Wedding, D. K., & Cios, K. J. (1996). Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model. Neurocomputing, 10(2), 149-168. google scholar
  • Welch, B. L. (1947). The generalization ofstudent’s’ problem when several different population variances are involved. Biometrika, 34(1/2), 28-35. google scholar
  • Wong, C. S., & Li, W. K. (2000). On a mixture autoregressive model. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 62(1), 95-115. google scholar
  • Yao, X. (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9), 1423-1447. google scholar
  • Yao, X., & Liu, Y. (1997). A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks, 8(3), 694-713. google scholar
  • Zhang, G., Patuwo, B. E., & Hu, M. Y. (1998). Forecasting with artificial neural networks:: The state of the art. InternationalJjournal of Forecasting, 14(1), 35-62. google scholar
  • Zhang, G. P. (2001). An investigation of neural networks for linear time-series forecasting. Computers & Operations Research, 28(12), 1183-1202. google scholar
  • Zhang, G. P. (2003). Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, 50, 159-175. google scholar
  • Zhang, G. P., Patuwo, B. E., & Hu, M. Y. (2001). A simulation study of artificial neural networks for nonlinear time-series forecasting. Computers & Operations Research, 28(4), 381-396. google scholar
  • Zou, H. F., Xia, G. P., Yang, F. T., & Wang, H. Y. (2007). An investigation and comparison of artificial neural network and time series models for Chinese food grain price forecasting. Neurocomputing, 70(16), 2913-2923. google scholar

Citations

Copy and paste a formatted citation or use one of the options to export in your chosen format


EXPORT



APA

Aras, S., Nguyen, A., White, A., & He, S. (2017). Comparing and Combining MLP and NEAT for Time Series Forecasting. Istanbul Business Research, 46(2), 147-160. https://doi.org/10.5152/iujsb.2017.001


AMA

Aras S, Nguyen A, White A, He S. Comparing and Combining MLP and NEAT for Time Series Forecasting. Istanbul Business Research. 2017;46(2):147-160. https://doi.org/10.5152/iujsb.2017.001


ABNT

Aras, S.; Nguyen, A.; White, A.; He, S. Comparing and Combining MLP and NEAT for Time Series Forecasting. Istanbul Business Research, [Publisher Location], v. 46, n. 2, p. 147-160, 2017.


Chicago: Author-Date Style

Aras, Serkan, and Anh Nguyen and Allan White and Shan He. 2017. “Comparing and Combining MLP and NEAT for Time Series Forecasting.” Istanbul Business Research 46, no. 2: 147-160. https://doi.org/10.5152/iujsb.2017.001


Chicago: Humanities Style

Aras, Serkan, and Anh Nguyen and Allan White and Shan He. “Comparing and Combining MLP and NEAT for Time Series Forecasting.” Istanbul Business Research 46, no. 2 (Mar. 2025): 147-160. https://doi.org/10.5152/iujsb.2017.001


Harvard: Australian Style

Aras, S & Nguyen, A & White, A & He, S 2017, 'Comparing and Combining MLP and NEAT for Time Series Forecasting', Istanbul Business Research, vol. 46, no. 2, pp. 147-160, viewed 10 Mar. 2025, https://doi.org/10.5152/iujsb.2017.001


Harvard: Author-Date Style

Aras, S. and Nguyen, A. and White, A. and He, S. (2017) ‘Comparing and Combining MLP and NEAT for Time Series Forecasting’, Istanbul Business Research, 46(2), pp. 147-160. https://doi.org/10.5152/iujsb.2017.001 (10 Mar. 2025).


MLA

Aras, Serkan, and Anh Nguyen and Allan White and Shan He. “Comparing and Combining MLP and NEAT for Time Series Forecasting.” Istanbul Business Research, vol. 46, no. 2, 2017, pp. 147-160. [Database Container], https://doi.org/10.5152/iujsb.2017.001


Vancouver

Aras S, Nguyen A, White A, He S. Comparing and Combining MLP and NEAT for Time Series Forecasting. Istanbul Business Research [Internet]. 10 Mar. 2025 [cited 10 Mar. 2025];46(2):147-160. Available from: https://doi.org/10.5152/iujsb.2017.001 doi: 10.5152/iujsb.2017.001


ISNAD

Aras, Serkan - Nguyen, Anh - White, Allan - He, Shan. “Comparing and Combining MLP and NEAT for Time Series Forecasting”. Istanbul Business Research 46/2 (Mar. 2025): 147-160. https://doi.org/10.5152/iujsb.2017.001



TIMELINE


Submitted17.08.2016
Last Revision17.09.2017
Accepted17.09.2017

LICENCE


Attribution-NonCommercial (CC BY-NC)

This license lets others remix, tweak, and build upon your work non-commercially, and although their new works must also acknowledge you and be non-commercial, they don’t have to license their derivative works on the same terms.


SHARE




Istanbul University Press aims to contribute to the dissemination of ever growing scientific knowledge through publication of high quality scientific journals and books in accordance with the international publishing standards and ethics. Istanbul University Press follows an open access, non-commercial, scholarly publishing.