Time series prediction with hierarchical recurrent model

Abstract

In this paper, we investigate the capability of modeling distant temporal interaction of Long Short-Term Memory (LSTM) and introduce a novel Long Short-Term Memory on time series problems. To increase the capability of modeling distant temporal interactions, we propose a hierarchical architecture (HLSTM) using several LSTM models and a linear layer. This novel framework is then applied to electric power consumption, real-life crime and financial data. We demonstrate in our simulations that this structure significantly improves the modeling of deep temporal connections compared to the classical architecture of LSTM and various studies in the literature. Furthermore, we analyze the sensitivity of the new architecture with respect to the hidden size of LSTM.