Next Article in Journal
Taxonomic Diversity of Fungi and Bacteria in Azoé-NP® Vertical Flow Constructed Wetlands
Previous Article in Journal
Characterizing Compost Rate Effects on Stormwater Runoff and Vegetation Establishment
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A New Rainfall-Runoff Model Using Improved LSTM with Attentive Long and Short Lag-Time

1
Key Laboratory of Geographic Information Science (Ministry of Education of China), East China Normal University, Shanghai 200241, China
2
School of Geographical Sciences, East China Normal University, Shanghai 200241, China
3
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai 200241, China
4
The Affiliated High School of Hangzhou Normal University, Hangzhou 310030, China
5
Key Laboratory of Marine Environment Monitoring and Information Processin, The School of Electronics and Information Engineering, Harbin Institute of Technology, Ministry of Industry and Information Technology, Harbin 150001, China
6
Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China
7
Henan Key Laboratory of Smart Lighting, Huanghuai University, Zhumadian 463000, China
8
School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China
*
Author to whom correspondence should be addressed.
Water 2022, 14(5), 697; https://doi.org/10.3390/w14050697
Submission received: 24 December 2021 / Revised: 7 February 2022 / Accepted: 9 February 2022 / Published: 23 February 2022

Abstract

:
It is important to improve the forecasting performance of rainfall-runoff models due to the high complexity of basin response and frequent data limitations. Recently, many studies have been carried out based on deep learning and have achieved significant performance improvements. However, their intrinsic characteristics remain unclear and have not been explored. In this paper, we pioneered the exploitation of short lag-times in rainfall-runoff modeling and measured its influence on model performance. The proposed model, long short-term memory with attentive long and short lag-time (LSTM-ALSL), simultaneously and explicitly uses new data structures, i.e., long and short lag-times, to enhance rainfall-runoff forecasting accuracy by jointly extracting better features. In addition, self-attention is employed to model the temporal dependencies within long and short lag-times to further enhance the model performance. The results indicate that LSTM-ALSL yielded superior performance at four mesoscale stations (1846~9208 km2) with humid climates (aridity index 0.77~1.16) in the U.S.A., for both peak flow and base flow, with respect to state-of-the-art counterparts.

1. Introduction

Rainfall-runoff modeling is of great importance for water resource management practices, such as flood protection, reservoir operation, inland shipping, irrigation, and drought mitigation [1,2,3,4,5,6]. Runoff forecasting models can be divided into three categories: conceptual models, physically-based models, and empirical models [7]. Conceptual and physically-based models simply represent real rainfall runoff and snowmelt runoff processes [8,9,10]. They have transparent model structures and exhibit quite good forecasting performance. However, the application of these models is difficult in many basins due to the multiple and complex physical parameters that require data that are not normally available. To solve these problems, empirical models have been widely developed for runoff forecasting. Empirical models use historical data and do not require the details of physical hydrology processes [11,12]. They consider the relationships between inputs and outputs from a statistical perspective.
At the initial development stages of empirical models, linear data-driven methods, such as the autoregressive moving average (ARMA) and autoregressive integrated moving average (ARIMA), were applied to runoff forecasting [13,14]. These models perform well when the input runoff time series are near linear, but they cannot explore the nonlinear properties hidden in the time series [15]. In recent years, many nonlinear machine learning methods have been used in runoff forecasting such as artificial neural networks (ANNs) [16,17,18] (Dawson and Wilby, 2001; Nourani, 2017; Tokar and Johnson, 1999), genetic programming (GP) [19,20,21] (Chadalawada et al., 2020; Kashid et al., 2010; Khu et al., 2001), support vector machines (SVMs) [22], support vector regression (SVR) [23], self-organizing maps (SOMs) [24], adaptive neuro fuzzy inference systems (ANFISs) [25], random forests (RFs) [26,27], generalized regression neural networks (GRNNs), and extreme gradient boosting (XGBoost) [28]. These nonlinear models capture more time series information and improve forecasting accuracy. However, due to computational limitations, traditional machine learning models have limited numbers of hidden layers [12].
Although single machine learning models have achieved good forecasting accuracy, these models can be further improved. To enhance the accuracy of runoff forecasting, many studies have developed hybrid models. Related methods such as singular spectrum analysis (SSA), ensemble empirical mode decomposition (EEMD) and wavelet transforms (WTs) are coupled with machine learning methods. Hybrid models have better accuracy in terms of runoff forecasting than single machine learning methods [29,30,31,32,33].
With the development of computational hardware resources, deep learning methods have been applied in runoff forecasting [34] and have made progress compared with traditional machine learning methods [35]. Li et al. applied convolutional deep belief networks (CDBNs) for rainfall-runoff simulation, and the CDBNs outperformed the traditional Xinanjiang model [36]. Long short-term memory (LSTM), a special type of recurrent neural network (RNN), was developed to deal with time series problems [37]. Tian et al. compared four RNNs in terms of runoff forecasting and found that nonlinear autoregressive exogenous inputs neural network (NARX) and LSTM outperformed the Elman recurrent neural network (ERNN) and echo state network (ESN) [38]. The LSTM model achieves better performance than traditional models [39]. An LSTM-based sequence-to-sequence model can forecast hourly runoff, and the LSTM-seq2seq model had better forecasting accuracy than the other six models [40]. A multi-timescale LSTM model was developed and compared with the US National Water Model [41]. The multi-timescale LSTM model exhibited better performance.
Despite the great improvements yielded by deep learning models in runoff forecasting, the temporal dependencies among the elements in a time series are still not well utilized. For example, short lag-times are beneficial to improving performance of runoff forecasting, because short lag-times are more correlated than longer lag-times with prediction results [42]. Unfortunately, no short lag-times have been involved in rainfall-runoff forecasting studies, to the best of our knowledge. In another words, we used historical runoff as input to forecast runoff in our previous study [42]. However, in this study, we used observed rainfall and historical runoff as an input to forecast runoff.
The dependencies between different lengths of input can also be calculated by self-attention to learn more internal information on the series [43], and self-attention has been widely used in sequence modeling, such as in machine translation, question answering, abstractive summarization, and image captioning [44,45,46]. The self-attention mechanism can capture temporal dependencies between its inputs and outputs, though it has not been used in rainfall-runoff forecasting until now.
In this study, a model named the LSTM-based rainfall-runoff model with attentive long and short lag-time (LSTM-ALSL) is proposed. The main contributions of LSTM-ALSL are listed as follows.
(1)
Short lag-times of rainfall-runoff input are explicitly considered in a rainfall-runoff model along with traditional long lag-time at the first time. The short lag-times are only involved in runoff modeling [42], and not employed in rainfall–runoff models;
(2)
To further improve rainfall-runoff forecasting performance, a self-attention mechanism is employed to simultaneously model the temporal dependencies within both long and short lag-times, by jointly extracting better features. The previous work only uses self-attention mechanism in short lag-times [42];
(3)
The LSTM-ALSL model is modified from SA-LSTM [42] to obtain better results. The LSTM-ALSL is compared with an SVR, convolutional neural network (CNN), random forest (RF), and LSTM models by forecasting runoff 1~7 days ahead. The results of the experiments, including both the base and peak flow results, over four stations in the U.S.A. demonstrate the effectiveness of LSTM-ALSL.

2. Methodology

LSTM-ALSL uses LSTM and self-attention modules, as used by [42]. When a (a hyperparameter) is less than b, the lag numbers of long lag-time and short lag-time are assumed to be b and a, respectively. The structure of LSTM-ALSL is shown in Figure 1. At the upper branch, the short previous time series is applied to LSTM first. Then, the self-attention module is used. At the lower branch, the long previous time series is applied to LSTM first, and then the self-attention module is used. The output matrix multiplies the result of the upper branch. The output is concatenated with the long previous time series by applying LSTM and linear transformation to create the final output.

3. Case Study

3.1. Study Area and Data

In this study, daily rainfall and runoff data were obtained from the Model Parameter Estimation Experiment (MOPEX) [47] (https://www.nws.noaa.gov/ohd/mopex/mo_datasets.html, assessed on 20 December 2021). The MOPEX datasets include the daily hydrometeorological data of 431 basins and the accompanying basin characteristic data. The data at stations 01127000, 05430500, 05518000 and 06192500 were used to verify the constructed models. The daily hydrometeorological data periods range from 1 January 1948 to 31 September 2002 (19,997 days). The statistical information regarding the stations is shown in Table 1. The average runoff and rainfall at station no. 01127000 are largest. According to the ratio between the standard deviation of the runoff and the average runoff, station no. 06192500 has the most variable runoff. The data at each station were divided into three parts: 60% for training, 20% for validation, and 20% for testing.

3.2. Forecasting Factor Selection

The selection of forecasting factors is essential for rainfall-runoff forecasting models. The correlations between the forecasting factors and runoff are computed by the partial autocorrelation function (PACF), autocorrelation function (ACF), and temporal lag cross correlation (TLCC) in this paper. The lag numbers of runoff inputs at four stations selected by the PACF and ACF are shown in Figure 2. The lag numbers of rainfall input at the four stations selected according to the TLCC are shown in Figure 3. As Figure 2 and Figure 3 show, the lag numbers of rainfall and runoff inputs are different at different stations. When the lag number is 6, the runoff data have high correlations at the four stations, as shown in Figure 2. Therefore, the previous six days are selected as the input variables of runoff at all stations. The lag number of rainfall data at stations 01127000, 05430500, 05518000 and 06192500 are 2, 9, 4, and 2, respectively. Considering prior research and the generalization abilities of models, the lag number of rainfalls is consistent with that of runoff. Therefore, the runoff and rainfall data for the previous six days are selected as the input variables at all stations in this study.

3.3. Optimization of Model Selection

During the training process, the numbers of short lag-times, hidden layers, and hidden neurons in the LSTM-ALSL model are selected. First, the long lag-time of the models was set to 6 according to the calculations in the previous section. Then, the numbers of short lag-times were set from 2 to 5. The sets of hidden layers and hidden neurons are shown in Table 2. The notation () represents the array of hidden layers and hidden neurons. For example, (3 6 2) indicates that the number of hidden layers is 3 and the numbers of neurons in the three layers are 3, 6, and 2. The forecasting performance of different structures at station no. 01127000 are compared. The results of the seven-day-ahead forecasts at station no. 01127000 when the numbers of short lag-time are 2 and 3 are shown as examples in Table 3. The R values of all structures are almost the same and are approximately 0.890 at station 01127000. At station 01127000, structure (4 6 5) with a 3-day short lag-time is the best among all structures. The NSE, NSE_ln, and MAE are optimal. The RMSE of this structure is the second lowest among all structures. Therefore, the model structure was set with 3 short lag-times, 3 hidden layers, and 4, 6, and 5 hidden neurons at each hidden layer.

4. Results and Discussion

4.1. Total Forecasting Accuracy Comparisons

4.1.1. Performance Metric Comparisons

The results of the performance metrics at the four stations are shown in Figure 4, Figure 5, Figure 6 and Figure 7. The SVR model is obviously the worst among all models. The R value of the SVR model is above 0.8 during the first few days of the lead time; then, the R value decreases rapidly. On the seventh day, the R value is higher than 0.6 at station 06192500 and less than 0.6 at the other three stations. The NSE value of the SVR model is higher than 0.8 on the first day at the three stations and decreases rapidly over the next few days. The NSE value is negative after the fourth day for all four stations. The RMSE and MAE of the SVR model are much higher than those of the other models.
On the seventh day, the R values of the LSTM-ALSL, RF, CNN, and LSTM models at station 01127000 are greater than 0.8, and the R values at the other three stations are greater than 0.9. The R values of the LSTM-ALSL model are the highest among all models. The NSE values of the LSTM-ALSL, RF, CNN, and LSTM models at stations 05430500, 05518000, and 06192500 are still greater than 0.8, and the NSE value of the LSTM-ALSL model at station 05430500 is greater than 0.9. The NSE values of the LSTM-ALSL model are 9.0%, 6.0%, 4.4%, and 3.4% higher than those of the LSTM model on the seventh day at the four stations, respectively. The RMSE and MAE of the LSTM-ALSL model are the lowest at all stations. On the seventh day, the RMSE of the LSTM-ALSL model is decreased by at least 8.1%, and the MAE is decreased by at least 10.6% relative to that of the LSTM model. This indicates that the short lag-time effectively improves the accuracy of runoff forecasting.

4.1.2. Comparisons of Forecasting Results

Figure 8, Figure 9 and Figure 10 show the time series of the forecasted runoff and observed runoff at the four stations on the first, fourth, and seventh days. In general, the LSTM-ALSL model accurately fits the observed runoff on the first, fourth, and seventh days, and the forecasting accuracy of this model is the best among all models. This indicates that a short lag-time improves the accuracy of runoff forecasting.
Figure 8 shows that the forecasted runoffs of all models fit well with the observed runoffs most of the time, except for that of the SVR model. As shown in Figure 8a, the CNN model is slightly worse than the other models on the first day at station 01127000. Figure 8b illustrates that the RF model is slightly worse than the other models on the first day at station 05430500. The accuracies of the SVR model at the four stations are low, especially at station 01127000.
Clearly, the errors of all models on the fourth day are larger than those on the first day upon comparing Figure 6 and Figure 7. The results of the SVR model at all four stations have large errors. Figure 9b shows that the accuracy of the RF model at station 05430500 is slightly lower than that of the other models. As shown in Figure 9c, the errors of the CNN and LSTM models at station 05518000 are slightly larger than those of the RF and LSTM-ALSL models. The LSTM model has negative forecasting values at station 06192500, and these negative values are set to zero in Figure 9d.
Figure 10 shows the forecasted runoff of all models and the observed runoff at the four stations on the seventh day. There is a lag between forecasted runoff and observed runoff. The LSTM-ALSL model has the highest accuracy, and the SVR model has the lowest accuracy. As shown in Figure 10a, the forecast errors of the CNN and RF models at station 01127000 are larger than those of the LSTM-ALSL and LSTM models. As shown in Figure 10c, the accuracy of the CNN and LSTM models at station 05518000 is slightly poor. The LSTM model has negative values at 06192500 station, and these negative values are set to zero in Figure 10d.

4.1.3. Scatter Plot Comparisons

The forecasted runoff of all models and the observed daily runoff scatter plots at the four stations on the seventh day are shown in Figure 11. The black line in Figure 11 is the line with the most ideal fit (y = x). The fit line of the LSTM-ALSL model is closest to the best-fit line at each of the four stations. The forecasting error of the SVR model is the largest. The SVR model overestimates most of the runoff values at the four stations. At stations 01127000, 054300500, and 05518000, the accuracy of the RF model is second highest after that of the LSTM-ALSL model. At station 06192500, the fit lines of the CNN and LSTM models are closest to the best-fit line. The LSTM-ALSL, RF, CNN, and LSTM models have higher accuracy and fit lines that are closer to the best-fit lines at stations with smaller average observed runoff values (such as stations 05430500 and 05518000), while these four models have lower accuracy at stations with larger average observed runoff values (such as stations 01127000 and 06192500). In conclusion, the overall forecasting accuracy of the LSTM-ALSL model at the four stations is the highest of all the models, which indicates that a short lag-time plays an important role in runoff forecasting.

4.2. Comparisons of Peak Flow Forecasting Accuracy

The 15 peak flow forecasting results of all models on the seventh day at the four stations are shown in Table 4, Table 5, Table 6 and Table 7. The mean ARE of the SVR model varies greatly at different stations. The mean AREs at stations 01127000 and 06192500 are the lowest. The mean ARE at station 05430500 is the second lowest and the mean ARE at station 06192500 is the highest. The peak flow forecasting accuracy of the SVR model at stations with large average runoff values is higher than that at stations with small average runoff values. The forecasting errors of the RF, CNN, LSTM. and LSTM-ALSL models are relatively large at stations with large average runoff values. The mean AREs of these models at stations 01127000 and 06192500 are significantly higher than those at stations 054300500 and 05518000. The mean AREs of the LSTM-ALSL model at stations 054300500 and 06192500 are the lowest, and the mean AREs at stations 01127000 and 06192500 are the second lowest after those of the SVR model. Compared with those of the LSTM model, the mean AREs of the LSTM-ALSL model decrease by 26.0%, 49.5%, 35.7%, and 16.9% at the four stations. The minimum AREs of the LSTM-ALSL model at stations 05430500 and 05518000 are less than 0.01. This indicates that the LSTM-ALSL model can forecast some peak flow values accurately. The minimum AREs of the SVR model at stations 01127000 and 06192500 are the lowest among all models. The minimum AREs of the RF model at the four stations are less than 0.1. The minimum AREs of the LSTM model fluctuate greatly and are larger than 0.2 at station 01127000 and less than 0.02 at the other three stations. The minimum ARE of the CNN model is larger than 0.2 at station 01127000 and less than 0.1 at the other three stations. The maximum ARE of each model varies greatly at different stations. The maximum AREs of the RF, CNN, LSTM, and LSTM-ALSL models at stations with large average runoff values are higher than those at stations with small average runoff values. The maximum AREs of the LSTM-ALSL and SVR models are less than 0.3 at station 05518000 and larger than 0.3 at other stations. The maximum AREs of the RF, CNN, and LSTM models are larger than 0.3 at all stations.
Figure 8, Figure 9 and Figure 10 show that the peak flow forecasting errors of the models increase gradually with increasing lead time. On the first day, the LSTM-ALSL, RF, CNN, and LSTM models forecast the peak flow well. On the fourth day and the seventh day, the forecasting errors of all models at some peak flow values are large. The LSTM-ALSL model is better than the RF, CNN, and LSTM models in terms of completely fitting the peak flow. The accuracy of the LSTM-ALSL model at stations 054300500 and 05518000 is higher than that at stations 01127000 and 06192500. Because most of the runoff values are overestimated, the SVR model forecasts accurately when the peak flow is large, as shown in Figure 11a. In conclusion, the accuracy of the LSTM-ALSL and SVR models is higher than that of the other models in terms of peak flow forecasting, and the accuracy of the LSTM-ALSL model is more stable. This indicates that a short lag-time effectively improves the accuracy of peak flow forecasting.

4.3. Comparisons of Base Flow Forecasting Accuracies

As shown in Figure 10, the SVR model clearly overestimates the base flow at station 01127000 and slightly overestimates the base flows at the other three stations. The accuracies of the LSTM-ALSL, RF, CNN, and LSTM models are high on the first day at all stations. As shown in Figure 11, all models forecast the base flow well on the fourth day except the SVR model. The SVR model overestimates the base flows at all four stations. Figure 11d shows that the LSTM model has negative base flow forecasting values at station 06192500. These negative values are set to zero. Figure 10 shows that the SVR model significantly overestimates most of the base flow values of the four stations on the seventh day. The CNN model underestimates part of the base flow at station 01127000 and overestimates part of the base flow at station 05518000. The LSTM model underestimates part of the base flow at station 05518000. The RF and LSTM-ALSL models can accurately fit most of the base flow values.
The NSE_ln of the LSTM-ALSL model is the highest among all models at the four stations. The NSE_ln of the LSTM-ALSL model is higher than 0.9 on the fifth day at the four stations and is still higher than 0.9 on the seventh day at stations 05518000 and 06192500. On the seventh day, the NSE_ln values of the RF and LSTM models are lower than 0.8 at station 05430500 and higher than 0.8 at the other three stations. The NSE_ln of the CNN model is 0.902 at station 06192500, which is the second highest value after that of the LSTM-ALSL model. Compared with that of the LSTM model, the NSE_ln of the LSTM-ALSL model increases by 9.0% at the four stations on average. This indicates that a short lag-time improves the base flow forecasting accuracy.

4.4. Epochs vs. Loss/NSE

Figure 12 shows the loss and the NSE of the LSTM-ALSL model at station 01127000 on the validation set with the gradual increase of epochs. The LSTM-ALSL model ran 10,000 epochs on the validation set and retained the best-performing model. In the first 45 epochs, the loss value decreased rapidly from 1.36 × 10−3 to 9.68 × 10−5. After the first 45 epochs, the loss value decreased slowly as the number of epochs increased. The loss value finally reached 5.05 × 10−5. In the first 36 epochs, the NSE value increased rapidly from 0.135 to 0.930. After the first 36 epochs, the NSE value increased slowly as the number of epochs increased. The NSE value finally reached 0.972.

5. Conclusions

In this paper, we combined long lag-time with short lag-time and proposed the LSTM-ALSL model for runoff forecasting. The LSTM-ALSL model applies a self-attention mechanism to both the long lag-time input and short lag-time inputs. The forecasting results of the LSTM-ALSL model were compared with those of the SVR, RF, CNN, and LSTM models for the four stations in the MOPEX dataset. The LSTM-ALSL model was superior to other state-of-the-art models and improved the accuracy of both peak flow and base flow forecasting. In future studies, we will increase the numbers of meteorological element inputs and basin characteristic attributes to further improve the data-driven rainfall-runoff models.

Author Contributions

Conceptualization: X.C. and H.G.; writing—original draft preparation: J.H.; Software: S.W. and X.C.; validation: J.H. and S.W.; supervision: X.C. and H.G.; writing—review and editing: X.C., S.W., H.G., G.Z., M.L., Y.Y., L.Z., Q.L. and H.Q. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Key Research and Development Project of China (2017YFE0100700), in part by the National Natural Science Foundation of China (Grant No. 41871340; 42122002; 42071081), in part by the Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, No. 2020KEY003, and in part by the Key Research Project of Shanghai Agricultural Science and Technology (SASTI-2018-2-1), the Fundamental Research Funds for the Central Universities.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

We thank anonymous reviewers for their valuable comments to improve the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Blöschl, G.; Hall, J.; Viglione, A.; Perdigão, R.A.P.; Parajka, J.; Merz, B.; Lun, D.; Arheimer, B.; Aronica, G.T.; Bilibashi, A.; et al. Changing climate both increases and decreases European river floods. Nature 2019, 573, 108–111. [Google Scholar] [CrossRef]
  2. Liu, Z.; Zhou, P.; Chen, X.; Guan, Y. A multivariate conditional model for streamflow prediction and spatial precipitation refinement. J. Geophys. Res. Atmos. 2015, 120, 10–116. [Google Scholar] [CrossRef] [Green Version]
  3. Nazeer, A.; Maskey, S.; Skaugen, T.; McClain, M.E. Simulating the hydrological regime of the snow fed and glaciarised Gilgit Basin in the Upper Indus using global precipitation products and a data parsimonious precipitation-runoff model. Sci. Total Environ. 2021, 802, 149872. [Google Scholar] [CrossRef]
  4. Palmer, M.; Ruhi, A. Linkages between flow regime, biota, and ecosystem processes: Implications for river restoration. Science 2019, 365, eaaw2087. [Google Scholar] [CrossRef] [Green Version]
  5. Wu, J.; Liu, Z.; Yao, H.; Chen, X.; Chen, X.; Zheng, Y.; He, Y. Impacts of reservoir operations on multi-scale correlations between hydrological drought and meteorological drought. J. Hydrol. 2018, 563, 726–736. [Google Scholar] [CrossRef]
  6. Yu, X.; Zhang, X.; Qin, H. A data-driven model based on Fourier transform and support vector regression for monthly reservoir inflow forecasting. J. Hydro Environ. Res. 2018, 18, 12–24. [Google Scholar] [CrossRef]
  7. Duan, Q.; Pappenberger, F.; Wood, A.; Cloke, H.L.; Schaake, J. Handbook of Hydrometeorological Ensemble Forecasting; Springer: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
  8. Birkel, C.; Soulsby, C. Advancing tracer-aided rainfall-runoff modelling: A review of progress, problems and unrealised potential. Hydrol. Process. 2015, 29, 5227–5240. [Google Scholar] [CrossRef]
  9. Seibert, J.; Vis, M.J.P.; Kohn, I.; Weiler, M.; Stahl, K. Technical note: Representing glacier geometry changes in a semi-distributed hydrological model. Hydrol. Earth Syst. Sci. 2018, 22, 2211–2224. [Google Scholar] [CrossRef] [Green Version]
  10. Gao, H.; Birkel, C.; Hrachowitz, M.; Tetzlaff, D.; Soulsby, C.; Savenije, H.H.G. A Simple Topography-Driven and Calibration-Free Runoff Generation Module. Hydrol. Earth Syst. Sci. 2019, 23, 787–809. [Google Scholar] [CrossRef] [Green Version]
  11. Liu, S.; Yao, X.; Guo, W.; Xu, J.; Shangguan, D.; Wei, J.; Bao, W.; Wu, L. The Contemporary Glaciers in China Based on the Second Chinese Glacier Inventory. Acta Geogr. Sin. 2015, 70, 3–16. [Google Scholar]
  12. Shen, C. A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists. Water Resour. Res. 2018, 54, 8558–8593. [Google Scholar] [CrossRef]
  13. Brath, A.; Montanari, A.; Toth, E. Neural networks and non-parametric methods for improving real-time flood forecasting through conceptual hydrological models. Hydrol. Earth Syst. Sci. 2002, 6, 627–639. [Google Scholar] [CrossRef] [Green Version]
  14. Toth, E.; Brath, A.; Montanari, A. Comparison of short-term rainfall prediction models for real-time flood forecasting. J. Hydrol. 2000, 239, 132–147. [Google Scholar] [CrossRef]
  15. Luo, X.; Yuan, X.; Zhu, S.; Xu, Z.; Meng, L.; Peng, J. A hybrid support vector regression framework for streamflow forecast. J. Hydrol. 2019, 568, 184–193. [Google Scholar] [CrossRef]
  16. Dawson, C.W.; Wilby, R.L. Hydrological modelling using artificial neural networks. Prog. Phys. Geogr. 2001, 25, 80–108. [Google Scholar] [CrossRef]
  17. Nourani, V. An Emotional ANN (EANN) approach to modeling rainfall-runoff process. J. Hydrol. 2017, 544, 267–277. [Google Scholar] [CrossRef]
  18. Tokar, A.S.; Johnson, P.A. Rainfall-runoff modeling using artificial neural networks. J. Hydrol. Eng. 1999, 4, 232–239. [Google Scholar] [CrossRef]
  19. Chadalawada, J.; Herath, H.M.V.V.; Babovic, V. Hydrologically Informed Machine Learning for Rainfall-Runoff Modeling: A Genetic Programming-Based Toolkit for Automatic Model Induction. Water Resour. Res. 2020, 56, e2019WR026933. [Google Scholar] [CrossRef]
  20. Kashid, S.; Ghosh, S.; Maity, R. Streamflow prediction using multi-site rainfall obtained from hydroclimatic teleconnection. J. Hydrol. 2010, 395, 23–38. [Google Scholar] [CrossRef]
  21. Khu, S.T.; Liong, S.-Y.; Babovic, V.; Madsen, H.; Muttil, N. Genetic programming and its application in real-time runoff forecasting. JAWRA J. Am. Water Resour. Assoc. 2001, 37, 439–451. [Google Scholar] [CrossRef]
  22. Li, X.-L.; Lü, H.; Horton, R.; An, T.; Yu, Z. Real-time flood forecast using the coupling support vector machine and data assimilation method. J. Hydroinformatics 2014, 16, 973–988. [Google Scholar] [CrossRef] [Green Version]
  23. Young, C.-C.; Liu, W.-C.; Wu, M.-C. A physically based and machine learning hybrid approach for accurate rainfall-runoff modeling during extreme typhoon events. Appl. Soft Comput. 2017, 53, 205–216. [Google Scholar] [CrossRef]
  24. Moradkhani, H.; Hsu, K.-L.; Gupta, H.; Sorooshian, S. Improved streamflow forecasting using self-organizing radial basis function artificial neural networks. J. Hydrol. 2004, 295, 246–262. [Google Scholar] [CrossRef] [Green Version]
  25. Talei, A.; Chua, L.H.C.; Wong, T.S. Evaluation of rainfall and discharge inputs used by Adaptive Network-based Fuzzy Inference Systems (ANFIS) in rainfall–runoff modeling. J. Hydrol. 2010, 391, 248–262. [Google Scholar] [CrossRef]
  26. Huang, H.; Liang, Z.; Li, B.; Wang, D.; Hu, Y.; Li, Y. Combination of Multiple Data-Driven Models for Long-Term Monthly Runoff Predictions Based on Bayesian Model Averaging. Water Resour. Manag. 2019, 33, 3321–3338. [Google Scholar] [CrossRef]
  27. Tongal, H.; Booij, M.J. Simulation and forecasting of streamflows using machine learning models coupled with base flow separation. J. Hydrol. 2018, 564, 266–282. [Google Scholar] [CrossRef]
  28. Cisty, M.; Soldanova, V. Flow Prediction Versus Flow Simulation Using Machine Learning Algorithms, International Conference on Machine Learning and Data Mining in Pattern Recognition; Springer: Berlin/Heidelberg, Germany, 2018; pp. 369–382. [Google Scholar]
  29. Alizadeh, M.J.; Kavianpour, M.R.; Kisi, O.; Nourani, V. A new approach for simulating and forecasting the rainfall-runoff process within the next two months. J. Hydrol. 2017, 548, 588–597. [Google Scholar] [CrossRef]
  30. Anctil, F.; Tape, D.G. An exploration of artificial neural network rainfall-runoff forecasting combined with wavelet decomposition. J. Environ. Eng. Sci. 2004, 3, S121–S128. [Google Scholar] [CrossRef]
  31. Nourani, V.; Parhizkar, M. Conjunction of SOM-based feature extraction method and hybrid wavelet-ANN approach for rainfall–runoff modeling. J. Hydroinformatics 2013, 15, 829–848. [Google Scholar] [CrossRef] [Green Version]
  32. Wang, Y.; Guo, S.; Xiong, L.; Liu, P.; Liu, D. Daily Runoff Forecasting Model Based on ANN and Data Preprocessing Techniques. Water 2015, 7, 4144–4160. [Google Scholar] [CrossRef] [Green Version]
  33. Wu, C.; Chau, K.W. Rainfall–runoff modeling using artificial neural network coupled with singular spectrum analysis. J. Hydrol. 2011, 399, 394–409. [Google Scholar] [CrossRef] [Green Version]
  34. Yokoo, K.; Ishida, K.; Ercan, A.; Tu, T.; Nagasato, T.; Kiyama, M.; Amagasaki, M. Capabilities of deep learning models on learning physical relationships: Case of rainfall-runoff modeling with LSTM. Sci. Total Environ. 2021, 802, 149876. [Google Scholar] [CrossRef]
  35. Klotz, D.; Kratzert, F.; Gauch, M.; Sampson, A.K.; Brandstetter, J.; Klambauer, G.; Hochreiter, S.; Nearing, G. Uncertainty Estimation with Deep Learning for Rainfall–Runoff Modelling. Hydrol. Earth Syst. Sci. Discuss. 2021, 1–32. [Google Scholar] [CrossRef]
  36. Li, X.; Du, Z.; Song, G. A Method of Rainfall Runoff Forecasting Based on Deep Convolution Neural Networks. In Proceedings of the 2018 Sixth International Conference on Advanced Cloud and Big Data (CBD), Lanzhou, China, 12–15 August 2018; pp. 304–310. [Google Scholar]
  37. Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
  38. Tian, Y.; Xu, Y.-P.; Yang, Z.; Wang, G.; Zhu, Q. Integration of a Parsimonious Hydrological Model with Recurrent Neural Networks for Improved Streamflow Forecasting. Water 2018, 10, 1655. [Google Scholar] [CrossRef] [Green Version]
  39. Hu, C.; Wu, Q.; Li, H.; Jian, S.; Li, N.; Lou, Z. Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation. Water 2018, 10, 1543. [Google Scholar] [CrossRef] [Green Version]
  40. Xiang, Z.; Yan, J.; Demir, I. A Rainfall-Runoff Model With LSTM-Based Sequence-to-Sequence Learning. Water Resour. Res. 2020, 56, e2019WR025326. [Google Scholar] [CrossRef]
  41. Gauch, M.; Kratzert, F.; Klotz, D.; Nearing, G.; Lin, J.; Hochreiter, S. Rainfall–runoff prediction at multiple timescales with a single Long Short-Term Memory network. Hydrol. Earth Syst. Sci. 2021, 25, 2045–2062. [Google Scholar] [CrossRef]
  42. Chen, X.; Huang, J.; Han, Z.; Gao, H.; Liu, M.; Li, Z.; Liu, X.; Li, Q.; Qi, H.; Huang, Y. The importance of short lag-time in the runoff forecasting model based on long short-term memory. J. Hydrol. 2020, 589, 125359. [Google Scholar] [CrossRef]
  43. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Processing Syst. 2017, 30, 5998–6008. [Google Scholar]
  44. Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv 2014, arXiv:1409.04730. [Google Scholar]
  45. Luong, M.-T.; Pham, H.; Manning, C.D. Effective approaches to attention-based neural machine translation. arXiv 2015, arXiv:1508.04025. [Google Scholar]
  46. Sukhbaatar, S.; Szlam, A.; Weston, J.; Fergus, R.J. End-to-end memory networks. Adv. Neural Inf. Processing Syst. 2015, 28, 1–9. [Google Scholar]
  47. Duan, Q.J.; Schaake, V.; Andréassian, S.; Franks, G.; Goteti, H.V.; Gupta, Y.M.; Gusev, F.; Habets, A.; Hall, L.; Hay, T.; et al. Model Parameter Estimation Experiment (MOPEX): An overview of science strategy and major results from the second and third workshops. J. Hydrol. 2006, 320, 3–17. [Google Scholar] [CrossRef] [Green Version]
Figure 1. The structure of the LSTM-ALSL model.
Figure 1. The structure of the LSTM-ALSL model.
Water 14 00697 g001
Figure 2. The PACFs and ACFs of stations 01127000, 05430500, 05518000, and 06192500.
Figure 2. The PACFs and ACFs of stations 01127000, 05430500, 05518000, and 06192500.
Water 14 00697 g002
Figure 3. The TLCCs of stations (a) 01127000, (b) 05430500, (c) 05518000 and (d) 06192500.
Figure 3. The TLCCs of stations (a) 01127000, (b) 05430500, (c) 05518000 and (d) 06192500.
Water 14 00697 g003
Figure 4. Comparisons of the daily runoff forecast performance of five models at station 01127000.
Figure 4. Comparisons of the daily runoff forecast performance of five models at station 01127000.
Water 14 00697 g004
Figure 5. Comparisons of the daily runoff forecast performance of five models at station 05430500.
Figure 5. Comparisons of the daily runoff forecast performance of five models at station 05430500.
Water 14 00697 g005
Figure 6. Comparisons of the daily runoff forecast performance of five models at station 05518000.
Figure 6. Comparisons of the daily runoff forecast performance of five models at station 05518000.
Water 14 00697 g006
Figure 7. Comparisons of the daily runoff forecast performance of five models at station 06192500.
Figure 7. Comparisons of the daily runoff forecast performance of five models at station 06192500.
Water 14 00697 g007
Figure 8. One-day-ahead forecasts and locally enlarged results of all models at stations 01127000 (a), 05430500 (b), 05518000 (c), and 06192500 (d).
Figure 8. One-day-ahead forecasts and locally enlarged results of all models at stations 01127000 (a), 05430500 (b), 05518000 (c), and 06192500 (d).
Water 14 00697 g008
Figure 9. Four-day-ahead forecasts and locally enlarged results of all models at stations 01127000 (a), 05430500 (b), 05518000 (c), and 06192500 (d).
Figure 9. Four-day-ahead forecasts and locally enlarged results of all models at stations 01127000 (a), 05430500 (b), 05518000 (c), and 06192500 (d).
Water 14 00697 g009
Figure 10. Seven-day-ahead forecasts and locally enlarged results of all models at stations 01127000 (a), 05430500 (b), 05518000 (c), and 06192500 (d).
Figure 10. Seven-day-ahead forecasts and locally enlarged results of all models at stations 01127000 (a), 05430500 (b), 05518000 (c), and 06192500 (d).
Water 14 00697 g010
Figure 11. Scatter plots and fit lines of the seven-day-ahead forecasting results of five models at stations (a) 01127000, (b) 05430500, (c) 05518000, and (d) 06192500.
Figure 11. Scatter plots and fit lines of the seven-day-ahead forecasting results of five models at stations (a) 01127000, (b) 05430500, (c) 05518000, and (d) 06192500.
Water 14 00697 g011
Figure 12. (a) Loss at each epoch at station 01127000; (b) NSE at each epoch at station 01127000.
Figure 12. (a) Loss at each epoch at station 01127000; (b) NSE at each epoch at station 01127000.
Water 14 00697 g012
Table 1. Statistical information on the four stations.
Table 1. Statistical information on the four stations.
Station IDLon.
(deg)
Lat.
(deg)
Drainage Area (km2)Ardity Index (PET/P)Average Runoff (mm/d)Standard Deviation of Runoff (mm/d)Average Rainfall (mm/d)Runoff Coefficient Station Location
01127000−71.98541.59818460.771.8011.8373.2690.551Quinebaug River at Jewett City, CT
05430500−89.07142.60986861.110.5800.4872.2140.262Rock River at Afton, WI
05518000−87.34341.18347960.990.9780.6162.5960.377Kankakee River at Shelby, IN
06192500−110.56545.59792081.161.0191.1741.8840.541Yellowstone River near Livingston, MT.
Table 2. Different LSTM-ALSL structures.
Table 2. Different LSTM-ALSL structures.
Hidden LayersHidden Neurons
1(3), (5), (7), (9), (11), (16)
2(2 3), (2 8), (8 2), (8 9), (16 5), (5 16)
3(3 6 2), (4 6 5), (5 7 3), (5 9 2), (7 9 5), (4 16 5)
4(3 5 6 2), (3 5 9 2), (3 7 9 5), (4 8 10 5)
5(3 5 9 7 2), (3 6 9 6 3), (4 7 11 5 2), (5 7 11 9 5)
Table 3. Seven-day-ahead forecasting performance of LSTM-ALSL with different structures at station 01127000.
Table 3. Seven-day-ahead forecasting performance of LSTM-ALSL with different structures at station 01127000.
Short Lag-TimeHidden Layers and Hidden NeuronRNSENSE_lnRMSEMAE
2(3)0.8720.7490.6250.8870.594
(5)0.8920.7440.7610.8970.491
(7)0.8820.7330.6480.9150.490
(9)0.8960.7530.6260.8800.635
(11)0.8870.7850.7430.8210.478
(16)0.8930.7880.8700.8160.495
(2 3)0.8900.7760.8590.8380.449
(2 8)0.8890.7890.8140.8130.463
(8 2)0.8920.7830.8240.8250.522
(8 9)0.8860.7550.8610.8780.569
(16 5)0.8800.7600.8220.8680.533
(5 16)0.8860.7820.8860.8270.487
(3 6 2)0.8920.6870.7190.9910.736
(4 6 5)0.8920.7340.8720.9130.488
(5 7 3)0.8870.7780.8660.8340.501
(5 9 2)0.8910.6690.6461.0190.799
(7 9 5)0.8890.7340.7580.9140.651
(4 16 5)0.8790.7080.8250.9570.630
(3 5 6 2)0.8950.7880.8260.8160.516
(3 5 9 2)0.8910.7810.6920.8290.459
(3 7 9 5)0.8550.7270.8760.9260.486
(4 8 10 5)0.8970.7480.8770.8900.462
(3 5 9 7 2)0.8900.7770.8860.8380.443
(3 6 9 6 3)0.8880.7780.8560.8340.480
(4 7 11 5 2)0.8860.7790.8570.8330.492
(5 7 11 9 5)0.8810.7110.8270.9520.644
3(3)0.8850.7500.6700.8860.636
(5)0.8890.7850.6510.8220.486
(7)0.8880.7800.8360.8310.526
(9)0.8910.7820.4280.8280.521
(11)0.8880.721-0.3260.9360.637
(16)0.8780.6270.8021.0820.742
(2 3)0.8960.7870.8070.8180.509
(2 8)0.8910.6410.5431.0620.667
(8 2)0.8910.7890.8690.8140.479
(8 9)0.8880.6900.7960.9860.547
(16 5)0.8830.6590.7031.0340.755
(5 16)0.9000.7450.5570.8950.686
(3 6 2)0.8890.7670.8500.8550.544
(4 6 5)0.8910.7890.8870.8140.438
(5 7 3)0.8860.7680.8780.8530.527
(5 9 2)0.8900.7510.8170.8850.595
(7 9 5)0.8850.7640.8320.8600.481
(4 16 5)0.8780.7550.0860.8770.505
(3 5 6 2)0.8970.7880.8700.8170.445
(3 5 9 2)0.8880.7050.7910.9620.548
(3 7 9 5)0.8850.7600.8800.8680.454
(4 8 10 5)0.8870.7850.8820.8220.464
(3 5 9 7 2)0.8910.7300.8830.9220.488
(3 6 9 6 3)0.8670.7300.7860.9220.591
(4 7 11 5 2)0.8770.6830.6620.9980.757
(5 7 11 9 5)0.8890.7840.7900.8230.443
Table 4. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 01127000.
Table 4. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 01127000.
Observed RunoffSVRCNNRFLSTMLSTM-ALSL
8.79110.5695.3285.4564.2286.132
11.7578.6443.9084.5324.0144.432
7.96410.6236.2625.4345.7445.954
11.4059.7785.2435.3003.9564.870
11.3378.8645.7948.7967.3639.090
14.49310.7446.7335.3616.5919.289
10.6338.4254.4834.7244.5205.633
8.76410.2755.5444.0303.6194.616
13.31510.1438.7627.8758.34812.099
12.3269.6947.8476.0465.2225.812
9.6038.8765.3106.5566.4066.841
9.19710.5036.4169.5766.3947.849
12.3399.1658.51410.7958.1918.850
10.1868.8515.18310.3357.6319.124
10.83610.5314.2483.3103.3125.288
Max ARE0.3340.6080.6940.6940.623
Min ARE0.0280.2140.0410.2510.104
Mean ARE0.1920.4430.3940.4690.347
Table 5. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 05430500.
Table 5. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 05430500.
Observed RunoffSVRCNNRFLSTMLSTM-ALSL
1.1691.7811.0621.1931.0071.110
1.1571.6541.0461.0130.9131.008
0.9691.5560.7760.5240.7620.821
3.0211.4312.3642.9072.4753.022
1.6501.4581.4001.3591.2801.415
1.0260.9960.7591.4550.7770.919
1.1571.8430.9271.0060.9111.100
2.4031.3881.9852.2442.4312.431
1.6501.7030.9470.9850.9941.128
0.9581.1780.8100.9330.8230.946
2.0291.9371.6721.8181.7621.971
1.0751.6190.9131.3140.8440.982
1.9891.4691.0471.5251.2921.323
2.6931.6461.8131.8572.2222.574
1.5311.5981.3411.7301.3951.571
Max ARE0.6060.4740.4590.3980.335
Min ARE0.0290.0910.0210.0120.0004
Mean ARE0.3170.2150.1910.1960.099
Table 6. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 05518000.
Table 6. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 05518000.
Observed RunoffSVRCNNRFLSTMLSTM-ALSL
1.9402.2971.6902.0121.6421.869
3.0212.5422.2312.8142.7932.934
2.5322.3202.1402.2831.9752.131
2.5322.4492.0422.3412.0882.259
2.9722.2952.1101.8742.0132.238
2.3912.5581.9231.8111.6311.802
2.5052.3962.3472.2862.2002.490
2.6622.3832.4452.4792.4112.657
2.9722.2121.6192.1193.0682.863
3.1682.4332.7232.7952.6852.764
2.9772.3932.1302.1052.0032.262
2.7662.4892.1342.4732.5652.724
2.8312.3862.0812.1171.9572.075
2.7932.5082.1112.1752.2972.503
2.7662.3962.1732.5012.7132.689
Max ARE0.2320.4550.3690.3270.267
Min ARE0.0330.0630.0370.0190.002
Mean ARE0.1390.2130.1610.1710.110
Table 7. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 06192500.
Table 7. Top 15 seven-day-ahead peak flow forecast results and ARE values of five models at station 06192500.
Observed RunoffSVRCNNRFLSTMLSTM-ALSL
3.3424.5911.8421.8292.3592.936
5.8895.1873.2353.2174.2884.051
4.0853.8441.6711.7022.5673.169
5.3855.6903.2233.1972.7583.549
8.9134.9554.4914.4454.5385.033
9.5505.5954.5114.4954.5874.565
2.7324.4731.4761.4862.2922.883
4.5894.2854.1644.1744.5704.200
5.2264.6711.9651.9673.0503.537
5.8365.3953.6973.6494.1543.882
4.6163.5152.2562.2673.1762.807
3.6613.7011.5081.5142.3953.266
2.0032.7361.5461.8041.2871.850
6.3934.0492.1172.1081.7811.607
4.8814.8544.4294.3504.9624.293
Max ARE0.6370.6690.6700.7210.749
Min ARE0.0050.0930.0900.0040.055
Mean ARE0.2230.4360.4300.3370.280
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chen, X.; Huang, J.; Wang, S.; Zhou, G.; Gao, H.; Liu, M.; Yuan, Y.; Zheng, L.; Li, Q.; Qi, H. A New Rainfall-Runoff Model Using Improved LSTM with Attentive Long and Short Lag-Time. Water 2022, 14, 697. https://doi.org/10.3390/w14050697

AMA Style

Chen X, Huang J, Wang S, Zhou G, Gao H, Liu M, Yuan Y, Zheng L, Li Q, Qi H. A New Rainfall-Runoff Model Using Improved LSTM with Attentive Long and Short Lag-Time. Water. 2022; 14(5):697. https://doi.org/10.3390/w14050697

Chicago/Turabian Style

Chen, Xi, Jiaxu Huang, Sheng Wang, Gongjian Zhou, Hongkai Gao, Min Liu, Ye Yuan, Laiwen Zheng, Qingli Li, and Honggang Qi. 2022. "A New Rainfall-Runoff Model Using Improved LSTM with Attentive Long and Short Lag-Time" Water 14, no. 5: 697. https://doi.org/10.3390/w14050697

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop