LSTM-Based Deformation Prediction Model of the Embankment Dam of the Danjiangkou Hydropower Station

Abstract

The Danjiangkou hydropower station is a water source project for the middle line of the South-to-North Water Transfer Project in China. The dam is composed of riverbed concrete dam and earth rock dam on both banks, with a total length of 3442 m. Once the dam is wrecked, it will yield disastrous consequences. Therefore, it is very important to evaluate the dam safety behavior in time. Based on the long-term and short-term memory (LSTM) network, the deformation prediction models of the embankment dam of the Danjiangkou hydropower station are constructed. The models contain two LSTM layers, adopt the rectified linear unit function as the activation function and determine the super parameters of the models with Bayesian optimization algorithm. According to the settlement monitoring data of LD12ZT01 measuring point (dam crest 0 + 648) on the left bank of the embankment dam of the Danjiangkou hydropower station from July 2013 to March 2022, the LSTM and bidirectional LSTM models are constructed. In total, 80% of the monitoring data are taken as the training set data and 20% of the monitoring data are taken as the test set data. The mean absolute error, root mean square error and mean square error for the test set are 0.42978, 0.56456 and 0.31873 for partial least squares regression (PLSR), 0.35264, 0.47561 and 0.22621 for LSTM and 0.34418, 0.45400 and 0.20612 for bidirectional LSTM, respectively. The results show that the bidirectional LSTM model can obtain better deformation prediction value than the LSTM model and the PLSR. Then, the bidirectional LSTM model is used to predict the settlement value of LD16YT01 measuring point (dam crest 0 + 658) on the right bank, and the mean absolute error, root mean square error and mean square error for the test set are 0.5425, 0.66971 and 0.4520, respectively. This shows the bidirectional LSTM model can effectively predict the settlement value of the embankment dam of the Danjiangkou hydropower station.

1. Introduction

Dam deformation, cracks and dam leakage seriously affect the safe operation of earth rock dams [1]. Dam deformation is characterized by uncertainty, diversity and time variability. Based on the prototype monitoring data, building a deformation prediction model is an important means to evaluate the dam operation safety.

The most commonly used monitoring model is the statistical model [2], which is simple to implement, and the accuracy can meet the needs of the project. Mata et al. [3] presented a hydrostatic-thermal-time statistical model for concrete dam deformation based on principal component analysis, in which the seasonal function is replaced with the recorded temperatures. Xi et al. [4] proposed an immune statistical model of dam deformation by coupling the statistical model with the immune algorithm. Sigtryggsdottir et al. [5] developed a hydrostatic-seasonal-time statistical model for predicting the settlement of concrete-faced rockfill dam during operation. The statistical models are also widely applied in civil and environmental problems. Tatin et al. [6] proposed a statistical model considering the water temperature profile, and a polynomial approximation of influence functions is imposed to constrain the statistical problem. Borzooei et al. [7,8] studied the effects of rainfall and daily precipitation on the influent flow rate and water quality constituents, and time series segmentation was applied through the sliding window algorithm. Borzooei et al. [9] proposed a stepwise approach for model-based energy optimization of the biological nutrient removal activated sludge system. Chatrabgoun et al. [10] investigated the risk and impacts of frost phenomenon in the vineyards by modeling the joint distribution of duration and severity factors. Noori et al. [11] predicted the total sediment load in rivers by developing a robust approach in terms of multiple linear regression and principal component analysis-based support vector regression models. However, when there is multicollinearity among factors, the prediction accuracy of the statistical model is not good [12].

With the development of computer technology, machine learning algorithms such as support vector machine [13], extreme learning machine [14], artificial neural network [15,16] are widely used in deformation prediction models. However, these machine learning algorithms have various shortcomings, such as over fitting, easy to fall into local extremum and difficult to determine model super parameters. In recent years, with the development of deep learning technology, long short-term memory (LSTM) [17] has effectively solved the problems of gradient explosion, gradient disappearance and long-term dependence of conventional recurrent neural network (RNN) models by introducing cell state and gating. The LSTM models have been widely used for various predictions, such as streamflow [18], well production [19], vehicle trajectory [20], dam deformation [21,22,23,24] and so on. Yang et al. [21] compared the LSTM model, the stepwise regression method and the partial least square regression method for deformation prediction of concrete dam. Liu et al. [22] respectively coupled the principal component analysis and moving average method with the LSTM to predict the long-term deformation of Lijiaxia arch dam. Xing et al. [23] forecasted the dam deformation with historical monitoring data using an LSTM network with dynamic update strategy. Qu et al. [24] studied single-point and multipoint deformation prediction models of concrete dam in terms of the LSTM network combined with the rough set theory.

The traditional LSTM is a one-way network that only receives the previous input information. The actual output results are often related to both the previous and subsequent inputs. To overcome the shortcomings of traditional LSTM model, a bidirectional LSTM model composed of forward LSTM layer and backward LSTM layer was proposed [25,26]. The bidirectional LSTM model has independent hidden layers in both directions (forward and backward), and can process sequences at the same time. Zhang et al. [27] predicted the displacements of the Bazimen and Baishuihe landslides in the Three Gorges, China with a variational mode decomposition-bidirectional LSTM model with optimized features. Wei et al. [28] presented a missing data processing method in terms of the partial distance fuzzy C-means model and bidirectional LSTM network. Le et al. [29] evaluated the performance of several deep learning models for streamflow forecasting, and the study shows that the LSTM-based models have better performance and stability than the feed-forward neural network (FFNN) and convolutional neural network (CNN) models. Lee and Kim [30] predicted the inflow rate with a sequence-to-sequence mechanism combined with a bidirectional LSTM. Li et al. [31] used a deep-stacked bidirectional LSTM neural network with a self-attention mechanism to capture the temporal dependencies of the original sensor data, and the method can deal with various missing data scenarios in dam monitoring system. Feizi et al. [32] proposed a hybrid deep learning inflow prediction-rolling window framework for inflow prediction.

Numerical methods can obtain highly accurate deformation of structure, but numerical methods have some shortcomings, such as complex preprocessing, difficult determination of material parameters, long calculation time and so on. To predict deformation in real time, statistical models and deep learning models are generally used in a dam monitoring system. The LSTM can effectively solve the gradient explosion, gradient disappearance and long-term dependent problems in conventional RNN models, it can reasonably consider the influence of early information in time series samples and effectively predict dam deformation, and it usually performs better than time RNN and hidden Markov model (HMM). In addition, LSTM can be used as a complex nonlinear element to construct a larger deep neural network. This work aims to establish the LSTM-based deformation prediction models of the embankment dam of the Danjiangkou hydropower station based on the monitoring data. According to the settlement monitoring data of measuring point LD12ZT01 (dam crest 0 + 648) on the left bank of the earth rock dam of the Danjiangkou hydropower station from July 2013 to March 2022, the partial least squares regression model, LSTM and bidirectional LSTM recurrent neural network models are constructed, and the results are compared. The results shows that the deformation prediction model based on LSTM has better prediction performance, and the prediction accuracy of bidirectional LSTM model is higher than that of LSTM model. Finally, the bidirectional LSTM model is used to predict the settlement of measuring point LD16YT01 on the right bank.

After the introduction, Section 2 introduces the deformation prediction theory of earth rock dam, Section 3 presents the deformation prediction model of the earth rock dam based on LSTM. Section 4 predicts the deformation of the earth rock dam of the Danjiangkou hydropower station. Finally, some conclusions are given.

2. Theory of Deformation Prediction for Earth Rock Dam

The vertical displacement (settlement) of earth rock dams is larger than the horizontal displacement, so the deformation monitoring model is generally based on the settlement model. For existing earth rock dams in nonalpine regions, water pressure and rheology are the main influencing factors of dam deformation.

The hydraulic deformation ${\delta }_{\mathrm{H}}$ is related to the first, second and third power of the upstream water depth $H$. Earth rock dams have obvious rheological properties. For simplicity, the time-dependent deformation can be described in the form of a straight line ${\delta }_{\theta }$. Set $t$ as cumulative monitoring days and $\theta =0.01t$, then the settlement $\delta$ of the earth rock dam can be expressed as [2]:

$$\delta ={\delta }_{\mathrm{H}}+{\delta }_{\theta }={\displaystyle \sum _{\mathrm{i}=1}^3{a}_i{H}^i}+(c_1\theta +c_2\ln \theta )$$

(1)

where $a_i$, $c_1$ and $c_2$ are regression coefficients.

3. Prediction Model of the Earth Rock Dam Deformation Based on LSTM

3.1. LSTM Model

LSTM is a kind of recurrent neural network model, which has excellent time series prediction performance. The structure of LSTM unit is shown in Figure 1. Information is transmitted through a cell state (memory unit). Forgetting gate, input gate and output gate control the update of the cell state. The forgetting gate and the input gate determine the information discarded and added from the cell state, respectively, and the output gate determines the output cell state. LSTM can effectively use long-distance time series data, so it is suitable for predicting dam deformation.

The more hidden layers of LSTM model, the higher the fitting accuracy, but the lower the model training efficiency. The study shows that the LSTM model with two hidden layers can effectively predict dam deformation [8]. In order to avoid over fitting (good model training effect and poor prediction performance), it is recommended to use dropout technology for network training.

3.2. Bidirectional LSTM Model

The traditional LSTM is a one-way network that only receives the input information. The bidirectional LSTM model can process the sequence in the forward and backward directions, so it can receive the front and rear input information. Figure 2 shows the bidirectional LSTM model structure. Storage units in the LSTM layer are updated at each time step $t$.

Set $x_t$ as input vector of time $t$, $h_{t-1}$ output vector of time $t-1$ and $\sigma (.)$ sigmoid activation function, then the activation value vectors $f_t$, $i_t$ and $o_t$ of the forgetting gate, input gate and output gate can be expressed as [12]:

$$f_t=\sigma \left(W_{xf}{x}_t+W_{hf}{h}_{t-1}+b_f\right)$$

(2)

$$i_t=\sigma \left(W_{xi}{x}_t+W_{hi}{h}_{t-1}+b_i\right)$$

(3)

$$o_t=\sigma \left(W_{xo}{x}_t+W_{ho}{h}_{t-1}+b_o\right)$$

(4)

where $W_{xf}$, $W_{hf}$, $W_{xi}$, $W_{hi}$, $W_{xo}$ and $W_{ho}$ are weight matrixes, $b_f$, $b_i$ and $b_o$ are deviation vectors.

The cell state vector $c_t$ at time $t$ is calculated by:

$$c_t=f_t\circ c_{t-1}+i_t\circ \tanh \left(W_{xc}{x}_t+W_{hc}{h}_{t-1}+b_c\right)$$

(5)

where $\circ$ is the Hadamard product.

The memory cell output vector $h_t$ of time $t$ can be expressed as:

$$h_t=o_t\circ \tanh \left(c_t\right)$$

(6)

In order to contain future information, the bidirectional LSTM consists of forward and backward LSTM layers. The output layer simultaneously processes the input from the LSTM layer. $h_t$ and ${\overline{h}}_t$ are the vectors of the backward and forward propagation layers, respectively, and $y_t$ is the vector of the output layer, so the update method of the neural network is as follows [12]:

$$h_t=H\left(W_1{x}_t+W_2{h}_{t-1}+b\right)$$

(7)

$${\overline{h}}_t=H\left(W_3{x}_t+W_5{\overline{h}}_{t-1}+\overline{b}\right)$$

(8)

$$y_t=H\left(W_4{x}_t+W_6{\overline{h}}_{t-1}+b_y\right)$$

(9)

where $b$, $\overline{b}$ and $b_y$ are deviation vectors and $W_1$~$W_6$ are weight coefficients.

3.3. Modeling Steps

The modeling steps of LSTM-based earth rock dam deformation prediction are as follows [9]: (i) preprocess the measured data: gross error processing of monitoring data, remove unreliable data, and retain reliable data; (ii) build a prediction model: the input of the model is the training sample of the reliable data set, and the Bayesian optimization algorithm is used to determine the super parameters of the model and build the trained prediction model; (iii) predict deformation: obtain the deformation value by inputting the independent variable factor data into the prediction model; (iv) evaluate the model performance: the mean absolute error $\mathrm{MAE}=\frac{1}{m}{\displaystyle \sum _{i=1}^m\left|y_i-{\widehat{y}}_i\right|}$, root mean square error $\mathrm{RMSE}=\sqrt{\frac{1}{m}{\displaystyle \sum _{i=1}^m{\left(y_i-{\widehat{y}}_i\right)}^2}}$ and mean square error $\mathrm{MSE}=\frac{1}{m}{\displaystyle \sum _{i=1}^m{\left(y_i-{\widehat{y}}_i\right)}^2}$($m$ is the number of samples and $y_i$ and ${\widehat{y}}_i$ are the true value and predicted value of the first sample $i$, respectively) are often applied to evaluate the model performance.

4. Deformation Prediction of the Earth Rock Dam of the Danjiangkou Hydropower Station

The Danjiangkou hydropower station is composed of a dam, power plant, ship lift and water diversion project. The total length of the dam is 3442 m, the riverbed is a concrete dam and the two banks are earth rock dams. The earth rock dam on the right bank is a clay core dam with a dam length of 877 m and a maximum dam height of 60 m. The earth rock dam on the left bank is a clay inclined wall and clay core dam, with a dam length of 1424 m and a maximum dam height of 70.6 m. According to the settlement monitoring data of measuring point LD12ZT01 on the left bank (dam crest 0 + 648) and measuring point LD16YT01 on the right bank (dam crest 0 + 658) from July 2013 to March 2022, the settlement prediction models of the two measuring points are established. The positive value is assumed as settlement.

According to the deformation prediction principle of earth rock dam, the output and input are dam deformation δ and its influence factors ($H$, $H^2$, $H^3$, $\theta$, $\ln \theta$), respectively. The number of input layer nodes is 1, and the number of output layer nodes is 5. The LSTM and bidirectional LSTM models for dam deformation prediction are constructed. The Bayesian optimization algorithm is used to determine the hyperparameters of the recurrent neural network, and the activation function is the rectifier linear unit function.

Firstly, the measuring point LD12ZT01 (dam crest 0 + 648) on the left bank is modeled and predicted. The initial parameters of LSTM and bidirectional LSTM models are set as maxepochs = 300, minibatchsize = 16, maxigrationnumber = 30 and dropoutvalue = 0.6. The super parameter range is: numoflayer = [1, 4], numofunits = [50, 150], initiallearnrate = [0.01, 1], l2regulation = [10⁻¹⁰, 0.01]. The optimized super parameters of the LSTM model are: numoflayer = 1, numofunits = 51, initiallearnrate = 0.02286, l2regulation = 0.00123. The optimized super parameters of the bidirectional LSTM model are: numoflayer = 1, numofunits = 130, initiallearnrate = 0.03528, l2regulation = 0.00613. The training set data are 80% of the total data, and the remaining data are test data. In order to compare the modeling and prediction accuracy of the model, the corresponding statistical prediction model is constructed by using partial least squares regression (PLSR). The partial least squares equation is:

$$y=1.537920-0.010800H-0.000078H^2-0.000000H^3+0.362265\theta +0.276584\ln \theta$$

(10)

Figure 3, Figure 4 and Figure 5 are the settlement process lines of LSTM, bidirectional LSTM and PLSR models of measuring point LD12ZT01, respectively.

Table 1. Comparison of errors of different models.

	RMSE	MAE	MSE
PLSR	0.56456	0.42978	0.31873
Training set of LSTM	0.52767	0.39556	0.27844
Test set of LSTM	0.47561	0.35264	0.22621
Total of LSTM	0.51768	0.38698	0.26799
Training set of bidirectional LSTM	0.54676	0.41253	0.29895
Test set of bidirectional LSTM	0.45400	0.34418	0.20612
Total of bidirectional LSTM	0.52951	0.39886	0.28038

compares the errors of different models. The values of RMSE, MAE and MSE are almost as large as those in the References [33,34], which verifies the accuracy of the present method. From the figures and table, it is found that the results of LSTM and bidirectional LSTM models are better than those of the PLSR model, and the results of bidirectional LSTM model on the test set are better than those of LSTM model. Therefore, the bidirectional LSTM model is more suitable to predict the deformation of the Danjiangkou earth rock dam. Figure 6 shows the settlement training and testing process line of measuring point LD16YT01 predicted by the bidirectional LSTM model, and the mean absolute error, root mean square error and mean square error for the test set are 0.5425, 0.66971 and 0.4520, respectively. This shows that the bidirectional LSTM model can obtain high prediction accuracy.

5. Conclusions and Outlook

This study presented the deformation prediction models of the Danjiangkou earth rock dam based on LSTM recurrent neural network, and compared the prediction accuracy of LSTM, bidirectional LSTM and PLSR models. The results shows that the deformation prediction model based on LSTM has better prediction performance, and the prediction accuracy of bidirectional LSTM model is higher than that of LSTM model. Finally, the bidirectional LSTM model was selected to predict the settlement value of LD16YT01 measuring point (dam crest 0 + 658) on the right bank, and the mean absolute error and root mean square error for the test set are 0.5425 and 0.66971, respectively. This shows that the bidirectional LSTM model can effectively predict the settlement value of the embankment dam of the Danjiangkou hydropower station.

The case study show the bidirectional LSTM model can establish the complex nonlinear relationship between the deformation of the earth rock dam and its influencing factors without overfitting. At present, the amount of measured data in Danjiangkou hydropower station is not large. The proposed bidirectional LSTM model will be further validated with the increase of the amount of measured data. In addition, the uncertainty and reliability of the present results will be investigated by training group method of data handling using extreme learning machine conceptions [35].