Assessing the applicability of Transformer-based architectures as rainfall-runoff models

Assessing the applicability of Transformer-based architectures as rainfall-runoff models

Mao, Kangmin (TU Delft Civil Engineering & Geosciences)

Taormina, R. (mentor)
Hrachowitz, M. (graduation committee)
De Stefani, J. (graduation committee)
Couasnon, Anaïs (graduation committee)
Dahm, Ruben (graduation committee)
Nuttall, Jonathan (graduation committee)

Delft University of Technology

Civil Engineering

2023-01-27

Modeling the relationship between rainfall and runoff is a longstanding challenge in hydrology and is crucial for informed water management decisions. Recently, Deep Learning models, particularly Long short-term memory (LSTM), have shown promising results in simulating this relationship. The Transformer, a newly proposed deep learning architecture, has also demonstrated the ability to outperform LSTM in machine translation, text classification, etc. However, there has been limited research on applying Transformers for rainfall-runoff modeling.

The research examined the performance of using Transformer architecture, including its time series forecasting variants, to develop rainfall-runoff models using the CAMELS (US) data set. These models were compared to the LSTM regional rainfall-runoff models, with a particular focus on snow-driven basins as the attention mechanism in Transformer is believed to allow it to attend to the earlier precipitation events in the meteorological forcing. Additionally, the Transformer's potential as a global rainfall-runoff model was also tested using the global Caravan data to determine if it could learn and generalize a wide range of rainfall-runoff behaviors, allowing it to potentially be applied in ungauged basins.

The results suggest that while Transformer and its variants may not be able to fully replace LSTM for rainfall-runoff modeling, the variant called Reformer has shown promise for daily discharge forecasting in snow-driven basins, particularly in terms of peak flow and low flow prediction. However, using the global Caravan data for building a global rainfall-runoff model was not successful due to uncertainty in the forcing data, particularly precipitation. The code for Transformer-based rainfall-runoff modeling is available publicly at https://github.com/Numpy-Panda/neuralhydrology_Transformer.

Rainfall-runoff relationship
Deep Learning
Hydrological modeling
Transformer
Time Series Forecasting
Ungauged basins

http://resolver.tudelft.nl/uuid:d225d073-0499-4f1c-8340-1e8f5f4a8401

https://github.com/Numpy-Panda/neuralhydrology_Transformer Repository link The GitHub repository of the Transformer-based rainfall-runoff modeling.

Student theses

master thesis

© 2023 Kangmin Mao