Ensemble Kalman Particle Filters for High-Dimensional Data Assimilation

Abstract

Data assimilation consists in estimating the state of a system, for example the atmosphere in numerical weather prediction (NWP), by combining information coming from the dynamical laws of the system with a stream of observations. Because of the presence of observational noise and uncertainty in the initial conditions, a probabilistic instead of a deterministic approach is to be preferred. The goal is thus to estimate the time evolution of the distribution of the system state conditioned on all the past observations. Ensemble data assimilation methods, such as the ensemble Kalman filter (EnKF), solve this problem by representing the distribution of the state with a finite sample of particles which follow the dynamical laws of the system. What makes data assimilation for geophysical applications particularly challenging is that the dimension of the state to estimate is extremely high (order of 100 millions), while the ensemble size is limited to less than 100 due to heavy computational costs. At the same time, the increasing resolution of the physical models makes Gaussian assumptions, on which the EnKF relies, less and less valid. In the present thesis we propose extensions to the ensemble Kalman particle filter (EnKPF), a hybrid algorithm which relaxes some of the Gaussian assumptions by combining the EnKF with the particle filter (PF). The goal of these extensions is to make the EnKPF suitable for very high-dimensional applications. The first contribution consists in proposing two localized versions of the algorithm: the naive-LEnKPF and the block-LEnKPF. The naive-LEnKPF, similar to the local EnKF (LEnKF), works by assimilating data in local windows and then patching the results together. It has the advantage to be simple and efficient, but it does not address the issue of discontinuities introduced by the PF part of the algorithm. The block-LEnKPF, on the other hand, assimilates the observations by blocks and limits their influence to a local area while smoothing out the introduced discontinuities. Both local EnKPFs are applied to an artificial model of cumulus convection of medium dimensionality. The results of the numerical experiments show that the new algorithms perform at a similar level to the LEnKF, and bring some noticeable improvements for non-Gaussian variables such as the precipitation field. The second main contribution of this thesis is to propose a new algorithm, the ensemble transform Kalman particle filter (ETKPF). It is based on a reformulation of the EnKPF in ensemble space, which allows it to be easily and efficiently implemented in an existing full-scale NWP data assimilation framework. Furthermore, the ETKPF replaces the stochastic part of the algorithm with a deterministic scheme, such that it has exact second moment instead of only on expectation. The algorithm was tested on a challenging high-dimensional application at convective scale with COSMO, in a setup similar to the one used operationally at MeteoSwiss. The results of the experiments show the feasibility of the new algorithm in real-world applications and encourage further developments in the direction of localized hybrid particle filters for high-dimensional data assimilation.