1. No category

How to use new information technologies for prediction:
Ensemble flow forecasting, verification and
postprocessing
Maik Renner
Micha Werner*, Albrecht Weerts*, Paolo Reggiani*
TU Dresden / Germany – Institute of Hydrology and Meteorology
* Deltares, Delft The Netherlands
PUB 2011 workshop - Putting PUB into practise
Canmore, Canada - 14th May 2011
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
1 / 32
Introduction
Background, motivation
Are ensembles a luxury or a prerequistite?
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
2 / 32
Introduction
Background, motivation
Are ensembles a luxury or a prerequistite?
1
The future is uncertain!
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
2 / 32
Introduction
Background, motivation
Are ensembles a luxury or a prerequistite?
1
The future is uncertain!
2
Models are not reality!
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
2 / 32
Introduction
Background, motivation
Are ensembles a luxury or a prerequistite?
1
2
3
The future is uncertain!
Models are not reality!
But:
Our models may be close by.
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
2 / 32
Introduction
Background, motivation
Are ensembles a luxury or a prerequistite?
1
2
3
The future is uncertain!
Models are not reality!
But:
Our models may be close by.
We can try to express our certitude of a prediction with a probability.
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
2 / 32
Introduction
Background, motivation
Are ensembles a luxury or a prerequistite?
1
2
3
The future is uncertain!
Models are not reality!
But:
Our models may be close by.
We can try to express our certitude of a prediction with a probability.
→ decision can be made by a decision maker instead of the forecaster
(Weerts et al. 2011)
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
2 / 32
Introduction
Building Ensemble forecasts
Construction of an ensemble forecast:
(ac)knowlegde uncertainties
input data, model simulation, model structure, future conditions
simulate from different initial states – meteorological ensemble
forecasts
sample probability distributions: e.g. parameter distributions of a
hydrologic model, sample inputs to the model
Expectations
expect that the ensemble is a good representation of the predictive
uncertainty
→ the ensemble is drawn from the same distribution as the uncertainties
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
3 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
such objectives are e.g. : the forecast horizon and the respective basin
(travel times, etc...)
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
such objectives are e.g. : the forecast horizon and the respective basin
(travel times, etc...)
2
Is an ensemble prediction accurate?
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
such objectives are e.g. : the forecast horizon and the respective basin
(travel times, etc...)
2
Is an ensemble prediction accurate?
3
How to compare forecasts?
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
such objectives are e.g. : the forecast horizon and the respective basin
(travel times, etc...)
2
3
4
Is an ensemble prediction accurate?
How to compare forecasts?
How to improve ensemble forecasts given observed data?
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
such objectives are e.g. : the forecast horizon and the respective basin
(travel times, etc...)
2
3
4
Is an ensemble prediction accurate?
How to compare forecasts?
How to improve ensemble forecasts given observed data?
Postprocessing methods
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
such objectives are e.g. : the forecast horizon and the respective basin
(travel times, etc...)
2
3
4
Is an ensemble prediction accurate?
How to compare forecasts?
How to improve ensemble forecasts given observed data?
Postprocessing methods
Data assimiliation
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Research questions
Research questions
1
Relevant sources of uncertainty for the given objective?
such objectives are e.g. : the forecast horizon and the respective basin
(travel times, etc...)
2
3
4
Is an ensemble prediction accurate?
How to compare forecasts?
How to improve ensemble forecasts given observed data?
Postprocessing methods
Data assimiliation
5
Transferability of forecast accuracy towards ungauged basins?
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
4 / 32
Introduction
Outline
Outline
1 Introduction
Background, motivation
Building Ensemble forecasts
Research questions
Outline
2 Ensemble forecast verification
Verification Rank Histogram
Threshold scores
3 Case study: River Rhine
Model scheme
Hydrological Ensemble generation
Hindcast set up
ECMWF-EPS verification results
Improving forecasts with downscaling
Improving forecasts with postprocessing
Summary verification
4 Dominant sources of uncertainty
Maik Recommendations
Renner (TU Dresden)
Ensemble forecasting
PUB 2011
5 / 32
Ensemble forecast verification
Ensemble (probabilistic) forecast verification
How well agree ensemble forecast with observed data?
water levels
discharge
economic losses
Identify problems and improve your forecast!
active research and application in meteorology
Wilks (2006), WMO (2010), mailing list vx-discuss@rap.ucar.edu
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
6 / 32
Ensemble forecast verification
6000
4000
Warning Threshold
2000
discharge, m3s
8000
10000
An ensemble flow forecast
0
Observed discharge
Simulation
Ensemble forecast with error correction
Jul 30
t0,start forecast
Aug 06
8 day lead time
Aug 13
Aug 20
Figure: One ensemble forecast, with observations and threshold
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
7 / 32
Ensemble forecast verification
Scalar Accuracy Measures
mean absolute error M AE of a set of m forecasts and observations:
M AE =
n
1X
n
(1)
|yi − oi |
i=1
yi is the deterministic forecast for issued at day i, for ensembles the
ensemble mean and sometimes the median is used, probabilistic forecast
the median is used
does not consider the ensemble spread!
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
8 / 32
Ensemble forecast verification
Verification Rank Histogram
Reliability of an ensemble forecast - Rank Histogram
Considers the question if the ensemble is drawn from the same distribution
as the predictive uncertainty
4000
5000
6000
ensemble forecast
observation
3000
discharge at lead time [m3s]
Determination of Rank of Observation in Ensemble
0
10
20
30
40
50
Rank of Observation in Ensemble
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
9 / 32
Ensemble forecast verification
Verification Rank Histogram
Reliability of an ensemble forecast - Rank Histogram
Considers the question if the ensemble is drawn from the same distribution
as the predictive uncertainty
200
0
50
100
Frequency
300
Rank Histogram
0
10
20
30
40
50
Rank of observation in ensemble
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
9 / 32
Ensemble forecast verification
Threshold scores
Threshold scores - The Brier score
Ensemble forecast at 8 days lead time
Transform ensemble and
observations into
probabilities (of an event)
yi= 5/51
oi= 0
5000
Warning Threshold
Goes back to Brier (1950)
4000
Analogue to the mean
squared error for probabilites
3000
discharge at lead time [m3s]
6000
ensemble forecast
observation
0
10
20
30
40
50
sorted ensembles and observation
Maik Renner (TU Dresden)
BS =
Ensemble forecasting
m
1 X
N
(yi − oi )2
i=1
PUB 2011
10 / 32
Ensemble forecast verification
Threshold scores
10000
Ranked Probability Score
6000
4000
m=3
m=2
considers the distance of the
forecast to the observations
general accuracy of a
probabilistic forecast
m=1
t0,start forecast
0
discharge, m3s
m=4
2000
8000
extension to the Brier Score
for several categories
Jul 30
Aug 06
Aug 13
8 day lead time
negatively orientated, 0 best
Aug 20
Figure: Definition of 4 prediction categories.
RP S =
Maik Renner (TU Dresden)
Ensemble forecasting
1
M
X
M − 1 m=1

m
X



yi  − 
i=1
PUB 2011
m
X
2
oi 
i=1
11 / 32
Ensemble forecast verification
Threshold scores
Skill scores
skill compared to a reference forecast e.g. climatology, persistence
Skillscore = 1−
Score
(3)
Scoreref erence
1 ... perfect skill
0 ... no skill, i.e. equivalent
< 0 ... less skill than reference
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
12 / 32
'
N
#
0
#
0
#
0
Hattingen
#
0#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
r
#
0
(
IN
#
0
#
0
2
#
0
#
0
3
#
0
#
0
#
*#0Maxau#0
(
#
*
(
1...5
'
N
#
0#
0
25
50
100 Km
Bern #
0
#
0
#
0
#
0
#
0
#
0
0
##
0
CH
#
0
#
0
#
0
#
0#
0
#
0
#
#0
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
0
#
0 #
#
0
#
0
#
0
#
0
#
0
#
0#
#
0
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
0
#
0 #
#
0
#
0
#
0
0
5
#
0
#
0
#
0#
0
#
0
#
0
#
0
#
0
#
0
'
N
#
0
#
0
#
0
#
*
(
#
0
#
0
#
0
#
0
#
0
Rheinfelden #
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
#
0
4
#
0
#
0
#
0
#
0
#
0 Stuttgart
'
N
N Strasbourg
#
0'
#
0
#
0
rain gauge
river gauge
isochrones [d]
main river
#
0
tributaries
rhine basin
0
lakes
#
0 #
#
0
states
#
0
#
0
cities
#
0
ck a r
#
0
Nancy
0
'
N#
#
0
#
0
Ne
#
0
#
0
in
#
0
#
* Rockenau
(
#
0
#
0
Ma
#
0
#
0
#
0
#
0
#
0
#
0
#
0
0
Frankfurt a. M. #
'
N
#
0
#
0
#
0
#
0
#
0
E
#
0
L #0
#
0
#
0
*
Cochem #
#
0
#
0
#
0
#
* #0
(
#
0
#
0
F
Rhine basin in Europe
#
0
Andernach
#
0
#
0
uh
#
0
#
0
#
0
#
0
#
0
#
0
1
#
0
#
0
BE
#
0
R
#
*
(
#
0
0
0 #
#
0 #
#
0
#
0
#
0
#
0
#
0
#
0
#
0
Lobith
#
*
(
#
0
#
0
#
0
0
#
0#
#
0
0
##
0
#
0
D
#
0
Amsterdam
H
Details may be found in:
Renner, M., Werner, M.,
Rademacher, S. and Sprokkereef,
E.: 2009, Verification of ensemble
flow forecasts for the River Rhine,
Journal of Hydrology
376(3-4), 463–475.
NL
R
Length 1,233 km (766 mi)
Basin 170,000 km² (65,637
sq mi)
Discharge - average 2,000
m3/s (70,629 cu ft/s)
Ú
M o sel
Flow forecasting at River
Rhine
Aim: Medium range flow
forecasting → 2 to 10 days
lead time
river navigation
#
0#
0
#
0
#
0
#
0
#
0
#
0
A
I
#
0#
0
#
0
#
0
Editing: K. Geidel and M. Renner, TU Dresden
Sources: Feature data: BfG,
DTM: SRTM http://srtm.csi.cgiar.org/
State:
January 2009
Case study: River Rhine
Model scheme
Model scheme
use conceptual hydrological
HBV-96 model
simple routing,
hydrodynamic also possible
calibrated for 134 sub basins
taken from
http:
//www.smhi.se/foretag/m/hbv_demo/html/end.html
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
14 / 32
Case study: River Rhine
Hydrological Ensemble generation
Hydrological Ensemble generation
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
15 / 32
Case study: River Rhine
Hydrological Ensemble generation
Hydrological Ensemble generation
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
15 / 32
Case study: River Rhine
Hindcast set up
Hindcast set up
baseline simulation (HBV) with observed meteorological inputs (1-6
hourly rain and temperature)
daily hindcasts, HBV forced with meteorological forecasts
ECMWF-EPS a global circulation model with a resolution of approx.
50 km and 51 ensemble members
verification period 6/2004 - 10/2007
COSMO-LEPS dynamic downscaling approach nested in
ECMWF-EPS over Europe; 10 km resolution, 16 ensemble members,
verification period 1/2007 - 10/2007
statistical error correction with autoregressive (AR) based on 2-10
days of observed data, then predicting the error (Broersen and Weerts
2005)
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
16 / 32
Case study: River Rhine
ECMWF-EPS verification results
Skill of ECMWF-EPS precipitation forecasts
Precipitation forecasts on sub basin scale show to have Ranked
Probability Skill Score (RP SS) between 0.1 and 0.3 compared
against climatology, the skill deterioates with lead time and there is no
skill after 5 to 7 days
52
52
51
51
50
50
0.6
longitude
longitude
0.45
49
49
48
48
47
47
0.3
0.15
46
5
6
7
8
9
latitude
10
11
12
46
5
6
7
8
9
10
11
12
0
latitude
Figure: RPSS of precipitation forecasts against “observed” subbasin precip. Left one day, right
panel 5 days lead time. A value of one indicates a perfect forecast, while 0 indicates no skill.
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
17 / 32
Case study: River Rhine
ECMWF-EPS verification results
Skill of ECMWF-EPS driven discharge ensemble
Skill of discharge forecasts at main river basin scale (from 4124 to
160,800 km2 ) against climatology
positive skill up to 9 days → long travel times
1.0
tendency of increasing skill with catchment area
0.6
●
●
0.4
●
lead time in days
●
●
0.0
0.2
Ranked probability skill score
0.8
●
●
●
Hattingen
Rockenau
Cochem
Rheinfelden
Maxau
Andernach
1
3
5
7
9
Lobith
Figure: RPSS of error corrected flow forecasts at main river gauges, sorted by catchment size.
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
18 / 32
Case study: River Rhine
Improving forecasts with downscaling
Precipitation ECMWF-EPS vs. COSMO-LEPS
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
19 / 32
Case study: River Rhine
Improving forecasts with downscaling
ECMWF-EPS and COSMO-LEPS precip skill at one day ahead
52
52
51
51
50
50
0.6
longitude
longitude
0.45
49
49
48
48
47
47
0.3
0.15
46
5
6
7
8
9
latitude
Maik Renner (TU Dresden)
10
11
12
46
5
6
7
8
9
10
11
12
0
latitude
Ensemble forecasting
PUB 2011
20 / 32
Case study: River Rhine
Improving forecasts with downscaling
●
●
●
●
●
●
−0.2
0.0
●
lead time in days
●
−0.4
Ranked probability skill score
0.2
Discharge skill of COSMO-LEPS compared with ECMWF-EPS
Hattingen
1
3
5
7
9
Rockenau
Maik Renner (TU Dresden)
Cochem
Rheinfelden
Ensemble forecasting
Maxau
Andernach
Lobith
PUB 2011
21 / 32
Case study: River Rhine
Improving forecasts with downscaling
Reliability of flow forecasts forced with ECMWF-EPS vs. COSMO-LEPS
Hattingen, lead time = 4 days
2007−01−16 to 2007−10−06
Lobith, lead time = 8 days
2007−01−20 to 2007−10−10
80
ECMWF−EPS
COSMO−LEPS
40
counts
80
60
20
40
0
20
0
counts
60
100
120
ECMWF−EPS
COSMO−LEPS
0.05
0.25
0.45
0.65
0.85
0.05
rank of observation in ensemble
Maik Renner (TU Dresden)
0.25
0.45
0.65
0.85
rank of observation in ensemble
Ensemble forecasting
PUB 2011
22 / 32
Case study: River Rhine
Improving forecasts with postprocessing
Improving forecasts with its own error: Ensemble postprocessing with
Bayesian Model Averaging
30 days training period, gaussian error model (Fraley et al. 2010)
Lobith, lead time = 8 days
2007−01−20 to 2007−10−10
80
Ensemble Rank Histogram
PIT ensemble BMA
Ensemble Rank Histogram
PIT ensemble BMA
40
counts
80
60
20
40
0
20
0
counts
60
100
120
Hattingen, lead time = 4 days
2007−01−16 to 2007−10−06
rank of observation in ensemble
Maik Renner (TU Dresden)
rank of observation in ensemble
Ensemble forecasting
PUB 2011
23 / 32
Case study: River Rhine
Summary verification
What we learnt from verification
1 medium range ensemble flow forecasts provide some skill
2
3
4
downscaling of precipitation forecasts improves skill in flow forecasts
at all basin sizes
calibration of ensemble forecasts increase reliability (and often the
spread)
for interesting warning threshold verification long hindcasts are
necessary
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
24 / 32
Dominant sources of uncertainty
Dominant sources of uncertainty
0.8
use separation of error (MAE) by the ratio of:
●
0.6
●
ratio
0.0
0.2
0.4
●
f orecast vs. simulation
f orecast vs. observation
●●
Altenahr
900 km2
●●●●●●● ●●●●●●●
●●●●●●●
●●●●
●●●
●●●●●●●●●●●●● ●●●●
●●●●●● ●●●●
Grolsheim 4013 km2
●● ●●●●●
●●● ●●
●●●●●●●●
● ●●
●●
●●●● ●●●●●●●
●●●●●●●●●●●●●●●● ●
●●●●●●●● ●●●●●●●● ●●
●●●●● ●● ●●
Hattingen 4124 km2
● ●●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●●●● ●●
●● ●● ●●●●●●●●●●●
●● ●●●●●●● ● ●●●
●●●●
●
●●●●●
●●● ●●●●
●
Rockenau 12616 km2
●●●●●
●●●●●●● ●●●●
●● ●●
●●●●●●●●●●●●●●●● ●●
●●●
●●●●●●●
●●●●●●
●●
●●●●
● ●●●
●●●●●● ●●● ●●●●●●● ●●●● ●
●●●●●●●●●●● ●●●●●
●●●●●●●●●●●●
●●●
●
●●
●●●
●
Raunheim 27142 km2
●●●●●●
●●
●
● ●●●●
●
●●●●●●●●●●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●●
●
●
●●●
●●●●● ●●●●●●●●●●●
●●●●●●
●●●●●● ●
●●●●●●
●●
● ●●
●●●●
Cochem
27262 km2
●●
●●●●●●●●●
●●●
●●●●● ●
●●
●●
●●
●●
●●●●●●●
●●
●●
●●●●
●● ●●
●●●●●●●
●●●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●●●●●
●●●
●●●●
●●●
●●
●●
●●
Rheinfelden 34550 km2
●● ●
●
●●● ●●●●●●●●●●●●●●●● ●● ●●●
●● ●●●●●
● ●●●
●●●●
●●●●●
●●●●●●●
●●●
●●
●●
●●
●●
●●●
●●●●
●●●●●●● ●
●●● ●
●●
●●
●●●●●●
●●●●●
●●
●●●●● ●●●●●●●●●● ●
●●●●●●●
Maxau
50196 km2
●●●
●●●
●●
●●
●●
●●●●●●●●●
●● ●●●●●●●●●●●
●●●●
●●● ●●●●●
●
●●● ●● ●●
●● ●●
●●●●● ●●●●●●●● ●●●●
●●●
● ●● ●●●
●●
●●●●●●●●●● ●
●●
Andernach 139549 km2 ●●● ●●●●●●●●●●●●●●●
● ● ● ●●● ●●●●● ●●●●●
●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ● ●●●
●●● ●
●
●●● ●●●●●●●● ● ●●●●
●●●●
●● ●●
●●●
●●
●● ●●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
Lobith
160800 km2 ●●● ●●● ●●●●
●●●●●● ●●●
●●
●
●●
●●●
●●
●
●●●
●●●
●
● ●●●
●●●●● ● ●●●●●●●●●●●
●●●●●●● ●
●●
●●
●●●●●●●
●●●●● ●● ●● ●●
●●●●
●●●●●●
●●●
●●●● ●●●●
●●●
●●●●
●● ●●●●
●●●
●●●●●●● ● ●●●●● ●●●
●●●●●● ●●● ●●●
●●● ● ●●●●●●●
●●●
●●●● ●●●● ●●● ●●●●●
●● ●●●
●●●
●●
●●●
●
●●●
●●
●●
●●●●●
●●●●●●●●
●●
●
●●
●
● ● ●●●● ●●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●●● ●●●● ●●● ●●●●●●●●●
●●
●
●●●●●●●●●●●●
●● ●
●● ●
●
●
●●
● ●● ●●●
●● ●● ●●
●●
●
●●
●●●● ●●●●●
●● ● ●●
●●●●●
●
● ●●●●
●●●●● ●●
●●●● ●
●●
●
●●
●●● ●●●●●●●●●
●●●● ●●●●
●●● ●●
●
●●●
●●
●●
●●
●●●●●
●
●●● ●●●●●●●● ●
●● ●●●
●●●●●● ●●
●●●●●●●●
● ●●●● ●●
●●●●●●●●●
●●
●● ● ●
●
●
●
●
●
●
●
●
● ●●
●●● ●● ●●●●
●
●● ●●●
●
●●●●●●
●●
●●● ●●●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
● ●●● ●●●●●
●●●●●● ●
●●
● ●
●●● ●●
●●
●●●●●●●●●
●●●● ●
●●
●● ●●
●● ●
●●●
●●
●●●●●●
●●●
●●
●●
● ●
●
●●
●●●
●●●●
●●
●●●●●● ●
●● ●
●●
● ●
●●●●●
●●●
●●
●●●●●●●●●
●●
●●●●
●●
● ●●
●●
●●● ●●
●●●
●●●●●
●●
●
●●● ●●
●●●
●●●●● ● ●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●● ●● ●●●●
●●
●●●●●●●●
●●●
●●
●●
●●●●●●
● ●●●●
● ●●●
●●●
●●
●●●
●●●●●
●
●
●●
●● ●●
●●
● ●●●
●●●●●
●●●
●●
● ●●●
●●
●
●●●●●●●
●●●
●●
●●●
● ●●
●●
●●●●●
●
●●
●●●
●●●
●●
●●
●●●
●●●●●
●
●● ●●●●
●
●
●
●
●
●●●
●
●
●●●
●
● ●●●●●●●●●●●
●● ●●●
●
●●●
●●● ●●
●●●●
●●
●●
●●
●
●●●
●● ●● ●●
●●●
●●
●● ●●
●●●●
●●
●●●●●●●
●●●●●
●
●● ●●●●●●●
●●●●●
●●●●●●●●
●●●●●●●●
●●●●● ●●● ●●
●●●●●● ●●●●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●● ● ●●●●● ●●●
●●
●●
●● ●●●●
●●
●●●●●●● ●●●●●●
●●
●
●●
●
●
●●
●●●●●●●●●●●●●●●●
●●
●●
●●
●●
●●
●●
●●●●●●
●●
●●
●●●
0
50
100
150
200
250
lead time [hours]
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
25 / 32
Dominant sources of uncertainty
Recommendations
Recommendations to improve flow forecasts
Future meteorological conditions are dominant:
1
force hydrological models with meteo. ensembles
2
downscaling further improves hydrologic forecasts
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
26 / 32
Dominant sources of uncertainty
Recommendations
Short term forecasts
data rich:
1 Error correction
statistical
model state updating with Ensemble Kalman Filter (Weerts and
El Serafy 2006, Vrugt and Robinson 2007)
2
3
Postprocessing, e.g. BMA, Model Output Statistics (MOS), ...
Hydrological Ensembles (Multimodel (Georgakakos et al. 2004,
Velázquez et al. 2011), Sampling Input Uncertainty
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
27 / 32
Dominant sources of uncertainty
Recommendations
Short term forecasts
data poor:
1
2
3
Hydrological Ensembles
Model state updating using other data, e.g. Remote sensed soil
moisture (Komma et al. 2008)
any PUB ideas?
some data:
Geostatistical interpolation of forecast accuracy or postprocessing
hydrodynamic modelling with updated states (Weerts et al. 2010)
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
28 / 32
Acknowledgments
Thank you for listening!
Questions?
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
29 / 32
References
References
Brier, G.: 1950, Verification of forecasts expressed in terms of probability, Monthly weather review 78(1), 1–3.
Broersen, P. and Weerts, A.: 2005, Automatic Error Correction of Rainfall-Runoff models in Flood Forecasting
Systems, Instrumentation and Measurement Technology Conference, 2005. IMTC 2005. Proceedings of the
IEEE, Vol. 2, pp. 963– 968.
Fraley, C., Raftery, A. E., Sloughter, J. M., Gneiting, T., of Washington With contributions from Bobby Yuen, U.
and Polokowski, M.: 2010, ensembleBMA: Probabilistic Forecasting using Ensembles and Bayesian Model
Averaging. R package version 4.5.
URL: http://CRAN.R-project.org/package=ensembleBMA
Georgakakos, K., Seo, D., Gupta, H., Schaake, J. and Butts, M.: 2004, Towards the characterization of streamflow
simulation uncertainty through multimodel ensembles, Journal of hydrology 298(1-4), 222–241.
Komma, J., Bloschl, G. and Reszler, C.: 2008, Soil moisture updating by Ensemble Kalman Filtering in real-time
flood forecasting, Journal of Hydrology 357(3-4), 228–242.
Velázquez, J., Anctil, F., Ramos, M. and Perrin, C.: 2011, Can a multi-model approach improve hydrological
ensemble forecasting? A study on 29 French catchments using 16 hydrological model structures, Advances in
Geosciences 29, 33–42.
Vrugt, J. and Robinson, B.: 2007, Treatment of uncertainty using ensemble methods: Comparison of sequential
data assimilation and Bayesian model averaging, Water Resour. Res 43, W01411.
Weerts, A. and El Serafy, G.: 2006, Particle filtering and ensemble Kalman filtering for state updating with
hydrological conceptual rainfall-runoff models, Water resources research 42(9), W09403.
Weerts, A., El Serafy, G., Hummel, S., Dhondia, J. and Gerritsen, H.: 2010, Application of generic data
assimilation tools (DATools) for flood forecasting purposes, Computers & Geosciences 36(4), 453–463.
Weerts, A., Winsemius, H. and Verkade, J.: 2011, Estimation of predictive hydrological uncertainty using quantile
regression: examples from the National Flood Forecasting System (England and Wales), Hydrol. Earth Syst. Sci
15, 255–265.
Wilks, D. S.: 2006, Statistical Methods in the Atmospheric Sciences, 2nd edn, International Geophysics Series, Vol.
59, Academic Press.
WMO: 2010, Forecast verification - issues, methods and faq, WWRP/WGNE Joint Working Group on Verification.
URL: http://www.cawcr.gov.au/projects/verification/
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
29 / 32
Appendix
Reliability at a given threshold
specific for a threshold of the target variable and a period T
given all cases, when a certain probability of exceedance of this
threshold was issued by the forecast, how often this exceedance has
been observed
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
29 / 32
forecast vs. baseline simulation
meteo. forecast uncertainty only
deviations: low probability of
exceedance (important for high risk
decisions) are underpredicted due to
meteo. forecasts!
0.6
0.4
0.2
0.8
0.0
●
0.6
●
●
●
Qcorr − Qobs
0.6
●
0.4
reliability diagram: close to 1-1 line
→ is relatively reliable
0.2
●
0.0
0.0
error corrected forecast vs. observed
full uncertainties
●
●
●
0.4
forecast probabilities histogram:
“sharp” but mostly 0 → low sample
size for probabilites in between
Observed relative frequency, o1
→ very long hindcast data set would be
rquired to verifiy forecasts for
warning thresholds!
●
●
Qforc − Qsim
0.2
80% quantile –> observed at 200
out of 1000 days (sample
climatologic probability)
1.0
evaluate threshold exceeding
1460 m3 /s at lead time of 8 days
at Maxau/Upper Rhine
0.8
Reliability of ECMWF-EPS driven discharge forecasts
0.0
0.2
0.4
0.6
0.8
1.0
Forecast probability, yi
Figure: Reliability diagram for Maxau at a lead time of
8 days exceeding 1460 m3 /s. The dark green line
marked with circles shows the reliability line for the
verification of the error-corrected flow forecast against
high exceedance probability predicted observations (Qcorr – Qobs ). The dark red dashed
Appendix
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
31 / 32
Appendix
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
32 / 32
Appendix
1
Maik Renner (TU Dresden)
Ensemble forecasting
PUB 2011
33 / 32