Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis
ABSTRACT Hydrological models (HMs) can be applied for different purposes, and a key step is model calibration using objective functions (OF) to quantify the agreement between observed and calculated discharges. Fully understanding the OF is important to properly take advantage of model calibration and interpret the results. This study evaluates 36 OF proposed in the literature, considering two watersheds of different hydrological regimes. Daily simulated streamflow time-series, using a distributed hydrological model (MGB-IPH), and ten daily streamflow synthetic time-series, generated from the observed and calculated streamflows, were used in the analysis of each watershed. These synthetic data were used to evaluate how does each metric evaluate hypothetical cases that present isolated very well known error behaviors. Despite of all NSE-derived (Nash-Sutcliffe efficiency) metrics that use the square of the residuals in their formulation have shown higher sensitivity to errors in high flows, the ones that use daily and monthly averages of flow rates in absolute terms were more stringent than the others to assess HMs performance. Low flow errors were better evaluated by metrics that use the flow logarithm. The constant presence of zero flow rates deteriorate them significantly, with the exception of the metrics TRMSE (Transformed root mean square error) did not demonstrate this problem. An observed limitation of the formulations of some metrics was that the errors of overestimation or underestimation are compensated. Our results reassert that each metric should be interpreted specifically thinking about the aspects it has been proposed for, and simultaneously taking into account a set of metrics would lead to a broader evaluation of HM ability (e.g. multiobjective model evaluation). We recommend that the use of synthetic time series as those proposed in this work could be useful as an auxiliary step towards better understanding the evaluation of a calibrated hydrological model for each study case, taking into account model capabilities and observed hydrologic regime characteristics.
| Main Authors: | , , |
|---|---|
| Format: | Digital revista |
| Language: | English |
| Published: |
Associação Brasileira de Recursos Hídricos
2020
|
| Online Access: | http://old.scielo.br/scielo.php?script=sci_arttext&pid=S2318-03312020000100404 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| id |
oai:scielo:S2318-03312020000100404 |
|---|---|
| record_format |
ojs |
| spelling |
oai:scielo:S2318-033120200001004042020-09-21Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysisFerreira,Paloma Mara de LimaPaz,Adriano Rolim daBravo,Juan Martín Model calibration Hydrologic simulation Performance measures MGB-IPH ABSTRACT Hydrological models (HMs) can be applied for different purposes, and a key step is model calibration using objective functions (OF) to quantify the agreement between observed and calculated discharges. Fully understanding the OF is important to properly take advantage of model calibration and interpret the results. This study evaluates 36 OF proposed in the literature, considering two watersheds of different hydrological regimes. Daily simulated streamflow time-series, using a distributed hydrological model (MGB-IPH), and ten daily streamflow synthetic time-series, generated from the observed and calculated streamflows, were used in the analysis of each watershed. These synthetic data were used to evaluate how does each metric evaluate hypothetical cases that present isolated very well known error behaviors. Despite of all NSE-derived (Nash-Sutcliffe efficiency) metrics that use the square of the residuals in their formulation have shown higher sensitivity to errors in high flows, the ones that use daily and monthly averages of flow rates in absolute terms were more stringent than the others to assess HMs performance. Low flow errors were better evaluated by metrics that use the flow logarithm. The constant presence of zero flow rates deteriorate them significantly, with the exception of the metrics TRMSE (Transformed root mean square error) did not demonstrate this problem. An observed limitation of the formulations of some metrics was that the errors of overestimation or underestimation are compensated. Our results reassert that each metric should be interpreted specifically thinking about the aspects it has been proposed for, and simultaneously taking into account a set of metrics would lead to a broader evaluation of HM ability (e.g. multiobjective model evaluation). We recommend that the use of synthetic time series as those proposed in this work could be useful as an auxiliary step towards better understanding the evaluation of a calibrated hydrological model for each study case, taking into account model capabilities and observed hydrologic regime characteristics.info:eu-repo/semantics/openAccessAssociação Brasileira de Recursos HídricosRBRH v.25 20202020-01-01info:eu-repo/semantics/reporttext/htmlhttp://old.scielo.br/scielo.php?script=sci_arttext&pid=S2318-03312020000100404en10.1590/2318-0331.252020190155 |
| institution |
SCIELO |
| collection |
OJS |
| country |
Brasil |
| countrycode |
BR |
| component |
Revista |
| access |
En linea |
| databasecode |
rev-scielo-br |
| tag |
revista |
| region |
America del Sur |
| libraryname |
SciELO |
| language |
English |
| format |
Digital |
| author |
Ferreira,Paloma Mara de Lima Paz,Adriano Rolim da Bravo,Juan Martín |
| spellingShingle |
Ferreira,Paloma Mara de Lima Paz,Adriano Rolim da Bravo,Juan Martín Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis |
| author_facet |
Ferreira,Paloma Mara de Lima Paz,Adriano Rolim da Bravo,Juan Martín |
| author_sort |
Ferreira,Paloma Mara de Lima |
| title |
Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis |
| title_short |
Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis |
| title_full |
Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis |
| title_fullStr |
Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis |
| title_full_unstemmed |
Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis |
| title_sort |
objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis |
| description |
ABSTRACT Hydrological models (HMs) can be applied for different purposes, and a key step is model calibration using objective functions (OF) to quantify the agreement between observed and calculated discharges. Fully understanding the OF is important to properly take advantage of model calibration and interpret the results. This study evaluates 36 OF proposed in the literature, considering two watersheds of different hydrological regimes. Daily simulated streamflow time-series, using a distributed hydrological model (MGB-IPH), and ten daily streamflow synthetic time-series, generated from the observed and calculated streamflows, were used in the analysis of each watershed. These synthetic data were used to evaluate how does each metric evaluate hypothetical cases that present isolated very well known error behaviors. Despite of all NSE-derived (Nash-Sutcliffe efficiency) metrics that use the square of the residuals in their formulation have shown higher sensitivity to errors in high flows, the ones that use daily and monthly averages of flow rates in absolute terms were more stringent than the others to assess HMs performance. Low flow errors were better evaluated by metrics that use the flow logarithm. The constant presence of zero flow rates deteriorate them significantly, with the exception of the metrics TRMSE (Transformed root mean square error) did not demonstrate this problem. An observed limitation of the formulations of some metrics was that the errors of overestimation or underestimation are compensated. Our results reassert that each metric should be interpreted specifically thinking about the aspects it has been proposed for, and simultaneously taking into account a set of metrics would lead to a broader evaluation of HM ability (e.g. multiobjective model evaluation). We recommend that the use of synthetic time series as those proposed in this work could be useful as an auxiliary step towards better understanding the evaluation of a calibrated hydrological model for each study case, taking into account model capabilities and observed hydrologic regime characteristics. |
| publisher |
Associação Brasileira de Recursos Hídricos |
| publishDate |
2020 |
| url |
http://old.scielo.br/scielo.php?script=sci_arttext&pid=S2318-03312020000100404 |
| work_keys_str_mv |
AT ferreirapalomamaradelima objectivefunctionsusedasperformancemetricsforhydrologicalmodelsstateoftheartandcriticalanalysis AT pazadrianorolimda objectivefunctionsusedasperformancemetricsforhydrologicalmodelsstateoftheartandcriticalanalysis AT bravojuanmartin objectivefunctionsusedasperformancemetricsforhydrologicalmodelsstateoftheartandcriticalanalysis |
| _version_ |
1756441152309952512 |