Quality control and filling of daily temperature and precipitation time series in Colombia

During the last several years, there has been an increase in the monitoring of meteorological phenomena. For accurate analysis of the spatial and temporal variability of precipitation as well as air temperature, recorded observations must be trustworthy. Therefore, one must implement effective methods for assuring data quality. This paper implements meteorological data quality controls using open-source Python code. We implemented six types of automatic quality-control processes for climatic data series. This work also introduces a quality index for meteorological time series data and uses three-dimensional kriging to fill in missing data. We apply these methods to daily historical data from 1980 to 2019 from the Institute of Hydrology, Meteorology, and Environmental Studies of Colombia. Applying the quality-control process to the available meteorological stations allows us to validate more than 90 % of the time series. We find that approximately 6.4 %, 4.4 %, 5.4 %, and 5.5 % of the data are flagged as atypical values in the time series of minimum temperature, mean temperature, maximum temperature, and precipitation, respectively. We verified the accuracy of the quality-control procedures by introducing multiplicative random errors and computing the probabilities of false positive and false negative errors. This procedure emphasizes the relationships between neighboring meteorological stations to detect errors in the data, even if they belong to different climatic regions. This analysis also implements a method of reconstructing missing data in the absence of a trustworthy reference time series.

Saved in:
Bibliographic Details
Main Authors: Teran Chaves, Cesar Augusto, Duarte Carvajalino, Julio Martin, Polo Murcia, Sonia Mercedes
Format: article biblioteca
Language:eng
Published: Schweizerbart'sche Verlagsbuchhandlung 2021-07-02
Subjects:Recursos hídricos y su ordenación - P10, Clima, Análisis espacial, Recursos hídricos, Transversal, http://aims.fao.org/aos/agrovoc/c_1665, http://aims.fao.org/aos/agrovoc/c_40da9d3b, http://aims.fao.org/aos/agrovoc/c_8325,
Online Access:https://www.schweizerbart.de/papers/metz/detail/30/100139/Quality_control_and_filling_of_daily_temperature_and_precipitation_time_series_in_Colombia
http://hdl.handle.net/20.500.12324/38927
Tags: Add Tag
No Tags, Be the first to tag this record!
id dig-bac-20.500.12324-38927
record_format koha
institution AGROSAVIA
collection DSpace
country Colombia
countrycode CO
component Bibliográfico
access En linea
databasecode dig-bac
tag biblioteca
region America del Sur
libraryname Biblioteca Agropecuaria de Colombia
language eng
topic Recursos hídricos y su ordenación - P10
Clima
Análisis espacial
Recursos hídricos
Transversal
http://aims.fao.org/aos/agrovoc/c_1665
http://aims.fao.org/aos/agrovoc/c_40da9d3b
http://aims.fao.org/aos/agrovoc/c_8325
Recursos hídricos y su ordenación - P10
Clima
Análisis espacial
Recursos hídricos
Transversal
http://aims.fao.org/aos/agrovoc/c_1665
http://aims.fao.org/aos/agrovoc/c_40da9d3b
http://aims.fao.org/aos/agrovoc/c_8325
spellingShingle Recursos hídricos y su ordenación - P10
Clima
Análisis espacial
Recursos hídricos
Transversal
http://aims.fao.org/aos/agrovoc/c_1665
http://aims.fao.org/aos/agrovoc/c_40da9d3b
http://aims.fao.org/aos/agrovoc/c_8325
Recursos hídricos y su ordenación - P10
Clima
Análisis espacial
Recursos hídricos
Transversal
http://aims.fao.org/aos/agrovoc/c_1665
http://aims.fao.org/aos/agrovoc/c_40da9d3b
http://aims.fao.org/aos/agrovoc/c_8325
Teran Chaves, Cesar Augusto
Duarte Carvajalino, Julio Martin
Polo Murcia, Sonia Mercedes
Quality control and filling of daily temperature and precipitation time series in Colombia
description During the last several years, there has been an increase in the monitoring of meteorological phenomena. For accurate analysis of the spatial and temporal variability of precipitation as well as air temperature, recorded observations must be trustworthy. Therefore, one must implement effective methods for assuring data quality. This paper implements meteorological data quality controls using open-source Python code. We implemented six types of automatic quality-control processes for climatic data series. This work also introduces a quality index for meteorological time series data and uses three-dimensional kriging to fill in missing data. We apply these methods to daily historical data from 1980 to 2019 from the Institute of Hydrology, Meteorology, and Environmental Studies of Colombia. Applying the quality-control process to the available meteorological stations allows us to validate more than 90 % of the time series. We find that approximately 6.4 %, 4.4 %, 5.4 %, and 5.5 % of the data are flagged as atypical values in the time series of minimum temperature, mean temperature, maximum temperature, and precipitation, respectively. We verified the accuracy of the quality-control procedures by introducing multiplicative random errors and computing the probabilities of false positive and false negative errors. This procedure emphasizes the relationships between neighboring meteorological stations to detect errors in the data, even if they belong to different climatic regions. This analysis also implements a method of reconstructing missing data in the absence of a trustworthy reference time series.
format article
topic_facet Recursos hídricos y su ordenación - P10
Clima
Análisis espacial
Recursos hídricos
Transversal
http://aims.fao.org/aos/agrovoc/c_1665
http://aims.fao.org/aos/agrovoc/c_40da9d3b
http://aims.fao.org/aos/agrovoc/c_8325
author Teran Chaves, Cesar Augusto
Duarte Carvajalino, Julio Martin
Polo Murcia, Sonia Mercedes
author_facet Teran Chaves, Cesar Augusto
Duarte Carvajalino, Julio Martin
Polo Murcia, Sonia Mercedes
author_sort Teran Chaves, Cesar Augusto
title Quality control and filling of daily temperature and precipitation time series in Colombia
title_short Quality control and filling of daily temperature and precipitation time series in Colombia
title_full Quality control and filling of daily temperature and precipitation time series in Colombia
title_fullStr Quality control and filling of daily temperature and precipitation time series in Colombia
title_full_unstemmed Quality control and filling of daily temperature and precipitation time series in Colombia
title_sort quality control and filling of daily temperature and precipitation time series in colombia
publisher Schweizerbart'sche Verlagsbuchhandlung
publishDate 2021-07-02
url https://www.schweizerbart.de/papers/metz/detail/30/100139/Quality_control_and_filling_of_daily_temperature_and_precipitation_time_series_in_Colombia
http://hdl.handle.net/20.500.12324/38927
work_keys_str_mv AT teranchavescesaraugusto qualitycontrolandfillingofdailytemperatureandprecipitationtimeseriesincolombia
AT duartecarvajalinojuliomartin qualitycontrolandfillingofdailytemperatureandprecipitationtimeseriesincolombia
AT polomurciasoniamercedes qualitycontrolandfillingofdailytemperatureandprecipitationtimeseriesincolombia
_version_ 1792484183643258880
spelling dig-bac-20.500.12324-389272024-02-23T03:01:58Z Quality control and filling of daily temperature and precipitation time series in Colombia Teran Chaves, Cesar Augusto Duarte Carvajalino, Julio Martin Polo Murcia, Sonia Mercedes Recursos hídricos y su ordenación - P10 Clima Análisis espacial Recursos hídricos Transversal http://aims.fao.org/aos/agrovoc/c_1665 http://aims.fao.org/aos/agrovoc/c_40da9d3b http://aims.fao.org/aos/agrovoc/c_8325 During the last several years, there has been an increase in the monitoring of meteorological phenomena. For accurate analysis of the spatial and temporal variability of precipitation as well as air temperature, recorded observations must be trustworthy. Therefore, one must implement effective methods for assuring data quality. This paper implements meteorological data quality controls using open-source Python code. We implemented six types of automatic quality-control processes for climatic data series. This work also introduces a quality index for meteorological time series data and uses three-dimensional kriging to fill in missing data. We apply these methods to daily historical data from 1980 to 2019 from the Institute of Hydrology, Meteorology, and Environmental Studies of Colombia. Applying the quality-control process to the available meteorological stations allows us to validate more than 90 % of the time series. We find that approximately 6.4 %, 4.4 %, 5.4 %, and 5.5 % of the data are flagged as atypical values in the time series of minimum temperature, mean temperature, maximum temperature, and precipitation, respectively. We verified the accuracy of the quality-control procedures by introducing multiplicative random errors and computing the probabilities of false positive and false negative errors. This procedure emphasizes the relationships between neighboring meteorological stations to detect errors in the data, even if they belong to different climatic regions. This analysis also implements a method of reconstructing missing data in the absence of a trustworthy reference time series. 2024-02-22T20:51:20Z 2024-02-22T20:51:20Z 2021-07-02 2021 article Artículo científico http://purl.org/coar/resource_type/c_2df8fbb1 info:eu-repo/semantics/article https://purl.org/redcol/resource_type/ART http://purl.org/coar/version/c_970fb48d4fbd8a85 https://www.schweizerbart.de/papers/metz/detail/30/100139/Quality_control_and_filling_of_daily_temperature_and_precipitation_time_series_in_Colombia 0026-1211 http://hdl.handle.net/20.500.12324/38927 10.1127/metz/2021/1077 reponame:Biblioteca Digital Agropecuaria de Colombia instname:Corporación colombiana de investigación agropecuaria AGROSAVIA eng Meteorologische Zeitschrift 30 6 489 501 Aieb, A., K. Madani, M. Scapra, B. Bonaccorso, K. Lef sih, 2019: A new approach for processing climate miss ing databases applied to daily rainfall data in Soummam watershed, Algeria. – Heliyon. 5, e01247, DOI:10.1016/ j.heliyon.2019.e01247. Cao, L., Y. Zhu, G. Tang, F. Yuan, Z. Yan, 2016: Climatic warming in China according to a homogenized data set from 2419 stations. – Int. J. Climatol. 36, 4384–4392. DOI:10.1002/ joc.4639. Durre, I., M.J. Menne, B.E. Gleason, T.G. Houston, R.S. Vose, 2010: Comprehensive Automated Quality Assur ance of Daily Surface Observations. – J. Appl. Meteor. Clima tol. 49, 1615–1633. DOI:10.1175/2010JAMC2375.1 Feng, S., Q. Hu, W. Qian, 2004: Quality control of daily me teorological data in China, 1951–2000: a new dataset. – Int. J. Climatol. 24, 853–870. DOI:10.1002/joc.1047. Gentilucci, M., M. Barbieri, P. Burt, F. D’aprile, 2018: Preliminary Data Validation and Reconstruction of Temper ature and Precipitation in Central Italy. – Geosci. 8, 202. DOI: 10.3390/geosciences8060202. Grzywna, A., A. Bochniak, A. Ziernicka-Wojtaszek, J. Kruzel, K. Jozwiakowski, A. Walaga, A. Cupak, A. Mazur, R. Obroslak, A. Serafin, 2020: The analysis of spatial variability of precipitation in Poland in the multi years 1981–2010. – J. Water Land Dev. 46, 105–111. DOI: 10.24425/jwld.2020.134202. Guan, H., J.L. Wilson, O. Makhnin, 2005: Geostatistical Map ping of Mountain Precipitation Incorporating Autosearched Effects of Terrain and Climatic Characteristics. – J. Hydrome teorol. 6, 1018–1031. DOI:10.1175/jhm448.1. Hoyos, I., B.A Rodríguez, 2020: Drawing the complexity of Colombian climate from non-extensive extreme behav ior. – Phys. A: Stat. Mech. Appl. 548, 123673. DOI:10.1016/ j.physa.2019.123673. Hubbard, K., J. You, M. Shulski, 2012: Toward a Better Qual ity Control of Weather Data. – In: Nezhad, M.S.F. (Ed.): Practical Concepts of Quality Control. DOI:10.5772/51632. Kim, J., J.H. Ryu, 2016: A Heuristic Gap Filling Method for Daily Precipitation Series. – Water Resour. Manag. 30, 2275–2294. DOI:10.1007/s11269-016-1284-z. Lanzante, J.R, 1996: Resistant, robust and non-parametric techniques for the analysis of climate data: theory and ex amples, including applications to historical radiosonde station data. – Int. J. Climatol. 16,1197–1226. DOI:10.1002/(SICI) 1097-0088(199611)16:11<1197:AID-JOC89>3.0.CO;2-L. Libertino, A., P. Allamano, F. Laio, P. Claps, 2018: Regional-scale analysis of extreme precipitation from short and fragmented records. Adv. Water Resour. 112, 147–159. DOI:10.1016/j.advwatres.2017.12.015. Llabrés-Brustenga, A., A. Rius, R. Rodríguez-Solá, M.C. Casas-Castillo, A. Redaño, 2019: Quality control process of the daily rainfall series available in Catalonia from 1855 to the present. – Theor. Appl. Climatol. 137, 2715–2729. DOI:10.1007/s00704-019-02772-5. Mahmood, R., S. Jia, 2016: Quality control and homogenization of daily meteorological data in the trans-boundary region of the Jhelum River basin. – J. Geogr. Sci. 26, 1661–1674. DOI: 10.1007/s11442-016-1351-7. Pérez, A., J. Ortiz, L. Bejarano, D. Otero D, L. Re strepo, H. Franco, 2018: Sea breeze in the Colombian Caribbean coast. – Atmósfera 31, 389–406. DOI:10.20937/ ATM.2018.31.04.06. Poveda, G., L. Jaramillo, L.F. Valljo, 2014: Seasonal precip itation patterns along pathways of South American low-level jets and aerial rivers. – Water Resour. Res. 50, 98–118. DOI: 10.1002/2013WR014087. Rühaak, W., 2015: 3-D interpolation of subsurface tempera ture data with measurement error using kriging. – Env. Earth Sci.73. 1893–1900. DOI:10.1007/s12665-014-3554-5. Sciuto, G., B. Bonaccorso, A. Cancelliere, G. Rossi, 2009: Quality control of daily rainfall data with neural networks. – J. Hydrol. 364, 13–22. DOI:10.1016/j.jhydrol.2008.10.008. Serrano-Notivoli, R., S. Beguería, M. Saz, L. Longares, M. De Luis, 2017: Spread: a high-resolution daily gridded precipitation dataset for Spain – An extreme events frequency and intensity overview. – J. Earth Syst. Sci. 9, 721–738. DOI: 10.5194/essd-9-721-2017. Štìpánek, P., P. Zahradníèek, P. Skalák, 2009: Data quality control and homogenization of air temperature and precip itation series in the area of the Czech Republic in the pe riod 1961–2007. – Adv. Sci. Res. 3, 23–26. DOI:10.5194/ asr-3-23-2009. Trewin, B., 2012: A daily homogenized temperature data set for Australia. – Int. J. Climatol. 33, 1510–1529. DOI:10.1002/ joc.3530. Veiga, H., N. Herrera, M. Skansi, G. Podestá, 2014: De scripción de controles de calidad de datos climáticos diarios implementados por el Centro Regional del Clima para el Sur de América del Sur, [Description of quality controls of daily climatic data implemented by the weather regional center for the south of south America], 2014. – Technical Report CRC SAS-2014-001, 54 p. https://www.crc-sas.org/es/pdf/reporte_ tecnico_CRC-SAS-2014-001.pdf Attribution-ShareAlike 4.0 International http://creativecommons.org/licenses/by-sa/4.0/ application/pdf application/pdf Schweizerbart'sche Verlagsbuchhandlung Stuttgart (Alemania) Meteorologische Zeitschrift; Vol. 30, Núm. 6 (2021): Meteorologische Zeitschrift (Julio);p. 489 -501.