Quality control and filling of daily temperature and precipitation time series in Colombia
During the last several years, there has been an increase in the monitoring of meteorological phenomena. For accurate analysis of the spatial and temporal variability of precipitation as well as air temperature, recorded observations must be trustworthy. Therefore, one must implement effective methods for assuring data quality. This paper implements meteorological data quality controls using open-source Python code. We implemented six types of automatic quality-control processes for climatic data series. This work also introduces a quality index for meteorological time series data and uses three-dimensional kriging to fill in missing data. We apply these methods to daily historical data from 1980 to 2019 from the Institute of Hydrology, Meteorology, and Environmental Studies of Colombia. Applying the quality-control process to the available meteorological stations allows us to validate more than 90 % of the time series. We find that approximately 6.4 %, 4.4 %, 5.4 %, and 5.5 % of the data are flagged as atypical values in the time series of minimum temperature, mean temperature, maximum temperature, and precipitation, respectively. We verified the accuracy of the quality-control procedures by introducing multiplicative random errors and computing the probabilities of false positive and false negative errors. This procedure emphasizes the relationships between neighboring meteorological stations to detect errors in the data, even if they belong to different climatic regions. This analysis also implements a method of reconstructing missing data in the absence of a trustworthy reference time series.
Main Authors: | , , |
---|---|
Format: | article biblioteca |
Language: | eng |
Published: |
Schweizerbart'sche Verlagsbuchhandlung
2021-07-02
|
Subjects: | Recursos hídricos y su ordenación - P10, Clima, Análisis espacial, Recursos hídricos, Transversal, http://aims.fao.org/aos/agrovoc/c_1665, http://aims.fao.org/aos/agrovoc/c_40da9d3b, http://aims.fao.org/aos/agrovoc/c_8325, |
Online Access: | https://www.schweizerbart.de/papers/metz/detail/30/100139/Quality_control_and_filling_of_daily_temperature_and_precipitation_time_series_in_Colombia http://hdl.handle.net/20.500.12324/38927 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
id |
dig-bac-20.500.12324-38927 |
---|---|
record_format |
koha |
institution |
AGROSAVIA |
collection |
DSpace |
country |
Colombia |
countrycode |
CO |
component |
Bibliográfico |
access |
En linea |
databasecode |
dig-bac |
tag |
biblioteca |
region |
America del Sur |
libraryname |
Biblioteca Agropecuaria de Colombia |
language |
eng |
topic |
Recursos hídricos y su ordenación - P10 Clima Análisis espacial Recursos hídricos Transversal http://aims.fao.org/aos/agrovoc/c_1665 http://aims.fao.org/aos/agrovoc/c_40da9d3b http://aims.fao.org/aos/agrovoc/c_8325 Recursos hídricos y su ordenación - P10 Clima Análisis espacial Recursos hídricos Transversal http://aims.fao.org/aos/agrovoc/c_1665 http://aims.fao.org/aos/agrovoc/c_40da9d3b http://aims.fao.org/aos/agrovoc/c_8325 |
spellingShingle |
Recursos hídricos y su ordenación - P10 Clima Análisis espacial Recursos hídricos Transversal http://aims.fao.org/aos/agrovoc/c_1665 http://aims.fao.org/aos/agrovoc/c_40da9d3b http://aims.fao.org/aos/agrovoc/c_8325 Recursos hídricos y su ordenación - P10 Clima Análisis espacial Recursos hídricos Transversal http://aims.fao.org/aos/agrovoc/c_1665 http://aims.fao.org/aos/agrovoc/c_40da9d3b http://aims.fao.org/aos/agrovoc/c_8325 Teran Chaves, Cesar Augusto Duarte Carvajalino, Julio Martin Polo Murcia, Sonia Mercedes Quality control and filling of daily temperature and precipitation time series in Colombia |
description |
During the last several years, there has been an increase in the monitoring of meteorological phenomena. For accurate analysis of the spatial and temporal variability of precipitation as well as air temperature, recorded observations must be trustworthy. Therefore, one must implement effective methods for assuring data quality. This paper implements meteorological data quality controls using open-source Python code. We implemented six types of automatic quality-control processes for climatic data series. This work also introduces a quality index for meteorological time series data and uses three-dimensional kriging to fill in missing data. We apply these methods to daily historical data from 1980 to 2019 from the Institute of Hydrology, Meteorology, and Environmental Studies of Colombia. Applying the quality-control process to the available meteorological stations allows us to validate more than 90 % of the time series. We find that approximately 6.4 %, 4.4 %, 5.4 %, and 5.5 % of the data are flagged as atypical values in the time series of minimum temperature, mean temperature, maximum temperature, and precipitation, respectively. We verified the accuracy of the quality-control procedures by introducing multiplicative random errors and computing the probabilities of false positive and false negative errors. This procedure emphasizes the relationships between neighboring meteorological stations to detect errors in the data, even if they belong to different climatic regions. This analysis also implements a method of reconstructing missing data in the absence of a trustworthy reference time series. |
format |
article |
topic_facet |
Recursos hídricos y su ordenación - P10 Clima Análisis espacial Recursos hídricos Transversal http://aims.fao.org/aos/agrovoc/c_1665 http://aims.fao.org/aos/agrovoc/c_40da9d3b http://aims.fao.org/aos/agrovoc/c_8325 |
author |
Teran Chaves, Cesar Augusto Duarte Carvajalino, Julio Martin Polo Murcia, Sonia Mercedes |
author_facet |
Teran Chaves, Cesar Augusto Duarte Carvajalino, Julio Martin Polo Murcia, Sonia Mercedes |
author_sort |
Teran Chaves, Cesar Augusto |
title |
Quality control and filling of daily temperature and precipitation time series in Colombia |
title_short |
Quality control and filling of daily temperature and precipitation time series in Colombia |
title_full |
Quality control and filling of daily temperature and precipitation time series in Colombia |
title_fullStr |
Quality control and filling of daily temperature and precipitation time series in Colombia |
title_full_unstemmed |
Quality control and filling of daily temperature and precipitation time series in Colombia |
title_sort |
quality control and filling of daily temperature and precipitation time series in colombia |
publisher |
Schweizerbart'sche Verlagsbuchhandlung |
publishDate |
2021-07-02 |
url |
https://www.schweizerbart.de/papers/metz/detail/30/100139/Quality_control_and_filling_of_daily_temperature_and_precipitation_time_series_in_Colombia http://hdl.handle.net/20.500.12324/38927 |
work_keys_str_mv |
AT teranchavescesaraugusto qualitycontrolandfillingofdailytemperatureandprecipitationtimeseriesincolombia AT duartecarvajalinojuliomartin qualitycontrolandfillingofdailytemperatureandprecipitationtimeseriesincolombia AT polomurciasoniamercedes qualitycontrolandfillingofdailytemperatureandprecipitationtimeseriesincolombia |
_version_ |
1792484183643258880 |
spelling |
dig-bac-20.500.12324-389272024-02-23T03:01:58Z Quality control and filling of daily temperature and precipitation time series in Colombia Teran Chaves, Cesar Augusto Duarte Carvajalino, Julio Martin Polo Murcia, Sonia Mercedes Recursos hídricos y su ordenación - P10 Clima Análisis espacial Recursos hídricos Transversal http://aims.fao.org/aos/agrovoc/c_1665 http://aims.fao.org/aos/agrovoc/c_40da9d3b http://aims.fao.org/aos/agrovoc/c_8325 During the last several years, there has been an increase in the monitoring of meteorological phenomena. For accurate analysis of the spatial and temporal variability of precipitation as well as air temperature, recorded observations must be trustworthy. Therefore, one must implement effective methods for assuring data quality. This paper implements meteorological data quality controls using open-source Python code. We implemented six types of automatic quality-control processes for climatic data series. This work also introduces a quality index for meteorological time series data and uses three-dimensional kriging to fill in missing data. We apply these methods to daily historical data from 1980 to 2019 from the Institute of Hydrology, Meteorology, and Environmental Studies of Colombia. Applying the quality-control process to the available meteorological stations allows us to validate more than 90 % of the time series. We find that approximately 6.4 %, 4.4 %, 5.4 %, and 5.5 % of the data are flagged as atypical values in the time series of minimum temperature, mean temperature, maximum temperature, and precipitation, respectively. We verified the accuracy of the quality-control procedures by introducing multiplicative random errors and computing the probabilities of false positive and false negative errors. This procedure emphasizes the relationships between neighboring meteorological stations to detect errors in the data, even if they belong to different climatic regions. This analysis also implements a method of reconstructing missing data in the absence of a trustworthy reference time series. 2024-02-22T20:51:20Z 2024-02-22T20:51:20Z 2021-07-02 2021 article Artículo científico http://purl.org/coar/resource_type/c_2df8fbb1 info:eu-repo/semantics/article https://purl.org/redcol/resource_type/ART http://purl.org/coar/version/c_970fb48d4fbd8a85 https://www.schweizerbart.de/papers/metz/detail/30/100139/Quality_control_and_filling_of_daily_temperature_and_precipitation_time_series_in_Colombia 0026-1211 http://hdl.handle.net/20.500.12324/38927 10.1127/metz/2021/1077 reponame:Biblioteca Digital Agropecuaria de Colombia instname:Corporación colombiana de investigación agropecuaria AGROSAVIA eng Meteorologische Zeitschrift 30 6 489 501 Aieb, A., K. Madani, M. Scapra, B. Bonaccorso, K. Lef sih, 2019: A new approach for processing climate miss ing databases applied to daily rainfall data in Soummam watershed, Algeria. – Heliyon. 5, e01247, DOI:10.1016/ j.heliyon.2019.e01247. Cao, L., Y. Zhu, G. Tang, F. Yuan, Z. Yan, 2016: Climatic warming in China according to a homogenized data set from 2419 stations. – Int. J. Climatol. 36, 4384–4392. DOI:10.1002/ joc.4639. Durre, I., M.J. Menne, B.E. Gleason, T.G. Houston, R.S. Vose, 2010: Comprehensive Automated Quality Assur ance of Daily Surface Observations. – J. Appl. Meteor. Clima tol. 49, 1615–1633. DOI:10.1175/2010JAMC2375.1 Feng, S., Q. Hu, W. Qian, 2004: Quality control of daily me teorological data in China, 1951–2000: a new dataset. – Int. J. Climatol. 24, 853–870. DOI:10.1002/joc.1047. Gentilucci, M., M. Barbieri, P. Burt, F. D’aprile, 2018: Preliminary Data Validation and Reconstruction of Temper ature and Precipitation in Central Italy. – Geosci. 8, 202. DOI: 10.3390/geosciences8060202. Grzywna, A., A. Bochniak, A. Ziernicka-Wojtaszek, J. Kruzel, K. Jozwiakowski, A. Walaga, A. Cupak, A. Mazur, R. Obroslak, A. Serafin, 2020: The analysis of spatial variability of precipitation in Poland in the multi years 1981–2010. – J. Water Land Dev. 46, 105–111. DOI: 10.24425/jwld.2020.134202. Guan, H., J.L. Wilson, O. Makhnin, 2005: Geostatistical Map ping of Mountain Precipitation Incorporating Autosearched Effects of Terrain and Climatic Characteristics. – J. Hydrome teorol. 6, 1018–1031. DOI:10.1175/jhm448.1. Hoyos, I., B.A Rodríguez, 2020: Drawing the complexity of Colombian climate from non-extensive extreme behav ior. – Phys. A: Stat. Mech. Appl. 548, 123673. DOI:10.1016/ j.physa.2019.123673. Hubbard, K., J. You, M. Shulski, 2012: Toward a Better Qual ity Control of Weather Data. – In: Nezhad, M.S.F. (Ed.): Practical Concepts of Quality Control. DOI:10.5772/51632. Kim, J., J.H. Ryu, 2016: A Heuristic Gap Filling Method for Daily Precipitation Series. – Water Resour. Manag. 30, 2275–2294. DOI:10.1007/s11269-016-1284-z. Lanzante, J.R, 1996: Resistant, robust and non-parametric techniques for the analysis of climate data: theory and ex amples, including applications to historical radiosonde station data. – Int. J. Climatol. 16,1197–1226. DOI:10.1002/(SICI) 1097-0088(199611)16:11<1197:AID-JOC89>3.0.CO;2-L. Libertino, A., P. Allamano, F. Laio, P. Claps, 2018: Regional-scale analysis of extreme precipitation from short and fragmented records. Adv. Water Resour. 112, 147–159. DOI:10.1016/j.advwatres.2017.12.015. Llabrés-Brustenga, A., A. Rius, R. Rodríguez-Solá, M.C. Casas-Castillo, A. Redaño, 2019: Quality control process of the daily rainfall series available in Catalonia from 1855 to the present. – Theor. Appl. Climatol. 137, 2715–2729. DOI:10.1007/s00704-019-02772-5. Mahmood, R., S. Jia, 2016: Quality control and homogenization of daily meteorological data in the trans-boundary region of the Jhelum River basin. – J. Geogr. Sci. 26, 1661–1674. DOI: 10.1007/s11442-016-1351-7. Pérez, A., J. Ortiz, L. Bejarano, D. Otero D, L. Re strepo, H. Franco, 2018: Sea breeze in the Colombian Caribbean coast. – Atmósfera 31, 389–406. DOI:10.20937/ ATM.2018.31.04.06. Poveda, G., L. Jaramillo, L.F. Valljo, 2014: Seasonal precip itation patterns along pathways of South American low-level jets and aerial rivers. – Water Resour. Res. 50, 98–118. DOI: 10.1002/2013WR014087. Rühaak, W., 2015: 3-D interpolation of subsurface tempera ture data with measurement error using kriging. – Env. Earth Sci.73. 1893–1900. DOI:10.1007/s12665-014-3554-5. Sciuto, G., B. Bonaccorso, A. Cancelliere, G. Rossi, 2009: Quality control of daily rainfall data with neural networks. – J. Hydrol. 364, 13–22. DOI:10.1016/j.jhydrol.2008.10.008. Serrano-Notivoli, R., S. Beguería, M. Saz, L. Longares, M. De Luis, 2017: Spread: a high-resolution daily gridded precipitation dataset for Spain – An extreme events frequency and intensity overview. – J. Earth Syst. Sci. 9, 721–738. DOI: 10.5194/essd-9-721-2017. Štìpánek, P., P. Zahradníèek, P. Skalák, 2009: Data quality control and homogenization of air temperature and precip itation series in the area of the Czech Republic in the pe riod 1961–2007. – Adv. Sci. Res. 3, 23–26. DOI:10.5194/ asr-3-23-2009. Trewin, B., 2012: A daily homogenized temperature data set for Australia. – Int. J. Climatol. 33, 1510–1529. DOI:10.1002/ joc.3530. Veiga, H., N. Herrera, M. Skansi, G. Podestá, 2014: De scripción de controles de calidad de datos climáticos diarios implementados por el Centro Regional del Clima para el Sur de América del Sur, [Description of quality controls of daily climatic data implemented by the weather regional center for the south of south America], 2014. – Technical Report CRC SAS-2014-001, 54 p. https://www.crc-sas.org/es/pdf/reporte_ tecnico_CRC-SAS-2014-001.pdf Attribution-ShareAlike 4.0 International http://creativecommons.org/licenses/by-sa/4.0/ application/pdf application/pdf Schweizerbart'sche Verlagsbuchhandlung Stuttgart (Alemania) Meteorologische Zeitschrift; Vol. 30, Núm. 6 (2021): Meteorologische Zeitschrift (Julio);p. 489 -501. |