Calibration and validation of land-use models

Land use is constantly changing. For example, urban areas expand as a result of population growth, cropping patterns change to fulfil the demand for bioenergy and natural vegetation recovers in locations where farmers cease to farm. Understanding these changes is pivotal to explore future land-use scenarios and to design spatial policies. Land-use models are increasingly being used for these purposes. They function as virtual laboratories in which scientists or policy analysts can conduct experiments. In order to reliably apply models for these purposes, they need to be calibrated, where calibration is the adjustment of parameters to improve the model’s performance. Consequently, the value of modelled land-use scenarios and policy assessments depends on the quality of the calibration. Assessment of the quality of the calibration is termed validation, and is ideally performed independently in the sense that the data that is used for validation has not been used for calibration. The development of a land-use model can be described by four sequential phases: analysis and conceptual modelling, computer programming of the conceptual model, calibration of the computerized model, and experimentation with the calibrated model. The first three phases all have their own evaluation procedures: conceptual validation, code verification and operational validation, respectively. Operational validation provides insights into the strengths and weaknesses of a particular model application, and sometimes it can suggest directions for improvement. However, available assessment methods have limitations for their application in land-use modelling. Therefore, there is a demand to develop and apply more appropriate methods. The work presented in this thesis first considers the properties of land-use models that are important for their assessment, and subsequently presents and applies several methods that can be used for this assessment. Many land-use changes are directly or indirectly the result of human decisions. However, human decisions are inherently uncertain, and therefore land-use models cannot be expected to simulate these land-use changes exactly. This is acknowledged by many land-use models as they use a stochastic component to simulate land-use changes. Therefore, land-use models should not only be validated on their predictive accuracy, their capacity to accurately allocate land-use changes on the map, but also on their process accuracy, their capacity to realistically simulate land-use change processes. Moreover, many models start from an initial land-use map and simulate changes relative to this map. The amount of change for a simulation is typically small relative to the entire map, which means that a large part of the result is caused by persistence. For this reason, a benchmark, such as a naive predictor, is required to properly asses the accuracy of simulation results. This benchmark can be implicit to the assessment method itself, or explicit, i.e. another land-use model which serves as a reference model. Outperforming the benchmark can be considered a minimum threshold to pass; however, it cannot directly inform whether a model is acceptable as this depends on the purpose of the model, the requirements of the study and the application domain. The predictive accuracy of a land-use model is typically assessed by comparing a simulation result with the actual land-use map at the end of a simulation. A common method for this is the Kappa statistic, which expresses the agreement between two land-use maps corrected for the expected agreement from a random allocation given the distribution of class sizes. However, this is not an appropriate reference level to assess the predictive accuracy of land-use models, because it does not account for the amount of change. This thesis presents Kappa Simulation, a new map-comparison method that is identical in form to Kappa, but which instead applies a more appropriate reference model based on random allocation of class transitions relative to the initial map. This implicitly accounts for the amount of change, which truly allows gauging the predictive accuracy of changes in land-use models. However, Kappa Simulation cannot differentiate between near-hits and complete misses, while this distinction is often very relevant for land-use modellers. This thesis therefore presents Fuzzy Kappa Simulation. This statistic is an improvement of Kappa Simulation, as it applies a fuzzy interpretation of class transitions and their locations. This means that it can account for near-hits, which makes it arguably the most suitable map comparison method to assess the predictive accuracy of land-use models. Because of the intrinsic uncertainty underlying land-use change processes, a realistic land-use model does not necessarily generate a high predictive accuracy, which justifies a separate assessment of its process accuracy. Ideally, process accuracy is assessed directly from the values of model parameters. However, it is often impossible to observe real-world values for these parameters because drivers for land-use changes are correlated or they cannot be measured. Therefore, the process accuracy is typically assessed indirectly from the land-use patterns generated by the model. Two groups of methods exist to characterize land-use patterns: landscape metrics and fractal metrics. Landscape metrics are a collection of algorithms that have been applied in landscape ecology to characterize land-use patterns. In this thesis, landscape metrics have been used to compare the simulated land-use map with the observed land-use map instead. Fractal metrics, which have their origin in complexity science, are another type of measures to characterize regularities in (urban) land-use patterns. Examples are power-law distributions for urban clusters and fractal dimensions of patches of urban land. Moreover, fractal metrics can be interpreted in absolute terms since they represent general regularities in urban systems for which values are known from literature. Therefore, fractal metrics also allow evaluation of the process accuracy of a synthetic application for which no observed land-use pattern is available for comparison. Neighbourhood rules represent the influence of the existing land-use distribution on the location of land-use changes. This includes the persistence, conversion and attraction/repulsion of land uses in the neighbourhood of a location. Because neighbourhood rules cannot be estimated directly from data, they need to be set in a calibration procedure. The work in this thesis indicates that agents consider their neighbourhood at different spatial scales: the direct vicinity of a location has a strong influence on the allocation of new urban land, but neighbourhood rules over larger distance – typically the size of urban regions – also improve the model performance. This thesis also discusses a special type of neighbourhood rules: rules describing the influence of the existing activity distribution on the allocation of activity changes, where activities denote a quantity or density related to a land use, such as inhabitants for residential land use. This study shows that relatively simple rules can grow a realistic urban settlement structure, which confirms that neighbourhood rules improve the process accuracy of a land-use model. It should be noted that while the methods presented and applied in this thesis are objective, the selection of assessment methods remains subjective. Moreover, because no method is yet capable of describing land-use patterns satisfactorily, more subjective methods such as visual assessment of simulation results or interpretation of parameter values remain of added value in the calibration and validation of land-use models.