Determining the Range of Predictions for Calibrated Agent-based Simulation Models.

Shi, DongFang (2008) Determining the Range of Predictions for Calibrated Agent-based Simulation Models. PhD thesis, Lancaster University.

PDF (11003739.pdf)
11003739.pdf - Published Version
Available under License Creative Commons Attribution-NoDerivs.
Download (9MB)

Abstract

Agent-based simulation is increasingly used to study systems in many areas of business and science nowadays. Agent-based simulation refers to simulations of systems that contain agent entities whose behaviour depends dynamically on the state of the system. This enables the agents to adapt their behaviour to changing conditions. For some applications, using agent-based simulation for prediction (rather than just for a better understanding) could be very powerful. For example, a company might wish to use a model of the population of their customers with word-of-mouth interactions to predict the sales of their product or the effect of an advertising campaign. However, the problem is that agent-based models typically have a very large number of parameters and many of these cannot be measured directly or estimated with sufficient precision. The result is that a wide range of sets of parameter values may give an acceptable fit and are therefore feasible values. However, they may give quite different predictions. Therefore, simply choosing a single set of parameter values that produces a good fit may mean that the model results are incorrect and very misleading. The inverse problem has been studied in other areas of science including groundwater modelling (Brooks et al., 1994), but it appears that this issue has not yet been investigated for agent-based simulation. In order to investigate the extent of this problem, in the research an agent-based consumer diffusion model was developed and treated as the real system. Selected output data from this model was used as measured values from the real world. In a pseudo-modelling exercise, this data was then used to calibrate agent-based models of the system, and a method similar to that of Brooks et al. (1994) was used to find the extent of the variations in predictions. The method had to be adapted since the model in this research is stochastic whereas the method had previously only been applied to deterministic groundwater models. In the model, a social network of individuals who interact with one another rather than a vast population of agents with many neutral contacts is represented. All agents are allocated to a diffusion social circle with a certain level of influence within the social network. These are constant attributes for that individual throughout the simulation. All agents initially have no knowledge or preference about the selected product. During the simulation, agents receive marketing communication messages (i.e. from company's advertisements, supermarkets, online search results etc.) and contact each other to exchange their knowledge and preferences about the product. There has been very little agent based modelling of this situation and the mechanisms developed represent a potential theoretical structure for this application. Sensitivity analysis was carried out and the model appears to produce realistic behaviour. The adapted method was applied to four experiments of different amounts of observed data (initial periods of 70, 105, 140 and 175 days) to find the range of predictions of total sales in each case. The total sales for the real system model were 124 and the range of predictions for the four experiments were [58, 376], [79, 319], [91, 277], [109, 187]. As expected, the prediction range narrows as more data is available. However, the range of predictions is very wide for all four experiments and therefore the model would have limited usefulness for predicting sales in this type of situation. In particular, choosing a single set of parameter values is not appropriate and could produce very misleading results.