A Moment of Mental Model Clarity : Response to Jones et al . 2011

Jones et al. (2011) review a variety of elicitation methods for identifying and describing stakeholders’ mental models that have been successfully deployed in a variety of natural resource management (NRM) contexts. These methods are broadly categorized into two classes. The first is direct elicitation, where stakeholders work in conjunction with an analyst to describe and produce a graphical representation of the model in an iterative and interactive fashion. This is distinguished from indirect elicitation, where a research team utilizes textual information from interviews, websites, and other documents to extract a graphical model via content analysis and/or the help of specially-designed computer programs. The authors provide an explanation of the theoretical underpinnings of the methods and challenges in applying the construct to natural resource management.

In our opinion there are three points where the information communicated by the authors may be unclear to an audience that is unfamiliar with current theory in cognitive science and mental modeling.First, Jones and colleagues (2011) state that past focus on "stakeholders' attitudes, preferences, and values" in the service of understanding behavior "fail to account for the human capacity to predict outcomes or analyze causeeffect relationships."They present mental model representations as a tool that better facilitates understanding of underlying cognition (the mental model) about a topic and therefore behavior in the way that knowledge about attitudes, etc. cannot.The authors may have preferred to say that attitudes, preferences, values, and related constructs are information that is still "necessary, but not sufficient" to fully understand and effectively predict the behavior of stakeholders.If the contents of the theoretical mental model are taken to be relevant information in the mind that is recalled when thinking about a NRM issue, these attitudes and values may be considered as a component of that representation.The authors' review does not make it clear that such an understanding of stakeholder attitudes and preferences may be comingled, along with factual knowledge, as part of a mental model, although the review suggests these may be part of cultural models or schemata.All of these components (attitudes, preferences, knowledge, etc.) and their interactions should be part of the mental model representation to the extent that this representation is designed to describe the information that influences decisions and behaviors in which stakeholders engage.Any adaptive management (Walker et al. 2006) or risk management (Wood et al. 2012b) practice should leverage both components of stakeholders' mental models and associated influences for successful stakeholder integration in the NRM decision making process.This is independent of whether these mental models and other constructs reside in a crystallized long-term memory structure, or a more flexible working memory store that facilitates reasoning and inference (i.e., Baddeley 1986).
On a second and related point, it is important to differentiate the construct of the mental model as a knowledge structure from the operations performed on that structure.While these processes may coincide in the dynamical evolution of mental models as described by the authors, the building of the knowledge structure is certainly not the sole endpoint that should be used for the quantification of policies (e.g., through influence diagrams) related to natural resources.Anderson (1978) notes that any mental operation can be described as either relying on a complex knowledge structure that is utilized by a simple process, or as a simple knowledge structure that is manipulated by a complex process.The mental model concepts presented by the authors include both a structure component (the knowledge in the model) and a process component (reasoning and other operations performed on that knowledge), but these components are presented in a way that seems undifferentiated, and does not characterize the relative contribution of each in failing to make behavioral predictions.This may be particularly confusing to readers who are new to cognitive science theory and mental modeling methods.
Said another way, the information contained in a stakeholder's stated belief may be too sparse to make a strong prediction of resulting behavior, or there may be a lack of understanding by researchers of the process by which stakeholders' belief information coproduces behavior that is the problem.Often, the truth will lie somewhere in between.For instance, when a stakeholder states an opinion about climate change, it could be a result of simply recalling a past declaration of this belief (structure explanation), it could be a result of making a reasoned inference based on what he or she knows about the physical climate change process (process explanation), or it could be from recalling a past belief about weather and extrapolating that to state an opinion (hybrid explanation).A focus on belief information alone uses the knowledge component of this equation, and ignores the processes that may be involved in producing behavior from this knowledge, thus highlighting the importance of mental models research that attempts to understand both.For instance, some researchers theorize that beliefs and perceptions related to hazards are developed on the fly by a process of analogy from a perceptually similar context (e.g., changes in weather) to the hazard context (e.g., climate change; Bostrom 2008).
Finally, the authors provide a strong review of methods for eliciting mental models, in spite of its brevity.While they cite Morgan et al. (2002), they fail to cite any of the advances that have built on this seminal decision analysis based mental models approach (e.g., Gregory et al. 2003, Darisi et al. 2005, Downs et al. 2008, Reynolds et al. 2010, Wood et al. 2012a;Wood et al. 2012b).This method is distinct from some of the other indirect methodologies listed in both Jones et al. and other papers in this feature volume in that it entails a science-based decision analysis of the stakeholders' decision problem, represented as an influence diagram.Although influence diagram representations are found in other participative modeling and direct elicitation methods, the explicit use of decision analysis to derive the initial coding scheme differentiates this type of method from those described in Jones et al. (and elsewhere).Further, because the influence diagram provides the backbone for the coding scheme it can be annotated once augmented with new ideas from the interviews, to represent individual or aggregate stakeholder mental models concerning the NRM problem of interest, using interview transcripts or other textual data.Such a representation may, however, misrepresent the native conditional dependencies within individual mental models, to the extent that those are evident from the interviews.This differs from Abel et al. (1998) in that interview queries are conceptually organized from broad queries to more focused ones in order to develop a holistic and initially unbiased view of the problem, though increasingly reactive over the course of the interview.By comparison, Abel and colleagues "lead the witness" literally by eliciting stakeholder opinion on a walk along several locations.Their elicitation technique builds creatively on the context in which the stakeholder engages with the domain on a daily basis, but it also imposes a temporal and topological structure that may be confounded with what a stakeholder knows about the problem.
Decision-analysis based mental models approaches have been used in numerous application contexts to compare expert-informed influence diagrams with layperson mental models in the service of developing risk communications and to compare differences in mental models between stakeholder groups (e.g., Wood et al. 2012b) in a way similar to Abel, Ross, and Walker (1998).In addition, many other advances have occurred since the work of Abel et al., including the development of software solutions (Cognitive Science Systems 2011, see http://www.decisionpartners.com) to accelerate content analysis and related analytic processes using Natural Language Processing tools.This software will eventually make visualization and annotation of mental model diagrams easier, and even incorporate questionnaire data in providing an evaluation of the strength of the relationship between two variables.Jones et al.'s (2011) review of the current state of the science and technology of mental models will surely prove valuable to many readers; we hope this response augments that value.

RESPONSES TO THIS ARTICLE
Responses to this article are invited.If accepted for publication, your response will be hyperlinked to the article.To submit a response, follow this link.To read responses already accepted, follow this link.