Combining participatory scenario planning and systems modeling to identify drivers of future sustainability on the Mongolian Plateau

• Abstract
• Introduction
• Background
  • Systems dynamics modeling
  • Scenario planning
• Methods and Results
  • Scenario planning workshop participants
  • Scenario planning workshop
  • Ranking of drivers and scenario development
  • System dynamics models
  • Application of scenarios in SDM
  • Model projections and insights
  • Evaluation and reflection
• Discussion
• Conclusions
• Responses to this article
• Acknowledgments
• Literature Cited

INTRODUCTION

Understanding and addressing the complexity and “wickedness” (Rittel and Webber 1973) of socio-environmental problems requires the expertise of researchers from multiple disciplines, and participation of stakeholders from multiple societal sectors, to generate new transdisciplinary knowledge and solutions (Brown et al. 2010). Further, to address complex and spatially extensive challenges in a way that results in tractable policy recommendations requires transdisciplinary approaches that incorporate nonscientist stakeholders from diverse social sectors together with researchers, in joint problem-framing and solution-seeking as part of the research process (Lang et al. 2012). Challenges to this type of collaboration and transdisciplinary dialogue include differing epistemologies, methodologies, vocabularies, values, and cultures among different disciplines, the assumptions researchers make about other fields and about nonresearchers (Lélé and Norgaard 2005, Stokols et al. 2008), and differential power within and between different disciplines (Cobb and Thompson 2012). These collaborations often face challenges with knowledge integration across disciplines (Pennington 2008), but have the potential to result in novel insights on the dynamics and feedbacks that occur within these systems (Chen et al. 2015a, Hull et al. 2105). Achieving these insights requires methods and tools that integrate diverse knowledge across disciplines and sectors (Lang et al. 2012, Chen and Liu 2014).

Although the need for greater interdisciplinary collaboration is gaining attention and momentum (Ledford 2015), best practices for achieving effective transdisciplinary science and problem solving are still needed to address inherent challenges (Heemskerk et al. 2003, Pennington et al. 2013), including building communication competence within and among interdisciplinary teams (Thompson 2009, Palmer et al. 2016). This transdisciplinary dialogue can be challenging because of differences in terminology and epistemology (Stokols et al. 2008). An important foundation for a successful interdisciplinary collaboration is joint problem definition and integrated conceptual framing of the phenomenon by participating team members (Pennington 2008, Knapp et al. 2011, Lang et al. 2012). Structured dialogue (Pennington 2008), participatory conceptual modeling (Heemskerk et al. 2003, Knapp et al. 2011), and facilitated scenario planning workshops (Winowiecki et al. 2011, Voinov et al. 2014) have been used to promote communication across disciplines and develop a shared understanding of a complex system so that problems can be studied more effectively. Two complementary approaches have been shown to help facilitate dialogue: the use of material artifacts, such as mental models and schematics as boundary objects for cocreation of research questions and articulating problems (Pennington 2010); and participatory modeling to integrate stakeholder knowledge into quantitative assessments (Van den Belt 2004, Voinov and Bousquet 2010, Voinov et al. 2016, Basco-Carrera et al. 2017).

We used participatory scenario planning and system dynamics modeling (SDM) to assist an interdisciplinary research group, together with other nonscientist stakeholders, in identifying key drivers and interactions in a complex transboundary social-ecological system (SES): the Mongolian Plateau. The overarching research question for the exercise asked the following: What factors will affect the future sustainability of the SES on the Mongolia Plateau? We implemented the scenario development and model projections as independent, equally important steps in a process of investigating potential impacts of future policy decisions and climate changes on a system (Kok et al. 2015, Withycombe Keeler et al. 2015). A key aspect of the socio-environmental context on the Mongolian Plateau derives from the striking differences between the country of Mongolia and the adjacent province of Inner Mongolia, China (Fig. 1). Although the two regions share a similar cultural and ecological history, they have experienced very different population growth rates, levels of market access, and land-tenure reforms (Chen et al. 2015a). At the same time, they are also facing similar sustainability challenges, including grassland degradation and desertification, drought and water scarcity, and increased frequency of dzud, a winter weather disaster that results in catastrophic loss of livestock (Chen et al. 2015a). The broader motivation for this work came from the need to articulate plateau-wide and region-specific knowledge about potential future issues and drivers of change, in order to build consensus on important research directions and to inform ongoing policy discussions within China and Mongolia.

Despite the growing number of examples of participatory modeling in the environmental sciences, a few authors have recently noted a lack of adequate reflection on the process (Seidl 2015, Elsawah et al. 2017). Here, we report on a project where we combined participatory scenario planning and system dynamic modeling to produce transdisciplinary knowledge about the Mongolian Plateau system and build a shared understanding of this complex system. We involved participants from different academic disciplines (ecology, physical science, economics, social science), nationalities/cultures, positions in the academic hierarchy (Masters and doctoral students, junior and senior researchers and faculty), and different social sectors (e.g., herder, local government, business, and NGO, as well as academia). We anticipated that the primary value of the scenario planning in this context would be to stretch participants’ thinking about system drivers and dynamics, i.e., to foster systems thinking, and to help synthesize the diverse knowledge of participants about the system. We predicted that the process would foster increased engagement and discussion by diverse members of the group. We also aimed at gaining insights on how to structure our quantitative and computational models of system dynamics, based on outputs from this engaged scenario planning activity and follow-up input from participants as we developed the models. We expected that these inputs would take the form of descriptions of system drivers, consideration of the most uncertain drivers, and possible alternative scenarios under which we could subsequently use model simulations to evaluate ecological and social outcomes of the scenarios.

Literature on participatory scenario development and modeling has largely emphasized the benefits of including stakeholder knowledge for framing research questions and conceptual models to foster model outputs that are tractable for knowledge users and policy makers (Kok 2009). Quantitative modeling or the iterative model development process can also point to system drivers that were overlooked in the original conceptual framing, but are important potential contributors to future dynamics.

Our objectives in this paper are the following: (a) to explicitly report and reflect on our iterative process of participatory scenario development and quantitative modelling, and (b) to highlight the knowledge gains that arose from our approach. Specifically, we trace the insights gained at four stages in the process. First, we demonstrate the value of participatory scenario development for fostering transdisciplinary knowledge production and eliciting system drivers and interactions to inform quantitative modeling. Second, we draw attention to the differences in the scenario details generated by the two groups, emphasizing the importance of soliciting expert knowledge in developing modeling scenarios. Third, we highlight some similarities in the system drivers identified in these scenarios, despite the different ways the scenarios were articulated and the utility of iterating between model development and narrative scenarios in order to refine quantitative projections (Mallampalli et al. 2016). Finally, we also demonstrate an underappreciated benefit of this iterative approach. In addition to the qualitative scenarios leading to refinement of the systems models, the quantitative modeling process also revealed aspects of the system that were overlooked by the participants during scenario development.

We use the terms “interdisciplinary” and “transdisciplinary” throughout this paper to refer to different aspects of the work, drawing from the discussion in Max-Neef (2005) and Pennington (2016). We refer to the team of researchers leading this project (ourselves) as interdisciplinary, in that we come from different disciplinary groundings, but are coordinating research across our specialties to answer higher level questions about system behavior. We use “transdisciplinary” to indicate that the process we are describing here has engaged specialists and stakeholders to interrogate potential futures in a systematic way and produce a new understanding of the system that incorporates both pragmatic and values-centered perspectives (Max-Neef 2005). For instance, the generation of scenario stories requires the participants to harness empirical and technological knowledge and insights, while also making claims and predictions that are rooted in value judgments. In summary, we use interdisciplinary to refer to the people involved in the work and transdisciplinary to refer to the process and outcomes.

After briefly introducing the concepts of system dynamics modeling and participatory scenario planning, we describe how we implemented them and report on the outputs of the scenario development and iteration of the models with the scenarios and participants. We then present reflections from the workshop participants on the process of developing the scenarios as well as our own reflections on the process of conducting the workshop and working through model iteration. Throughout this paper we return to our initial objectives of highlighting four avenues of knowledge gained in this process: participant knowledge integration, scenario differences, system driver similarities, and emergent insights on drivers.

BACKGROUND

Systems dynamics modeling

System dynamics models (SDMs) can be used to understand emergent, system-level behaviors that result from the interaction of variables in different subsets of complex systems. The approach was developed for addressing issues in industrial manufacturing (Forrester 1961, 1994), but is now increasingly applied to model economic and ecological dynamics, particularly for social-ecological systems that exhibit complex feedbacks among variables (Holmberg 2000, Rasmussen et al. 2012). SDM can be used to synthesize diverse data types to understand how feedbacks produce nonlinearities in complex systems (Forrester 1961, 1994). They are particularly useful for SESs such as rangelands, where changes in climatic and socioeconomic forces interact to influence human decision making regarding land use and resource management (Shen et al. 2009, Rasmussen et al. 2012). Once parameterized with empirical data and relationships among the variables, SDMs can be used to evaluate the ecological and socioeconomic outcomes of different potential futures such as those outlined by participants in our workshop (Allington et al. 2017). Simulations based on model scenarios can then reveal counterintuitive behavior of key system variables over time, which might not have been anticipated by the original workshop participants. Thus, one can learn about the systems themselves from the simulations of different scenarios, the trajectories of the key variables over time, and the interactions among the variables in different parts of the system.

Scenario planning

Scenario planning is a method of thinking strategically about the future in situations of high complexity and uncertainty and low controllability (Peterson et al. 2003a). The approach has its origins in business and the military (Shoemaker 1995), but has been applied across a range of sectors, including recently to conservation and natural resource management. In these contexts, scenario planning has been used to explore future ecosystem services in the Millenium Ecosystem Assessment (Peterson et al. 2003b, Bohensky et al. 2006), for climate change adaptation planning in national parks (Ernst and van Riemsdijk 2013) and arctic communities (Picketts et al. 2012), and to consider the future of natural resource-based livelihood systems such as mobile pastoralism in Spain (Oteros-Rozas et al. 2013), and livestock ranching in Australia (Puig et al. 2011). A hallmark of scenario planning in these cases is that it brings together diverse stakeholders from different disciplines, social sectors or roles, and with different expertise and ways of knowing, to think together about a question of mutual interest. Participants collaborate to produce charts of relevant drivers that act as negotiated boundary objects to describe the system in question, a process which, in and of itself, has been shown to improve group dynamics (Pennington 2010). We expected that, when used in the context of interdisciplinary research, participatory scenario planning would serve a valuable function by facilitating dialogue and knowledge integration across disciplines and sectors (Harris and Lyon 2013, Palmer et al. 2016), and the development of transdisciplinary framing for quantitative systems modeling (Tress et al. 2005). The collaborative scenario development process forces participants from different backgrounds and experiences to make their assumptions and tacit knowledge of the system explicit (Seidl and Le 2012), and to explain the rationale and logic behind their ranking of system factors based on importance and uncertainty. In an organizational context, scenario planning has been evaluated through various theoretical lenses, including organizational learning (Haeffner et al. 2012) and organizational resilience and system innovation (Cobb and Thompson 2012), among others. Here, we report on the application of participatory scenario planning and the associated scenario development as a tool to facilitate interdisciplinary and cross-sectoral communication and collaboration, and demonstrate its value in fostering transdisciplinary knowledge production and eliciting system drivers and interactions to inform systems dynamic modeling. We use the resulting scenario stories as a basis for setting the parameters for future scenarios in the model and also as a framework to interpreting model projections.

METHODS AND RESULTS

Scenario planning workshop participants

The Scenario planning workshop was held in Ulaanbaatar, Mongolia in June 2014. Participants were attendees at a larger scientific synthesis meeting to which they were invited because of their involvement with one of a number of ongoing research projects in the region, or because of their knowledge of the region. The projects all address some aspect of social-ecological dynamics in the region and involve international collaborations that include researchers and stakeholders in the region. Project leaders, collaborators, and participating students were invited. Funding was available to support travel, thereby increasing the ability of invitees to participate. Twenty-six people participated in the workshop, 19 men and seven women. The majority were of Chinese (11) and Mongolian (9) ethnicity, followed by researchers native to the U.S. (3), Russia (1), Germany (1), and India (1). Twenty-two participants were researchers representing the life sciences (8), geography/planning/remote sensing (6), economics (5), and physical sciences (2). One researcher was not identified by discipline. Four participants (all Mongolian) were from social sectors outside of academia, including local and provincial government and agriculture, i.e., herding.

Participants were assigned to one of four groups, each composed of six to eight people (Fig. 2). Assignments were made based on the primary language of the participants (monolingual Mongolian, Chinese, English, or bilingual speakers), and disciplinary background, with an attempt to have representatives of biological, physical, and social sciences in every group. Each group had at least one female participant. The monolingual Mongolian speakers, who included all of the nonscientists, were placed into one group and instructed to focus their scenarios on Mongolia. One non-Mongolian speaker was also in this group, but this individual has extensive past experience in Mongolia. Thus, although the workshop overall was dominated by researchers, half of the members of the Mongolia group, which produced one of two scenarios used in the SDM, were not scientists. A second group composed of native Chinese speakers was directed to focus their scenarios on the Inner Mongolian Autonomous Region (IMAR) of China. The remaining two groups included a mix of national/linguistic backgrounds (U.S., German, Russian, Chinese, Mongolian) and both focused on the entire Mongolian Plateau region, including both the Republic of Mongolia and IMAR, and spoke in English. The facilitator was a member of the research group with experience facilitating scenario planning. The facilitator maintained the role of process guide throughout but did not participate directly in any of the groups except to clarify instructions, answer questions and redirect as necessary. Tables in the room were arranged into four large rectangular clusters, each group working together around its own table.

Scenario planning workshop

We used a shortened version of the scenario planning process outlined by Chermack (2011), which has been applied widely to business and organizational change, as well as more recently to natural resource management planning, especially related to climate change (Cobb and Thompson 2012). The facilitator was experienced using this process to teach system thinking and natural resource management planning to advanced undergraduate students, but had not facilitated it with researchers or other stakeholders. However, she was experienced in facilitating multistakeholder workshops for participatory modeling, knowledge integration, and adaptive management planning.

The scenario planning workshop session consisted of the following steps, which are abbreviated as Stages 1 through 3 (Fig 2):

Stage 1:

1. An initial brief presentation on the purpose and process of scenario planning;
2. Identification of the focal question. The workshop leaders determined the following focal question in advance for all groups to guide their planning: What policies would support long-term sustainability of the social-ecological systems on the Mongolian Plateau?

Stage 2:

1. Brainstorm of potential factors that affect the focal question. Participants were instructed to think broadly and to come up with as many different factors as possible, making sure that each group member contributed at least three factors to the discussion;
2. Ranking of factors by impact on the focal question. Participants ranked factors from most to least important in their influence on the focal question;
3. Ranking of factors by uncertainty. Participants ranked each factor from highly uncertain to highly certain;

Stage 3:

1. Identification of “critical uncertainties”: those factors in the impact-by-uncertainty matrix that are in the high impact-high uncertainty quadrant (Appendix 1);
2. Exploration of potential scenarios by combining pairs of unrelated critical uncertainties to think about possible alternative futures for the system. Participants were instructed to define the extremes of the axis for each uncertainty and to select the two critical uncertainties that result in a set of divergent but plausible set of scenarios that are substantially different from the conventional wisdom. The aim was to identify scenarios that are “good to think with”; and
3. Description of what the future would look like under each of the four resulting combinations. Participants were asked to avoid identifying “best case,” “worst case,” and “status quo” scenarios and rather to try to describe both the positive and negative aspects of each possible future (Fig. 2).

These steps loosely approximate the first three steps of Alcamo’s approach to scenario development (Alcamo 2008). In an interactive and participatory process using many index cards and large pieces of paper, each subgroup generated a list of critical uncertainties. From this list they selected and used two of these uncertainties to identify four contrasting plausible future scenarios. Each group then described the key features of each of their scenarios on a chart where the vertical and horizontal axes represented two extremes of each selected critical uncertainty, and each resulting quadrant represented a different alternative future (Fig. 2, Stage 3, Appendix 1). For example, one axis might represent climate variability and the other potentially divergent policy options for rural development. After the subgroups drafted their scenario outlines, they presented them to the entire group, followed by a short Q&A and comment period on each outline.

For the IMAR group, the first axis concerned precipitation trends under future climate, ranging from more wet to more dry. The second axis described policy decisions that focused on short-term versus long-term development outcomes. The former set of policies focused on short-term economic growth, rather than on a development strategy that had long-term capacity development as a key goal.

For the Mongolia group, the axes focused on defining directions of future economic development and capital investment (Fig. 2, Stage 3). The first axis concerned current debates over developing the industrial and mining sectors of the economy versus prioritizing agricultural livelihoods and pastoralism. The second axis divided futures focused on private markets and continued integration into global capital markets from those on sustainable resource development via investment in communal pastoral systems and rural institutions.

The quadrants defined by each pair of axes resulted in four distinct scenarios for each group (Fig. 2, Stage 3). The IMAR group, comprising entirely researchers from social science and environmental fields, labeled their scenarios with nicknames: Grassland Paradise, Grassland Pendulum, Desertification, and Proactive Adaptation. Grassland Paradise described a future where plentiful rainfall and long-term policy lead to improved ecological conditions, optimal livestock numbers, increased prosperity and social harmony, attracting migrants that may eventually lead to overpopulation and social instability. In Proactive Adaptation, farsighted policy in the face of declining moisture leads to conservative stocking policies and a strong social safety net, providing social stability in the face of frequent natural disasters. Grassland Pendulum plays out when high rainfall is coupled with short-term policy perspectives, leading to cyclical overexploitation of the environment, and a volatile and unpredictable economic situation. In Desertification, environmental degradation makes continued livestock husbandry infeasible, leading to poverty, social instability and emigration. Over the longer term, the change in the economic structure could lead to technological innovation—a focus on education and economic diversification.

The Mongolia group, comprising a mixed group of researchers and local government and business representatives, also described four contrasting scenarios. These ranged from a future where mining dominates the economy under a socially oriented green development policy leading to prosperity, ecological restoration, and waste management, but absent a strong livestock sector, to a future where agriculture dominates the economy under capitalist market-driven conditions leading to more specialized livestock enterprises, conflicts between crop agriculture and livestock production, soil loss, dust storms, pollution, and increased greenhouse gas emissions. In the scenario characterized by an agricultural economy under a green development policy, small-scale pastoralism persists with improved rural living conditions and rangeland management. In the scenario where market-driven policies dominate in a mining-oriented economy, the group foresaw a future of increased degradation, pollution, and rural-urban migration, fueled by a higher gross domestic production (GDP). Across all scenarios, the group focused on the consequences for living conditions and environmental health.

Ranking of drivers and scenario development

One of the main objectives of the scenario planning activity was to identify a “short-list” of key socio-environmental drivers on the Plateau in general, to serve as a basis for further quantitative modeling activities by the research team. The scenario planning process generated a long list of drivers with eight de facto priority drivers, the two critical uncertainties selected by each group as the axes for their scenarios. However, the group expressed that these selections may have overlooked other critical drivers in the region, including those that have high certainty and high impact. Therefore, we used a full group discussion to solicit additional major drivers that were omitted from the list of eight. The groups began by returning to the master list of factors to recommend the key drivers. This process led to a list of 23 drivers (Appendix 2). To rank these drivers, we used a modified nominal group technique and asked each participant to allocate five votes in any fashion they chose among the 23 drivers. They could cast one vote for each of five different drivers, or multiple votes (but no more than five total) for fewer drivers. Through this process we identified those drivers that were perceived as the most important overall by members of the group, as well as those that were seen as least important.

The drivers identified by the participants fall into six themes: climate, environmental condition, economic development at the country or regional scale, social and economic mobility at the household scale, infrastructure, and technology. The division of individual drivers among those categories varied by stakeholder group. For those representing Mongolia, the most common drivers were environmental and social conditions, which accounted for 60% of the list. For IMAR, social and policy drivers were identified as the most important for the future, followed by the economy (Appendix 2, Fig 2).

System dynamics models

After the scenario planning exercise, several participating researchers (including two authors of this paper) used outputs from the workshop to formulate a SDM for the Mongolian Plateau (Table 1). The conversations at the workshop informed the initial model structure and the outcomes of the scenario development were the grounding for the future scenarios applied to the model. We based the work on a conceptual model for the entire plateau, based on three interacting subsectors (Fig. 3): human, environment, and land use. Based on this conceptual model, we built separate system dynamics models using data from two different case study areas on the Mongolian Plateau: Xilingol League in Inner Mongolia, PR China; and Suhkbaatar Aimag in Mongolia (Fig. 1). The SDMs represent the system as a set of stocks (e.g., human population or total grassland area) and flows (e.g., movement of units in and out of the stock). Independent exogenous variables, such as precipitation or policy, represent inputs to the model, while dependent endogenous variables connect the various subsystems and produce the feedback loops. Models were fit with a series of differential equations, which remain the same across all subsequent scenario simulations; parameters such as constants and coefficients were modified based on assumptions of the individual scenarios (Table 1, Appendix 3). Thus, we were able to compare the projections for key system variables, e.g., grassland area, livestock population, between the alternate future scenarios in the next stage. We estimated parameters using data synthesized from several researchers who were present at the workshop, from primary sources, available literature, and functions that we derived ourselves from the empirical data (Allington et al. 2017).

Application of scenarios in SDM

We translated the participant-generated storylines into discrete future scenarios that could be represented as sets of model parameters and inputs to explore the potential outcomes of the policy and climate drivers identified by the groups (Table 1). We isolated distinct trajectories along the four axes that we could approximate by making changes to the model parameters (Fig. 2; Stage 4). For details on how specific parameters were modified within each model to project each scenario into the future, see Appendix 3 and Allington et al. (2017). For each scenario, we projected the models out to 2050 to examine differences in both temporal dynamics and final model projections for key system variables, e.g., grassland area, livestock population. Model simulations are not intended to be empirical predictions; rather, they serve as a way to compare how the rangeland system might behave under different modifications of policy, climate, or the economy. Because our objective was to compare directly the dynamics in Mongolia vs Inner Mongolia for this first round of modeling, we only used the storylines from those groups; we did not use the stories developed by the groups that focused on the Plateau as a whole. The simulated future scenarios we applied to the models are described in Figure 2. The specific parameters that were modified based on each scenario are detailed in Appendix 3. Details of the initial system dynamics model were presented at a meeting in Ulaanbaatar in the summer of 2015, which was attended by many of the participants of the original participatory scenario planning workshop, as well as additional researchers, stakeholders, and NGOs. Input gleaned from feedback at the meeting was used to further refine the model structure (Fig. 2; Stage 5), which was discussed with a subset of workshop participants at subsequent smaller meetings throughout 2015 and 2016. The final model outputs based on simulations for each scenario were then used as a starting point for a discussion at a final project meeting in May 2016. This final meeting aimed at synthesis understanding of the dynamics of social-ecological systems on the Mongolian Plateau, and was attended primarily by researchers with backgrounds in rangeland ecology, remote sensing, economics, ecosystem modeling, and rural sociology, as well as a staff member of the Mongolian Academy of Science Institute of Geography and Geoecology.

Model projections and insights

We used the final models to examine differences in both temporal changes and final model projections for key system variables, such as grassland area and livestock population under each scenario, through 2050. The most important drivers of change over time and variability among individual scenario projections were consistent with those identified by workshop participants. In IMAR the continuance of environmental protection policies regarding land use and restoration had the most significant impacts on long-term stability of the system. For Mongolia, the relative amount of investment in rural infrastructure versus prioritization of the industrial sector structured the long-term dynamics of the grasslands. These correspond to some of the main axes identified in storyline development (Fig. 2). Interestingly, climate trends (e.g., precipitation) were not a significant determinant of long-term grassland sustainability in IMAR, compared to the impacts of policies; this is in contrast to the predicted key uncertainties identified by the scenario planning process.

In both IMAR and Mongolia, the results of the model projections revealed insights about factors of the system that had not been the focus of the scenario descriptions of either region. These largely overlooked aspects included uncertainty about future urbanization trends and the relationship between rural out-migration and livestock population dynamics. For IMAR, some scenarios projected increases in available biomass, despite continued loss of grassland, which was initially counterintuitive. This was caused by the loss of herders from rural areas and associated declines in livestock population. Initial projected trends in biomass and livestock in Mongolia also revealed the importance of rural out-migration on future grazing intensity. However, these trends in Mongolia were based on historic relationships between herder population and stocking rates, which have more recently been decoupled as nationwide livestock population is reaching record highs despite high rates of urbanization. Conversations with conference participants in 2015 prompted us to make refinements to the model specifications to more accurately represent contemporary livestock population dynamics (Figs. 2 and 4).

A full presentation of the projections derived from the model simulations is beyond the scope of this particular article; a few summaries are presented in Fig. 4 and details are available in Allington et al. (2017). Our focus here is not on the specific outcomes of each simulation but rather on the interplay between the qualitative storylines and the quantitative model projections, and the way in which the outcomes of the simulations reflect back on the storylines themselves.

Evaluation and reflection

After presenting and discussing the draft scenario outlines, the participants in the June 2014 participatory scenario planning workshop were asked to answer five evaluation questions about the scenario planning process that probed what they learned from the process. Evaluation questions focused on the most interesting and valuable parts of the process; how they might apply the process or scenarios in their future work; the biggest challenge; the biggest frustration; and how the process could be improved. The written evaluation was followed by a short reflective discussion by the group.

In workshop evaluations, the participants noted two aspects of the process that they found the most interesting or informative. The first was the experience itself; participants found the process of going through the scenario planning exercise to be valuable and interesting. Second, several participants noted that their perspectives on the system were expanded by being exposed to ideas raised by other group members with different experiences. They noted that seeing the specific factors or number of factors identified as influencing the focal question by other members of the group was informative (Table 2). Participants also mentioned gaining appreciation for system complexity and dynamics such as path dependence, learning about how others see the world and think about the future.

Participants overwhelmingly reported that the most valuable aspect of the scenario planning workshop was the diversity of ideas and viewpoints expressed and the opportunity to learn about the views and ways of thinking from other disciplines, e.g., cultures and sectors. One respondent focused specifically on the value of analyzing multiple interacting processes with an interdisciplinary team, highlighting the value of the process for bringing diverse disciplinary knowledge to bear on understanding system complexity (Schoemaker 1991, Schmitt Olabisi et al. 2010). Some specifically mentioned the value of the ranking portion of the activity.

Most participants anticipated applying what they learned from the workshop in their future work. Researchers anticipated applying this process in diverse ways, including as a way to facilitate comprehensive discussions across diverse disciplines, to identify critical factors and issues for future research, and as a useful way to think about problems. Government and practitioner participants anticipated using the process with local stakeholders as a way to facilitate public involvement in planning and to develop more realistic management plans.

The challenges that participants reported were largely the mirror images of the values: the difficulty in combining or reconciling diverse disciplinary and cultural perspectives and arranging factors on the axes of importance and uncertainty. The meaning of some key terms, especially “uncertainty” created a challenge, as did envisioning each scenario and its outcomes. The main suggestions for improvement included the need for more preworkshop materials and preparation for the participants, including examples of other scenario plans; involving an even greater diversity of participants; focusing on a narrower focal question; and providing more specific definitions of key terms.

DISCUSSION

Engaging a group with diverse scientific and sector expertise in dialogue about system dynamics is one way to generate the system-level understanding needed to evaluate current complex systems and to begin to consider the implications of future changes and interventions (Schmitt Olabisi et al. 2010, Alberti et al. 2011, An 2012). We used a combination of participatory scenario planning and system dynamics modeling to generate new knowledge about a complex system, while also revealing significant knowledge gaps that should be considered by researchers and policy makers in the future. Scenario planning as we applied it in this workshop proved a useful process for synthesizing diverse knowledge systems, including both disciplinary scientific knowledge and the embedded cultural and experiential knowledge of individuals from different national origins and occupations, and stimulating meaningful discussion grounded in these distinct scientific and cultural perspectives. These dialogues led to identification of a wide range of plausible future scenarios for the Mongolian Plateau, which informed further dynamic systems modeling. From the perspective of interdisciplinary collaboration and communication, participants found value in the structured interactive process of brainstorming, ranking, and exploring scenarios that enabled them to view a problem in new ways, and revealed their disciplinary blind spots. A further value of the process was the opportunity for participants to learn about different ways of thinking about the focal question around future sustainability of the Mongolian Plateau system, through interactions with individuals from different disciplines and cultures. The engaged, discussion-based, and problem-focused nature of the scenario planning dialogue differs from traditional formats of knowledge exchange in interdisciplinary scientific meetings, such as speaker presentations followed by Q&A. The scenario planning process provided all participants an opportunity to share and discuss relevant information, based on science, previous knowledge, experience, or culture. This made the process especially effective at integrating nonscientist knowledge, in addition to scientific knowledge from different disciplines, because it created opportunities for meaningful learning that builds upon the authentic experience of participants (Novak 2010, Baival and Fernández-Giménez 2012). For this reason, several participants, including the nonresearchers, grasped its value as a process for stakeholder engagement, and its salience for local land use and resource management planning. Indeed, based on the feedback from this exercise, one of the coauthors subsequently applied the same process in two regional workshops in Mongolia to facilitate the application of research results together with local knowledge to community conversations about the future of Mongolian rangelands, and policy and management options for community resilience building.

We expected that the novelty of the process and structured facilitation might engender some resistance among participants. For example, we anticipated that physical scientists might resist the qualitative nature of the process. Further, based on previous discussions with researchers from one of the represented nationalities, we anticipated that some non-Western participants might find the process alien to their local context and norms of interaction, leading to less engagement. As such, this use of scenario development also represents an example of how it can be effective as a research and engagement tool in the context of transdisciplinary, international, collaborative research addressing complex dynamics of SES.

It is important to note that we adapted a scenario planning process that was designed for use in planning for business, government, or NGOs (Chermack 2011), but in a different context and for a distinct purpose. As such, our scenario planning process was necessarily a much-abbreviated version of the full process, and thus did not fully explore the implications of the scenarios or develop complete scenario narratives. Rather than testing policies to determine risk and benefit under various scenarios through a process of “wind tunneling” (Chermack 2011), we instead used dynamic systems modeling to explore policy implications. Further, as conventionally implemented, scenario planning would engage an entire group in developing a single plan. In order to facilitate discussion and participation of all workshop attendees, we broke participants into smaller groups, resulting in a variety of scenarios. Although this would be less useful in an actual planning process, it was helpful in our context, both in facilitating interdisciplinary and cross-cultural dialogue, and in eliciting a wider range of drivers, critical uncertainties, and potential future system trajectories.

Comparing future projections under alternate scenarios can be very informative, but the specific lessons depended on the assumptions of the scenarios themselves and the ways they are implemented in the model. In the absence of the workshop, the future scenarios explored by the modeling team would have likely been different, not as a representative of stakeholder experiences, and likely would have been more similar across the two case study areas. Indeed, the future scenarios articulated by the two groups were quite different, despite their ecological and environmental similarities. The system dynamics model allowed us to evaluate the outcomes of storylines generated during the workshop. The model projections are a way to further explore the implications of stories outlined in the scenarios (Alcamo 2008). At the same time, they also recursively informed the scenario planning exercise by further validating that certain factors and policies do indeed have impacts on system function. By integrating this model development with stakeholder-generated scenarios, it enabled us to go beyond simply just reporting technical details of model outputs and fostered more productive communication (Elsawah et al. 2017).

By basing future assumptions on the specific stakeholder-generated scenarios, we ensured that the pulses or trends that we applied to the model to simulate the future of the Plateau were reflective of (a) likely occurrences, or (b) concerns and questions currently being articulated/under debate in the society being modeled, rather than superimposed by us as external researchers. It is important to note that we do not intend the model projections to be taken as predictions, per se (Peterson et al. 2003a, Schmitt Olabisi et al. 2010). Rather, we used the endpoints of model projections as a useful way to explore relative changes in key system variables, e.g., grass or livestock, that might result from alternative conditions in the future. More specifically, the model is a way to refine and formalize the qualitative conceptualizations of scenarios outlined by stakeholders, and to explore how differences between the scenarios can lead to similar or divergent futures (Voinov et al. 2014).

Simulations can help to identify areas of a model that may need additional refinement in order to better represent drivers identified by participants. For example, one driver of future change identified by participants from the Inner Mongolia group is the frequency of natural disasters, such as drought or dzud, a severe winter following a drought summer that often results in significant livestock losses, a phenomenon unique to this region. We modeled this part of the system by introducing a stochastic element to the baseline livestock death rate based on an assumption that sporadic natural disasters result in livestock die-off (Allington et al. 2017), but future efforts should refine this by coupling the death rate variable to specific climate variables and biomass, i.e., functional responses to dzud.

It is worth noting that although there were some similarities in how the groups described the Plateau, e.g. highlighted desertification, and there was some consensus on key drivers, the ways they described the system and potential futures were quite different. This could be attributed to the differences in the disciplinary backgrounds on the group members and actual differences in current and historical political and economic factors. Despite these differences, and differences in the direction of projected trajectories for some key variables, e.g. livestock populations, the two SDMs highlighted several similar dynamics in both regions: (1) uncertainty about urbanization dynamics in the near and distant future, and (2) the evolving relationship between rural population and the population of livestock, at national and regional scales. Interestingly, these were not the focus of the articulated uncertainties for either group. Although workshop participants considering Inner Mongolia identified future social and economic policies as important, and those considering Mongolia specified economic development as important, neither group specified the effects of those policies on urbanization rates as being potentially instrumental in structuring future dynamics. Because the model structure required consideration of human population size, it required specification of rural-urban migration rates as a key determinant of the population in the rural areas. The formalization of qualitative scenarios that is necessary for their translation into quantitative model structure is an important link strengthening the complementarity of these two approaches. The quantitative modeling process, together with the evaluation results, revealed unanticipated system behavior and key drivers not identified or overlooked by stakeholders.

The modeling process also brought to light key sensitivities and vulnerabilities in the system, which the participants in the scenario planning exercise had not identified. Initial model runs indicated that urbanization was interacting strongly with other variables, such as biomass availability and livestock population dynamics. We discussed these interactions with group members from the original planning workshop during subsequent smaller meetings, and gathered feedback about social-political dynamics driving rural out-migration. These discussions revealed that future empirical efforts need to focus more specifically on better understanding the evolving relationship between total rural herder population and livestock populations (Fernández-Giménez et al. 2017). The baseline models assumed that the historic strong relationships between rural human population and livestock populations will continue in the future. However, this relationship has become decoupled in recent years as livestock population growth rates are increasingly rapidly, while rural populations are declining (Chen et al. 2015b). Discrepancies between model projections and trends on the ground highlight the need for additional research to understand this phenomenon. Although this reality limits our ability to make projections about livestock population into the future, the revelation itself underscores the utility of this kind of iterative participatory modeling process to reveal untested assumptions and areas of rapid change and uncertainty.

Although there have been some critiques of this iterative approach, based on a perceived “methodological imbalance” between the qualitative and quantitative approaches (e.g., Kosow and Gaßner 2008), our experience highlights how this can create a format to bridge the two (Klein 2008). The qualitative scenarios informed the model testing and simulation, and the future projections also revealed knowledge gaps that need substantially more attention (Peterson et al. 2003a), either through additional dialog and synthesis or new empirical research. In the absence of this structure and associated process, it is likely that the modeling team would have simply articulated Plateau-wide future scenarios that were applied to the models of each country. However, by explicitly seeking the input of knowledgeable individuals, we revealed the country-specific drivers of interest and concern relative to future dynamics. We then adapted the models and simulations based on those specifics.

CONCLUSIONS

The future sustainability of dynamic social-ecological systems, such as the Mongolian Plateau, will require cross-disciplinary dialogue and collaboration between stakeholders and researchers from the social and natural sciences, but also participation by stakeholders who are part of these systems, i.e., engagement. Comparison of system dynamics across ecological gradients and institutional settings provides an opportunity for learning and experimentation about possible solutions (Brown et al. 2013).

We used scenario planning as a way to facilitate dialogue across disciplines, to provide a platform for individuals to share ideas across boundary objects and for participants to process their own ideas about how individual drivers of change may impact the Plateau in the future, without the imposition of a need for consensus (Star 2010). The scenario planning exercise helped to establish a productive dynamic among members of the group, which has promoted a collaborative process moving forward. This experience thus demonstrates the value of scenario planning as a process to facilitate interdisciplinary, cross-cultural, and cross-sectoral dialogue if the aim is to elicit and exchange a diversity of perspectives on a problem, and to promote increased awareness of different ways of thinking about it grounded in distinct disciplinary or cultural experiences. It also ensured the integration of place-based knowledge into scenario development for subsequent quantitative modeling exercises. The emergent knowledge gained from this process underscores the utility of pairing the qualitative scenarios with quantitative simulations to reveal unanticipated system behavior or key drivers not identified or overlooked by stakeholders.

LITERATURE CITED

Alberti, M., H. Asbjornsen, L. A. Baker, N. Brozovic, L. E. Drinkwater, S. A. Drzyzga, C. A. Jantz, J. Fragoso, D. S. Holland, T. A. Kohler, J. Liu, W. J. McConnell, H. D. G. Maschner, J. D. A. Millington, M. Monticino, G. Podestá, R. G. Pontius, C. L. Redman, N. J. Reo, D. Sailor, and G. Urquhart. 2011. Research on coupled human and natural systems (CHANS): approach, challenges, and strategies. Bulletin of the Ecological Society of America 92:218-228. http://dx.doi.org/10.1890/0012-9623-92.2.218

Alcamo, J. 2008. The SAS approach : combining qualitative and quantitative knowledge in environmental scenarios. Pages 123-150 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands. http://dx.doi.org/10.1016/S1574-101X(08)00406-7

Allington, G. R. H., W. Li, and D. G. Brown. 2017. Urbanization and environmental policy effects on the future availability of grazing resources on the Mongolian Plateau: modeling socio-environmental system dynamics. Environmental Science & Policy 68:35-46. http://dx.doi.org/10.1016/j.envsci.2016.11.005

An, L. 2012. Modeling human decisions in coupled human and natural systems: review of agent-based models. Ecological Modelling 229:25-36. http://dx.doi.org/10.1016/j.ecolmodel.2011.07.010

Baival, B., and M. E. Fernández-Giménez. 2012. Meaningful learning for resilience-building among Mongolian pastoralists. Nomadic Peoples 16(2):53-77. http://dx.doi.org/10.3167/np.2012.160205

Basco-Carrera, L., A. Warren, E. van Beek, A. Jonoski, and A. Giardino. 2017. Collaborative modelling or participatory modelling? A framework for water resources management. Environmental Modelling & Software 91:95-110. http://dx.doi.org/10.1016/j.envsoft.2017.01.014

Bohensky, E. L., B. Reyers, and A. S. Van Jaarsveld. 2006. Future ecosystem services in a Southern African river basin: a scenario planning approach to uncertainty. Conservation Biology 20(4):1051-1061. http://dx.doi.org/10.1111/j.1523-1739.2006.00475.x

Brown, D. G., A. Agrawal, D. A. Sass, J. Wang, J. Hua, and Y. Xie. 2013. Responses to climate and economic risks and opportunities across national and ecological boundaries: changing household strategies on the Mongolian plateau. Environmental Research Letters 8(4):45011. http://dx.doi.org/10.1088/1748-9326/8/4/045011

Brown, V. A., J. A. Harris, and J. Y. Russell. 2010. Tacking wicked problems through the transdisciplinary imagination. Earthscan, London, UK.

Chen, J., R. John, C. Shao, Y. Fan, Y. Zhang, and A. Amarjargal. 2015b. Policy shifts influence the functional changes of the CNH systems on the Mongolian plateau. Environmental Research Letters 10(8):85003. http://dx.doi.org/10.1088/1748-9326/10/8/085003

Chen, J., R. John, Y. Zhang, C. Shao, D. G. Brown, O. Batkhishig, A. Amarjargal, Z. Ouyang, G. Dong, D. Wang, and J. Qi. 2015a. Divergences of two coupled human and natural systems on the Mongolian Plateau. BioScience 65(6):559-570. http://dx.doi.org/10.1093/biosci/biv050

Chen, J. and Y. Liu. 2014. Coupled natural and human systems: a landscape ecology perspective. Landscape Ecology 29:1641-1644. http://dx.doi.org/10.1007/s10980-014-0125-9

Chermack, T. J. 2011. Scenario planning in organizations: How to create, use, and assess scenarios; an excerpt. Berrett-Koehler, San Francisco, California, USA.

Cobb, A. N., and J. L. Thompson. 2012. Climate change scenario planning: a model for the integration of science and management in environmental decision-making. Environmental Modelling & Software 38:296-305. http://dx.doi.org/10.1016/j.envsoft.2012.06.012

Elsawah, S., S. A. Pierce, S. H. Hamilton, H. van Delden, D. Haase, A. Elmahdi, and A. J. Jakeman. 2017. An overview of the system dynamics process for integrated modelling of socio-ecological systems: lessons on good modelling practice from five case studies. Environmental Modelling and Software 93:127-145. http://dx.doi.org/10.1016/j.envsoft.2017.03.001

Ernst, K. M., and M. van Riemsdijk. 2013. Climate change scenario planning in Alaska’s national parks: stakeholder involvement in the decision-making process. Applied Geography 45:22-28. http://dx.doi.org/10.1016/j.apgeog.2013.08.004

Fernández-Giménez, M. E., N. H. Venable, J. Angerer, S. R. Fassnacht, R. S. Reid, and J. Khishigbayar. 2017. Exploring linked ecological and cultural tipping points in Mongolia. Anthropocene 17:46-69. http://dx.doi.org/10.1016/j.ancene.2017.01.003

Forrester, J. W. 1961. Principles of systems. Productivity Press, Portland, Oregon, USA.

Forrester, J. W. 1994. System dynamics, systems thinking, and soft OR. System Dynamics Review 10(2-3):245-256. http://dx.doi.org/10.1002/sdr.4260100211

Haeffner, M., D. Leone, L. Coons, and T. Chermack. 2012. The effects of scenario planning on participant perceptions of learning organization characteristics. Human Resource Development Quarterly 23(4):519-542. http://dx.doi.org/10.1002/hrdq.21147

Harris, F., and F. Lyon. 2013. Transdisciplinary environmental research: building trust across professional cultures. Environmental Science & Policy 31:109-119. http://dx.doi.org/10.1016/j.envsci.2013.02.006

Heemskerk, M., K. Wilson, and M. Pavao-Zuckerman. 2003. Conceptual models as tools for communication across disciplines. Conservation Ecology 7(3):8. http://dx.doi.org/10.5751/ES-00554-070308

Holmberg, S. 2000. A systems perspective on supply chain measurements. International Journal of Physical Distribution & Logistics Management 30(10):847-868. http://dx.doi.org/10.1108/09600030010351246

Hull, V., M.-N. Tuanmu, and J. Liu. 2015. Synthesis of human-nature feedbacks. Ecology and Society 20(3):17. http://dx.doi.org/10.5751/ES-07404-200317

Klein, J. T. 2008. Evaluation of interdisciplinary and transdisciplinary research. A literature review. American Journal of Preventive Medicine 35(2):S116-S123. http://dx.doi.org/10.1016/j.amepre.2008.05.010

Knapp, C. N., M. Fernandez-Gimenez, E. Kachergis, and A. Rudeen. 2011. Using participatory workshops to integrate state-and-transition models created with local knowledge and ecological data. Rangeland Ecology & Management 64(2):158-170. http://dx.doi.org/10.2111/REM-D-10-00047.1

Kok, K. 2009. The potential of fuzzy cognitive maps for semi-quantitative scenario development, with an example from Brazil. Global Environmental Change 19(1):122-133. http://dx.doi.org/10.1016/j.gloenvcha.2008.08.003

Kok, K., I. Bärlund, M. Flörke, I. Holman, M. Gramberger, J. Sendzimir, B. Stuch, and K. Zellmer. 2015. European participatory scenario development: strengthening the link between stories and models. Climatic Change 128(3-4):187-200. http://dx.doi.org/10.1007/s10584-014-1143-y

Kosow, H., and R. Gaßner. 2008. Methods of future and scenario analysis. Deutsches Institut für Entwicklungspolitik, Bonn, Germany.

Lang, D. J., A. Wiek, M. Bergmann, M. Stauffacher, P. Martens, P. Moll, M. Swilling, and C. J. Thomas. 2012. Transdisciplinary research in sustainability science: practice, principles, and challenges. Sustainability Science 7(S1):25-43. http://dx.doi.org/10.1007/s11625-011-0149-x

Ledford, H. 2015. How to solve the world’s biggest problems. Nature 525(7569):308-311. http://dx.doi.org/10.1038/525308a

Lélé, S., and R. B. Norgaard. 2005. Practicing interdisciplinarity. BioScience 55(11):967–975. http://dx.doi.org/10.1641/0006-3568(2005)055[0967:PI]2.0.CO;2

Mallampalli, V. R., G. Mavrommati, J. Thompson, M. Duveneck, S. Meyer, A. Ligmann-Zielinska, C. G. Druschke, K. Hychka, M. A. Kenney, K. Kok, and M. E. Borsuk. 2016. Methods for translating narrative scenarios into quantitative assessments of land use change. Environmental Modelling and Software 82:7-20. http://dx.doi.org/10.1016/j.envsoft.2016.04.011

Max-Neef, M. A. 2005. Foundations of transdisciplinarity. Ecological Economics 53:5-16. http://dx.doi.org/10.1016/j.ecolecon.2005.01.014

Novak, J. D. 2010. Learning, creating, and using knowledge: concept maps as facilitative tools in schools and corporations. Journal of E-Learning and Knowledge Society 6(3):21-30.

Oteros-Rozas, E., B. Martín-López, C. A. López, I. Palomo, and J. A. González. 2013. Envisioning the future of transhumant pastoralism through participatory scenario planning: a case study in Spain. Rangeland Journal 35:251-272. http://dx.doi.org/10.1071/RJ12092

Palmer, M. A., J. G. Kramer, J. Boyd, and D. Hawthorne. 2016. Practices for facilitating interdisciplinary synthetic research: the National Socio-Environmental Synthesis Center (SESYNC). Current Opinion in Environmental Sustainability 19:111-122. http://dx.doi.org/10.1016/j.cosust.2016.01.002

Pennington, D. D. 2008. Cross-disciplinary collaboration and learning. Ecology and Society 13(2):8. http://dx.doi.org/10.5751/ES-02520-130208

Pennington, D. D. 2010. The dynamics of material artifacts in collaborative research teams. Computer Supported Cooperative Work 19(2):175-199. http://dx.doi.org/10.1007/s10606-010-9108-9

Pennington D. 2016. A conceptual model for knowledge integration in interdisciplinary teams: orchestrating individual learning and group processes. Journal of Environmental Studies and Sciences 6:300-312. http://dx.doi.org/10.1007/s13412-015-0354-5

Pennington, D. D., G. L. Simpson, M. S. McConnell, J. M. Fair, and R. J. Baker. 2013. Transdisciplinary research, transformative learning, and transformative science. BioScience 63(7):564-573. http://dx.doi.org/10.1525/bio.2013.63.7.9

Peterson, G. D., T. D. Beard Jr., B. E. Beisner, E. M. Bennett, S. R. Carpenter, G. S. Cumming, C. L. Dent, and T. D. Havlicek. 2003b. Assessing future ecosystem services: a case study of the Northern Highlands Lake District, Wisconsin. Conservation Ecology 7(3):1. http://dx.doi.org/10.5751/ES-00557-070301

Peterson, G. D., G. S. Cumming, and S. R. Carpenter. 2003a. Scenario planning: a tool for conservation in an uncertain world. Conservation Biology 17(2):358-366. http://dx.doi.org/10.1046/j.1523-1739.2003.01491.x

Picketts, I. M., A. T. Werner, T. Q. Murdock, J. Curry, S. J. Déry, and D. Dyer. 2012. Planning for climate change adaptation: lessons learned from a community-based workshop. Environmental Science & Policy 17:82-93. http://dx.doi.org/10.1016/j.envsci.2011.12.011

Puig, C.J., R. Greiner, C. Huchery, I. Perkins, L. Bowen, N. Collier, and S. T. Garnett. 2011. Beyond cattle: potential futures of the pastoral industry in the Northern Territory. Rangeland Journal 33:181-194. http://dx.doi.org/10.1071/RJ10043

Rasmussen, L. V., K. Rasmussen, A. Reenberg, and S. Proud. 2012. A system dynamics approach to land use changes in agro-pastoral systems on the desert margins of Sahel. Agricultural Systems 107:56-64. http://dx.doi.org/10.1016/j.agsy.2011.12.002

Rittel, H. W. J., and M. M. Webber. 1973. Dilemmas in a general theory of planning. Policy Sciences 4(2):155-169. http://dx.doi.org/10.1007/BF01405730

Schmitt Olabisi, L. K., A. R. Kapuscinski, K. A. Johnson, P. B. Reich, B. Stenquist, and K. J. Draeger. 2010. Using scenario visioning and participatory system dynamics modeling to investigate the future: lessons from Minnesota 2050. Sustainability 2(8):2686-2706. http://dx.doi.org/10.3390/su2082686

Schoemaker, P. J. H. 1991. When and how to use scenario planning: a heuristic approach with illustration. Journal of Forecasting 10(6):549-564. http://dx.doi.org/10.1002/for.3980100602

Schoemaker, P. J. H. 1995. Scenario planning: a tool for strategic thinking. Sloan Management Review 36:25-40.

Seidl, R. 2015. A functional-dynamic reflection on participatory processes in modeling projects. Ambio 44(8):750-765. http://dx.doi.org/10.1007/s13280-015-0670-8

Seidl, R., and Q. B. Le. 2012. Modelling human-environment systems in transdisciplinary processes. Pages 1811-1818 in R. Seppelt, A.A. Voinov, S. Lange, D. Bankamp, editors. iEMSs 2012 - Managing Resources of a Limited Planet: Proceedings of the Sixth Biennial Meeting of the International Environmental Modelling and Software Society. International Environmental Modelling and Software Society, Manno, Switzerland.

Shen, Q., Q. Chen, B. Tang, S. Yeung, Y. Hu, and G. Cheung. 2009. A system dynamics model for the sustainable land use planning and development. Habitat International 33(1):15-25. http://dx.doi.org/10.1016/j.habitatint.2008.02.004

Star, S. L. 2010. This is not a boundary object: reflections on the origin of a concept. Science, Technology & Human Values 35(5):601-617. http://dx.doi.org/10.1177/0162243910377624

Stokols, D., S. Misra, R. P. Moser, K. L. Hall, and B. K. Taylor. 2008. The ecology of team science: understanding contextual influences on transdisciplinary collaboration. American Journal of Preventive Medicine 35(2):S96-S115. http://dx.doi.org/10.1016/j.amepre.2008.05.003

Thompson, J. N. 2009. The coevolving web of life. American Naturalist 173(2):125-40. http://dx.doi.org/10.1086/595752

Tress, B., G. Tress, and G. Fry. 2005. Defining concepts and the process of knowledge production in integrative research. From Landscape Research to Landscape Planning: Aspects of Integration, Education and Application 12:13-26.

Van den Belt, M. 2004. Mediated modeling: a system dynamics approach to environmental consensus building. Island, Washington, D.C., USA.

Voinov, A., and F. Bousquet. 2010. Modelling with stakeholders. Environmental Modelling & Software 25(11):1268-1281. http://dx.doi.org/10.1016/j.envsoft.2010.03.007

Voinov, A., N. Kolagani, M. K. McCall, P. D. Glynn, M. E. Kragt, F. O. Ostermann, S. A. Pierce, and P. Ramu. 2016. Modelling with stakeholders - next generation. Environmental Modelling and Software 77:196-220. http://dx.doi.org/10.1016/j.envsoft.2015.11.016

Voinov, A., R. Seppelt, S. Reis, J. E. M. S. Nabel, and S. Shokravi. 2014. Values in socio-environmental modelling: persuasion for action or excuse for inaction. Environmental Modelling and Software 53:207-212. http://dx.doi.org/10.1016/j.envsoft.2013.12.005

Winowiecki, L., S. Smukler, K. Shirley, R. Remans, G. Peltier, E. Lothes, E. King, L. Comita, S. Baptista, and L. Alkema. 2011. Tools for enhancing interdisciplinary communication. Sustainability: Science, Practice, & Policy 7(1):74-80. http://dx.doi.org/10.1080/15487733.2011.11908067

Withycombe Keeler, L., A. Wiek, D. D. White, and D. A. Sampson. 2015. Linking stakeholder survey, scenario analysis, and simulation modeling to explore the long-term impacts of regional water governance regimes. Environmental Science and Policy 48:237-249. http://dx.doi.org/10.1016/j.envsci.2015.01.006