Increasing the effectiveness of participatory scenario development through codesign

• Abstract
• Introduction
• Case Study: Advancing Sustainable Land-use Futures in New England
  • Problem context
  • The New England Landscape Futures project
• Collaborative Design Process and Outcomes
  • Codesign of the scenario development process
  • Codesign process decisions
  • Implementation of the scenario development process
  • Evaluation of the scenario development process
• Discussion
  • Importance of tailoring the scenario development process
  • The codesign product: strengths and weaknesses of the designed process
  • Benefits of codesign for informing scenario process development
  • Improving the process of codesign: critical reflections on the New England Landscape Futures Project (NE-LFP) case study
• Conclusion
• Responses to This Article
• Acknowledgments
• Literature Cited
• Erratum

INTRODUCTION

Scenario planning is a method for structured exploration of multiple hypothetical futures that provides a powerful way to explore and understand social-ecological systems while explicitly acknowledging their inherent uncertainty (Peterson et al. 2003). Scenarios are consistent and coherent storylines that reflect different hypotheses about how the future might unfold. By exploring and testing assumptions about possible futures, scenarios can broaden conventional thinking, enhance understanding, and generate new insights relevant to taking meaningful action in complex, dynamic systems (Henrichs et al. 2010, Wilkinson et al. 2013). As such, scenario planning has become popular in social-ecological research and environmental planning as a means for analyzing complex problems and facilitating transformative change (e.g., Rothman 2008, Thompson et al. 2012, Chaudhury et al. 2013, Carpenter et al. 2015, Daw et al. 2015, Oteros-Rozas et al. 2015).

Scenario planners working in environmental systems are increasingly enlisting stakeholders to help develop scenarios (Seppelt et al. 2011, Oteros-Rozas et al. 2015). Following Reed et al. (2013:346) stakeholders can be defined as “those who are affected by or can affect a decision or action.” Engaging stakeholders in scenario development has a wide range of potential benefits, including: improving the quality and relevancy of the scenarios by incorporating diverse perspectives and local knowledge, empowering stakeholders, fostering shared sense making, and helping to enhance the perceived legitimacy and ownership of the results (Berkhout et al. 2002, Cash et al. 2003, Pahl-Wostl 2008). The popularity of participatory approaches may be credited to a growing awareness that the closer scenario development processes are linked to the actual actors involved, the more likely they are to generate relevant insights and to result in successful uptake and action (e.g., Johnson et al. 2012, Vervoort et al. 2014, Bennett 2017).

The result has been a proliferation of methods for, and applications of, participatory environmental scenarios (Volkery et al. 2008, Seppelt et al. 2011, Oteros-Rozas et al. 2015). Although all participatory practices engage stakeholders, they vary in terms of the timing, amount, and type of engagement (Reed et al. 2013, de Vente et al. 2016). For example, stakeholder involvement can range from primarily one-way consultation processes, which dominate the environmental scenario literature (Oteros-Rozas et al. 2015), to more collaborative processes in which researchers and stakeholders codesign the scenario development process to ensure the result meets their needs (Wollenberg et al. 2000, Pahl-Wostl 2008, Volkery and Ribeiro 2009, Henrichs et al. 2010).

The design of a participatory scenario development process involves balancing competing legitimacy, credibility, and saliency demands (Cash et al. 2003, Henrichs et al. 2010, Kunseler et al. 2015). Trade-offs exist, for example, between the amount of involvement required and the ability of stakeholders to participate, and between process complexity and transparency (Volkery et al. 2008, Rounsevell and Metzger 2010, Wright et al. 2013). The imperative for explicit scenario design and the need for balancing competing demands, however, has not yet translated into greater attention to the design of the scenario development process itself. Most studies appear to design or select their scenario approach in an ad-hoc manner (Alcamo 2008a, Oteros-Rozas et al. 2015), with little front-end stakeholder engagement when choosing scenario development processes (Alcamo 2008a, Kok et al. 2011). Scenario applications in the literature, for example, rarely describe the design of the scenario development process or provide rationales for the methods selected (though exceptions exist, e.g., Mitchell et al. 2016). Applications of land-use scenario processes are still in the early stages of learning how to effectively combine stakeholder and scientific (model-based) inputs (Booth et al. 2016, Mallampalli et al. 2016) and have yet to move beyond this exploratory phase toward demands for increased rigor or guidelines in the methods selected for scenario development (Rounsevell et al. 2012, Van Berkel and Verburg 2012, Capitani et al. 2016).

This lack of attention to the scenario development design process is an important gap that risks undermining the level of support offered for the scenario process by the stakeholders and by the people and institutions who authorize them to act (i.e., authorizing environment; Table 1), and therefore the legitimacy and ultimate uptake of results (Kok et al. 2011, Kirchhoff et al. 2013, Mauser et al. 2013). It also risks limiting the utility of the resulting scenarios to researchers (Alcamo 2008b) and the ability of scenario practitioners to learn from and build on past practices and modify participatory scenario processes to better achieve their stated objectives (Oteros-Rozas et al. 2015).

We use the New England Landscape Futures Project as a case study to (1) illustrate a process for collaboratively designing a land-use scenario development process in conjunction with both stakeholders and simulation modelers, (2) evaluate the strengths and weaknesses of the resulting participatory scenario development process, (3) explore how a codesigned process can promote the cooperative ownership of the scenario process and enhance the credibility, salience, and legitimacy of project outcomes. We highlight how the scenario development techniques should follow from project objectives (including research objectives), the problem context, and stakeholder preferences for engagement activities. We also make the case for codesigning the scenario development process using methods that are transparent and replicable. Finally, we reflect on our experience with codesign and potential avenues for improving the application of scenario process codesign in the future. Note that although definitions may vary, we use “codesign” to refer to a collaborative approach that actively involves stakeholders in the design process to provide a joint framing of objectives and challenges, and to help ensure the process and results meet their needs and are usable (Mauser et al. 2013).

CASE STUDY: ADVANCING SUSTAINABLE LAND-USE FUTURES IN NEW ENGLAND

Problem context

New England is an 18-million hectare region in the northeastern United States that includes 6 states throughout which total forest cover exceeds 80%, but ranges from 50% (Rhode Island) to 90% (Maine). Eighty percent of the region’s forests are privately owned, including the nation’s largest contiguous block of private commercial forestland (> four million ha) and hundreds of thousands of family forest owners with small to mid-sized parcels totaling > seven million hectares (Butler et al. 2016). After 200 years of forest regrowth following abandonment of colonial-era agriculture expansion, all of the New England states are now losing forest cover (Olofsson et al. 2016). The majority of forest loss is associated with low-density residential development. No centralized authority exists in New England to regulate land use. Instead, it is loosely coordinated through a patchwork of regional planning entities, state policies and permits, and local planning boards. In addition, land trusts and other types of conservation organizations operate to protect valued resources through easements and fee acquisition (approximately, 23% of the region is protected from development) and promote smart growth and other land-use planning measures. This dispersed problem context in which land-use decisions are in the hands of hundreds of thousands of individual land owners defies a predictive approach to understanding and analyzing future landscape conditions and lends itself to a participatory scenarios process that engages diverse stakeholders from across the region in elaborating a range of possible futures.

The New England Landscape Futures Project

The New England Landscape Futures Project (NE-LFP) is an initiative led by the Harvard Forest as a focus of its Long-Term Ecological Research program (LTER) and an associated Research Coordination Network (landscape scenarios, ecosystem services, and benefits to society; S³ RCN). Both the Harvard Forest LTER and the S³ RCN have the dual objectives of advancing research and informing sustainable land-use policy and planning in New England by facilitating knowledge coproduction and collaborative action with practitioners from the public, private, and nonprofit sectors (Foster 2013, Foster et al. 2014). The primary organizing questions of the NE-LFP are (1) how might the New England landscape change over the next 50 years?; (2) what are the possible consequences for people and nature?; and (3) what actions could help sustain important resources in the face of change? The NE-LFP aims to answer these questions by engaging scientists, business owners, government officials, landowners, and nonprofit representatives in the development of a set of alternative landscape futures (scenarios) for New England, as a tool through which to coproduce legitimate and salient knowledge about the consequences of different land-use trajectories for ecosystem services that can inform land-use planning, conservation, and management decisions (e.g., Thompson et al. 2016). Specially, the project aims to:

1. forge a collaborative regional network of researchers and practitioners to better understand land-use challenges facing New England in a time of global change;
2. synthesize and catalyze research about the consequences of alternative land-use futures by codeveloping and sharing scenario narratives and simulations;
3. apply insights to near-term policy, planning, and land management and conservation decisions to help policy makers and practitioners prepare for a suite of novel future land-use challenges; and
4. develop scenario narratives, simulation models, interactive communication products, and re-engagement processes to support that application.

Achieving these goals involves a multiphase process (Appendix 1), and we report primarily on the lessons learned from the initial scenario development phase of the project. The next phase of the project involves the modeling and analysis of the consequences of these stakeholder-developed land-use scenarios for ecosystem services and is still in progress and a planned “scenario application” phase that involves engaging with stakeholders to use the simulation results to collaboratively design and implement shared strategies for sustainable land use is still in development. A summary of the NE-LFP’s goals and problem context is provided in Table 1.

COLLABORATIVE DESIGN PROCESS AND OUTCOMES

Codesign of the scenario development process

The design of any scenario exercise requires decisions about a number of process variables, including the type of scenarios, degree and form of stakeholder involvement, desired complexity, use of qualitative and/or quantitative methods, and the techniques that will be used to generate the scenarios (van Notten et al. 2003, Börjeson et al. 2006, Rounsevell and Metzger 2010). Our aim in codesigning the scenario development process was to tailor the activities and approach to the stated preferences of stakeholders and scientists who were likely to be involved in the project based on their knowledge of the local context and research needs. Our specific process objectives for scenario development were to: (1) design a scenario development process that strengthened support for the process and outcomes by relevant stakeholders and institutions to promote the uptake and use of the results, (2) elicit plausible and decision-relevant scenarios of landscape change at the regional scale with sufficient detail for modeling and analysis, and (3) build the capacity of practitioners to plan for and adapt to multiple futures and the capacity of researchers to coproduce knowledge with stakeholders. It is worth noting that these process objectives cut across each of the three main dimensions of scenario use, i.e., (1) scientific exploration and research, (2) education and capacity building, and (3) decision support and strategic planning, and demanded the design of a process able to balance the different and sometimes competing scenario exercise elements involved (e.g., Henrichs et al. 2010, Kunseler et al. 2015).

The codesign process took place over 10 months in 2014-2015 and incorporated input from relevant stakeholders and scientists through an iterative process that combined workshops, semistructured interviews, and open-ended conversations. This approach is reflective of other codesign applications reported on in the literature (e.g., Mauser et al. 2013, Binder et al. 2015, Reyers et al. 2015, Iwaniec et al. 2016, Page et al. 2016). Codesign activities were conducted as part of implementing the broader S³ RCN program and aimed to inform both the scenario development process and to help establish a shared problem framing and research objectives for the NE-LFP. Participants were identified via a purposive snowball approach that drew on the knowledge and contacts of the core network of S³ RCN collaborators (Creswell and Plano Clark 2007, Foster et al. 2014) and that was geared toward building a community of practitioners, scientists, and policymakers (in line with the aims of the S³ RCN network). The results of this purposive snowball sampling were consolidated using stakeholder mapping along axes of interest and influence (Bryson 2004) and used to inform stakeholder selection for the (1) initial design workshop, (2) interviews, and (3) scenario development workshops. Invited stakeholders could determine their preferred level of involvement, including receiving products derived from the project, participating in scenario development workshops, hosting workshops as partner organizations, and participating as full collaborators actively involved in shaping the ongoing research and engagement efforts.

The codesign process began with a two-day workshop that brought together 35 researchers and stakeholders from across New England with the intent of developing a shared understanding for how best to apply scenario development toward the coproduction of actionable science for use in informing sustainable land-use policy, planning, and stewardship in the region. Attendees at the initial codesign meeting were selected to provide a balance of stakeholders, researchers, and experienced scenario practitioners who could provide insights about challenges and best practices in conducting participatory scenario processes. Activities at the workshop were structured with the intent of facilitating knowledge exchange, building relationships and trust between scientists and stakeholders, and enhancing collaborative capacity (Lemos and Morehouse 2005, Godemann 2008, Cockburn et al. 2016). They included presentations by stakeholders and scientists, panel discussions, facilitated dialogue, small working group sessions, informal conversations, and training sessions. The first day of the workshop focused on establishing a shared knowledge base around (1) the process of conducting transdisciplinary research and determining what “success” looks like from the point of view of stakeholders versus scientists (e.g., Lang et al. 2012), and (2) the application of participatory scenario processes to inform transdisciplinary research and the challenges involved, as informed by the perspectives and recommendations of experienced scenario practitioners (Table 2). The second day of the workshop built on this knowledge foundation and focused on refining a vision for the role of the NE-LFP and for how participatory scenarios research could be used to address the specific land-use challenges facing New England.

The initial blueprint for the scenario development process established at the workshop was further refined over a period of several months via informal stakeholder consultation and an additional 57 semistructured interviews conducted with stakeholder representatives identified from each of the six states (three of which were also attendees at the initial codesign workshop). As with the workshop, the goal of the interviews was improving understanding of both stakeholder preferences regarding scenario development and process outputs as well as their knowledge and concerns surrounding the future of land use in New England. On the basis of the workshops and stakeholder consultations, the broad form that the scenario development process would take was established: a one day workshop held in each state to develop explorative, plausibility-based scenarios, with stakeholders driving the initial scenario development from scratch and remaining actively engaged throughout the process (i.e., Table 3, decisions 1-6). The reasoning that drove each of these decisions will be outlined in greater detail in the next section. To avoid placing undue demands on the stakeholders’ time, the core research team, drawing on both the stakeholders’ inputs and the knowledge and resources available in the scenario planning literature, then developed scenario development materials within this broad process framework.

Codesign process decisions

Collaborative design is an iterative, nonlinear process (Lemos and Morehouse 2005, Sarkki et al. 2015) and rather than attempting to document the decision process in its entirety, we instead focus on detailing several high-priority decisions that strongly influenced the final process. This means that although the decision descriptions that follow may suggest a segmented, sequential process, in practice the design process proceeded in a more iterative fashion, with interactions and overlap present between several of the decisions (e.g., Pahl-Wostl 2008, Henrichs et al. 2010). We framed each focal decision in terms of its relevance to the challenges that exist in the application of foresight-based tools, like scenario planning, within a participatory process aimed at management and policy impact (Table 3). For each decision, we also focused on highlighting the broad trade-offs involved and the major classes of scenario techniques that were considered for each decision rather than listing all possible options, which given the multitude of scenario development techniques available would be impractical (e.g., Bradfield et al. 2005, Bishop et al. 2007).

Decision: type of scenarios

Key trade-offs: supporting a specific decision context versus invoking more explorative and transformative thinking.

The term “scenario” encompasses multiple types of hypothetical futures, most commonly differentiated in terms of whether they inspire normative (what do we want to happen?), exploratory (what may happen?), or predictive (what will happen?) styles of thinking (see van Notten et al. 2003 for more details on the different types of scenarios that exist). Normative scenarios are usually concerned with directly supporting decision making. They are used to examine possible paths for reaching different desired futures to help identify suitable policy options. Not all scenarios need be directly decision focused, however. Scenarios can also act as powerful tools for exploring more general possibilities, and exploratory scenarios can act as a backdrop for strategic conversations that can help to challenge and sharpen the mental models of stakeholders by generating new ideas and insights into the way societal and environmental processes influence one another (van der Heijden 1996). Approaches that combine explorative scenarios with a normative back-casting or policy exploration stage are also gaining traction as a means for coupling the exploration of long-term plausible futures with their implications for short-term (normative) decision making (e.g., Kok et al. 2011, Vervoort et al. 2014).

For the NE-LFP, we opted for a combined approach: developing explorative scenarios along with a planned application stage in which the scenarios would be used as tools to explore alternative decision strategies for achieving sustainable land-use outcomes. Focusing the initial development stage on explorative rather than normative scenario development suited the NE-FLP’s objective for developing the capacity of participants to envision and adapt to multiple futures. Given the multiple and diverse actors influencing land-use decisions in the region, developing a single set of normative scenarios equally relevant to all parties in the time provided would have been challenging. In particular, we lacked a strong enough authorizing environment (e.g., commissioning of the study by a public agency) across the region to allow for the development of normative scenarios that would be widely perceived as having adequate legitimacy. However, this choice to focus on strategy exploration at a subsequent stage in the NE-LFP meant forgoing the more immediate decision relevance that a normative scenario exercise can provide during scenario development (Rounsevell et al. 2012). This left a higher barrier to overcome in establishing the utility, and thus the relevancy and saliency, of long-term exploratory scenarios for stakeholders during this first stage of the engagement process.

Decision: duration of the participation process

Key trade-offs: balancing stakeholders’ time commitment with (1) maximizing stakeholder involvement in the process, (2) providing enough time to grapple with the challenging nature of the thinking involved in building scenarios for research, and (3) providing enough time for stakeholders to develop trust and a shared understanding of the problem.

Within scenario research, the time-consuming nature of a scenario development process has been identified as one of the key challenges to successful implementation (Rickards et al. 2014). Time is required for those involved in the scenario development process to establish trust and build relationships and to develop a shared understanding of the problem, an issue often further heightened by the diverse actors and epistemologies involved (Polk 2015, Reyers et al. 2015, Vervoort et al. 2015, Bennett 2017). Although higher levels of active participation are usually linked to higher levels of engagement and more useful and influential outputs (e.g., Newig and Fritsch 2009, Evely et al. 2011, Clark et al. 2016, Posner et al. 2016), requiring too much time can restrict the ability of stakeholders to participate and may also limit the diversity of stakeholders engaged (Polk 2015). This is especially true of stakeholders for which the scenario project does not fall directly within the purview of their day-to-day tasks and in which the process is not specifically linked to policy development or decision making, as with the NE-LFP (and further heightened by the choice to pursue explorative rather than normative scenario development).

In designing our scenario development process, one of the key guidelines communicated during the initial consultation workshop was to be mindful of not overextending the time commitments being asked of stakeholders and risk limiting their ability to participate. To address this, we reached out to different stakeholder groups and received feedback that a one-day workshop was regarded as the maximum time commitment it was appropriate to ask stakeholders to make. The resulting challenge for the NE-LFP was then how to ensure strong, meaningful engagement with a large, diverse group of stakeholders when working within such a limited time period.

Decision: approach to plausibility

Key trade-offs: maintaining relevancy to stakeholders and specific decision contexts versus invoking more explorative and transformative thinking; challenging (and sometimes uncomfortable) nature of envisioning very different and perhaps idiosyncratic futures versus the need to find common areas of concern and not make too many incursions on stakeholders’ time.

Scenario practices encompass techniques for developing probable, plausible and/or possible futures (Wilkinson 2009). Plausibility-centric approaches, i.e., the more widely used approach in participatory environment applications, aim to articulate multiple alternative futures in a way that explores the limits of possibility rather than make any attempt at forecasting the future (Bishop et al. 2007, Ramírez and Selin 2014). The tendency of people to perceive the future as being largely like the past (Bryson et al. 2016), however, means that relying on preconceptions of what is plausible can be problematic (Ramírez and Selin 2014). An important part of scenario work thus lies in “stretching” thinking about the future by widening the range of possibilities considered plausible (Wright et al. 2013, Bryson et al. 2016). This reperceiving (Wack 1985) must be carefully balanced against the risk of scenarios being deemed too uncomfortable, implausible, or pessimistic to be relevant (Schwartz and Ogilvy 1998).

A variety of approaches can be used to stretch participant thinking beyond standard conceptions of the future. For example, incorporating “shocks” or discontinuities (e.g., Carpenter et al. 2015, Daw et al. 2015) is often seen as one way of achieving a very high level of divergence from existing assumptions about the future (van Notten et al. 2005), in which shocks are usually understood as being significant changes that depart qualitatively from one’s expectations about a phenomenon and what is actually observed. Less extreme options for pursuing the development of divergent scenarios also exist, such as with the widely applied scenario matrix method (e.g., Schwarz 1991, van Notten et al. 2003). Alternatively, Ramírez and Selin (2014) and others have argued for refocusing scenario development around maximizing discomfort and uncertainty to widen perceptions of what is plausible, rather than around strict scenario divergence.

Given that the more divergent scenarios are, the less plausible and thus less relevant to end-users they may be perceived to be, approaches that really push beyond current perceptions of the future may only be suitable when there is sufficient time for deep reflection and exploration (Vervoort et al. 2015). For the NE-LFP, we opted not to explicitly strive for extremely divergent scenarios or to incorporate exercises to generate shocks for inclusion in the scenarios, but instead chose to focus on encouraging participants to embrace uncertainty, discomfort, and to recognize contrasting perspectives about what the future might hold (e.g., across different stakeholder groups). This decision was made in response to feedback received from stakeholders about the types of scenarios they considered most relevant, and were therefore most likely to use and implement. We were also mindful of the time constraints, which meant that more extreme approaches to pushing stakeholders to extend their thinking beyond existing conceptions of plausibility were not necessarily suitable.

Decision: spatial scale for scenarios and the scenario development process

Key trade-offs: local relevance versus needs of scientists, and scale of impact for research.

The spatial scale for a scenario exercise, from local, to regional, to global, is recognized as being of particular concern in the design of participatory scenario processes (Biggs et al. 2007, Henrichs et al. 2010). The choice of spatial scale for a scenario analysis influences the nature of the processes and relationships within social-ecological systems that can be represented. Whether scenarios can be put to effective use is also often a question of whether the scenarios address and display trends at an appropriate scale, that is, the level of interest and relevance to the intended audience or political decision makers (Kok et al. 2017). As such, a tension often arises in environmental assessments between the desire to capture multiscale ecosystem processes and the benefits of tailoring a scenario exercise to particular contexts to address local heterogeneity and enhance relevancy for local stakeholders. Increasingly, multiscale approaches that link scenarios across different geographical scales are being used to reconcile these competing demands, thus enabling the representation of multiscale social-ecological dynamics and allowing for greater saliency across key stakeholders at multiple scales (Zurek and Henrichs 2007, Alcamo 2008b). Options for the development of multiscale scenarios can range considerably, from iterative to sequential processes, and via top down (regional scenarios contextualizing local scenarios) or bottom up (local scenarios informing regional scenarios) approaches. The resulting multilevel scenarios can be either loosely or tightly coupled, depending on project demands, development style, and the feasibility of linking variables and processes across different scales (see Zurek and Henrichs 2007 for more on the different degrees of scenario linkage that are possible). Despite the benefits that multiscale methods can provide for environmental scenario assessments, their use entails a greater investment of time and resources and must be managed carefully to avoid the risk of reduced relevance at any individual scale.

For the NE-LFP, although the scientific aims of the project necessitated a region-wide perspective, the feedback we received from stakeholders was that as a region dominated by “home rule,” i.e., strong local governments, and in which most stakeholders operate within state boundaries, engagement at the state rather than New England-wide level would be important for ensuring local relevance and credibility. An additional consideration was the level of variation across New England from state to state and the fact that a state-based scale was more cognitively manageable for participants and would be better able to draw on their specific knowledge and insights.

To balance these concerns against the broader scientific aims of the project, a decision was made to use a bottom up multilevel approach and develop a set of coordinated state-based scenarios that would then feed into the development of a set of New England-wide scenarios. This led to a shift from the initial plan of holding two north and south New England workshops, to holding workshops in each of the six New England states, with the aim of then assimilating the developed state scenarios into a coherent region-wide scenario to meet the scientific requirements for the process. Although the option of providing stakeholders with a set of global/regional scenarios and then asking them to develop regional/local scenarios in response to these scenarios was considered, this was ultimately rejected over concerns that this would be too cumbersome for a one-day process, and it was also thought that stakeholders might be less willing to accept and work within the constraints of a set of predeveloped scenarios.

Decision: qualitative or quantitative scenarios

Key trade-offs: accessibility to stakeholders versus scientific credibility and the needs of scientists; time required of stakeholders and scientists versus faithfully and transparently representing the stakeholders’ input; potential for reduced interaction and engagement given stakeholders often have less ability to engage with the modeling process.

Techniques for depicting scenarios range from non-numerical qualitative descriptions, like stories and pictures, through to methods involving the extensive use of quantitative and computational tools. Combinations of both qualitative and quantitative techniques are also common (Alcamo 2008b). Quantitative representations deal with numerical information and allow for the explicit representation and testing of underlying assumptions. They are often an important part of scenario work aimed at scientific audiences. However, although using models provides many advantages, their use can also reduce the transparency of scenario outputs to nonscientists and restrict the kinds of issues that can be explored. Qualitative scenarios, in contrast, have the advantage of being easily accessible to stakeholders. Not requiring representation with models also potentially allows for more creative, unrestricted thinking and for representing the views and complexity of many different interests (van Vliet et al. 2012, Freeth and Drimie 2016). Combination “Story and Simulation” style approaches aim to capitalize on the strengths of each format, though at the cost of a lengthier process, and the need to balance the input requirements and constraints of the selected quantitative approach with the need to be transparent and faithful to the underlying scenario descriptions. Although a variety of methods for this exist (Mallampalli et al. 2016), successful translation is ultimately best accomplished through a process that involves multiple iterations between scientists and stakeholders (Alcamo 2008c). Tensions thus arise between the time constraints of the broader process, the complexity of the selected modeling and translation process, and the need to adequately maintain stakeholder understanding of and engagement with the process.

For the NE-LFP, a combination Story and Simulation approach was chosen to help meet the needs of both a research driven process using complex land-use models to explore the implications of different futures for landscape features, such as ecosystem services, and of a successful participatory process that engages and involves stakeholders throughout. In particular, given the desire to make use of an existing framework of land-cover change, ecosystem processes, and ecosystem services (sensu Blumstein and Thompson 2015, Thompson et al. 2016), there was a desire to involve stakeholders at a level that was commensurate with their experience and interests. For the translation step, we opted to involve the stakeholders directly in translating their initial qualitative scenarios into semiquantitative estimates for the set of land-use model inputs during the planned one-day workshops. To help ground their quantitative estimates in science, stakeholders were first provided with relevant information on recent land-use trends in their state. The use of a simple worksheet (see Appendix 2) then allowed participants to specify their estimates in a variety of ways, i.e., numbers, arrows, drawings, words (whichever suited them best), helping to maximize the accessibility of the translation process for stakeholders and foster their continued ownership over the subsequent quantitative translations.

Decision: dividing participation and responsibilities between stakeholders and scientists.

Key trade-offs: balancing stakeholders’ time commitment, expertise, and level of engagement with scientific credibility and the information needs of researchers.

Determining in which stages of the scenario development that stakeholders will take part in and how and which will be completed by the research team is distinct from decisions regarding the duration of the engagement process. It is not uncommon, for example, for researchers to opt to develop initial draft scenarios that are then provided to stakeholders for review and refinement, or alternatively for a process to engage with stakeholders to elicit their knowledge and concerns regarding possible futures, but for the research team to be responsible for scenario construction based on this initial stakeholder input (e.g., Carpenter et al. 2015). Although high levels of stakeholder interaction are encouraged in scenario development (e.g., Schwartz and Ogilvy 1998, Oteros-Rozas et al. 2015), in many instances this must be balanced against the time-intensive nature of developing scenarios and the desire to incorporate scientific knowledge and tools (e.g., Volkery et al. 2008, Bohensky et al. 2011).

Given the recommendations from the initial planning workshop and the aims of the NE-LFP, the direct and ongoing involvement of stakeholders was treated as a priority to ensure maximal stakeholder engagement and ownership of the outcomes. This resulted in a decision to aim for a process that that would walk the stakeholders through the full scenario development process from “empty page” through to semiquantitative descriptions of four scenarios. The desire to enhance the transparency of the scenario modeling stage motivated a choice to involve the stakeholders in translating their initial qualitative scenarios into semiquantitative estimates for input into the set of land-use models. In addition, postworkshop webinars were planned as a means of keeping the stakeholders engaged and involved in the complete scenario development and translation process.

Decision: deductive versus inductive process

Key trade-offs: challenging nature of envisioning very different futures versus the need for a time- and engagement-intensive process.

A wide variety of approaches exist for developing plausibility-focused scenarios (van Notten et al. 2003, Bishop et al. 2007), among which a key distinction lies in whether they enlist deductive (general-to-specific) or inductive (specific-to-general) techniques for identifying the focal uncertainties and plot lines around which the scenarios will be based, a.k.a. the scenario logic (e.g., van der Heijden 1996, Schwartz and Ogilvy 1998, Davis 2002). Deductive processes develop scenarios via a general framework by first identifying the most influential and uncertain drivers of future change and then structuring scenarios around these critical uncertainties to deduce the scenarios. Such an approach provides a platform to support divergent thinking and ensures that appropriately distinct and variable scenarios are developed. In contrast, inductive approaches are much more open-ended and exploratory, with the scenarios emerging from in depth discussions about individual events or plot elements, around which larger scenario storylines are then developed organically (Gallopín and Rijsberman 2000). By building scenarios around individual plot elements relevant to the particular case study, this approach has the potential to yield compelling plot lines with direct links to relevant strategic decisions in the present (van Vliet et al. 2012, Bowman et al. 2013). However, the unstructured nature also results in a process that is both more opaque and more reliant on the creativity and imagination of the participants; in turn driving greater time and facilitation demands to ensure success (Volkery and Ribeiro 2009).

For the NE-LFP, the preworkshop interviews with stakeholders revealed a lack of obvious candidates for key drivers of change or plot elements around which to build up scenarios, creating concern that an inductive process would fail to generate enough divergence. In addition, the time constraints and need to eventually integrate the state-based scenarios into New England-wide scenarios suggested that a structured, repeatable process yielding comparable results would be advantageous. These reasons led to the decision to proceed with a highly structured, deductive style approach, wherein the step-by-step nature of the process would act as scaffolding around which appropriately divergent scenarios could be developed. Such a choice also allowed for the balancing of the need to maximize stakeholder engagement in the process and develop group derived state level scenarios from workshop participants with the need to accomplish the scenario development process within a one-day workshop. Moreover, the structured nature of the process made it easily replicable across the six state workshops so that each workshop generated similar outputs that could then be combined into a single New England-wide scenario set through a logical and transparent process.

Decision: choice of scenario development method

Key trade-offs: challenging nature of envisioning very different futures versus the need for a time- and engagement-intensive process; balancing stakeholders’ time commitment, expertise, and level of engagement with scientific credibility and information needs of researchers.

The vast majority of environmental scenario assessments are derived from the intuitive logics (IL) school of scenario development, though a variety of alternative methodologies for scenario development outside this school also exists (Bishop et al. 2007, Amer et al. 2013). The intuitive logics model is a plausibility-based approach that enables participants, usually within a workshop setting, to create narratives describing unfolding causal processes, resulting in a set of distinct, alternative possible future worlds. Its popularity stems from its accessibility, providing a good mix of sophistication and ease of use for both project organizers and process participants (Wright et al. 2013, Bowman 2016). Its flexibility also lends itself to a wide range of scenario purposes and allows for easy adaptation to a wide variety of contexts. By far the most common instance of IL approach is the deductive-style two-axis/matrix approach developed and popularized by Royal Dutch Shell/Global Business Network (Bradfield et al. 2005). This approach constructs scenarios around two drivers with two extreme states each, resulting in a set of four divergent scenarios that aim to explore the limits of possibility (Schwarz 1991, Rounsevell and Metzger 2010).

Despite its widespread use, there are a number of drawbacks associated with the two-axis/matrix approach. The imposition of a 2 x 2 matrix and axes extremes can drive unnecessary polarization in thinking, for example, and pre-emptively restrict exploration of the future possibility space to around only two uncertainty drivers (Wright et al. 2013, Parker et al. 2015, Bryson et al. 2016, Lord et al. 2016). These criticisms have spurred a growing body of tools for augmenting the standard IL approach, particularly through the use of quantitative decision support tools that can be used to enhance the choice of appropriately divergent scenarios and help counteract the potential shortcomings of purely judgment-driven methods for scenario development (e.g., Parker et al. 2015, Bryson et al. 2016). For example, methods such as morphological analysis (MA) and field anomaly relaxation (FAR) represent alternatives in which a greater number of alternative uncertainty states can be searched and considered (Bishop et al. 2007). However, they can also create additional decision-support complications (e.g., Parker et al. 2015) and are thus less commonly implemented despite their potential benefits (Bradfield 2008, Bryson et al. 2016).

For the NE-LFP, given our limited timeframe and preference for participatory-derived and plausibility-based deductive scenarios, we opted for a condensed version of the deductive two-axis approach from the IL school (Schwarz 1991, Van der Heijden 1996). It fitted well with our objective of maximizing stakeholder inclusion and engagement across a diverse range of participants, and our aims of developing a process that was engaging, creative, encouraged full participation and gave stakeholders authorship of their scenarios, while balancing this with the need to eventually arrive at quantitative scenarios that could be used to inform the simulation modeling. The designed process included the following stages: (1) introduction and orientation to the process, (2) identification of driving forces, (3) selection of key drivers, (4) development of the scenario matrix, (5) fleshing out of the scenario narratives, (6) presentation on recent trends, and (7) conversion of storylines to semiqualitative land-use change estimates for use in modeling (Fig. 1; Table 4).

Implementation of the scenario development process

Scenario development workshops

The codesigned process was implemented through six scenario development workshops held during September-November 2015 and convened in each of the six New England states. At each 1-day workshop, approximately 20-25 stakeholders were in attendance. These stakeholders differed from those attending the initial planning workshop (with the exception of 4 individuals in attendance at both) and overlapped partially with the stakeholders interviewed during the codesign process (16 of the stakeholders attending 1 of the workshops were also 1 of the 57 stakeholders interviewed, and 1 individual was represented at all 3 engagement activities). The 128 attendees were drawn from a mix of the private sector (e.g., real-estate development and forestry), government agencies, nongovernmental organizations operating in the region, and universities.

Regional landscape scenarios generated by stakeholders

As a result of following the process outlined in Figure 1, each state workshop resulted in a 2 x 2 scenario matrix, a set of four qualitative scenario outlines, and rudimentary quantification of the inputs required for the modeling stage of the project (Fig. 2; Appendix 2).

Combing the six sets of state scenario outputs into a single set of four New England-wide scenarios (i.e., regionalization) required a three-tiered approach: (1) aligning state matrices and merging driving forces to create a regional, overarching matrix; (2) integrating, and where necessary, resolving, characteristics across the six states for each of the four resulting scenario quadrants and using these to construct the regional scenario narratives; and similarly (3) aligning the results of the quantification stage across each state for each of the scenario quadrants. For step (1), the process of merging the individual state drivers to generate the regional, overarching drivers is summarized in Figure 3, and the resulting land-use change rates are summarized in Figure 4. This additional feedback led to a number of substantive revisions to the scenarios, including shortening of length, a greater focus on describing land-use change plot elements rather than socioeconomic developments, the addition of plot summaries, and editing for tone to reduce the perception of any one scenario being perceived as the favorite. The final scenario narratives are included in Appendix 3.

Post-workshop feedback and iteration

Following the workshops, two 90-minute interactive webinars were held (each with identical content) during April-May 2016 to provide feedback on the process of integrating the individual state scenario outputs into a combined set of four New England-wide scenarios and to engage the stakeholders again around the translation of the narratives into models. During the webinars, feedback on the integration process was solicited using interactive voting software, together with additional input on how to further flesh out the scenario narratives, the translation of the scenario narratives into models, and candidates for the regional scenario names. For the full list of interactive questions presented during the webinar see Appendix 4. This process acted as a second phase in which the stakeholders could take ownership of the scenarios and played a critical role in maintaining transparency and stakeholder engagement throughout the process of combining workshop outputs into a single set of regional scenarios and translating those storylines into rules for simulation. Results from the webinar voting and feedback were used to revise the scenario narratives and to further refine the process of translating the narratives to model simulations (Fig. 4, b). Following these revisions, a complete draft set of scenario narratives was circulated to the full stakeholder group for comment, continuing the cycle of iteration between stakeholders and scientists in the process of developing the scenarios.

Evaluation of the scenario development process

The relative success of the scenario development process to date was assessed using the participant feedback collected after each of the six scenario elicitation workshops and during the webinars. We note however, that this evaluation gathers feedback at a relatively early stage in the overall process and perceptions may change as additional products are provided to stakeholders (Walter et al. 2007, Roux et al. 2010). Workshop evaluation forms asked participants to list what they had liked, learned, and would change about the process. We used qualitative content analysis to identify themes (Corbin and Strauss 2014) and then organized these in relation to the design trade-offs identified for scenario processes in Table 2.

Postworkshop evaluations suggested that most participants enjoyed the process and that it successfully walked participants through the scenario development process and expanded their thinking about the future. However, many comments also acted as a good illustration of the design trade-offs in action (Table 5). For example, although many participants commented favorably on the organized and efficient nature of the process, others commented that the process felt rushed at times and that more time to further develop the scenarios would have been useful. And although almost all participants reported finding the scenario building process useful as a structure for thinking more expansively about the future that managed to “...tease out an increased understanding” (Maine academic) and encouraged them “...to really think outside the present” (Rhode Island forester) and “...com[e] away with a broader spectrum of what the future look like” (Maine nonprofit director), there were also comments regarding the difficult nature of the task, and that for example, “...[it] was tough to wrap your arms around a 50-year scenario” (Vermont nonprofit director). Additional issues raised included concerns about the legitimacy of the process, “...[we] needed more diverse participants in the room” (Rhode Island land manager), about the credibility, “.... [it] seems like too much of a ‘back of the envelope’ approach on which to base detailed modeling” (New Hampshire nonprofit researcher), and saliency, “...more specifics on how to use this tool in my own organization on a day to day basis. And how this will be used by us in the future” (Maine nonprofit coordinator). Similar feedback was received from both stakeholders who did and did not take part directly in the codesign process, though codesign process participants did not raise any concerns over its relevancy for decision making (i.e., Table 5, row 1). Feedback collected during the webinar on the utility of the scenario products developed revealed that participating stakeholders (n = 26) saw the final regional scenario matrix as being relevant (62%) or somewhat relevant (35%) to their work (Table 6). Similarly, they reported the regional scenario storylines as being relevant (69%) or somewhat relevant (31%) to their work.

DISCUSSION

Importance of tailoring the scenario development process

In any scenario project, there are competing demands on the development process that must be balanced. With the New England Landscape Futures Project, we advanced a framework for codesigning a scenario development process and illustrated how the choice of scenario development techniques should follow from the project objectives, the problem context, and stakeholder preferences. As described in the codesign descriptions, the choice to be informed by stakeholder and scientists’ preferences for a scenario building process influenced the type of scenarios developed, the duration and scale for engagement, the development stages in which stakeholders were involved, the style of scenario building process, and the outputs developed from the workshop (Table 3). Explicit attention to process design helped us to recognize and address the presence of important trade-offs in all participatory scenario processes, i.e., considerations such as (1) time efficiency and level of detail, (2) creative versus analytical thinking, and (3) striving for immediate policy relevance versus more long-orientated planning (Schoemaker 1998, Henrichs et al. 2010, Rounsevell and Metzger 2010, Kunseler et al. 2015, Cairns et al. 2016). Negotiating these trade-offs via a collaborative approach allowed us to design a process for eliciting divergent scenarios that satisfied the research needs of scientists while ensuring stakeholders could be involved throughout the full development process (Table 6).

For the NE-LFP, three priorities drove the majority of decisions made: constructing draft scenarios through a one-day process, engaging stakeholders in all steps of the scenario development process, and generating the information required by researchers. For other settings, a different set of core priorities will shape choices. The protocol implemented in Cairns et al. (2016), for example, was specifically designed to prioritize the ability of senior individuals from industry and government with very limited time availability to participate. This guided the development of an engagement process of three 90-minute workshop sessions with stakeholders in conjunction with the project team completing the bulk of scenario development outside the workshops. This is a different approach to ours, and one that was possible in part because of a stronger authorizing environment and more direct relevance to upcoming policy decisions, but the takeaway in both cases is that initial consideration of engagement objectives and constraints, followed by a deliberate and informed process of design that draws on the experiences and knowledge of the literature and the stakeholders and context, can usefully inform design decisions.

The codesign product: strengths and weaknesses of the designed process

For many planning contexts, the willingness of stakeholders to commit time is likely to be directly proportional to the relevance of the process and outcomes to their professional responsibilities. Thus, for the NE-LFP, characterized by a diffuse authorizing environment and a diverse set of stakeholders (Table 1), the codesign process led to the creation of a robust and highly structured one-day scenario development process that was mostly well received by stakeholders. Even the most well-planned and strategic processes, however, have limits to how well they can quickly develop divergent, nuanced scenarios with a group of diverse and inexperienced participants (Tables 2, 3). We judge that although a one-day process was the correct choice for our project, more time would likely have permitted greater detail and nuance to be incorporated into the scenarios. Alternatively, given that the one-day process was a design priority, including fewer steps in the process and relying on the expertise of researchers to develop the first draft of the quantitative rules of the model simulation could have been a more productive use of stakeholders’ time and interest.

Despite eliciting the required outcomes, implementation across the six workshops also revealed limitations to our scenario development process (Tables 3, 5). Most notably, the choice to involve stakeholders in the initial translation of the qualitative narratives to quantitative inputs for use in the simulation was too much to ask. This stage in the process was geared at balancing the needs of the scientists with transparency and accessibility of the modeling stage for stakeholders. Although the outputs were able to inform the simulation models, stakeholders and facilitator feedback indicated that the inclusion of this quantification stage was challenging for stakeholders to perform in the time permitted. This was also the stage of the regionalization process for which the most revision was required to operationalize the stakeholder’s inputs. Our conclusion was that asking stakeholders to develop semiquantitative estimates may have been pushing them too far beyond their experience, and that a better, equally legitimate approach would have had the scientists converting the storylines into model simulation rules, and then sharing their recommendations with the stakeholders for commentary. This perhaps reflects the importance of engaging individuals in the capacity through which they are best able to contribute and not trying to push them too far beyond that simply for the sake of participation.

Benefits of codesign for informing scenario process development

To what degree does our case study support the use of codesign over more standard modes of scenario process design? A lack of a control study limits our ability to draw any strong conclusions. Nonetheless, one means by which we can explore potential benefits is by comparing stakeholder experiences with our process to that of stakeholder experiences in projects with less of an explicit focus on scenario codesign. A comprehensive review of feedback from participatory scenario studies is beyond our scope. However, Oteros-Rozas et al. (2015) reviewed experiences from 23 participatory scenario processes across a range of applications and acts as a good subsample of studies with which to make comparisons. The stakeholder feedback described across these 23 case studies revealed many commonalities with that received for the NE-LFP. For example, the most frequently cited concerns for the NE-LFP are common across participatory processes and not specific to our case study, such as the time required and the difficulties in compacting engagement processes into a one-day framework (e.g., Oteros-Rozas et al. 2015, Polk 2015, Cockburn et al. 2016, Page et al. 2016). Similarly, for feedback received about the lack of diversity among stakeholders, there is a tendency for certain stakeholder groups, such as industry representatives, to remain underrepresented in transdisciplinary research programs (Johnson et al. 2012, Oteros-Rozas et al. 2015, de Vente et al. 2016). One area in which our process did appear to differ and possibly offer an advantage was in avoiding the more strongly negative reactions to scenario processes reported for some of the 23 case studies (Oteros-Rozas et al. 2015; and see also Reed et al. 2013). The fact that stakeholders who directly participated in the codesign process were less likely to question the usefulness of the scenario development outputs is also promising (see also de Vente et al. 2016).

The postworkshop and webinar questionnaires were intended as formative evaluations and thus provide only a partial and imperfect metric of the relative success of the codesigned scenario development process, owing in part to the biases and limitations with self-reported data, and to the fact that the NE-LFP is still ongoing (Chan 2009, Roux et al. 2010). They also address only a single measure of success, stakeholder perspectives on the scenario workshop process and its outputs. Additional evaluations are planned for the NE-LFP with the intention of elucidating a more holistic understanding of the relative benefits that a codesign approach can provide across a broader range of dimensions, including: quality/effectiveness of the process and products, changes in understanding (e.g., increased knowledge, or awareness, skills, or attitude change), and impacts on practice and policy (Walter et al. 2007, Fazey et al. 2014).

Improving the process of codesign: critical reflections on the New England Landscape Futures Project (NE-LFP) case study

As noted in the process objectives, our intention in codesigning the scenario development process was to tailor the activities and approach to the stated preferences of stakeholders and scientists. A possible limitation with this approach, not considered at the outset, is the risk of developing a process that largely caters to participants’ pre-existing ideas about their needs for long-term orientated land-use policy and planning, missing the opportunity to challenge assumptions about foresight exercises and generate more transformative insights. Our stakeholders, for instance, indicated that one day for scenario development would be all that it would be reasonable to ask people to commit, a design requirement that runs counter to the sufficient time required for scenarios to prompt deeper reflection and exploration (see also Table 2). This risk is particularly relevant when codesigning scenario-based processes, given the disconnect between typical preferences toward pragmatic and time driven outcomes with a concrete focus versus the embracing of uncertainty and openness for exploration and new ideas through which scenarios are most truly successful (Burt and Chermack 2008, Ramírez and Selin 2014, Vervoort et al. 2015, Bowman 2016).

Given that postworkshop feedback included a desire for more time, this raises the question of to what degree a codesign process should only respond to stakeholders’ preferences versus actively trying to inform stakeholders about process requirements that may run counter to their initial expectations (such as more time might be needed than the participants may at first think is necessary). Although our initial codesign workshop included presentations and discussion panels with experienced scenario practitioners to inform stakeholders and scientists about the requirements for successful outcomes, an even greater focus on using codesign as a learning opportunity through which to better establish the motivation for scenario analysis may be beneficial. And because participants directly involved in the codesign process appeared to have a stronger grasp of how scenarios could be eventually applied than those not directly involved, it appears that collaborative design may be particularly useful as a platform through which to establish the utility of foresight knowledge, enhancing its ability to generate actionable science in support of sustainable futures (Page et al. 2016).

A related issue is whether the scope of the scenario codesign process will be limited by the process organizer’s knowledge and understanding of scenario methods. First, any methods the project team are not aware of will necessarily not receive adequate consideration. More subtly, the way in which the project organizers perceive and communicate the techniques and trade-offs to stakeholders will influence the codesign process. This suggests that developing a greater awareness of the wide and ever-evolving range of methodologies available, particularly outside the standard two-axis/matrix approach that dominates in environmental scenario practice (e.g., Rounsevell and Metzger 2010), may be important in extending the scope of a codesign process. For the NE-LFP, despite our best efforts to draw from literature and consult with experienced scenario practitioners operating in the region, we still felt at times that the design process was limited by our lack of knowledge regarding the different methodologies available and how their drawbacks and advantages would play out for our particular context. Some of this could be corrected by greater efforts to report on and evaluate the benefits of different scenario methods, an area of the environmental scenario literature that remains underdeveloped (Oteros-Rozas et al. 2015).

Other aspects of our codesign process worth reflecting on include the balance of time spent on codesign versus actual implementation. A codesign approach entails additional time commitments from participants that might otherwise have been spent on scenario development. The literature on transdisciplinary research suggests that this is time well spent and that participatory processes benefit from a lengthy project scoping and codesign stage as a means to build trust, ownership, and process legitimacy (Kirchhoff et al. 2013, Meadow et al. 2015, Polk 2015, Reyers et al. 2015, Clark et al. 2016, Cockburn et al. 2016, Page et al. 2016). A less straightforward question is what level and type of stakeholder participation in the design process is necessary to ensure success, given the amount of work involved in designing a scenario process when combined with the need to adequately inform stakeholders about scenario practice research. Our approach to navigating this issue was to develop the framework for the process in direct collaboration with stakeholders, while leaving choices about the specific scenario development techniques and materials within this broad process framework to the core research team. The workshop feedback suggests this approach had drawbacks, with details of the process that stakeholders were not directly involved in specifying, such as the process for translating the narratives to semiquantitative outputs, discovered to be problematic upon implementation. The conclusion to draw is not necessarily that the choice to use this approach was incorrect, but rather that by more fully involving stakeholders in this decision process and enhancing their understanding of the reasoning and motivations behind its inclusion, they may have come to view the translation stage as challenging but ultimately worthwhile.

The limitations of the codesign process experienced in our case study suggest that a worthwhile avenue to explore in the future would be a more cyclical approach to scenario exercise codesign, which makes use of multiple iterations of process design and execution (Wilkinson and Eidinow 2008, Vervoort et al. 2014, Sarkki et al. 2015). Codesign processes are integrative in nature and best implemented via an iterative, reflexive cycle (Mauser et al. 2013, Page et al. 2016). Embedding scenario development codesign and implementation into a sustained social learning process would allow participants to learn through experience and application (Lemos and Morehouse 2005, Vervoort et al. 2014, Sarkki et al. 2015). In doing so, it would represent a move away from the use of scenarios as a once only exercise and toward a tool for ongoing adaptive organizational learning (e.g., van der Heijden 1996, Burt and Chermack 2008, Wilkinson and Eidinow 2008, Kok et al. 2011). Current usage suggests that environmental applications may also benefit from giving greater emphasis to the collaborative sense-making aspect of scenario building, rather than the products-based focus that has tended to dominate (Parson 2008, Oteros-Rozas et al. 2015, Ramírez and Wilkinson 2016, Kok et al. 2017).

Finally, any attempt at codesign requires balancing a diversity of needs, preferences, and expectations across different stakeholder groups (Lemos et al. 2014, Clark et al. 2016). Attempting to engage a diverse body of stakeholders within a single process is unlikely to satisfy all participants, and we believe that at least some of the conflicting workshop feedback received (e.g., for longer or shorter processes) reflects this fact. Evolving toward a method that codesigns multiple scenario development processes with individual stakeholder groups would be one approach to overcoming this issue. For the NE-LFP, we did not set out to restrict choices to a single method for scenario development (though this may have been an implicit assumption) and we would have been open to the use of multiple scenario development methods should it have emerged as necessary for balancing competing engagement demands and/or preferences from stakeholders. The codesign of individualized processes would also need to be balanced against the loss of the opportunity to bring together individuals from different sectors for knowledge exchange and partnership building. The greater resources and time commitments required to tailor the process to individual stakeholder groups would also place additional demands on what are likely already limited resources and capacity, an ongoing challenge in the mainstreaming of knowledge coproduction systems (e.g., Cowling et al. 2008, Kirchhoff et al. 2013, Meadow et al. 2015, Polk 2015). As we noted earlier, more targeted stakeholder-specific engagement processes are in planning for the application stage of the NE-LFP.

CONCLUSION

The outcomes from the NE-LFP application suggest that we may be able to increase the odds of fulfilling the transformative potential of participatory scenario planning activities by engaging in a collaborative design process that considers the needs of both researchers and stakeholders from the outset and throughout the process. By actively involving the stakeholders in the design of the scenario development process, we were able to define clear objectives for scenario development with stakeholders and give adequate consideration to how participants could be empowered through the scenario development activities. In the process of walking through a codesign process, we have highlighted the major trade-offs that ought to be considered in the design process, and we have shown how giving adequate consideration to design can have a substantial impact on the resulting scenario development process. Further, our experiences suggest that involving stakeholders in process codesign can act as a shared learning experience with the potential to not only inform the tailoring of the process to the needs of diverse user groups, but to also facilitate a greater understanding among stakeholders about the role for scenario development in generating relevant knowledge to inform land-use planning and decision making. Given the impact that the methods used have on process outcomes, establishing a broader community of practice that aims to share methods, challenges, and outcomes in a comparative way will be important next step for advancing participatory scenario practice.

LITERATURE CITED

Alcamo, J. 2008b. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.

Alcamo, J. 2008c. The SAS approach: combining qualitative and quantitative knowledge in environmental scenarios. Pages 123-150 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.

Alcamo, J. 2008a. Towards guidelines for environmental scenario analysis. Pages 13-35 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.

Amer, M., T. U. Daim, and A. Jetter. 2013. A review of scenario planning. Futures 46:23-40. http://dx.doi.org/10.1016/j.futures.2012.10.003

Bennett, E. M. 2017. Research frontiers in ecosystem service science. Ecosystems 20(1):31-37. http://dx.doi.org/10.1007/s10021-016-0049-0

Berkhout, F., J. Hertin, and A. Jordan. 2002. Socio-economic futures in climate change impact assessment: using scenarios as ‘learning machines’. Global Environmental Change 12(2):83-95. http://dx.doi.org/10.1016/S0959-3780(02)00006-7

Biggs, R., C. Raudsepp-Hearne, C. Atkinson-Palombo, E. Bohensky, E. Boyd, G. Cundill, H. Fox, S. Ingram, K. Kok, S. Spehar, M. Tengö, D. Timmer, and M. Zurek. 2007. Linking futures across scales: a dialog on multiscale scenarios. Ecology and Society 12(1):17. http://dx.doi.org/10.5751/ES-02051-120117

Binder, C. R., I. Absenger-Helmli, and T. Schilling. 2015. The reality of transdisciplinarity: a framework-based self-reflection from science and practice leaders. Sustainability Science 10(4):545-562. http://dx.doi.org/10.1007/s11625-015-0328-2

Bishop, P., A. Hines, and T. Collins. 2007. The current state of scenario development: an overview of techniques. Foresight 9:5-25. http://dx.doi.org/10.1108/14636680710727516

Blumstein, M., and J. R. Thompson. 2015. Land-use impacts on the quantity and configuration of ecosystem service provisioning in Massachusetts, USA. Journal of Applied Ecology 52(4):1009-1019. http://dx.doi.org/10.1111/1365-2664.12444

Bohensky, E., J. R. A. Butler, R. Costanza, I. Bohnet, A. Delisle, K. Fabricius, M. Gooch, I. Kubiszewski, G. Lukacs, P. Pert, and E. Wolanski. 2011. Future makers or future takers? A scenario analysis of climate change and the Great Barrier Reef. Global Environmental Change 21(3):876-893. http://dx.doi.org/10.1016/j.gloenvcha.2011.03.009

Booth, E. G., J. Qiu, S. R. Carpenter, J. Schatz, X. Chen, C. J. Kucharik, S. P. Loheide, II, M. M. Motew, J. M. Seifert, and M. G. Turner. 2016. From qualitative to quantitative environmental scenarios: translating storylines into biophysical modeling inputs at the watershed scale. Environmental Modelling and Software 85:80-97. http://dx.doi.org/10.1016/j.envsoft.2016.08.008

Börjeson, L., M. Höjer, K.-H. Dreborg, T. Ekvall, and G. Finnveden. 2006. Scenario types and techniques: towards a user’s guide. Futures 38(7):723-739. http://dx.doi.org/10.1016/j.futures.2005.12.002

Bowman, G. 2016. The practice of scenario planning: an analysis of inter-and intra-organizational strategizing. British Journal of Management 27(1):77-96. http://dx.doi.org/10.1111/1467-8551.12098

Bowman, G., R. B. MacKay, S. Masrani, and P. McKiernan. 2013. Storytelling and the scenario process: understanding success and failure. Technological Forecasting and Social Change 80:735-748. http://dx.doi.org/10.1016/j.techfore.2012.04.009

Bradfield, R., G. Wright, G. Burt, G. Cairns, and K. Van Der Heijden. 2005. The origins and evolution of scenario techniques in long range business planning. Futures 37(8):795-812. http://dx.doi.org/10.1016/j.futures.2005.01.003

Bradfield, R. M. 2008. Cognitive barriers in the scenario development process. Advances in Developing Human Resources 10(2):198-215. http://dx.doi.org/10.1177/1523422307313320

Bryson, J. M. 2004. What to do when stakeholders matter: stakeholder identification and analysis techniques. Public Management Review 6(1):21-53. http://dx.doi.org/10.1080/14719030410001675722

Bryson, S., M. Grime, A. Murthy, and G. Wright. 2016. Behavioral issues in the practical application of scenario thinking: cognitive biases, effective group facilitation and overcoming business-as-usual thinking. Pages 195-212 in M. Kunc, J. Malpass, and L. White, editors. Behavioral operational research: theory, methodology and practice. Palgrave Macmillan, London, UK. http://dx.doi.org/10.1057/978-1-137-53551-1_10

Burt, G., and T. J. Chermack. 2008. Learning with scenarios: summary and critical issues. Advances in Developing Human Resources 10(2):285-295. http://dx.doi.org/10.1177/1523422307313334

Butler, B., J. H. Hewes, B. J. Dickinson, K. Andrejczyk, S. M. Butler, and M. Markowski-Lindsay. 2016. USDA Forest Service National Woodland Owner Survey: national, regional, and state statistics for family forest and woodland ownerships with 10+ acres, 2011-2013. Forest Service Research Bulletin NRS-99. U.S. Department of Agriculture, Forest Service, Northern Research Station, Newtown Square, Pennsylvania, USA. [online] URL: https://www.fs.fed.us/nrs/pubs/rb/rb_nrs99.pdf

Cairns, G., G. Wright, and P. Fairbrother. 2016. Promoting articulated action from diverse stakeholders in response to public policy scenarios: a case analysis of the use of ‘scenario improvisation’ method. Technological Forecasting and Social Change 103:97-108. http://dx.doi.org/10.1016/j.techfore.2015.10.009

Capitani, C., K. Mukama, B. Mbilinyi, I. Malugu, P. K. T. Munishi, N. D. Burgess, P. J. Platts, S. M. Sallu, and R. Marchant. 2016. From local scenarios to national maps: a participatory framework for envisioning the future of Tanzania. Ecology and Society 21(3):4. http://dx.doi.org/10.5751/es-08565-210304

Carpenter, S. R., E. G. Booth, S. Gillon, C. J. Kucharik, S. Loheide, A. S. Mase, M. Motew, J. Qiu, A. R. Rissman, J. Seifert, E. Soylu, M. Turner, and C. B. Wardropper. 2015. Plausible futures of a social-ecological system: Yahara watershed, Wisconsin, USA. Ecology and Society 20(2):10. http://dx.doi.org/10.5751/es-07433-200210

Cash, D. W., W. C. Clark, F. Alcock, N. M. Dickson, N. Eckley, D. H. Guston, J. Jäger, and R. B. Mitchell. 2003. Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences 100(14):8086-8091. http://dx.doi.org/10.1073/pnas.1231332100

Chan, D. 2009. So why ask me? Are self-report data really that bad? Pages 309-336 in C. E. Lance and R. J. Vandenberg, editors. Statistical and methodological myths and urban legends: doctrine, verity and fable in the organizational and social sciences. Taylor and Francis, New York, New York, USA.

Chaudhury, M., J. Vervoort, P. Kristjanson, P. Ericksen, and A. Ainslie. 2013. Participatory scenarios as a tool to link science and policy on food security under climate change in East Africa. Regional Environmental Change 13(2):389-398. http://dx.doi.org/10.1007/s10113-012-0350-1

Clark, W. C., L. van Kerkhoff, L. Lebel, and G. C. Gallopin. 2016. Crafting usable knowledge for sustainable development. Proceedings of the National Academy of Sciences 113(17):4570-4578. http://dx.doi.org/10.1073/pnas.1601266113

Cockburn, J., M. Rouget, R. Slotow, D. Roberts, R. Boon, E. Douwes, S. O’Donoghue, C. T. Downs, S. Mukherjee, W. Musakwa, O. Mutanga, T. Mwabvu, J. Odindi, A. Odindo, Ş. Procheş, S. Ramdhani, J. Ray-Mukherjee, Sershen, M. C. Schoeman, A. J. Smit, E. Wale, and S. Willows-Munro. 2016. How to build science-action partnerships for local land-use planning and management: lessons from Durban, South Africa. Ecology and Society 21(1):28. http://dx.doi.org/10.5751/es-08109-210128

Corbin, J., and A. Strauss. 2014. Basics of qualitative research: techniques and procedures for developing grounded theory. Sage, Thousand Oaks, California, USA.

Cowling, R. M., B. Egoh, A. T. Knight, P. J. O’Farrell, B. Reyers, M. Rouget, D. J. Roux, A. Welz, and A. Wilhelm-Rechman. 2008. An operational model for mainstreaming ecosystem services for implementation. Proceedings of the National Academy of Sciences 105(28):9483-9488. http://dx.doi.org/10.1073/pnas.0706559105

Creswell, J. W., and V. L. Plano Clark. 2007. Designing and conducting mixed methods research. Sage, Thousand Oaks, California, USA.

Davis, G. 2002. Scenarios as a tool for the 21st century. Probing the future conference. Strathclyde University, Glasgow, Scotland. [online] URL: https://www.pik-potsdam.de/avec/peyresq2005/talks/0921/leemans/literature/davis_how_does_shell_do_scenarios.pd https://www.pik-potsdam.de/avec/peyresq2005/talks/0921/leemans/literature/davis_how_does_shell_do_scenarios.pdf

Daw, T. M., S. Coulthard, W. W. L. Cheung, K. Brown, C. Abunge, D. Galafassi, G. D. Peterson, T. R. McClanahan, J. O. Omukoto, and L. Munyi. 2015. Evaluating taboo trade-offs in ecosystems services and human well-being. Proceedings of the National Academy of Sciences 112(22):6949-6954. http://dx.doi.org/10.1073/pnas.1414900112

de Vente, J., M. S. Reed, L. C. Stringer, S. Valente, and J. Newig. 2016. How does the context and design of participatory decision making processes affect their outcomes? Evidence from sustainable land management in global drylands. Ecology and Society 21(2):24. http://dx.doi.org/10.5751/es-08053-210224

Evely, A. C., M. Pinard, M. S. Reed, and I. Fazey. 2011. High levels of participation in conservation projects enhance learning. Conservation Letters 4(2):116-126. http://dx.doi.org/10.1111/j.1755-263x.2010.00152.x

Fazey, I., L. Bunse, J. Msika, M. Pinke, K. Preedy, A. C. Evely, E. Lambert, E. Hastings, S. Morris, and M. S. Reed. 2014. Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Global Environmental Change 25:204-220. http://dx.doi.org/10.1016/j.gloenvcha.2013.12.012

Foster, D. R. 2013. New science, synthesis, scholarship, and strategic vision for society. Harvard forest lter V 2012-2018. Harvard Forest, Harvard, University, Cambridge, Massachusetts, USA. [online] URL: http://harvardforest.fas.harvard.edu/sites/harvardforest.fas.harvard.edu/files/publications/pdfs/LTERV-2012-proposal.pdf

Foster, D. R., K. Fallon Lambert, and J. R. Thompson. 2014. Integrating land-use scenarios, ecosystem services, and linkages to society. National Science Foundation, Harvard Forest, Harvard University, Cambridge, Massachusetts, USA.

Freeth, R., and S. Drimie. 2016. Participatory scenario planning: from scenario ‘stakeholders’ to scenario ‘owners’. Environment: Science and Policy for Sustainable Development 58(4):32-43. http://dx.doi.org/10.1080/00139157.2016.1186441

Gallopín, G. C., and F. Rijsberman. 2000. Three global water scenarios. International Journal of Water 1(1):16-40. http://dx.doi.org/10.1504/ijw.2000.002055

Godemann, J. 2008. Knowledge integration: a key challenge for transdisciplinary cooperation. Environmental Education Research 14(6):625-641. http://dx.doi.org/10.1080/13504620802469188

Henrichs, T., M. Zurek, B. Eickhout, K. Kok, C. Raudsepp-Hearne, T. Ribeiro, D. van Vuuren, and A. Volkery. 2010. Scenario development and analysis for forward-looking ecosystem assessments. Pages 151-220 in N. Ash, H. Blanco, K. Garcia, T. Tomich, B. Vira, M. Zurek, and C. Brown, editors. Ecosystems and human well-being: a manual for assessment practitioners. Island, Washington, D.C., USA.

Iwaniec, D. M., G. S. Metson, and D. Cordell. 2016. P-FUTURES: towards urban food and water security through collaborative design and impact. Current Opinion in Environmental Sustainability 20:1-7. http://dx.doi.org/10.1016/j.cosust.2016.03.001

Johnson, K. A., G. Dana, N. R. Jordan, K. J. Draeger, A. Kapuscinski, L. K. Schmitt Olabisi, and P. B. Reich. 2012. Using participatory scenarios to stimulate social learning for collaborative sustainable development. Ecology and Society 17(2):9. http://dx.doi.org/10.5751/es-04780-170209

Kirchhoff, C. J., M. Carmen Lemos, and S. Dessai. 2013. Actionable knowledge for environmental decision making: broadening the usability of climate science. Annual Review of Environment and Resources 38:393-414. http://dx.doi.org/10.1146/annurev-environ-022112-112828

Kok, K., M. van Vliet, I. Bärlund, A. Dubel, and J. Sendzimir. 2011. Combining participative backcasting and exploratory scenario development: experiences from the SCENES project. Technological Forecasting and Social Change 78(5):835-851. http://dx.doi.org/10.1016/j.techfore.2011.01.004

Kok, M. T. J., K. Kok, G. D. Peterson, R. Hill, J. Agard, and S. R. Carpenter. 2017. Biodiversity and ecosystem services require IPBES to take novel approach to scenarios. Sustainability Science 12(1):177-181. http://dx.doi.org/10.1007/s11625-016-0354-8

Kunseler, E.-M., W. Tuinstra, E. Vasileiadou, and A. C. Petersen. 2015. The reflective futures practitioner: balancing salience, credibility and legitimacy in generating foresight knowledge with stakeholders. Futures 66:1-12. http://dx.doi.org/10.1016/j.futures.2014.10.006

Lang, D. J., A. Wiek, M. Bergmann, M. Stauffacher, P. Martens, P. Moll, M. Swilling, and C. J. Thomas. 2012. Transdisciplinary research in sustainability science: practice, principles, and challenges. Sustainability Science 7(1):25-43. http://dx.doi.org/10.1007/s11625-011-0149-x

Lemos, M. C., C. J. Kirchhoff, S. E. Kalafatis, D. Scavia, and R. B. Rood. 2014. Moving climate information off the shelf: boundary chains and the role of RISAs as adaptive organizations. Weather, Climate, and Society 6(2):273-285. http://dx.doi.org/10.1175/wcas-d-13-00044.1

Lemos, M. C., and B. J. Morehouse. 2005. The co-production of science and policy in integrated climate assessments. Global Environmental Change 15(1):57-68. http://dx.doi.org/10.1016/j.gloenvcha.2004.09.004

Lord, S., A. Helfgott, and J. M. Vervoort. 2016. Choosing diverse sets of plausible scenarios in multidimensional exploratory futures techniques. Futures 77:11-27. http://dx.doi.org/10.1016/j.futures.2015.12.003

Mallampalli, V. R., G. Mavrommati, J. Thompson, M. Duveneck, S. Meyer, A. Ligmann-Zielinska, C. G. Druschke, K. Hychka, M. A. Kenney, K. Kok, and M. E. Borsuk. 2016. Methods for translating narrative scenarios into quantitative assessments of land use change. Environmental Modelling and Software 82:7-20. http://dx.doi.org/10.1016/j.envsoft.2016.04.011

Mauser, W., G. Klepper, M. Rice, B. S. Schmalzbauer, H. Hackmann, R. Leemans, and H. Moore. 2013. Transdisciplinary global change research: the co-creation of knowledge for sustainability. Current Opinion in Environmental Sustainability 5(3):420-431. http://dx.doi.org/10.1016/j.cosust.2013.07.001

Meadow, A. M., D. B. Ferguson, Z. Guido, A. Horangic, G. Owen, and T. Wall. 2015. Moving toward the deliberate coproduction of climate science knowledge. Weather, Climate, and Society 7(2):179-191. http://dx.doi.org/10.1175/wcas-d-14-00050.1

Mitchell, M., M. Lockwood, S. A. Moore, and S. Clement. 2016. Building systems-based scenario narratives for novel biodiversity futures in an agricultural landscape. Landscape and Urban Planning 145:45-56. http://dx.doi.org/10.1016/j.landurbplan.2015.09.003

Newig, J., and O. Fritsch. 2009. Environmental governance: participatory, multi-level and effective? Environmental Policy and Governance 19(3):197-214. http://dx.doi.org/10.1002/eet.509

Olofsson, P., C. E. Holden, E. L. Bullock, and C. E. Woodcock. 2016. Time series analysis of satellite data reveals continuous deforestation of New England since the 1980s. Environmental Research Letters 11(6):064002. http://dx.doi.org/10.1088/1748-9326/11/6/064002

Oteros-Rozas, E., B. Martín-López, T. Daw, E. L. Bohensky, J. Butler, R. Hill, J. Martin-Ortega, A. Quinlan, F. Ravera, I. Ruiz-Mallén, M. Thyresson, J. Mistry, I. Palomo, G. D. Peterson, T. Plieninger, K. A. Waylen, D. Beach, I. C. Bohnet, M. Hamann, J. Hanspach, K. Hubacek, S. Lavorel, and S. Vilardy. 2015. Participatory scenario planning in place-based social-ecological research: insights and experiences from 23 case studies. Ecology and Society 20(4):32. http://dx.doi.org/10.5751/ES-07985-200432

Page, G. G., R. M. Wise, L. Lindenfeld, P. Moug, A. Hodgson, C. Wyborn, and I. Fazey. 2016. Co-designing transformation research: lessons learned from research on deliberate practices for transformation. Current Opinion in Environmental Sustainability 20:86-92. http://dx.doi.org/10.1016/j.cosust.2016.09.001

Pahl-Wostl, C. 2008. Participation in building environmental scenarios. Pages 105-122 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.

Parker, A. M., S. V. Srinivasan, R. J. Lempert, and S. H. Berry. 2015. Evaluating simulation-derived scenarios for effective decision support. Technological Forecasting and Social Change 91:64-77. http://dx.doi.org/10.1016/j.techfore.2014.01.010

Parson, E. A. 2008. Useful global-change scenarios: current issues and challenges. Environmental Research Letters 3(4):045016. http://dx.doi.org/10.1088/1748-9326/3/4/045016

Peterson, G. D., G. S. Cumming, and S. R. Carpenter. 2003. Scenario planning: a tool for conservation in an uncertain world. Conservation Biology 17(2):358-366. http://dx.doi.org/10.1046/j.1523-1739.2003.01491.x

Polk, M. 2015. Transdisciplinary co-production: designing and testing a transdisciplinary research framework for societal problem solving. Futures 65:110-122. http://dx.doi.org/10.1016/j.futures.2014.11.001

Posner, S. M., E. McKenzie, and T. H. Ricketts. 2016. Policy impacts of ecosystem services knowledge. Proceedings of the National Academy of Sciences 113(7):1760-1765. http://dx.doi.org/10.1073/pnas.1502452113

Ramírez, R., and C. Selin. 2014. Plausibility and probability in scenario planning. Foresight 16(1):54-74. http://dx.doi.org/10.1108/fs-08-2012-0061

Ramírez, R., and A. Wilkinson. 2016. Strategic reframing: the Oxford scenario planning approach. Oxford University Press, Oxford, UK. http://dx.doi.org/10.1093/acprof:oso/9780198745693.001.0001

Reed, M. S., J. Kenter, A. Bonn, K. Broad, T. P. Burt, I. R. Fazey, E. D. G. Fraser, K. Hubacek, D. Nainggolan, C. H. Quinn, L. C. Stringer, and F. Ravera. 2013. Participatory scenario development for environmental management: a methodological framework illustrated with experience from the UK uplands. Journal of Environmental Management 128:345-362. http://dx.doi.org/10.1016/j.jenvman.2013.05.016

Reyers, B., J. L. Nel, P. J. O’Farrell, N. Sitas, and D. C. Nel. 2015. Navigating complexity through knowledge coproduction: mainstreaming ecosystem services into disaster risk reduction. Proceedings of the National Academy of Sciences 112(24):7362-7368. http://dx.doi.org/10.1073/pnas.1414374112

Rickards, L., J. Wiseman, T. Edwards, and C. Biggs. 2014. The problem of fit: scenario planning and climate change adaptation in the public sector. Environment and Planning C: Government and Policy 32(4):641-662. http://dx.doi.org/10.1068/c12106

Rothman, D. S. 2008. A survey of environmental scenarios. Pages 37-65 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.

Rounsevell, M. D. A., and M. J. Metzger. 2010. Developing qualitative scenario storylines for environmental change assessment. Wiley Interdisciplinary Reviews: Climate Change 1(4):606-619. http://dx.doi.org/10.1002/wcc.63

Rounsevell, M. D. A., B. Pedroli, K.-H. Erb, M. Gramberger, A. G. Busck, H. Haberl, S. Kristensen, T. Kuemmerle, S. Lavorel, M. Lindner, H. Lotze-Campen, M. J. Metzger, D. Murray-Rust, A. Popp, M. Pérez-Soba, A. Reenberg, A. Vadineanu, P. H. Verburg, and B. Wolfslehner. 2012. Challenges for land system science. Land Use Policy 29(4):899-910. http://dx.doi.org/10.1016/j.landusepol.2012.01.007

Roux, D. J., R. J. Stirzaker, C. M. Breen, E. C. Lefroy, and H. P. Cresswell. 2010. Framework for participative reflection on the accomplishment of transdisciplinary research programs. Environmental Science and Policy 13(8):733-741. http://dx.doi.org/10.1016/j.envsci.2010.08.002

Sarkki, S., R. Tinch, J. Niemelä, U. Heink, K. Waylen, J. Timaeus, J. Young, A. Watt, C. Neßhöver, and S. van den Hove. 2015. Adding ‘iterativity’ to the credibility, relevance, legitimacy: a novel scheme to highlight dynamic aspects of science-policy interfaces. Environmental Science and Policy 54:505-512. http://dx.doi.org/10.1016/j.envsci.2015.02.016

Schoemaker, P. J. 1998. Twenty common pitfalls in scenario planning. Pages 422-431 in L. Fahey and R. M. Randall, editors. Learning from the future: competitive foresight scenarios. John Wiley and Sons, New York, New York, USA.

Schwarz, P. 1991. The art of the long view: planning for the future in an uncertain world. Doubleday, New York, New York, USA.

Schwartz, P., and J. Ogilvy. 1998. Plotting your scenarios. Pages 57-80 in L. Fahey and R. M. Randall, editors. Learning from the future: competitive foresight scenarios. John Wiley and Sons, New York, New York, USA.

Seppelt, R., C. F. Dormann, F. V. Eppink, S. Lautenbach, and S. Schmidt. 2011. A quantitative review of ecosystem service studies: approaches, shortcomings and the road ahead. Journal of Applied Ecology 48(3):630-636. http://dx.doi.org/10.1111/j.1365-2664.2010.01952.x

Thompson, J. R., K. F. Lambert, D. R. Foster, E. N. Broadbent, M. Blumstein, A. M. A. Zambrano, and Y. Fan. 2016. Four land-use scenarios and their consequences for forest ecosystems and services they provide. Ecosphere 7(10):e01469. http://dx.doi.org/10.1002/ecs2.1469

Thompson, J. R., A. Wiek, F. J. Swanson, S. R. Carpenter, N. Fresco, T. Hollingsworth, T. A. Spies, and D. R. Foster. 2012. Scenario studies as a synthetic and integrative research activity for long-term ecological research. BioScience 62(4):367-376. http://dx.doi.org/10.1525/bio.2012.62.4.8

Van Berkel, D. B., and P. H. Verburg. 2012. Combining exploratory scenarios and participatory backcasting: using an agent-based model in participatory policy design for a multi-functional landscape. Landscape Ecology 27(5):641-658. http://dx.doi.org/10.1007/s10980-012-9730-7

van der Heijden, K. 1996. Scenarios: the art of strategic conversation. John Wiley and Sons, Chichester, UK.

van Notten, P. W. F., J. Rotmans, M. B. A. van Asselt, and D. S. Rothman. 2003. An updated scenario typology. Futures 35(5):423-443. http://dx.doi.org/10.1016/s0016-3287(02)00090-3

van Notten, P. W. F., A. M. Sleegers, and M. B. A. van Asselt. 2005. The future shocks: on discontinuity and scenario development. Technological Forecasting and Social Change 72(2):175-194. http://dx.doi.org/10.1016/j.techfore.2003.12.003

van Vliet, M., K. Kok, A. Veldkamp, and S. Sarkki. 2012. Structure in creativity: an exploratory study to analyse the effects of structuring tools on scenario workshop results. Futures 44(8):746-760. http://dx.doi.org/10.1016/j.futures.2012.05.002

Vervoort, J. M., R. Bendor, A. Kelliher, O. Strik, and A. E. R. Helfgott. 2015. Scenarios and the art of worldmaking. Futures 74:62-70. http://dx.doi.org/10.1016/j.futures.2015.08.009

Vervoort, J. M., P. K. Thornton, P. Kristjanson, W. Förch, P. J. Ericksen, K. Kok, J. S. I. Ingram, M. Herrero, A. Palazzo, A. E. S. Helfgott, A. Wilkinson, P. Havlík, D. Mason-D’Croz, and C. Jost. 2014. Challenges to scenario-guided adaptive action on food security under climate change. Global Environmental Change 28:383-394. http://dx.doi.org/10.1016/j.gloenvcha.2014.03.001

Volkery, A., and T. Ribeiro. 2009. Scenario planning in public policy: understanding use, impacts and the role of institutional context factors. Technological Forecasting and Social Change 76:1198-1207. http://dx.doi.org/10.1016/j.techfore.2009.07.009

Volkery, A., T. Ribeiro, T. Henrichs, and Y. Hoogeveen. 2008. Your vision or my model? Lessons from participatory land use scenario development on a European scale. Systemic Practice and Action Research 21(6):459-477. http://dx.doi.org/10.1007/s11213-008-9104-x

Wack, P. 1985. Scenarios: uncharted waters ahead. Harvard Business Review September 1985. [online] URL: https://hbr.org/1985/09/scenarios-uncharted-waters-ahead

Walter, A. I., S. Helgenberger, A. Wiek, and R. W. Scholz. 2007. Measuring societal effects of transdisciplinary research projects: design and application of an evaluation method. Evaluation and Program Planning 30(4):325-338. http://dx.doi.org/10.1016/j.evalprogplan.2007.08.002

Wilkinson, A. 2009. Scenarios practices: in search of theory. Journal of Futures Studies 13(3):107-114.

Wilkinson, A., and E. Eidinow. 2008. Evolving practices in environmental scenarios: a new scenario typology. Environmental Research Letters 3(4):045017. http://dx.doi.org/10.1088/1748-9326/3/4/045017

Wilkinson, A., R. Kupers, and D. Mangalagiu. 2013. How plausibility-based scenario practices are grappling with complexity to appreciate and address 21st century challenges. Technological Forecasting and Social Change 80:699-710. http://dx.doi.org/10.1016/j.techfore.2012.10.031

Wollenberg, E., D. Edmunds, and L. Buck. 2000. Using scenarios to make decisions about the future: anticipatory learning for the adaptive co-management of community forests. Landscape and Urban Planning 47(1):65-77. http://dx.doi.org/10.1016/s0169-2046(99)00071-7

Wright, G., R. Bradfield, and G. Cairns. 2013. Does the intuitive logics method - and its recent enhancements - produce “effective” scenarios? Technological Forecasting and Social Change 80:631-642. http://dx.doi.org/10.1016/j.techfore.2012.09.003

Zurek, M. B., and T. Henrichs. 2007. Linking scenarios across geographical scales in international environmental assessments. Technological Forecasting and Social Change 74(8):1282-1295. http://dx.doi.org/10.1016/j.techfore.2006.11.005