Systematic evaluation of scenario assessments supporting sustainable integrated natural resources management : evidence from four case studies in Africa

Scenarios have become a key tool for supporting sustainability research on regional and global change. In this study we evaluate four regional scenario assessments: first, to explore a number of research challenges related to sustainability science and, second, to contribute to sustainability research in the specific case studies. The four case studies used commonly applied scenario approaches that are (i) a story and simulation approach with stakeholder participation in the Oum Zessar watershed, Tunisia, (ii) a participatory scenario exploration in the Rwenzori region, Uganda, (iii) a model-based prepolicy study in the Inner Niger Delta, Mali, and (iv) a model coupling-based scenario analysis in upper Thukela basin, South Africa. The scenario assessments are evaluated against a set of known challenges in sustainability science, with each challenge represented by two indicators, complemented by a survey carried out on the perception of the scenario assessments within the case study regions. The results show that all types of scenario assessments address many sustainability challenges, but that the more complex ones based on story and simulation and model coupling are the most comprehensive. The study highlights the need to investigate abrupt system changes as well as governmental and political factors as important sources of uncertainty. For an in-depth analysis of these issues, the use of qualitative approaches and an active engagement of local stakeholders are suggested. Studying ecological thresholds for the regional scale is recommended to support research on regional sustainability. The evaluation of the scenario processes and outcomes by local researchers indicates the most transparent scenario assessments as the most useful. Focused, straightforward, yet iterative scenario assessments can be very relevant by contributing information to selected sustainability problems.


INTRODUCTION
Typically, human-natural systems are exposed to multiple, interlinked stresses (Peters et al. 2008).Local consumption, production patterns, and well-being, for instance, often depend not only on the region's economic and social conditions, but also on the capacities of ecosystems in other regions (Adger et al. 2009).Complex systems often react in a nonlinear way to stresses (Scheffer et al. 2001) because of, for example, crossing system thresholds such as environmental limits or feedback processes (e.g., Steffen et al. 2015).Assessments of future changes are subject to uncertainties that are associated with an incomplete understanding of complex systems, their unpredictable behavior, or human choices to be made in future, i.e., volition (e.g., Raskin et al. 2002, Ash 2010).There is still the question of how science can explore future changes in the human-natural system with its inherent complexity and uncertainty in order to effectively support society for sustainable management and equitable human development (e.g., Gibbons 1999, Kates at al. 2001, Pahl-Wostl 2007).
Africa, with its great diversity of cultures and environments (UNEP 2006), has become the continent with the fastest growing economy (AfDB 2013).Increasing demands for natural resources have caused their widespread overexploitation and strong competition for these critical assets (UNEP 2006, Jalloh et al. 2012).Many African countries will need to make huge efforts to reach food and water security (Fader et al. 2013, Müller 2013, FAO 2015) because of high population growth (UN 2013), widespread low agricultural performance (Mueller et al. 2012), and potential climate change impacts (Niang et al. 2014).The magnitude of expected change and the low adaptive capacity of many communities in Africa pose a serious threat to local livelihoods.Strengthening capacities to maintain and improve well-being are therefore needed (Ludi et al. 2012).
In the face of the diversity, complexity, and pace of environmental and social change, the use of scenarios has become a key tool to analyze sustainability problems (Biggs et al. 2007).Scenario assessments analyze future changes using a variety of approaches.Quantitative scenario assessments provide detailed information that is often required for planning strategies and exploring current and future trade-offs (Hulme and Dessai 2008).An advantage of using numerical models is their ability to process big data sets, crucial in tasks such as climate change impact assessments (Shukla et al. 2009, Dessu and Melesse 2013, Aich et al. 2014).The advance in high-performance computing allows the study of impacts from a range of drivers on interrelated domains or subsystems using model coupling (Guan et al. 2015, Clarke et al. 2017).The evaluation of different sources of uncertainty in impact projections has been done by, for example, applying an ensemble of impact models (e.g., Kassie et al. 2015, Vetter et al. 2016).Apart from quantitative assessments, qualitative scenario The purpose of this study is to compare and evaluate four scenario assessments on urgent sustainability problems in four African case studies that were carried out within the framework of integrated natural resources management (INRM).The scenario assessments are evaluated against a set of indicators directly linked to the research challenges by Swart et al. (2004) and by means of a survey of local researchers on their perception of the usefulness of these scenario assessments to promote sustainability in their case study areas.In doing so, we aim to contribute to sustainability research by feeding into the ongoing debate on developing effective scenario assessments.
The case studies were selected for having strong local partners and established networks with stakeholders and authorities.All four are based in rural areas covering a diversity of ecoregions.Agricultural activities provide a major income source in all case studies, with growing demands leading to strong competition between different natural resource users, increasing degradation of natural resources, and increasing vulnerability of the local population.
The Oum Zessar watershed (OZW) experiences high water competition between different users because of water scarcity (Sghaier and Genin 2003).Groundwater, an important and reliable water resource for many sectors, is heavily exploited (Romagny et al. 2004).Pressures on water resources are increasing through population growth, urbanization, and land-use intensification (Nesheim et al. 2014).Water competition is likely to increase in the future, as climate change simulations project decreasing precipitation and increasing temperatures for northern Africa (Niang et al. 2014), further challenging water management (Omrani and Burger 2012).
The Rwenzori region (RWR) has one of the world's fastest growing populations; Uganda experiences annual growth rates of 3.2% (Baguwemu et al. 2013), and very high population densities locally (NEMA 2010).Competition for land and access to protected areas is increasing (Atukwatse et al. 2012), and meeting the growing food demand will be one of the challenges in this area (KRC 2012).Poor land-use planning and inappropriate land management practices such as bush burning, inappropriate forest cover clearing, or crop husbandry increase pressures on natural resources.(Kabaseke et al. 2012).
In the Inner Niger Delta (IND), natural resources such as bourgou pastures, fish, as well as rice strongly support local livelihoods (Zwarts and Diallo 2005) and are intensively exploited (van der Kamp et al. 2005).Their production depends mainly on the magnitude and duration of inundation in the inland delta (Zwarts and Kone 2005), which, because of low annual rainfalls locally, is a function of the inter-and intra-annual variability in river inflows.This is driven by external factors such as climate variability in the wetter upstream area and changes in the management of the upstream river basin.Because Mali and the other Niger River riparian countries experience pressures to meet the growing food and energy demand, several dams and reservoirs in the upstream catchment have been installed and more are planned, with supposedly huge impacts on the timing and amount of river discharge and thus food production in the delta (NBA 2007, Zwarts and Frerotte 2012, Liersch et al. 2013).
The uThukela basin (UTH) faces severe land degradation processes (Blignaut et al. 2010) because of high stocking rates, a symptom of inappropriate grazing management, and high population densities in rural areas (DWAF 2004, SANBI 2014).In Kwazulu Natal, 28.2% of the households are engaged in agriculture, the majority of them, i.e., in livestock farming (SSA 2011).Land degradation, together with climate variability (risk of droughts, flooding, hails), endangers food and water security in the region (Department of Environmental Affairs andTourism 2006, Osbahr et al. 2010).

Description of scenario assessment processes
We used four stakeholder groups that have an interest in and/or manage natural resources within the project areas and that potentially affect and/or could be affected by project activities or the planning process.Participants are from policy communities (policy maker, donor), networks (farmer organizations, private sector), advocacy coalitions (e.g., NGOs), and epistemic communities (research institutions).The workshops of OZW, RWR, and IND comprised key stakeholders, namely regional experts with a particular field of expertise and influence that is relevant to the research question.These regional experts participated in three workshops in OZW and four workshops in RWR.Regional and national key stakeholders took part in a workshop in IND and a fourth workshop in OZW.A UTH workshop was attended by regional experts and interested nonexperts (civic citizens).The different stakeholder groups were engaged or at least informed during the scenario assessments (see Table A1.2 to Table A1.5 for workshop details).Schwartz (1996) was followed (a) identifying the most important drivers, (b) ranking them by their importance and unpredictability, (c) defining the scenario logics using two axes, (d) fleshing out narratives and trends, (e) discussing implications, and (f) defining leading indicators and signposts | SWOT analysis, where internal Strengths and Weaknesses as well as extePrintable Version...rnal Opportunities and Threats are analyzed, developed by the Harvard Business School in the 1960s (see Andrews 1980, as cited in Kotler et al. 2010)   ¶ WEAP model (Water Evaluation And Planning; http://www.weap21.org/) is a water allocation model (Yates et al. 2005).It was first used by Raskin et al. (1992) in the Aral Sea region and has been applied widely for evaluating and assessing water management strategies taking an integrated perspective (see http://www.weap21.org/index.asp?action=216).# SWIM model (Soil and Water Integrated Model) is a semidistributed continuous-time eco-hydrological model (Krysanova et al. 1998(Krysanova et al. , 2005)).It has been widely applied for simulating river-basin scale hydrological processes, vegetation growth, erosion, and nutrient dynamics (Krysanova et al. 2015), reservoir management, and irrigation water uptake (Koch et al. 2013) as well as inundation processes (Liersch et al. 2013).† † SITE model (SImulation of Terrestrial Environments) is a generic framework for integrated land-use modeling.It produces spatially explicit land-use dynamics through a specified set of rules (Schweitzer et al. 2011).The SITE model has been applied in various contexts (Priess et al. 2011, Das et al. 2012).
or research scope on which to develop scenarios.The OZW and RWR focused on sustainable development by improving INRM of specific natural resources under pressure.Research in the IND and UTH assessed the impacts of climate change and various land and water management options on natural resource availability.Each case study selected scenario drivers, either as a first step in scenario development by involving stakeholders, developing concept maps, and using the Social-Technical-Economic-Environmental-Political (STEEP) framework (OZW, RWR), or from earlier research activities (IND, UTH; e.g., Liersch et al. 2013).
The type of scenario approach applied in the case studies was mainly determined by the role and objectives of the scenario assessment in the INRM decision-making and planning process, key drivers, available resources, and data availability.The four scenario approaches encompass a story and simulation approach (OZW), a participatory scenario exploration (RWR), a modelbased prepolicy study (IND), and a scenario exploration based on model coupling (UTH).
The participatory qualitative scenarios in the OZW and RWR were developed using the scenario axes technique inspired by Schwartz (1996).Concept maps were elaborated to define the https://www.ecologyandsociety.org/vol23/iss1/art5/main factors, drivers, and processes.In the OZW, participatory scenarios were evaluated qualitatively according to the Strengths-Weaknesses-Opportunities-Threats (SWOT) framework (see Andrews 1980, as cited in Kotler et al. 2010) and simulated using the Water Evaluation and Planning (WEAP) model (Yates et al. 2005).In the IND, the eco-hydrological model Soil & Water Integrated Model (SWIM; Krysanova et al. 1998Krysanova et al. , 2005) ) was used to simulate the scenarios, similar to a previous study by Liersch et al. (2013).In the UTH, the SWIM model and the land-use model SImulation of Terrestrial Environments (SITE; Schweitzer et al. 2011) were coupled in order to simulate the scenarios (see van der Kwast et al. 2013).
The aim of the participatory scenario assessments (OZW, RWR) was to assess qualitative impacts related to a number of indicators.In quantitative assessments, impacts were projected on either one (OZW), two (IND), or three indicators (UTH).To account for climate change impacts, the scenario periods ranged from today until 2050.More information on the climate change scenarios can be found in Text Box A1.1.

METHODS
We descriptively evaluate the performance of the four case studies for each of the nine research challenges from Swart et al. (2004) and their effectiveness in potentially enhancing sustainability.The evaluation on the research challenges was done by the researcher team involved in all scenario assessments.The scientific effects of the scenario assessments such as a contribution to new scientific insights and methods (Walter et al. 2007) were collated from a survey of local researchers.
The research challenges were addressed by systematically evaluating scenario products (such as driver definition, narratives) and scenario process features (such as degree of stakeholder engagement, type of tools used) on the basis of an indicator set.Drawing on a literature review, the researcher team designed an indicator set that characterized the processes and outcomes of scenario assessments according to a range of aspects (van Notten et al. 2003, Swart et al. 2004, Biggs et al. 2007, Niemeijer and de Groot 2008, Albert 2013).Two indicators were specified for each research challenge (see Table 2).The specification took into account the measurability, relevance, and intelligibility of the indicator set for scenario assessments as well as the data availability across all case studies (see, e.g., Walz 2000, Niemeijer andde Groot 2008).Using two indicators for each research challenge increases the robustness of the measure and highlights different levels and diverse aspects of implementation.
In five of the nine research challenges (1, 2, 6-8), the two indicators within each research challenge reflect different levels of implementation.The first of the two indicators always describes the lowest possible level of implementation, for example, in research challenge 7 the integration over two spatial scales.Thus, it asks whether a research challenge was addressed or not.The second indicator is always used to describe a more complex level of implementation, for example, two-directional scale interactions in research challenge 7. The indicators of research challenges 3 to 5 and 9 represent two different aspects or ways of implementation in scenario assessments.For example, the first indicator of research challenge 4 characterizes the integration across themes and issues regarding drivers and the second one regarding impact analysis.
The case studies were compared by the researcher team using the score of indicators per research challenge.A minimum score of zero (no indicator of a research challenge was addressed by a case study) and a maximum of two (both indicators were addressed) was possible for each research challenge.
For the survey, local researchers were chosen because they often act as knowledge brokers in their regions.They were selected because of their knowledge and personal interest in the region, and also their degree of formal education.They are important influential persons in their regional networks who provide policy advice, initiating and coordinating (participatory) projects, and maintain contact with other stakeholders, besides advancing scientific knowledge within the region (see Reyers et al. 2015).They were the only stakeholders who had an overview over the whole research process.
The self-administered surveys were carried out approximately one year after finalizing the scenario process (more details on the evaluation in Table A2.1).Groups of two to four researchers per case study were surveyed once and provided an overall evaluation on behalf of their team.
Building on existing literature, the survey consisted of 33 questions, spanning credibility, salience, legitimacy, and capacity building as criteria or principles to bridge the divide of science and nonscience (Cash et al. 2002, Hegger et al. 2012, Chaudhury et al. 2013, Belcher et al. 2016).Credibility describes the technical quality and adequacy of information, salience its relevance for decision making, and legitimacy whether the whole process was fair and respectful of stakeholders (Clark et al. 2016).Capacity building is required to be able to adapt to changes (Kates et al. 2001).The questions could be scored on an ordinal scale between 1 (I absolutely do not agree) and 5 (I absolutely agree; Table A2.2).Each research challenge was operationalized with at least one question in the survey in order to evaluate the benefit from the local researcher's side (see Table A3.1 for details).The remaining questions of the survey were used as additional information to evaluate the overall effectiveness of the four scenario assessments.
The comparison and evaluation of the four scenario assessments was done mostly qualitatively using radar charts and bar plots for visualising results.Figure 2 shows the different research steps.Text Box A3.1 provides details on the contribution of the research team members to this evaluation.

Summary of the scenario building process in each case study
The scenario assessments were used to analyze the following scenario drivers: in OZW the transition period and economic reorientation after the Tunisian revolution in 2010/2011, in RWR the level of environmental awareness of the local population and governmental effectiveness in implementing laws and policies, in IND a new ensemble of climate change scenarios and upstream management options, and in UTH, climate change as well as reservoir and grazing management (Fig. 3).Scenario drivers were analyzed regarding possible changes in future water availability and regional development prospects (OZW and IND) as well as food provisioning (IND).The RWR and UTH studied land degradation with impacts on food production (RWR) and food and water provisioning (UTH).External drivers impact on regional processes External and regional drivers impact on regional processes External drivers impact on regional processes External and regional drivers impact on regional and local processes 7.2.Scenarios included bidirectional scale interactions (multiscale scenarios with feedbacks) Interlinked national and regional narratives 8. Accounting for temporal inertia and urgency 8.1.Analysis of drivers and processes regarding their (different)   Of all research challenges, research challenge 4 (integrating themes) was the most addressed with both indicators being relevant to all case studies and research challenges 1 and 7-9 (combining qualitative and quantitative analysis, spatial scales, temporal scales, uncertainty) the least.Research challenges 1 and 8 were not addressed by all case studies.

Combining qualitative and quantitative analysis
The OZW and UTH applied qualitative and quantitative tools, RWR followed a qualitative approach, and IND was a quantitative model-based analysis.The RWR and OZW produced narratives and OZW subsequently simulated three out of four scenarios using a water allocation model.The UTH first analyzed the main regional processes to develop a qualitative framework for model coupling (van der Kwast et al. 2013), which was then implemented (Pilz 2013, Yalew et al. 2014).

Engaging stakeholders
All case studies informed stakeholders at the beginning of the project about the research plans and consulted them to define the focal issue.In OZW and RWR, stakeholders were actively involved in developing qualitative scenarios, whereas assessments in IND and UTH were science-driven.All case studies transferred scenario outcomes to stakeholders except UTH.

Reflecting multiple stresses and functional complexity
All case studies analyzed multiple stresses (two to three drivers) but with a varying degree of complexity.In the participatory assessments of OZW and RWR, participants selected the two most important and uncertain drivers for the focal issue and described them using a set of characteristics.For instance, the transition period after the Tunisian revolution (OZW), either short or long, was characterized by, for example, the length of time without a constitution, the stability of the political situation, corruption levels, a preference for short-vs long-term measures, or (in)effective governance.Participants then developed ways in which the two drivers might interact (example in Box A2.1).In the model-based exercises (IND, UTH, OZW), drivers encompassed a range of numbers on NRM (Natural Resources Management) and climate change projections.In IND, upstream management assumptions included the settings of built and planned reservoirs, current and future extensions of the seven main irrigation areas, and different assumptions pertaining to water use efficiency.In OZW it encompassed a range of assumed future land and water uses.In contrast to the other three, UTH defined driver assumptions straightforwardly, whereby grazing management (variation of levels of grazing intensity and different grazing lengths) and reservoir management (variation of levels of water abstraction) were tested under climate change conditions.

Integrating across themes and issues
Scenarios in OZW, RWR (qualitative parts), and UTH integrated different themes more than in IND.In these three case studies the developed conceptual models (concept maps, model coupling framework) served as the discussion and description of the key factors and processes over a range of domains (example in Figure A2.1).The IND, however, was strongly focused on hydrology although different technical management details were simulated using an integrated model.Food provisioning was estimated through functional relationships to the projected maximum inundated area and river discharges (see Zwarts et al. 2005, Liersch et al. 2013).The simulation in OZW required a substantial reduction of complexity.It was based on a quantifiable subset of water allocation and management trends, which were logically linked to the development trajectories of the narratives.

Accounting for volition
All case studies assessed the potential implications of a type of human decision that is outside of the stakeholder's sphere of influence: impacts related to the priorities and efficiency of national-scale governance and policy making, and/or climate change.Three case studies also analyzed volition as an internal driver or factor that was controllable by local people.In OZW, for instance, the level of social commitment and cooperation (number of people involved in NGOs, the quality of social networks) introduced differences within the scenarios.In UTH, rules for a decision-based land-use change were used, conceptualized in Figure 3 as "Rules."Rules were introduced for spatially explicit grazing suitability, which declines, for example, with distance from a grid cell of a water class and therefore also relates to socioeconomic factors such as water infrastructure and population growth.

Recognizing a wide range of outlooks, including values and preferences
Scenarios in all case studies spanned various regional-scale futures.Those for OZW and RWR also reflected different values and preferences of the local people.In OZW this is depicted in the development of, for example, agriculture (irrigated vs traditional techniques) or lifestyle (more globalized vs. more traditional).In RWR, cultural values and traditions for implementing adaptation strategies were discussed.Scenarios for IND, in contrast, reflected a system that is completely dependent on the management of the upstream catchment.However, they can support (national) decision makers' choice of one management strategy over another.The assessment for UTH was a sensitivity analysis designed to study system behavior.

Spanning spatial scales
All scenarios covered a variety of scales with mostly external drivers impacting the regional-scale ecosystem and livelihood.
The UTH assessed impacts also at the local scale by producing site-specific information.
Unidirectionally means that drivers impact regional processes, but not the other way around.In contrast, bidirectional interactions in OZW introduced feedback (represented in Fig. 3 through NRM at both scales).An example is the first Tunisian scenario "Liberalization and market orientation," focusing on rapid economic growth.The scenario team evaluated a rapid economic growth until 2050 as implausible mainly because of four reasons.First, Tunisia's economy would be increasingly vulnerable to external shocks because of its strong exportorientation.Second, income disparity was assumed to increase, leading to higher poverty rates and social discontent.Third, Tunisia's limited water resources constrain exploitable natural resources.Fourth, because of the relatively high environmental awareness of the Tunisian population, high environmental degradation would not be tolerated.Based on these arguments, a paradigm change along the scenario time frame was assumed.

Accounting for temporal inertia and urgency
To analyze causes and effects that potentially occur on different time scales was the goal in all case studies except RWR.However, only OZW and to some extent UTH considered feedbacks / an adaption of factors that rather affect the short term (management options to be pursued now) to those causing changes over long periods (like climate change).In OZW, assumptions on scale and sector interactions and available water resources caused changes (dynamics) in regional NRM within the scenario periods.In UTH, land use was a function of climate change and of a threshold of sustainable grassland management.In contrast, NRM in UTH and upstream NRM in IND remained constant during the simulation runs.
It was challenging to find a suitable solution to integrate climate change issues in the participatory exercises (RWR, OZW).Finally, knowledge of climate change projections was presented and indirectly considered by participants as additional information on future changes in general.

Reflecting uncertainty by incorporating surprises, unexpected trajectories, and critical thresholds
Surprises, abrupt changes, or unexpected trajectories were a focus in OZW, RWR and IND.In OZW, the abrupt change after the Tunisian revolution was assessed along four transformation pathways.The RWR scenario team analyzed the possible impacts of natural catastrophes (earthquakes and floods) by adapting existing scenarios.In IND, an extreme scenario was simulated to show potential impacts if all land that was allocated to large-scale farming investors in the past 10 years were to be developed for irrigation farming (see Hertzog et al. 2012).Moreover, the implementation of new reservoirs implies abrupt changes with an immediate impact on the hydrological regime.
Only UTH included an explicitly defined ecological threshold on sustainable grassland management, which is a minimum biomass that is not grazed (Yalew et al. 2014).Stakeholders in OZW assumed a threshold of sustainable water abstraction to develop sustainable future pathways.NRM was changed when they expected longer term unsustainable water abstraction and intolerable land degradation.Preliminary simulation results indicated a fast depletion of groundwater resources in the first scenario because of unrealistic increasing water demands, especially for irrigation.Initial test runs in UTH using the coupled SITE and SWIM models indicated that 35% of the grassland areas do not produce sufficient biomass for sustainable grazing, i.e., they do not surpass the recommended threshold.Grazing pressures may thus lead to soil and grassland degradation (Yalew et al. 2014).

Evaluation of the scenario process and outcomes by local researchers
In general, the surveyed researchers of all case studies evaluated the scenario process and its outcomes as being beneficial for sustainability research within the framework of INRM (Fig. 5, Table A2.2).The evaluations of RWR and IND were the most positive, OZW ranks after them, and UTH last.The results of the surveys suggest that out of the four criteria related to the potential of our research to bridge science and nonscience, the relevance of the scenario process and outcomes is perceived as most different between case studies.
Comparing the respective responses per research challenge with the score of indicators shows no evidence that a higher score for the indicators is related to a better evaluation of the process and outcomes for sustainability research in the case studies (Fig. 6).
The surveys show a similar high agreement across all case studies on the contribution to research challenges 1, 3, and 4. For the other research challenges, the responses are more diverse.In the following, the results of the surveys and the indicator scores are compared for each research challenge.
Results for research challenge 1 show that the quality of scenarios and tools are evaluated high despite the use of different scenario approaches by the different case studies.A reason for the equally positive results for research challenges 3 and 4, the integration of Fig. 5. Evaluation of the local researchers regarding the credibility of scenarios, the relevance of the scenario process and outcomes, the salience of the scenario process, and the contribution of the scenario process and outcomes to capacity building.Dark blue represents the maximum score for each group and red the lowest score.More details on the survey are given in Table A2.1 and Table A2.In UTH, scenarios are found to lack relevance, because the time schedules for the scenario assessment and the regional planning document did not coincide.For this reason, scenario outcomes could be shared with the scientific community but not with the broader public.
Researchers of RWR and IND perceive the most value for understanding the implications of volition (research challenge 5).
Although the OZW scenarios addressed many different types of volition, these resulted in only a low perceived gain in capacity.
Regarding research challenge 6, recognizing a wide range of outlooks, the UTH records the lowest value of all case studies on people's values and preferences for future development, an issue that was not addressed in their scenarios.Researchers of the OZW also evaluate the assessment as not very beneficial in this point but, unlike for UTH, it did form part of the scenarios.
Despite the development of complex scale interactions, the lowest contribution for understanding scale dependencies (research challenge 7) is given by scientists of OZW.As with spatial scales, a more complex level of implementing temporal inertia and urgency (research challenge 8) does not reflect in a more positive evaluation in OZW.Again, RWR and IND score the highest for knowledge gain, although in RWR this issue was not explicitly analyzed.However, it should be mentioned that research challenges 7 and 8 are only covered by one response in the survey.
All case study teams evaluate the assessments as being beneficial for reflecting uncertainty (research challenge 9) despite a considerable range in responses.In OZW, the analysis of the abrupt change after the Tunisian revolution was perceived as highly relevant, hence, that of thresholds not sufficient.By contrast, researchers from IND admit to high capacity building related to thresholds for future research but feel there was a lack of analysis related to the security situation after the violent conflict in Mali.

Strengths and limitations of the applied methods
All of the case studies have complex sustainability problems originating from climate change, population dynamics, land degradation, and/or poor governance.They all involved research into sustainable development amid regional and global change using scenario assessments.The indicator set across all research challenges was designed to compare scenario assessments with respect to a range of aspects related to sustainability science.By comparing case studies, different ways of exploring research challenges in scenario assessments can be shown.This raises awareness of the complexity of sustainability problems, and of the variety of scenario approaches possible that are required to address them.Moreover, research challenges that are of concern across case studies, like the exploration of system thresholds in this study, can be exposed and may indicate the need for more research.Using different and/or more indicators would highlight additional aspects and is therefore encouraged.Comparing scenario assessments, for instance, in terms of contributing to the prioritization, assessment, and implementation of management options could bring additional insights for sustainability studies.2) and the survey responses of local researchers on the perceived usefulness of scenario assessments to contribute to the research challenges (see Table A2.2 and Table A3.Transdisciplinary research processes are usually designed for a variety of goals and stakeholder compositions so that general evaluation schemes are difficult to develop (Walter et al. 2007, Hegger et al. 2012).We used four criteria to evaluate the potential of our research to bridge the divide of science and nonscience by means of a survey (credibility, salience, legitimacy, and capacity building) that have been applied in a number of studies before (e.g., Chaudhury et al. 2013, Kunseler et al. 2015).
Comparing the results of the survey with the evaluation according to the framework of Swart et al. (2004) helped to verify benefits and shortcomings of the scenario assessments for advancing sustainability.A limitation in this regard comes from the small number of responses for some research challenges.Because research challenges 3, 4, and 9 intertwine, they were covered by overlapping response questions.Surveys are, moreover, prone to different response styles and if self-administered there is a risk of interpreting the questions or response selection differently.Both factors affect the validity of the results (Fowler 2013, Roberts 2016).Because of the small number of participants in the surveys and the mentioned limitations of surveys in general, a comparison between case studies must be treated with caution.Another potential weakness of the sampling procedure is the background of the researchers.
Because of the range of knowledge, interests, interpretations, and expertise, and the norms for evaluating the credibility, legitimacy, and salience, the acceptance of produced knowledge varies among actors with different professional backgrounds (White et al. 2010, Kunseler et al. 2015).The majority of researchers of the survey had an environmental background, a tendency that was also observed in the workshops.An engagement of more stakeholders from social sciences and other groups could have enhanced the production of socially robust knowledge on complex Ecology and Society 23(1): 5 https://www.ecologyandsociety.org/vol23/iss1/art5/sustainability problems, supported the development of research strategies to address them, and increased scientific, and societal, effects of the scenario assessments (Gibbons 1999, Raymond et al. 2010).

Implementing the research challenges in the scenario approaches
The comparison of the four case studies shows that not all research challenges highlighted for sustainability science by Swart et al. (2004) were fully met.The most comprehensive case study was OZW, where all research challenges were addressed.The developed scenarios included qualitative and quantitative analysis and a number of stakeholders was actively engaged in the whole process.
The case studies used three strategies of knowledge integration (see Mollinga 2010): active engagement of stakeholders from different domains in qualitative exercises (OZW, RWR), a quantitative model-based analysis (IND, UTH, OZW), and the development of a model coupling framework (UTH).The main advantage of participatory approaches (OZW, RWR) was the possibility of integrating issues with the level of complexity and focus preferred by participants (see also Kok and van Delden 2009).Experienced shortcomings of qualitative approaches were the poor spatial explicitness and the difficulties of addressing climate change adequately, which hampers the qualitative assessment of management options.Similar to other studies, a difficulty in dealing with nonenvironmental drivers or factors such as governance was their translation from narratives into model input due to the required reduction in complexity (Walz et al. 2007, Mason-D'Croz et al. 2016) and the lack of time-series data to calibrate and validate the models in this regard.
The development of conceptual models such as the model coupling framework (UTH) in expert groups or concept maps in participatory processes (OZW, RWR) facilitated the selection and visualization of interactions between relevant system drivers and processes over a range of domains (see Reyers et al. 2015).As is discussed by Birkmann et al. (2015), the benefit of using complex numerical models was the level of detail and the possibility of simulating climate change effects.Drawbacks in this regard were the restriction to quantifiable research questions and data requirements.
Regarding the research challenges, three research needs were expressed across case studies -namely, to investigate system thresholds, to determine spatially explicit local information, and to analyze political and governmental factors.
All case studies required relevant system thresholds to ensure regional sustainability.This experience is reflected in the growing discussion to define a safe operating space for humanity by studying planetary boundaries (e.g., Steffen et al. 2015).In line with Dearing et al. (2014), we argue that boundaries should be developed also for the regional scale where natural resources are mostly managed in order to increase policy relevance.
The case studies analyzed dependencies from external drivers but also needed spatially explicit local information for addressing upstream-downstream issues for example, or planning local adaptation measures.In this regard, the UTH approach for analyzing spatially explicit ecosystem service changes is promising.Producing spatially explicit information as well as linking information across scales has been widely recognized as increasing policy relevance (e.g., Godar et al. 2015, Capitani et al. 2016), but this requires good data availability, resources, and the implementation of appropriate assessment tools, which were not necessarily available in all case studies.Three out of the four case studies show that important uncertainties arise from political or governmental issues; a finding that is also reported by Kok et al. (2007) and Chaudhury et al. (2013).The case studies considered power relations in governance systems, especially land tenure systems, essential for understanding sustainability problems.These were subject of analysis in OZW and RWR.According to Berbés-Blázquez et al. (2016) ecosystem assessments need a stronger focus on power relations because they effect the management and the access to natural resources and thus the social and regional equity in gains and losses from producing ecosystem services.Power dynamics and the inherent power relations of the iterative decision-making process moreover may contribute to difficulties in implementing scientific knowledge in policy (Cáceres et al. 2016).The role of governance for successful resource management has long been recognized (e.g., Acheson 2006) and its better representation in quantitative scenarios was demanded (van Ruijven et al. 2014).There are attempts to quantify dampening effects of weak governance on policy implementation (McNeill et al. 2014) and the conditions that influence the effectiveness of environmental regimes (de Vos et al. 2013).However, the most common approach has been to translate different specifications of governance parameters into model input (e.g., Berkhout et al. 2002) because numerical models are less equipped to analyze governmental issues (de Vos et al. 2013).To date, (participatory) qualitative approaches are therefore probably the most adequate for in-depth analyses of governmental and political issues and their possible developments in the future.
A benefit that could not be clearly attributed to one of the research challenges is that scenario development reveals potential cognitive biases in the judgement of participants (see Tversky and Kahnemann 1974).The OZW team was more reluctant to develop unpleasant scenarios because of the long and unstable transition period after the revolution; an experience that has also been noted by Schoemaker (1993) and Kok et al. (2007).Recognizing the influence of such cognitive biases on scenario development is important for studying potential adaptation measures that should be robust to a range of possible futures (see Metzger et al. 2010).According to Meissner and Wulf (2013), scenario planning does not only reveal but also decreases cognitive biases and therefore improves decision quality; however, this potential benefit needs further analysis.
Scenarios can be more than just a tool for analyzing a range of research challenges.Focusing on interlinkages between a set of research challenges can deliver additional insights to manage sustainable transition.De Vries et al. (2009), for instance, analyzed the connection between different worldviews (values, interests) of people and their human choices and how to translate these into scenarios.The experiences of this study suggest a combination of temporal inertia and urgency and thresholds for analysis.The case studies developed either static or dynamic future trajectories, and both benefit from using thresholds.Analyzing static trajectories with fixed management/driver assumptions, allows the identification of different points in time when the system of analysis passes a threshold and reaches an https://www.ecologyandsociety.org/vol23/iss1/art5/unsupportable state.Analyzing the impact of new dams and reservoirs in this way, such as done in IND, can be directly relevant to policy.The development of dynamic management pathways benefits from thresholds in order to adapt driver assumptions once a critical value is reached.In doing so, short-term decision making needs and long-term climate change effects can be taken into account simultaneously (see Kok et al. 2007).

Evaluation of scenario assessments through local researchers
Combining the comparison of the implementation of the sustainability research challenges with the survey of local researchers made evident that the singularity of every case study influences the type of research challenges and the way they were addressed, but may not directly affect the perceived efficiency of the scenario assessments.The poor agreement of the evaluation with the score of indicators in some research challenges might be partly attributed to the mentioned methodological shortcomings but also to case study-specific reasons.
Reasons for the perceived moderate usefulness of the OZW and UTH assessments encompass the difficulty to understand, to agree on, and to adapt the complex scenario approaches to local contexts.In OZW, the detailed and long process of developing participatory scenarios (see also Kok et al. 2007 andHatzilacou et al. 2007) might have compromised the transparency of the results so that it was hard for end-users to grasp the complexity of scenarios.Rounsevell and Metzger (2010) discuss the difficulty of validating complex qualitative scenarios and this might have contributed to the lack of plausibility and consistency perceived by the OZW scientists (see also Hatzilacou et al. 2007).Model coupling in UTH was a long and IT-driven process and was also not easy to follow for the people who were not involved in this particular process or from this research discipline.
For RWR, it was the first time that the case study team had been involved in such work and therefore the start of science-practice collaboration on INRM.The main reason for the very positive survey responses is that the process likely allowed for the enhancement of basic knowledge regarding many issues and brought together key stakeholders and decision makers.
Whereas RWR developed the big picture as a first step, a detailed analysis of the system's response to selected natural and societal pressures was conducted in IND.For IND, the AFROMAISON project was a follow-up activity of the WETWIN project (Johnston et al. 2013, Shamir andVerhoeven 2013) where important tools (the SWIM model, Liersch et al. 2013) and the broader context had already been worked out (qualitative assessments, see Zsuffa and Cools 2011).Such iterative scenario assessments can contribute or complete information required for evidence-based decision making (see Schoemaker 1995, Wilkinson and Eidinow 2008, Mahmoud et al. 2009).This indicates that breaking down the system complexity into smaller parts can be required because of the ongoing lack of system understanding, even for clearly defined interactions between society and natural systems, as in IND.

SUMMARY AND CONCLUSIONS
Scenario assessments have to address complex human-natural systems when studying sustainability problems amid regional and global change.They are faced with the challenge of recognizing the inherent complexity of system behavior and to reduce but not oversimplify it for analysis.There is a need to integrate knowledge and stakeholders into the process and to raise awareness on complexity but also to make it applicable for end users.Scenario assessments that are well-tailored to the needs of the policy environment can provide an effective support for society for making decisions about complex sustainability problems.We aim with this study to make a contribution to such an effort.
We evaluated four case studies that used scenarios in their sustainability research in the context of INRM.The evaluation combined an analysis of whether and how the case studies implemented Swart et al.'s (2004) research challenges of sustainability and a survey of local researchers to discover the scientific effects of the scenario assessments.The survey was carried out approximately one year after the local projects were finalized.
The limitations of this study are mainly related to the sampling procedures to assess the potential of our research to bridge the divide of science and nonscience.The survey was limited to a small number of local researchers, mainly from natural sciences.It would be extremely valuable to invite feedback from a higher number of local stakeholders and to include the perceptions of different stakeholder groups in future research.Such evaluations could contribute to better understanding of future research needs in a specific decision context as well as opportunities and barriers of implementing research in policy.
One outcome of the survey showed that the comprehensiveness of a scenario assessment in analyzing research challenges does not necessarily mean that it is perceived as useful by scenario users.The application of complex approaches, as in OZW or UTH, need to be carefully planned because they are very resource and time intensive.The expected lengthy process has to be coordinated with policy schedules and the amount of information produced made sufficiently transparent, specific, and usable.
The findings of this study encourage the development of holistic narratives with active stakeholder participation (OZW, RWR) for analyzing new and unknown socio-political situations with potentially high impacts on many stakeholder groups.This facilitates the selection of key processes and indicators for more specific future research.A good overall system understanding (because of earlier research activities, comprehensive cause-effect diagrams, and on the condition that no fundamental system changes occur) allows the system of analysis to be broken down into its components.This is a precondition for studying specific research challenges in sufficient detail.Moreover, the interlinkages of research challenges should be analyzed to increase the understanding of processes related to sustainability problems.
This study adds to the current debate on defining environmental limits for regional human-natural systems.Using thresholds strongly supports the development of useful and long-term management strategies and helps to show temporal mismatches between different drivers, which are generated when short-term management needs confront research recommendations on longterm climate change effects or sustainability goals.
Regional scenarios build a bridge between the national (or higher) scale, where policies are often formulated, and the local scale, where adaptation options are planned and implemented.Their https://www.ecologyandsociety.org/vol23/iss1/art5/relevance is likely to increase if scales are linked by providing spatially explicit outcomes and/or bringing subnational perspectives to higher levels.
Although the findings of this study cannot be generalized given the small number of case studies, the results highlight that individual, locally adapted scenario procedures usually lead to scenarios that are perceived as useful by scenario participants and users.A scenario building process has to be flexible, with a strong connection to previous activities related to the key issue, and make use of data and earlier collaborative work.
Responses to this article can be read online at: http://www.ecologyandsociety.org/issues/responses.php/9728 Box A1.1: Description of climate input data used in scenario assessments For the reference periods in three of the case studies (OZW, IND, UTH) WATCH ERA40 forcing data were used (Weedon et al. 2011).

Details about the workshops on scenario assessments in the case studies
Oum Zessar watershed, Tunisia  The scenario building approach was easy to understand The scenario process contributed to the process of assessing the impact of management options 2 4 5 2 13 The scenario process contributed to the process of implementing management options 2 5 3 1 14 The scenario process improved the knowledge to deal with surprising events / abrupt changes 5 5 2 3

Fig. 2 .
Fig. 2. Research steps followed during this study to systematically evaluate the case studies based on different sources of materials.The left side of the circle describes the steps for analyzing scenario products and scenario process features and the right side visualizes the steps for collecting data through surveys.

Fig. 3 .
Fig. 3. Scenario contents of each case study.Specified interactions between drivers are marked by arrows.Horizontal boxes (OZW, IND): only external scale scenario drivers used; vertical boxes (RWR, UTH): external and internal scale scenario drivers used.Green colors signify the natural system, black the human system, orange surprises, and orange italic colors thresholds.Grey boxes show where volition was addressed.Underlined concepts: quantitative parts of scenarios, else qualitative.CC = climate change, NRM = Natural Resources Management, OZW = Oum Zessar watershed, RWR = Rwenzori region, IND = Inner Niger Delta, UTH = upper Thukela / uThukela basin.
Fig.5.Evaluation of the local researchers regarding the credibility of scenarios, the relevance of the scenario process and outcomes, the salience of the scenario process, and the contribution of the scenario process and outcomes to capacity building.Dark blue represents the maximum score for each group and red the lowest score.More details on the survey are given in TableA2.1 and TableA2.2; the number of questions (n) in each criterion is indicated below the subfigures.OZW = the Oum Zessar watershed, RWR = Rwenzori region, IND = Inner Niger Delta, UTH = upper Thukela / uThukela basin.

Fig. 6 .
Fig. 6.Comparison between the implementation of research challenges in the case studies evaluated by the researcher team (see Table2) and the survey responses of local researchers on the perceived usefulness of scenario assessments to contribute to the research challenges (see TableA2.2 and TableA3.1).Boxes represent the median of the responses per research challenge and case study (n corresponds to the number of responses taken into account for each research challenge).The blue whiskers give the minimum and maximum response value for each research challenge.Filling lines represent the score of indicators evaluated by the researcher team (see Fig.4).OZW = Oum Zessar watershed, RWR = Rwenzori region, IND = Inner Niger Delta, UTH = upper Thukela / uThukela basin.
Fig. 6.Comparison between the implementation of research challenges in the case studies evaluated by the researcher team (see Table2) and the survey responses of local researchers on the perceived usefulness of scenario assessments to contribute to the research challenges (see TableA2.2 and TableA3.1).Boxes represent the median of the responses per research challenge and case study (n corresponds to the number of responses taken into account for each research challenge).The blue whiskers give the minimum and maximum response value for each research challenge.Filling lines represent the score of indicators evaluated by the researcher team (see Fig.4).OZW = Oum Zessar watershed, RWR = Rwenzori region, IND = Inner Niger Delta, UTH = upper Thukela / uThukela basin.

Table 1
summarizes the main outcomes of each scenario assessment.As a first step, each case study defined a focal issue https://www.ecologyandsociety.org/vol23/iss1/art5/

Table 1 .
Summary of the outcomes of the four scenario assessments.

Table 2 .
Main findings regarding the implementation of the nine research challenges in the four case studies.CC = climate change, NRM = Natural Resources Management, OZW = Oum Zessar watershed, RWR = Rwenzori region, IND = Inner Niger Delta, UTH = upper Thukela / uThukela basin.

Table A1 . 2 :
Overview of the workshops undertaken in the Oum Zessar watershed organised by the Institut des Régions Arides, Observatoire du Sahara et du Sahel and Potsdam Institute for Climate Impact Research

Table A1 . 5 :
Overview of the workshop undertaken for the uThukela region organised by the Institute of National Resources Association

Table A2 . 2 :
Survey of local researchers on the usefulness of the scenario process and outcomes for sustainability research to feed into the INRM process in the case studies.The scores represent the overall evaluation of each group of researchers.They are on an ordinal scale from 1 to 5 with 1: I absolutely do not agree, 2: I do not agree, 3: I partially agree, 4: I agree, and 5: I fully agree.