Copyright © 2014 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf
version of this article
The following is the established format for referencing this article:
S. Munaretto, G. Siciliano, and M. E. Turvani. 2014. Integrating adaptive governance and participatory multicriteria methods: a framework for climate adaptation governance. Ecology and Society 19(2): 74.
, part of a special feature on The Governance of Adaptation
Integrating adaptive governance and participatory multicriteria methods: a framework for climate adaptation governance
1Institute for Environmental Studies (IVM), VU University Amsterdam, 2Centre for Development, Environment and Policy (CeDEP), School of Oriental and African Studies (SOAS), University of London, 3Department of Design and Planning in Complex Environments, University IUAV of Venice
Climate adaptation is a dynamic social and institutional process where the governance dimension is receiving growing attention. Adaptive governance is an approach that promises to reduce uncertainty by improving the knowledge base for decision making. As uncertainty is an inherent feature of climate adaptation, adaptive governance seems to be a promising approach for improving climate adaptation governance. However, the adaptive governance literature has so far paid little attention to decision-making tools and methods, and the literature on the governance of adaptation is in its infancy in this regard. We argue that climate adaptation governance would benefit from systematic and yet flexible decision-making tools and methods such as participatory multicriteria methods for the evaluation of adaptation options, and that these methods can be linked to key adaptive governance principles. Moving from these premises, we propose a framework that integrates key adaptive governance features into participatory multicriteria methods for the governance of climate adaptation.
Key words: adaptive capacity, climate change, decision-making tools, experimentation, learning, participation
Adaptation is necessary to prevent and mitigate climate change impacts (Intergovernmental Panel on Climate Change 2007). In its 2001 assessment report, the Intergovernmental Panel on Climate Change suggests that, when evaluating adaptation options, the question of how good these options are according to a plurality of values and dimensions of analysis needs to be addressed (Intergovernmental Panel on Climate Change 2001). This question frames climate change adaptation more as a dynamic social and institutional process than as a problem requiring technical solutions (Hinkel et al. 2010). Indeed, van Nieuwaal et al. (2009:8) argue that “dealing with climate adaptation not only demands a rethink of how we arrange our social–ecological or social–technical systems but also how we govern them.”
Governance, as opposed to government control and steering, refers to “the whole of public as well as private interactions taken to solve societal problems and create societal opportunities. It includes the formulation and application of principles guiding those interactions and care for institutions that enable them” (Kooiman and Bainck 2005:17). The growing number of failed governance attempts to deliver efficient, reliable, and optimal ecosystem goods and services (Walters 1997) has led to calls for more adaptive governance regimes capable of dealing with the inherent complexity and uncertainty of social–ecological systems’ (Dietz et al. 2003, Folke et al. 2005, Armitage et al. 2007). Adaptive governance is a continuous problem-solving process (Plummer and Armitage 2007) “by which institutional arrangements and ecological knowledge are tested and revised in a dynamic, ongoing, self-organized process of learning by doing” (Folke et al. 2005:448). This approach suggests a fundamental paradigm shift from understanding a single part of an ecological system for controlling its variables to understanding the dynamics of the whole social–ecological system for managing its capacity to absorb change and dealing with uncertainty and surprise (Gunderson and Holling 2002, Dietz et al. 2003, Folke et al. 2005, Brunner et al. 2006, Olsson et al. 2006, Lebel et al. 2006). Whereas adaptive governance proponents acknowledge problems of ambivalence of goals, uncertainty of knowledge, asymmetric distribution of power, and additional management costs, such proponents emphasize participation, experimentation, and collective learning as key elements of governance (Folke et al. 2005, Brunner et al. 2006, Armitage 2007, Huitema et al. 2009). With its promise to deal with uncertainty and change, the adaptive governance approach is in keeping with the climate adaptation approach. However, the idea of adaptive governance as a mechanism for climate adaptation is still embryonic. Furthermore, the adaptive governance literature has thus far paid little attention to decision-making tools and methods, focusing more on enabling conditions and design principles (Armitage et al. 2007) whereas the literature on the governance of adaptation is in its infancy in this regard.
Participatory multicriteria methods (PMCM) have been identified as suitable decision support tools for the prioritization of adaptation options, the application of which is growing in different contexts (see, e.g., Qin et al. 2008, Reeder and Ranger 2011, United Nations Climate Change Secretariat 2011, United Nations Environment Programme 2011, Haque et al. 2012, Viguié and Hallegatte 2012, Porthin et al. 2013). These methods help decision makers to evaluate and choose among alternative options that are ranked based on multiple criteria and stakeholder preferences. Stakeholder involvement in the identification of options, criteria, and weights can be limited to just experts, as in the technical decision-making approach, or can be open to nonexperts such as private actors, citizens, and other collectives as in the societal decision-making approach. In technical decision-making processes, “science and technology intersect with the political domain because the issues are of visible relevance to the public” (Collins and Evans 2002:236). However, over the last two decades, the need to include a broad spectrum of stakeholders in decision making has gained societal consensus, especially in relation to urgent societal–environmental problems, such as adaptation to climate change (O’Neill 2001, Parkins and Mitchell 2005, Renn 2006). The idea of societal decision-making approaches is that members of the public can provide their “special technical expertise in virtue of experience that is not recognized by degrees or other certificates as experience-based experts” (Collins and Evans 2002:238; see this full article for further discussion on the notions of expertise and experience in decision making).
We argue that the inclusion of adaptive governance principles in participatory multicriteria evaluations would provide a systematic and yet flexible decision-making approach capable of bridging the gap between governance and action in the climate adaptation literature. The literature is mostly silent on this matter, with the exception of a few examples integrating adaptive management, i.e., the more operational dimension of adaptive governance, and multicriteria methods (Norton and Steinemann 2001, Linkov et al. 2006, Reeder and Ranger 2011, Runge et al. 2011). The framework proposed here aims to go beyond the technical aspects of multicriteria methods (i.e., the aggregation algorithm) and adaptive management, paying attention to stakeholders’ inclusiveness, experimentation, and iterative learning throughout the adaptation decision-making process.
CLIMATE ADAPTATION: KEY ISSUES AND CHALLENGES
Adaptation is a process of deliberate change in anticipation of, or in reaction to, external stimuli and stress (Nelson et al. 2007). Its value in moderating the adverse effects of climate change is increasing recognized within the scientific and policy-making communities (Intergovernmental Panel on Climate Change 2007). Climate change adaptation encompasses a wide range of technical, institutional, legal, educational, and behavioral actions often embedded within sectoral initiatives such as water management planning, agriculture policy, and risk reduction strategies.
The degree of adaptation that is required within the current historical, infrastructural, and technological context is indicated by the degree of vulnerability of a social–ecological system (Wheaton and Maciver 1999). The more adaptive capacity a social–ecological system possesses, the less its vulnerability (Gallopìn 2006, Engle 2011). Adaptive capacity depends on the ability of actors to manage and influence social–ecological resilience and facilitate transformation, that is, to maintain three fundamental systems’ characteristics: the ability to absorb disturbances and still maintain core attributes, the ability to self-organize, and the capacity for learning (Walker et al. 2004, Nelson et al. 2007). Adaptation capacity is dynamic, and is influenced by economic and natural resources, technology, infrastructure, social networks, human resources, and institutions (Füssel and Klein 2006, Intergovernmental Panel on Climate Change 2007, Nelson et al. 2007).
There is scientific uncertainty about future climate conditions on the scale relevant for making adaptation decisions. One way of accounting for this uncertainty is to design adaptation responses according to the principles of flexibility, incrementality, and reversibility so that they can perform under a wide range of climatic conditions or be relatively easily adjusted to withstand more severe climatic shocks (Fankhauser 1999, Intergovernmental Panel on Climate Change 2007, Füssel 2007). Examples of such adaptation options include precautionary and/or mid-lifetime adjustments in the design of long-lived infrastructure, e.g., dams, irrigation systems, bridges, etc. Designing such adaptation responses come at a cost. However, it will often be cheaper than running the risk of expensive modernization or premature replacement (Fankhauser 1999, Intergovernmental Panel on Climate Change 2007).
Decision-making processes on climate adaptation involve many stakeholders. Calls for public participation and the intensive collaboration of experts with different backgrounds in the formulation of adaptive responses are prominent (see, e.g., United Nations Climate Change Secretariat 1992, Intergovernmental Panel on Climate Change 2007, International Institute for Sustainable Development et al. 2003, International Federation of the Red Cross and Red Crescent Societies 2004, Füssel 2007). Collaboration among experts with different backgrounds is advocated as a way to address uncertainty coupled with the multiple interactions taking place attributed to climate change and other stresses on social–ecological systems (Füssel 2007). Simultaneously, a continuous dialog with relevant stakeholders is indispensable for evaluating the compatibility of adaptation options with other policy goals such as sustainable development and economic diversification (Fankhauser 1999, Füssel 2007). Furthermore, active participation in adaptation decision making has the potential to enhance adaptive capacity, increase support for decisions, and ensure that the distribution of burdens and benefits is equitable among different social groups (see, e.g., International Institute for Sustainable Development et al. 2003, Few et al. 2007, Intergovernmental Panel on Climate Change 2007, Collins and Ison 2009). Finally, learning is considered a normative goal of participation in adaptation decisions (Collins and Ison 2009). An assessment of adaptation options can benefit from the different types of knowledge and experience that participants bring to the discussion; e.g., addressing past adaptation.
Scientific research is an important instrument for adaptation. It can improve adaptation in two ways: by providing more reliable information about climate change and its impacts; and by developing and testing improved adaptation options and technologies (Fankhanser 1999, Füssel 2007). The latter implies incorporating the acquisition of new knowledge as one of the goals in the design of adaptation responses. Adaptation measures with such a goal are experimental in nature as they are explicitly meant as a learning tool to gain a better understanding of social–ecological system responses to climate stimuli. This learning function of research in climate adaptation is of crucial importance as our knowledge about social–ecosystem resilience and adaptive capacity to climate stimuli is still limited.
ADAPTIVE GOVERNANCE AND ITS RELEVANCE FOR CLIMATE ADAPTATION
The adaptive governance literature emphasizes experimentation, learning, and participation as attributes of governance that are actively planned for in decision making. For a complete overview of attributes, see Table 1 and the literature cited therein. These three key features are in keeping with the conceptualization of the climate adaptation problem, suggesting that adaptive governance holds potential for addressing the complexity of the adaptation decision-making process.
Experimentation refers to the idea of considering policies and management actions as experiments that, at the same time as managing, also yield new knowledge about ecosystem function and responses to management actions (Walters and Holling 1990, Lee 1993, Huitema et al. 2009). The essence of such an experimental approach is that decision makers seek strategies that can be modified once new information becomes available. In this context, flexible, reversible, and incremental solutions are preferred as way of dealing with uncertainty of knowledge (Lee 1993). These solutions in fact allow for adjustments when more information becomes available and diminish the risk of regretting decisions taken in light of insufficient information. As discussed, this approach is also suggested in climate adaptation decisions. For instance, Porthin et al. (2013) indicate the flexibility of those responding to the principle of scalability of constructions. For example, coastal protection or water storage infrastructure can be designed to allow for midterm adjustments to more severe storms and floods. Yet, Qin et al. (2008) argue that flexible adaptation responses in the water sector should maintain or improve the functioning of the target system under multiple water supply/demand conditions, e.g., by increasing the capacity of water storage systems in anticipation of future droughts. Finally, the case of the adaptation plan for the Thames Estuary (Environment Agency 2012) is an example of an incremental, reversible, and flexible adaptation strategy. Rather than suggesting decision makers make an irreversible decision at the present time about the one or two “best” adaptation options to cope with climate change, the plan adopts a “what if” approach where incremental adaptation occurs over time while maintaining as much flexibility as is desirable about future options (Reeder and Ranger 2011).
Learning, as a normative goal in environmental management and climate adaptation, has gained increasing support in the literature (see, e.g., Lee 1993, International Panel on Climate Change 2007, Armitage 2008, Pelling et al. 2008, Pahl-Wostl 2009, Diduck et al. 2010), although critiques about vague and sometimes uncritical applications of core concepts, assumptions, and approaches have challenged its theoretical underpinning (Armitage 2008, Diduck et al. 2010). Scholars have placed much emphasis on the importance of social learning in environmental management, intended as a property emerging from social interaction when actors collaborate on management actions, or participate in joint decision-making exercises, for example (Armitage et al. 2008, Pahl-Wostl 2009). Learning may concern routines and practices, ideas, and values, and/or critical questions and assumptions about underlying governance systems, referred to as multiple-loop learning; see, e.g., Armitage et al. 2008, Pahl-Wostl 2009. Huitema et al. (2009) also suggest enhanced trust and improved understanding of others’ mindsets as an important form of learning stemming from interactions, that is, relational learning. Improved trust and knowledge accumulates in the collective social memory, strengthening societal adaptive capacity to deal with change (Plummer and Baird 2013). In this vein, learning itself can be considered a kind of adaptive behavior (Pelling et al. 2008). However, the process through which actors learn to be adaptive has just started to be understood (Armitage et al. 2011).
Public participation, intended to be individuals or their collectives taking part in governing processes, may lead to an improved understanding of problems and solutions, greater accountability, transparency, and support for decisions (Few et al. 2007, Reed 2008, Pahl-Wostl 2009, Collins and Ison 2009). However, participatory processes are inherently complex (Few et al. 2007, Reed 2008, Collins and Ison 2009). In discussing modes and degrees of inclusion, Arnstein (1969) argues that simply informing or consulting the public without providing feedbacks may be frustrating for those who invest time in participating, thus putting the credibility of participation at risk. Active participation may be problematic too. For example, pre-existing power asymmetries may persist and reinforce existing privileges, and discourage minority perspectives from being expressed (Reed 2008, Collins and Ison 2009). Some authors have also shown how participatory processes are manipulated by the organizations promoting them (Davos and Lajano 2001, Few et al. 2007). Moreover, scholars have noted that social actors have differing capacities to articulate their concerns (Few et al. 2007), and differing access to participatory processes where their views are not always fairly represented (Bloomfield et al. 2001, Reed 2008). Further, not all relevant stakeholders may want to participate because of negative past experiences and time costs involved in participation (Bloomfield et al. 2001, Few et al. 2007). Despite all these difficulties, the adaptive governance literature reports evidence that well-structured dialogs among policy makers, scientists, and the concerned public can lead to improved natural resource management (Folke et al. 2005, Lebel et al. 2006, Olsson et al. 2006). As discussed, participation is considered of paramount importance also in climate adaptation decision making.
Examples of adaptive governance implemented as response strategies to building adaptive capacity to climate change are rare in the literature. Brunner and Lynch (2010) assess cases to reveal a basic trend toward adaptive governance in climate adaptation, consisting of an increase in community-based initiatives, some of which evolve within networks when leaders become aware of shared interests. They also observed that leaders of these initiatives tend, at least in the beginning, to rely on trust, respect, skill in interpersonal relations, and knowledge of the particular community, and only later on do they look for political support, funds, and other resources needed to implement specific projects. However, some of these initiatives have made considerable progress, and some have not. Plummer and Baird (2013) argue that the implementation of adaptive governance in the context of climate adaptation may be enhanced by having realistic expectations, fitting adaptive governance to the specific situation, and emphasizing conditions for success; e.g., well-defined, small-scale resource systems, stakeholders access to tools for management, presence of leaders supporting the process, explicit support of public agencies, and/or presence of resources and capacity building. Limitations have also been pointed out when applying an adaptive governance approach to climate adaptation. For instance, Evans (2012) argues that adaptive governance may not be suited for large-scale, e.g., national or international, problems as the system may become too complex. Ostrom (2007) argues that adaptive governance may not be suited to respond to the rapid alterations that anthropogenic climate change could cause because it has essentially been used as an approach to build resilience to gradual change. Yet, as Brunner and Lynch (2010) have shown, even with these limitations adaptive governance can be a successful approach for addressing climate change impacts at a local scale. However, questions about how to organize and what methods and tools can support the adaptation decision-making process still remain open in the literature.
PARTICIPATORY MULTICRITERIA METHODS AND THEIR RELEVANCE FOR CLIMATE ADAPTATION
Multicriteria decision making (MCDM) was initially developed as a method to support decisions based on a comparison of alternative options to obtain an order of preference for possible plans of actions. For a general overview of MCDM, see Arrow and Raynaud 1986, Bana and Costa 1990, Munda 1995, and Gal et al. 1999. Progressively, the idea of incorporating public participation into MCDM to improve pluralism, legitimacy, transparency, and flexibility gained popularity and led to the development of participatory multicriteria methods (PMCM). Participation in this context is regarded as an “open exercise” in which mapping preferences and reporting the findings obtained is as much part of the outcomes as the ranking of alternatives itself (Banville et al. 1998, Munda 2004, Proctor 2006, Stirling 2006).
Participatory multicriteria methods have been used in a wide variety of contexts including climate adaptation. According to the United Nations Framework Convention on Climate Change (UNFCC) guidelines, the application of PMCM is particularly relevant for the identification of National Adaptation Programs of Action and the prioritization of adaptation measures, especially in cases in which climate change has multifaceted impacts that are difficult to convert into monetary terms or quantitative evaluations (United Nations Climate Change Secretariat 2011). In Table 2, we review a number of case studies on climate change adaptation according to the complexity of the aspects analyzed and degree of participation in the multicriteria analysis. In relation to stakeholder involvement, including experts and nonexperts, i.e., social actors, various degrees of public participation can be identified that reflect the distinction between technical and societal decision making. Technical decision making involves only experts as active participants in the decision-making process; societal decision making involves both experts and social actors participating in the decision-making process. For example, the cases of water management in Yemen (United Nations Environment Programme 2011) and the assessment of national adaptation options in Bhutan (United Nations Climate Change Secretariat 2011, National Environment Commission 2011) represent technical decision making, where social actors are either not involved or, when involved, they are only requested to provide general information which is then used by experts to frame the problem or they are simply informed about the analyzed issue and the results obtained.
In a contrasting example of societal decision making, in the case of flood risk assessment in Bangladesh (Haque et al. 2012), social actors contribute to the identification of management options, criteria, objectives, and goals, and/or provide judgments on scores and criteria weights. An example falling in-between these two extremes is the case of the assessment of national adaptation options in the Netherlands, where different stakeholders were involved in the identification of adaptation options, while criteria and weights were obtained through experts’ judgement only (de Bruin et al. 2009). A different approach was used in the case of the Thames Estuary (Environment Agency 2012), where the general public was requested to provide feedback on adaptation measures previously identified by experts (Reeder and Ranger 2011). Other studies have used public surveys to acquire knowledge for multicriteria analysis in adaptation planning (e.g., Qin et al. 2008).
As indicated in Table 2, most of the reviewed studies have applied a technical decision-making approach, combining different kinds of qualitative and quantitative information, and all of them have included multiple criteria in the evaluation process to tackle the complexity of the adaptation problem. Even though societal decision-making approaches are preferred in principle when the issues at stake are highly complex, technical decision making is useful when there is no need to include the public because the issues are predominantly technical and/or resources are scarce (see, e.g., Porthin et al. 2013). In fact, although technical evaluations are generally inexpensive—apart from gathering information with ad hoc surveys, the application of societal multicriteria methods is usually resource-, data-, and time-intensive (Rauschmayer and Wittmer 2006, Qin et al. 2008). Furthermore, the inclusion of different stakeholder perspectives and interests in the decision-making process is not an easy task, as different individuals or groups of individuals can have conflicting views and perceptions over the same problem (Septer et al. 2012). However, in PMCM, various techniques can be used to aggregate different perspectives to allow for a ranking of the alternatives which reflects both a compromise solution and a mediation of conflicting interests and values, instead of providing the optimal choice in relation to only one group or individual. On the application of these techniques from both a theoretical and empirical perspective see Saaty 1980, Jelassi et al. 1990, Forman and Peniwati 1998, Mendoza and Martins 2006, Jabeur and Martel 2007, and Garmendia and Gamboa 2012, among others.
Based on the above, we maintain that participatory multicriteria methods are, in principle, appropriate frameworks of analysis in climate adaptation decision making, as also suggested by the UNFCCC. However, we are also aware of the limitations of the approach in terms of: (1) identifying the boundaries of stakeholder participation; i.e., various degrees of participation can be applied; (2) aggregating different interests; and (3) the potential higher cost and time needed to adopt consensus building and participatory approaches in decision making. Table 3 summarizes the key features, as well as main strengths and weaknesses, of the participatory multicriteria approach discussed above.
LINKING ADAPTIVE GOVERNANCE PRINCIPLES TO PARTICIPATORY MULTICRITERIA METHODS: AN INTEGRATED FRAMEWORK FOR CLIMATE ADAPTATION DECISION MAKING
Key commonalities between adaptive governance and PMCM include participation, learning, integration of different types of knowledge, collective deliberation, a variety of problem frames and solutions, flexibility, and adaptability. As previously discussed, all these features are also particularly valued in the climate adaptation literature. Consequently, we argue that because of these commonalities, PMCM are suitable tools to operationalize adaptive governance in climate adaptation decision-making practice. In fact, PMCM offer a structured platform for participatory decision making where different stakeholders’ knowledge and preferences about adaptation options can be discussed, actions can be designed as experiments, and lessons learned from feedbacks form the basis for re-evaluating and adjusting adaptation options.
It is possible that adaptive governance principles are already implicitly integrated into PMCM, given the common attributes. Indeed, we found evidence of this in some case studies where multicriteria analysis was applied to evaluate climate adaptation options (see Table 2). For example, the principle of flexibility is integrated into the multicriteria analysis of flood management in the UK and Finland (Reeder and Ranger 2011, Porthin et al. 2013) and water management in Canada (Qin et al. 2008). Different types of knowledge were integrated in the multicriteria evaluation of flood risk in Bangladesh and in the UK (Reeder and Ranger 2011, Haque et al. 2012).
However, despite the existence of these cases, we argue that the systematic and consistent integration of adaptive governance attributes into PMCM should be more widely pursued in adaptation decision-making processes as way to improve societal adaptive capacity to climate change. There is no evidence in the literature of applications of a societal decision-making approach that also integrates all of the adaptive governance principles previously highlighted. In the following, we illustrate how this can be done in an integrated framework.
The framework is based on three phases of the adaptation decision-making process (see also Fig. 1): (1) identification of the “starting conditions” of the system; (2) the “process” of decision; and (3) the “responses” of the system to adaptation actions (Smith and Lenhart 1996, Füssel 2007). These phases trace a typical multicriteria analysis; that is, ex-ante assessment, multicriteria evaluation, and ex-post evaluation.
Assessment of the starting conditions of the system
The starting conditions relate to different factors such as economic, social, institutional, and technological circumstances. These determinants can facilitate or constrain the development and implementation of adaptation strategies and options. In this phase, vulnerabilities and adaptive capacity of the system are assessed and systemic thresholds are identified. This phase includes: (1) the assessment of the socioeconomic and environmental state and climate vulnerabilities of the system; and (2) the identification of the relevant stakeholders to be involved. This phase may be problematic as choices that are made at this stage may negatively affect later stages of the decision-making process. For example, if some stakeholders are left out, the legitimacy of the final decision may be questioned by excluded parties at a later stage. This may delay action and make the whole adaptation process more expensive. Therefore, careful selection of relevant stakeholders is needed. This deserves some reflection about who initiates the process, who are relevant stakeholders, and how they are selected. Initiators of participatory decision-making processes may be different depending on problems, interests, resources, and capacity. In response to mounting public pressure, public agencies may decide to initiate a participatory investigation of adaptation options. It may also be the case that local communities, e.g., farmer associations or citizens groups, concerned with a specific issue organize stakeholder dialogs. Finally, the research community is often a promoter of participatory platforms within research-funded projects (see, e.g., Plummer and Baird 2013). As to who participates, identifying stakeholders is usually an iterative process, where stakeholders are added as the analysis continues. Expert opinion, focus groups, semistructured interviews, snowball sampling, or a combination of these methods are used for stakeholder analysis (Reed 2008).
The composition of the initial set of stakeholders may vary depending on whom initiate the process. In general, if a good stakeholder analysis is conducted, expert-driven processes tend to have a more complete initial set of stakeholders—this being in the interest of the research outcome. In contrast, participatory platforms initiated by public agencies or local communities may be biased in the initial phase toward excluding relevant groups based on past disputes, negative past experiences working together, or a lack of information. This problem is usually solved when stakeholders are actively engaged in identifying who is included in the process (Reed 2008). Mitchell et al. (1997) argue that the basis upon which stakeholders are identified as being relevant for the issue at hand depends on them possessing one or more of the following attributes: (1) power (coercive, material, or influential) to carry out their own will despite resistance; (2) legitimacy, meaning that their actions are socially appropriate and accepted; and (3) urgency, indicating that the issue is important for them at that specific moment in time.
The process phase entails: (1) problem definition and goal setting; (2) identification of adaptation alternatives; (3) identification of evaluation criteria and weights; and (4) ranking alternatives. Constructing an open, transparent, and fair stakeholder dialog presents fundamental methodological and relational challenges in this phase. From a methodological perspective, a variety of strategies can be implemented to elicit stakeholder ideas, understanding underling attitudes, and explore issues raised in debates including brainstorming in groups (Leys and Vancley 2011), cognitive mapping (Septer et al. 2012), and data tabulation and simulation modeling (Bousquet et al. 2004). From a relational perspective, initial tension among stakeholders is inevitable as power and interest dynamics are not explicit in the beginning. In particular, the risk that power asymmetries amplify initial social inequities cannot be neglected (Few et al. 2007, Reed 2008, Barnaud and Paassen 2013). Different strategies can be used to account for power asymmetries, including alternating individual and subgroup discussion with plenary debates, so as to devote attention to both the powerless and the powerful separately, highlighting the diversity of interests, and favoring integrative negotiation processes that try to identify win–win solutions (Barnaud and Paassen 2013). In addition, there are some participatory approaches that are more focused on taking power asymmetries into account, such as the companion modeling approach (e.g., see Bousquet et al. 2004).
Another critical point in this phase is the evaluation of adaptation alternatives according to a plurality of values and dimensions of analysis. In addressing this issue, a set of quantitative and qualitative criteria is identified and weighted against stakeholders’ preferences. At this stage of the process, in addition to the economic, environmental, and social criteria selected according to the characteristics of the adaptation problem under investigation, the different options will also be evaluated with respect to the criteria of experimentality, flexibility, reversibility, and incrementality. In this lies the novelty of this framework, as these evaluation criteria, reflecting key adaptive governance principles, are systematically incorporated in any participatory multicriteria exercise evaluating adaptation options. In practical terms, the evaluation of adaptation options responds to the questions: (1) to what extent will the option yield new understanding about social–ecological systems functioning and responses to management? (experimentation); (2) to what extent is the option able to withstand a range of possible climatic conditions and/or can be adjusted over time? (flexibility); (3) has an incremental approach to adaptation been adopted in the design of this option? and (4) to what extent is the option reversible? The role of scientists in this phase is to help the involved parties understand the importance of embedding these criteria in the evaluation of adaptation options as way to incorporate uncertainty about future climate change and its impacts on the decision.
The process phase ends with a ranking of adaptation options, which is a compromise that reflects stakeholders’ preferences and values. The authority in charge of making the final decision about what option to implement decides based on the ranking of alternatives and the outcome of the discussion with stakeholders. Different aggregation algorithms are available; the choice to use one or the other depends on the main features of the multicriteria problem under investigation, for a summary of the various models used, we refer the reader to Montis et al. 2004, Figueira et al. 2005, Munda 2005, and Linkov et al. 2006.
Finally, the response phase involves implementing the selected option(s), along with monitoring and evaluating decisions. Implementation, monitoring, and evaluation of adaptation options are tasks usually placed in the hands of experts. However, there is increasing recognition of the value for iterative learning of involving participants in experimentally monitoring and evaluating their decisions and adapting them accordingly (Estrella and Gaventa 2000, Gunderson and Holling 2002, Reed 2008). Lessons learnt from monitoring and evaluation feedbacks may lead to re-ranking adaptation options, abandoning some options, and/or including new ones, or to challenging more fundamental assumptions about problems and goals. Re-evaluating options may be less time- and resource-consuming, as stakeholders who have already engaged in the ex-ante exercise are familiar with the process and with each other, and often have developed relationships of trust.
DISCUSSION AND CONCLUSION
We have proposed an adapted version of the participatory multicriteria approach where participation, knowledge integration, and iterative learning are inherently part of the process, and the key adaptive governance principles of experimentation, flexibility, reversibility, and incrementality are systematically included as evaluation criteria. Given that all of these attributes are of crucial importance in climate adaptation decision making, such a participatory multicriteria approach becomes a tool for supporting adaptation decisions and enhance societal adaptive capacity.
We have argued that decision making regarding climate adaptation requires tools that are inclusive and open to different values and preferences as well as the ability to incorporate uncertainty into decisions. In light of a review of the challenges of adaptation decision making and of the key features of both adaptive governance and PMCM, we have demonstrated that the latter can be tailored to the specificity of the climate adaptation decision-making context. This can be done by including the principles of experimentality, flexibility, incrementality, and reversibility as evaluation criteria in the participatory multicriteria exercise. In so doing, our framework responds to two fundamental questions in adaptation decision making: (1) “how good are the adaptation options according to a plurality of values and dimensions of analysis?” (Intergovernmental Panel on Climate Change 2001); and (2) are adaptation options able to deal with a range of future climate-change scenarios? On the one hand, PMCM provides a ranking of options which represents a compromise solution reflecting stakeholders’ preferences, thus addressing the first of these questions. On the other hand, by including adaptive governance principles as evaluation criteria, the compromise solution resulting from the assessment is, in principle, adequate for the task of providing protection against a wide range of possible climate regimes, as uncertainty is incorporated into the decision-making process, thus addressing the second question.
We recognize the limitation of not having yet applied this framework to an empirical case study. However, we have discussed at length the application of the adaptive governance and the PMCM approaches to adaptation decisions, and proposed how to integrate the two. This analysis suggests that there is potential for systematically integrating the two approaches into one framework. In fact, in our investigation of case studies, we have found promising attempts at incorporating uncertainty into adaptation decision making where multicriteria methods were used, e.g., the Thames Estuary 2100 plan (Environment Agency 2012). We argue that further applications should more explicitly consider adaptive governance principles and active participation in the identification, evaluation, and implementation of climate adaptation measures.
Yet, any decision making implies the existence of trade-offs. Although the framework proposed here is based on a deliberative approach that allows participants to define problems and to make trade-offs explicit, we are aware that the ensuing decisions can penalize some stakeholders more than others. In these situations, either appropriate measures should be designed to provide compensation to the most penalized ones, or decision makers should consider the possibility of implementing second-best solutions that reduce trade-offs. In this way, fairness of decisions would improve and additional costs that might arise from potential post-decision conflicts would be reduced.
Participatory multicriteria management offers a suitable decision-making framework for bridging the gap between the soft, qualitative adaptive governance paradigm and the more structured, quantitative PMCM paradigm. Frameworks that integrate these two, such as the one presented here, have the potential to more adequately accommodate the inherent complexity of climate adaptation, by incorporating uncertainty related to future climatic changes along with the multitude of stakeholders’ concerns.
The authors acknowledge the support received from University IUAV of Venice, Department of Design and Planning in Complex Environments for conducting most of this research.
Arnstein, S. R. 1969. A ladder of citizen participation. Journal of the American Planning Association.
35(4):216–224. [online] URL: http://www.tandfonline.com/doi/abs/10.1080/01944366908977225
Armitage, D. R., F. Berkes, A. Dale, E. Kocho-Schellenberg, and E. Patton. 2011. Co-management and the co-production of knowledge: learning to adapt in Canada’s Arctic. Global Environmental Change
Armitage, D. R., M. Marschke, and R. Plummer. 2008. Adaptive co-management and the paradox of learning. Global Environmental Change.
Armitage, D. R., F. Berkes, and N. Doubleday. 2007. Introduction: moving beyond co-management. Pages 1–15 in D. Armitage, F. Berkes, and N. Doubleday, editors. Adaptive co-management: collaboration, learning, and multi-level governance.
University of British Columbia Press, Vancouver, British Columbia, Canada.
Arrow, K. J. and H. Raynaud. 1986. Social choice and multicriterion decision making.
MIT Press, Cambridge, Massachusetts, USA.
Bana e Costa, C. A. 1990. An additive value function technique with a fuzzy outranking relation for dealing with poor intercriteria preference information. Pages 351–382 in
C. A. Bana e Costa, editor. Readings in multiple criteria decision aid.
Springer, Berlin, Germany. http://dx.doi.org/10.1007/978-3-642-75935-2_16
Banville, C., M. Landry, J. M. Martel, and C. Boulaire. 1998. A stakeholder approach to MCDA. Systems Research and Behavioral Science
Barnaud, C., and A. van Paassen. 2013. Equity, power games and legitimacy: dilemmas of participatory natural resource management. Ecology and Society
18(2): 21 [online] URL: http://www.ecologyandsociety.org/vol18/iss2/art21/
Barnaud, C., A. Van Paassen, G. Trebuil, T. Promburom, and F. Bousquet. 2010. Dealing with power games in a companion modelling process: lessons from community water management in Thailand highlands. The Journal of Agricultural and Extension Education
Bloomfield, D., K. Collins, C. Fry, and R. Munton. 2001. Deliberation and inclusion: vehicles for increasing trust in UK public governance? Environment and Planning C
19(4):501–513. [online] URL: http://www.envplan.com/abstract.cgi?id=c6s
Bousquet, F., and C. Le Page. 2004. Multi-agent simulations and ecosystem management: a review. Ecological Modelling
Brunner, R. D., T. A. Steelman, L. Coe-Juell, C. M. Cromley, C. M. Edwards, and D. W. Tucker. 2005. Adaptive governance: integrating Science, Policy, and Decision Making.
Columbia University Press, New York, New York, USA.
Brunner, R.D., T.D. Steelman, L. Coe-Juell, C.M. Cromley, C.M. Edwards, and D.W. Tucker. 2006. Adaptive governance: integrating science policy and decision making.
Columbia University Press, New York, New York, USA.
Brunner, R., and A. Lynch. 2010. Adaptive governance and climate change.
American Meteorological Society, Boston, Massachusetts, USA. http://dx.doi.org/10.1007/978-1-935704-01-0
Collins, H. M., and R. Evans. 2002. The third wave of science studies: studies of expertise and experience. Social Studies of Science
Collins, K., and R. Ison. 2009. Jumping off Arnstein’s ladder: social learning as a new policy paradigm for climate change adaptation. Environmental Policy and Governance
Davos, C. A., and R. P. Lajano. 2001. Analytical perspectives of cooperative coastal management. Journal of Environmental Management
Diduck, A. 2010. The learning dimension of adaptive capacity: untangling the multi-level connections. Pages 199–223 in
D. Armitage, and R. Plummer, editors. Adaptive Capacity and Environmental Governance.
Springer, Berlin/Heidelberg, Germany. http://dx.doi.org/10.1007/978-3-642-12194-4_10
Dietz, T., E. Ostrom, and P. C. Stern. 2003. The struggle to govern the commons. Science
de Bruin, K., R. B. Dellik, A. Ruijs, L. Bolwidt, A. van Buuren, J. Graveland, R. S. de Groot, P. J. Kuikman, S. Reinhard, R. P. Roetter, V. C. Tassone, A. Verhagen, and E. C. van Ierland. 2009. Adapting to climate change in the Netherlands: an inventory of climate adaptation options and ranking of alternatives. Climatic Change
Engle, N. L. 2011. Adaptive capacity and its assessment. Global Environmental Change
Environment Agency (EA). 2012. Managing flood risk through London and the Thames estuary. TE2100 Plan, Thames Estuary 2100.
Environment Agency, Charlton, London, United Kingdom. [online] URL: http://a0768b4a8a31e106d8b0-50dc802554eb38a24458b98ff72d550b.r19.cf3.rackcdn.com/LIT7540_43858f.pdf
Estrella, M., Gaventa, J., 2000. Who counts reality? Participatory monitoring and evaluation: a literature review.
IDS Working Paper 70, Institute of Development Studies, Brighton, United Kingdon. [online] URL: http://www.ids.ac.uk/files/Wp70.pdf
Evans, J. P. 2012. Environmental governance.
Routledge, New York, New York, USA.
Fankhauser, S., J. B. Smith, R. S. J. Tol. 1997. Weathering climate change: some simple rules to guide adaptation decisions. Ecological Economics
Few, R., K. Brown, and E. M. Tompkins. 2007. Public participation and climate change adaptation: avoiding the illusion of inclusion. Climate Policy
7: 46–59. http://dx.doi.org/10.3763/cpol.2007.0704
Figueira, J., S. Greco, and M. Ehrgott. 2005. Multiple-criteria decision analysis. State of the art surveys.
Springer, New York, New York, USA.
Folke, C., T. Hahn, P. Olsson, and J. Norberg. 2005. Adaptive governance of social–ecological systems. Annual Review of Environment and Resources.
Forman, E., and K. Peniwati. 1998. Aggregating individual judgments and priorities with the analytic hierarchy process. European Journal of Operational Research
Füssel, H. M. 2007. Adaptation planning for climate change: concepts, assessment approaches, and key lessons. Sustainability Science.
Füssel, H.M., and R.J.T. Klein. 2006. Climate change vulnerability assessments: an evolution of conceptual thinking. Climatic Change
Gal T., T. J. Stewart and T. Hanne. 1999. Multi criteria decision making: advances in MCDM models, algorithms, theory and applications.
Kluwer, Dordrecht, The Netherlands.
Gallopìn, G. C. 2006. Linkages between vulnerability, resilience, and adaptive capacity. Global Environmental Change
Garmendia, E., G. Gamboa, J. Franco, J. M. Garmendia, P. Liria, and M. Olazabal. 2010. Social multi-criteria evaluation as a decision support tool for integrated coastal zone management. Ocean and Coastal Management
Garmendia, E., and G. Gamboa. 2012. Weighting social preferences in participatory multi-criteria evaluations: a case study on sustainable natural resource management. Ecological Economics
Gunderson, L. H., and C. S. Holling, editors. 2002. Panarchy: understanding transformations in human and natural systems.
Island, Washington, D.C., USA.
Haque, A. N., S. Grafakos, M. Huijsman. 2012. Participatory integrated assessment of flood protection measures for climate adaptation in Dhaka. Environment and Urbanization
Hinkel, J., A. Bisaro, T. E. Downing, M. Hofmann, K. Lonsdate, D. McEvoy, and D. Tabara. 2010. Learning to adapt: re-framing climate change adaptation. Pages 113–134 in
M. Hulme, and H. Neufeld, editors. Making climate change work for us: European perspectives on adaptation and mitigation strategies.
Cambridge University Press, Cambridge, UK.
Huitema, D., E. Mostert, W. Egas, S. Moellenkamp, C. Pahl-Wostl, and R. Yalcin. 2009. Adaptive water governance: assessing the institutional prescriptions of adaptive (co-) management from a governance perspective and defining a research agenda. Ecology and Society
14(1): 26. [online] URL: http://www.ecologyandsociety.org/vol14/iss1/art26/
Intergovernmental Panel on Climate Change (IPCC). 2001. Climate Change 2001. Impacts, adaptation, and vulnerability. Contribution of working group II to the third assessment report of the Intergovernmental Panel on Climate Change.
Cambridge University Press, Cambridge, UK. [online] URL: http://www.ipcc.ch/ipccreports//tar/wg2/index.php?idp=0
IPCC. 2007. Climate Change 2007. Impacts, adaptation and vulnerability. Contribution of working group II to the fourth assessment report of the Intergovernmental Panel on Climate Change.
Cambridge University Press, Cambridge, UK. [online] URL: http://www.ipcc.ch/publications_and_data/ar4/wg2/en/contents.html
International Federation of Red Cross and Red Crescent Societies (IFRC). 2004. World disasters report: focus on community resilience.
IFRC, Geneva, Switzerland. [online] URL: http://www.ifrc.org/PageFiles/89755/WDR2004/58000-WDR2004-LR.pdf
International Institute for Sustainable Development (IISD). 2003. Livelihoods and climate change: combining disaster risk reduction, natural resource management and climate change adaptation in a new approach to the reduction of vulnerability and poverty.
IISD, Winnipeg, Manitoba, Canada. [online] URL: http://www.iisd.org/pdf/2003/natres_livelihoods_cc.pdf
Jabeur, K., and J. M. Martel. 2007. A collective choice method based on individual preferences relational systems (p.r.s.). European Journal of Operational Research
Janssen, M. A., J. M. Anderies, and E. Ostrom. 2007. Robustness of social–ecological systems to spatial and temporal variability. Society and Natural Resources: An International Journal
Jelassi, T., G. Kersten, and S. Ziont. 1990. An introduction to group decision and negotiation support. Pages 537–568 in
C. A. Bana e Costa, editor. Readings in multiple criteria decision aid.
Springer, Berlin, Germany. http://dx.doi.org/10.1007/978-3-642-75935-2_23
Kooiman, J., and M. Bainck. 2005. The governance perspective. Pages 11–25 in
J. Kooiman, M. Bavinck, S. Jentoft, and R. Pullin, editors. Fish for life: interactive governance for fisheries.
Amsterdam University Press, Amsterdam, The Netherlands. http://dx.doi.org/10.1007/978-3-642-75935-2_23
Lebel, L., J. M. Anderies, B. Cambell, C. Folke, S. Hatfield-Dodds, T. P. Hughes, and J. Wilson. 2006. Governance and the capacity to manage resilience in regional social–ecological systems. Ecology and Society
11(1): 19. [online] URL: http://www.ecologyandsociety.org/vol11/iss1/art19/
Lee, K. N. 1993. Compass and gyroscope. Integrating science and politics for the environment.
Island Press, Washington, D.C., USA.
Leys, A. J., and J. K. Vanclay. Social learning: a knowledge and capacity building approach for adaptive co-management of contested landscapes. Land Use Policy
28:574–584. [online] URL: http://www.sciencedirect.com/science/article/pii/S0264837710001171
Linkov, I., F. K. Satterstrom, G. Kiker, C. Batchelor, T. Bridges, and E. Ferguson. 2006. From comparative risk assessment to multi-criteria decision analysis and adaptive management: recent developments and applications. Environment International.
32(8):1072– 1093. http://dx.doi.org/10.1016/j.envint.2006.06.013
Mendoza, G. A., and H. Martins. 2006. Multi-criteria decision analysis in natural resource management: a critical review of methods and new modelling paradigms. Forest Ecology and Management
Mitchell, R. K., R. A. Bradley, D. J. Wood. 1997. Toward a theory of stakeholder identification and salience: defining the principle of who and what really counts. Academy of Management Review
Montis, A., P. De Toro, B. Droste-Franke, I. Omann, and S. Stagl. 2004. Assessing the quality of different MCDA methods. Pages 99–133 in
M. Getzner, C. Spash, and S. Stagl, editors. Alternatives for environmental valuation.
Routledge, New York, New York, USA.
Munda, G. 1995. Multicriteria evaluation in a fuzzy environment: theory and applications in ecological economics.
Physica, Heidelberg, Germany. http://dx.doi.org/10.1007/978-3-642-49997-5
Munda, G. 2004. Social multi-criteria evaluation (SMCE): methodological foundations and operational consequences. European Journal of operational Research
158(3):662–677. [online] URL: http://www.sciencedirect.com/science/article/pii/S0377221703003692
Munda, G. 2005. Multi-criteria evaluation. Pages 130–154 in
J. Proops, and P. Safonov, editors. Modelling in ecological economics.
Edward Elgar, Northampton, Massachusetts, USA.
Munda, G. 2006. Social multi-criteria evaluation for urban sustainability policies. Land Use Policy
National Environment Commission (NEC). 2011. Bhutan national adaptation programme of action.
National Environment Commission, Royal Government of Bhutan, Thimphu, Bhutan. [online] URL: http://unfccc.int/resource/docs/napa/btn01.pdf
Nelson, D. R., W. N. Adger, and K. Brown. 2007. Adaptation to environmental change: contributions of a resilience framework. Annual Review of Environment and Resources
Norton, B. G., and A. C. Steinemann. 2001. Environmental values and adaptive management. Environmental Values
Olsson, P., L. H. Gunderson, S. R. Carpenter, P. Ryan, L. Lebel, C. Folke, and C. S. Holling. 2006. Shooting the rapids: navigating transitions to adaptive governance of social–ecological systems. Ecology and Society
11(1): 18. [online] URL: http://www.ecologyandsociety.org/vol11/iss1/art18/
Ostrom, E. 1990. Governing the commons. The evolution of institutions for collective actions.
Cambridge University Press, New York, New York, USA. http://dx.doi.org/10.1017/CBO9780511807763
Ostrom, E. 2007. Sustainable social–ecological systems: an impossibility?
Social Science Research Network, Rocherster, New York, USA. [online] URL: http://dx.doi.org/10.2139/ssrn.997834
O’Neill, J. 2001. Representing people, representing nature, representing the world. Environment and planning C
19(4):483–500. [online] URL: http://www.envplan.com/abstract.cgi?id=c12s
Pahl-Wostl, C. 2009. A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes. Global Environmental Change.
Parkins, J. R., and R. E. Mitchell. 2005. Public participation as public debate: a deliberative turn in natural resource management. Society and Natural Resources
Pelling, M., C. High, D. Dearing, and D. Smith. 2008. Shadow spaces for social learning: a relational understanding of adaptive capacity to climate change within organisations. Environment and Planning A
40(4) :867–884. http://dx.doi.org/10.1068/a39148
Plummer, R., and J. Baird. 2013. Adaptive co-management for climate change adaptation: considerations for the Barents region. Sustainability
Plummer, R., and D. Armitage. 2007. A resilience-based framework for evaluating adaptive co-management: linking ecology, economics and society in a complex world. Ecological Economics
Porthin, M., T. Rosqvist, A. Perrels, and R. Molarius. 2013. Multi-criteria decision analysis in adaptation decision-making: a flood case study in Finland. Regional Environmental Change
Proctor, W., and M. Drechsler. 2006. Deliberative multicriteria evaluation. Environmental and Planning C
Qin, X. S., G. H. Huang, A. Chakma, X. H. Nie, and Q. G. Lin. 2008. A MCDM-based expert system for climate-change impact assessment and adaptation planning. A case study for the Georgia Basin, Canada. Expert Systems with Applications
34(3):2164–2179. [online] URL: http://www.sciencedirect.com/science/article/pii/S0957417407000942
Rauschmayer, F., and H. Wittmer. 2006. Evaluating deliberative and analytical methods for the resolution of environmental conflicts. Land Use Policy
Reed, M. S. 2008. Stakeholder participation for environmental management: a literature review. Biological Conservation
Reeder, T., and N. Ranger. 2011. How do you adapt in an uncertain world? Lessons from the Thames Estuary 2100 project.
World Resources Report, Washington, D.C., USA. [online] URL: http://www.wri.org/sites/default/files/uploads/wrr_reeder_and_ranger_uncertainty.pdf
Reen, O. 2006. Participatory processes for designing environmental policies. Land Use Policy
Roy, B. 1985. Méthodologie multicritere d’ aide à la decision.
Economica, Paris, France.
Runge, M. C., S. J. Converse, and J. E. Lyons. 2011. Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program. Biological Conservation
Saaty, T. L. 1980. The analytic hierarchy process.
McGraw-Hill, New York, New York, USA.
Septer, T. J., J. Dijkstra, and F. N. Stokman. 2012. Detecting and measuring crucial differences between cognitive maps. Rationality and Society
Simon, H. A. 1976. From substantive to procedural rationality. Pages 129–148 in
J. S. Latsis, editor. Methods and appraisal in economics.
Cambridge University Press, New York, USA. http://dx.doi.org/10.1017/CBO9780511572203.006
Smith, J. B., and S. S. Lenhart. 1996. Climate change adaptation policy options. Climate Research
Stagl, S. 2006. Multicriteria evaluation and public participation: the case of UK energy policy. Land Use Policy
Stirling, A. 2006. Analysis, participation and power: justification and closure in participatory multi-criteria analysis. Land Use Policy
23:95–107. [online] URL: http://www.sciencedirect.com/science/article/pii/S0264837704000948
United Nations Environment Programme (UNEP). 2011. A practical framework for planning pro-development climate policy. [online] URL:
United Nations Climate Change Secretariat (UNFCCC). 1992. United Nations framework convention on climate change (UNFCCC).
United Nations Climate Change Secretariat, Bonn, Germany. [online] URL: http://unfccc.int/resource/docs/convkp/conveng.pdf
United Nations Climate Change Secretariat (UNFCCC). 2011. Assessing the costs and benefits of adaptation options. An overview of approaches.
United Nations Climate Change Secretariat, Bonn, Germany. [online] URL: http://unfccc.int/files/.../2011_nwp_costs_benefits_adaptation.pdf
Van Nieuwaal, K., Driessen, P., Spit, T., Termeer, C. 2009. A state of the art governance literature on adaptation to climate change: towards a research agenda. Report N. kfc 003/2009. Dutch National Research Programme Knowledge for Climate, Utrecht University, Wageningen University, The Netherlands. http://dx.doi.org/10.1088/1755-1307/6/36/362019
Viguié, V., and S. Hallegatte. 2012. Trade-offs and synergies in urban climate policies. Nature and Climate Change
Walker, B., C. S. Holling, S. R. Carpenter, and A. Kinzig. 2004. Resilience, adaptability and transformability in social–ecological systems. Ecology and Society.
9(2):5. [online] URL: http://www.ecologyandsociety.org/vol9/iss2/art5
Walters C. 1997. Challenges in adaptive management of riparian and coastal ecosystems. Conservation Ecology 1(2):1. [online] URL: http://www.consecol.org/vol1/iss2/art1/
Walters, C., and C. S. Holling. 1990. Large-scale management experiments and learning by doing. Ecology.
Wheaton, E. E., and D. C. Maciver. 1999. A framework and key questions for adapting to climate variability and change. Mitigation and Adaptation Strategies for Global Change