Copyright © 2013 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf
version of this article
The following is the established format for referencing this article:
Berkley, J. 2013. Opportunities for collaborative adaptive management progress: integrating stakeholder assessments into progress measurement. Ecology and Society 18(4): 69.
, part of a special feature on Exploring Opportunities for Advancing Collaborative Adaptive Management (CAM): Integrating Experience and Practice
Opportunities for Collaborative Adaptive Management Progress: Integrating Stakeholder Assessments into Progress Measurement
1U.S. Environmental Protection Agency
Collaborative Adaptive Management (CAM) program stakeholders informally assess program progress through subjective assessments regularly. Each stakeholder does this by individually selecting objective progress indicators based on their needs, values, and preferences. They do this even though there may be a stakeholder group agreed-on set of progress objectives. Individual stakeholder indicators may be a subset of the group set or outside of the agreed-on set. This is because many factors influence behavior, and stakeholders may act differently in group settings as opposed to individual settings. These assessments can provide valuable information about stakeholder needs that are not being met, and potential motivations for stakeholders circumventing a CAM process. They can also provide information, beyond the normal measures, about the importance of system components and relationships that are keys to progress and action. Progress is important to continued support for these publically funded CAM programs. The Glen Canyon Dam Adaptive Management Program (AMP) and the Missouri River Recovery Program (MRRP) were used to explore the idea of integrating stakeholder assessments into CAM progress measurement. A study using a framework with AMP stakeholders was conducted to measure progress, whereas examples from MRRP were used to explain how the framework could be used to understand real scenarios of stakeholder behavior related to unmet needs and individual measures of progress. Integration of stakeholder attitudes and behavior in CAM progress evaluation can yield important results. Stakeholders’ attitudes and corresponding behaviors can affect a CAM program’s progress. Gathering data on their attitudes can help decision makers understand stakeholders’ perceptions of progress and avoid potential blocks to progress.
There are differences among stakeholders in the indicators they consider as relevant to the assessment of progress. Elucidating these differences can provide useful information about system components and relationships that are important to public support of a CAM program and progress. One of the sources of differences in progress assessments among stakeholders comes from their diverse perceptions about the desired and current states of the social–ecological systems. Stakeholder behavior can be inconsistent between group and individual settings. Individually they may make plans, based on their assessments, that do not conform to the group plan because of their unique interests and preferences. The results of this study need to be further tested. The framework should be used through multiple cycles to determine whether the information gathered with this approach results in additional progress as compared with past approaches. In particular, it would be helpful to test whether gathering such information resulted in a decrease in stakeholders electing to go outside of the CAM process to get their needs met.
Key words: adaptive management, Adaptive Management Working Group, AMP, AMWG, attitudes, behavior, collaborative adaptive management, Glen Canyon Dam Adaptive Management Program, Missouri River Recovery Program, MRRP, progress, stakeholders
Adaptive management (AM) is an evaluative approach to natural resources planning and policies (Holling 1978). Plans and policies are treated as temporary in this type of management program. They are created and revised in response to the results of ongoing scientific monitoring. Adaptive management was initially designed to help manage and address problems in complex natural systems (Holling 1978). The approach eventually evolved to include social systems and the two were combined and became known as “social–ecological system” (Berkes 2000). Adaptive management programs have been established for a variety of places, such as the Glen Canyon Dam Adaptive Management Program
(AMP), the Missouri River Recovery Program
(MRRP), and others (please see Wescoat and White 2003 for a listing of additional AM projects).
Many AM processes are stakeholder driven, AMP and MRRP included. We define a stakeholder as someone who is either affected by or can affect the outcome of an AM process. This definition includes private interests who have a stake in outcomes, for example, landowners, agriculture producers, recreationists, municipal water suppliers, and nongovernmental organizations (NGOs). It also includes federal, state, tribal, and local government representatives, who may be stakeholders by necessity through regulatory or management responsibility; e.g., the U.S. Bureau of Reclamation, U.S. Fish and Wildlife Service, or a tribal government. Decision makers’ motivations for involving stakeholders include the realization that they can affect the implementation and outcomes of proposed management actions and policies, the fact that implicit public trust of experts has decreased over time (Ludwig 2001), and the idea that more direct stakeholder involvement may help in rebuilding trust. Stakeholders are motivated to participate in processes, so that they can hear and evaluate expert information directly and influence process outcomes.
Collaboration has become an integral part of many AM programs. Collaborative planning emphasizes the involvement of all stakeholder groups. Collaborative adaptive management (CAM) is the combination of AM and collaborative planning (Susskind et al. 2012). Public support for CAM is necessary for continuation of such programs, which are federally funded in the United States. Progress is necessary for public support (Harwell 1999). However, there are differences among stakeholders in how they assess progress, and their resultant attitudes and behavior (Berkley 2009). Because stakeholders’ attitudes and corresponding behaviors can affect a CAM program’s progress, gathering data on their attitudes can help decision makers understand stakeholders’ perceptions of progress and avoid potential blocks to progress.
Progress is defined here as narrowing the gap between the definition of what the social–ecological system “should be” and “what it is” (Fig. 1). Part of the source of differences in progress assessments among stakeholders comes from their diverse perceptions about the status of “what should be” and “what is” of socio–ecological systems.
Stakeholder behavior can be inconsistent between group and individual settings. Although there may be a set of goals and objectives agreed upon by a group for a particular CAM program (e.g., AMP or MRRP), this does not assure that individual stakeholders will use these criteria to measure progress (Berkley 2009). One of the reasons for this is that some stakeholders behave differently or exhibit different attitudes depending on whether they are in a group setting or by themselves (Barker 1978). Another reason is that individuals are always assessing and making plans. Individually, they may make plans based on their personal assessments that do not conform to the group plan because of their unique interests and preferences (Rittel 1984, Berkley 2009).
In many AM programs, collaboration has become the main method of incorporating stakeholders into the process. Collaborative adaptive management processes vary; both AMP and MRRP use a combination of public meetings and advice from a facilitated formal stakeholder group. Other programs, such as the Comprehensive Everglades Restoration Plan
(CERP) or the Department of the Army, U.S. Army Corps of Engineers Civil Works Program
, use public meetings and workshops. Such programs have not collected data on stakeholder progress-related attitudes and behavior thus far.
Here, AMP and MRRP will be used as examples to explain how collaboration occurs and progress is currently measured, to describe how stakeholders’ progress assessments and associated attitudes and behavior can vary, and to evaluate how that can affect CAM progress. This approach will be applied to the AMP with a brief summary of the results and a description of how this approach might apply to MRRP. The discussion will then focus on how gathering information on stakeholder progress assessments can help in identification of systems and relationships relevant to progress and why this is important (Cash et al. 2002). The final discussion addresses how this information should be incorporated into practice and how the exclusion of this information can act as barriers to CAM progress.
THE GLEN CANYON DAM ADAPTIVE MANAGEMENT PROGRAM AND THE MISSOURI RIVER RECOVERY PROGRAM
As is the case of many CAM programs in the USA, the U.S. Endangered Species Act (ESA) is the motivation for action. The AMP program is addressing four endangered species: the humpback chub (Gila cypha
), razorback sucker (Xyrauchen texanus
), willow flycatcher (Empidonax traillii
), and kanab ambersnail (Oxyloma haydeni kanabensis
). The MRRP is addressing three endangered or threatened species: the interior least tern (Sternula antillarum
), piping plover (Charadrius melodus
), and pallid sturgeon (Scaphirhynchus albus
). Both of these programs are similar in that they are focused on downstream river management and impacts of multipurpose dams for diverse stakeholder interests. The AMP is focused on management of Glen Canyon Dam, Lake Powell Recreation Area and the 482.8 km (300 mile) stretch of the Colorado River below the dam to Lake Mead. The MRRP is focused on management of six mainstem dams on the Missouri River beginning with the Fort Peck Dam and stretching approximately 3218.6 km (2000 miles) downriver to the Mississippi River near St. Louis, Missouri.
The multiple purposes for which the Colorado River is managed include hydropower generation and revenue, recreation, environmental attributes, state water supply, Native American interests, and cultural resources. The Missouri River is managed for eight authorized purposes: navigation, water supply, flood control, water quality, irrigation, recreation, fish and wildlife, and hydropower generation.
Both CAM programs have created representative stakeholder groups to advise the federal government on management of the river with respect to their interests and endangered species. The AMP’s Adaptive Management Workgroup
(AMWG) has 25 members representing the various interests on the river (based on the Grand Canyon Protection Act of 1992
). The recommendations made by this group to the government are based on a two-thirds majority vote, although consensus is preferred (Orenstein et al. 2010). Federal participants in the AMP are voting members.
The MRRP consists of a nearly 70-member stakeholder group, the Missouri River Recovery Implementation Committee
(MRRIC). Recommendations are made to the government are by consensus. In its Charter
, the MRRIC defines consensus as unanimity. Federal participants in this group are nonvoting members. These groups meet regularly and provide advice to the federal government on endangered species management actions as they relate to their interests.
AMP and MRRP progress metrics
Progress measurement of the AMP is based on the 1995 U.S. Department of the Interior Bureau of Reclamation Environmental Impact Statement
(EIS) for the Operation of Glen Canyon Dam: habitat function and quality, plant and animal species abundance, and economic values (Berkley 2009).
Progress measurement for the MRRP is based primarily on goals and objectives developed from 2000 and 2003 U.S. Fish and Wildlife Service Biological Opinions
: area of emergent sand bar habitat for the birds, area of shallow water habitat, and simulated spring flows for the fish. It is assumed by both programs that stakeholder collaboration addresses the social part of the CAM program.
PROGRESS MEASUREMENT METHODS
A typical method of measuring progress in a CAM program is represented in Fig. 2. Objective indicators of “what is” are compared to agency assessment indicators of “what should be.” The agency assesses whether the comparison merits changes to management or additional study, and makes a management decision about what action should be taken.
This progress measurement approach assumes that participating stakeholders consider these agency, and sometimes stakeholder, group identified indicators as valid measures of progress and that these indicators are being used to measure progress through an assessment as presented in Fig. 2. In an example from AMP, a measurable objective is to “maintain or attain 4330 to 4811 [humpback chub] individuals with a mean of 4508 individuals” (see the AMP Strategic Plan
In the MRRP example, one of the objective indicators is to restore 20% of the historical shallow water habitat that existed in the lower 1210.2 km (752 miles) of the [Missouri] river downstream of Gavins Point Dam. This is equivalent to restoring 0.08–0.12 km2
(20–30 acres) of shallow water habitat per 1.6 km (1 mile) of river, which must be in place by 2020 (see the 2003 U.S. Fish and Wildlife Service Biological Opinion
As the humpback chub population or shallow water habitat restoration approaches the numbers identified in the goal, a common assumption is that stakeholders will consider this satisfactory progress, such that they would further support the program. This may not hold true, as explained below.
The adaptive management programs discussed above, as well as other CAM programs, have recognized that the variables affecting progress are complicated and lie beyond simply considering objective measures. Stakeholder values are important variables to consider in decision making and progress. Decision support methods used in CAM processes, such as structured decision making (SDM), recognize the importance of including stakeholder values by incorporating them in the tool (see the U.S. Fish and Wildlife Service’s 2008 Structured Decision Making Fact Sheet
). However, values alone are not the only metric that affect stakeholder progress assessments, attitudes, and behavior. Stakeholder values, along with needs and preferences, are related to stakeholder attitudes and behaviors (Berkley 2009; Fig. 3).
Figure 3 is identical to Fig. 2 except with respect to how progress is measured. Objective indicators of “what is” are compared to stakeholder-relevant subjective indicators of “what should be“ in Fig 3. Each stakeholder in Fig. 3 assesses whether the comparison merits a change to management or additional study, and makes a judgment about what action should be taken. Rarely do individual stakeholders use all CAM identified objective indicators to measure progress, hence the previous reference to stakeholder-relevant indicators (Berkley 2009). The main reason is that selection of what is relevant for progress measurement is dependent on each stakeholder’s unique needs, values, and preferences.
Needs, values, and preferences vary across a stakeholder group (Berkley 2009). In referring to Fig. 3, one can see that group and individual assessments of progress can be different even though the individual is a part of that group (Barker 1978, Berkley 2009). Additionally, stakeholders may consider relevant objective indicators in their assessments that are not currently used as a measure of progress by the CAM program. Gathering information on these objective indicators allows one to determine what stakeholders are using to measure progress. Measuring what is relevant to a stakeholder is important in gaining support for action (Cash et al. 2002, Berkley 2009). Measurement of these indicators does not guarantee agreement to action by the particular stakeholder. However, it can provide information to decision makers about why a particular stakeholder does not think progress is sufficient (Berkley 2009). In the case where stakeholders are conducting subjective assessments of experimental results, as presented in Fig. 3, Fig. 1 explains relationships that show how this can result in diverse stakeholder perceptions and needs related to action and progress.
The framework using objective and subjective indicators was tested in a study of the AMP program (Fig. 3; Berkley 2009). Stakeholders were interviewed to determine their assessments of AMP progress. Interviews were conducted to collect data on the complex nature of stakeholder behavior. These data were used as subjective indicators. Stakeholder participants were interviewed using a questionnaire with 10 standard questions in both open- and closed-ended formats (Appendix 1). The interviews were conducted either in person or by telephone. Each interview was documented using an audio recorder with the informant’s knowledge and consent. The interviews were transcribed from audio to written format. Once the interviews were transcribed, they were returned to the informant to check for accuracy of content and then returned to the researcher.
Stakeholders were asked to describe their AMP-related needs. As a follow-up question, they were asked if any of their needs were not met. Their identification of “needs not met” was used to identify problems with AMP progress. Particular questions were asked for purposes of gaining an explicit understanding of the individual and collective stakeholder contexts in which their “needs not met” were expressed. Having this contextual information is important for examining the complexity of behavior in AM, and is helpful for explaining stakeholder assessments of progress. The information can lead to recommendations for resolution of problems. Questions were asked about stakeholder perceptions of the fairness of management of the resource with respect to all uses, on the AMP as a form of management, on the condition of the resource, on the sufficiency of progress, and on longitudinal AMP/Glen Canyon Environmental Studies (GCES) progress. These questions were asked to gather information about informational credibility, salience, and legitimacy. Glen Canyon Environmental Studies represent pre-AMP efforts to understand the nature of the downstream impacts of Glen Canyon Dam. The science program for the AMP was essentially a continuation of GCES efforts.
Fairness of process is important in governance because it is one of the bases of process legitimacy. Legitimacy of governmental process is a factor in whether a decision maker’s authority is accepted by those in the process (Adger et al. 2004). Perceived fairness may affect the level of stakeholder good-faith participation. Stakeholder perceptions of AMP as a form of management provide important contextual information for the understanding of attitudinal predispositions about the program. Because the evaluation in this study is for a program that is focused on restoration of ecosystem function and recovery of species, it makes sense that it would be important to understand stakeholder perceptions of the condition of the resource that the program is designed to manage. These perceptions could then be included in the evaluation. Stakeholder perceptions of sufficiency of progress are also critical for signaling an opportunity for improvement and problem resolution. Understanding stakeholder perceptions of the longitudinal progress informs how stakeholders formulate attitudes about the overall progress of the program. Their attitude may affect their assessment of progress and is related to their perceived credibility, salience of information, and legitimacy of process (Berkley 2009).
Members of the AMWG were also given a survey on AMWG developed AMP strategic plan goals and objectives (please see the example survey page in Appendix 2). This was used to assess the relative importance (i.e., ranked low, medium, high) of each goal to the stakeholder and to understand the variation among individual stakeholder perceptions about which goals are most relevant for use in progress measurement. The AMP Strategic Plan consists of 12 goals and 56 associated management objectives. The survey was administered via an electronic form, which was e-mailed to each respondent, filled out, and then returned by the respondent (Berkley 2009).
A root-cause analysis was performed on interview responses to determine AMP stakeholder assessments of progress through analyzing their perceptions on the root cause of problems with progress. Unmet needs were used to identify problems with progress and were coded. Coded stakeholder responses were aggregated and fell under 18 problem categories (Berkley 2009). For example, certain problems were identified by three AMWG members and aggregated as “clarity.” This problem was originally expressed in the interview responses by one member as: “So I’ve articulated two needs...more concrete boundaries and goals that everyone understands. Right now different people think that adaptive management of the Grand Canyon means different things.” A second member expressed their concern about “clarity” in the following quote: “And they also need to make it clear to the adaptive management members that that’s all they are doing is making recommendations, that they don’t have control over the program.” The third stakeholder response aggregated in the “clarity” category was:
What we need from this process is clear, incontrovertible scientific evidence that ecosystems and the species, not only those that are in trouble in the systems but also those that have been extirpated, that there’s active progress towards restoring them. That means there’s credible science for that recovery process and that there’s a clear and direct path to getting those species and ecosystems back to health.
After problems were identified, a root-cause analysis was conducted by searching interview transcripts for stakeholder explanations about the root causes of the problems. Root causes were coded and aggregated into categories. Three main themes were identified during the root-cause analysis: (1) funding inequity, (2) institutional rigidity or sluggishness, and (3) understanding of tribal needs and perspectives. These themes are represented by the following three respective quotes:
- “If you just look at the key resources that the program has been focusing on: sediment to build beaches, backwater, sand bars, and humpback chub, those two parts of the program probably get close to half the program funding (i.e., Glen Canyon Monitoring and Research Center’s funding) (AMP Stakeholder 1).”
- “We also would like to see...some additional funding from other sources besides power revenues (AMP Stakeholder 5).”
- “I don’t think a lot of those values that we present at the table...are being regarded; therefore, I don’t think...the need for support from AMP is not being met for our tribe (AMP Stakeholder 7).”
Objectives were created to address each problem’s root cause. The details used to focus each objective were based on a deeper reading of the root causes and needs coding reports. For example, in the case of the root cause identified as “insufficient understanding and integration of tribal needs and cultural perspective,” a deeper reading helped identify the need to integrate traditional knowledge and western science:
[Need] more of a conduit for the Native American perspective, I mean something that can flow along with Native American concerns and issues. Science has not addressed that, so you know if, how do you blend western science with Native American traditional knowledge and practices and is there a way that you can collaboratively come up with something that is going to be addressed in a collaborative manner, from both sides because it’s this huge gap in between and, you know, how do you form that bridge?
The United States Geological Survey (USGS) also conducted a progress evaluation, known as the SCORE report (Gloss et al. 2005), for the AMP using natural and cultural resources, and recreation objective indicator criteria, developed in the 1995 U.S. Department of the Interior Bureau of Reclamation Environmental Impact Statement
(EIS) for the Operation of Glen Canyon Dam.
The results of the AMP progress evaluations using subjective assessments were compared to the SCORE report for the AMP. The outcome was that the framework which integrated subjective indicators identified four stakeholder measures that the SCORE report did not: (1) equitable funding among Grand Canyon resources, (2) timely management action based on linked monitoring and experimental results and credible scientific and legal criteria, (3) commitment to tribal participation through the integration of Native American traditional knowledge and practices and western science; and timely implementation of an Indian/non-Indian designed Glen Canyon Dam AMP participation capacity building plan (including but not limited to financial and logistical needs), and (4) equitable sharing of responsibility for funding the Glen Canyon Dam AMP based on credible scientific and legal criteria (Berkley 2009).
These four identified measures were not addressed in the EIS-defined objective indicators. The information was useful in that it identified components of the social–ecological system that that weren’t being measured. By identifying those components, it also identified relationships important to stakeholder perceptions of progress.
Integrating stakeholder attitude and behavior information into CAM progress measurement can yield important information about yet-unidentified system components and relationships that may be keys to progress and public support. Gathering this knowledge (e.g., a stakeholder “needs not met”) and acting on it can provide an opportunity for decision makers to address concerns that would otherwise result in “outside the collaborative process” moves by stakeholders.
Because most CAM processes are facilitated, this type of information can be useful to the facilitator in problem solving. As previously mentioned, these integrated assessments can specifically identify stakeholder needs not met. The facilitator can use this information to focus problem-solving discussions.
The following paragraphs provide explanation of some aspects of this information that may be useful to consider in its application. Stakeholder involvement, whether through a group (AMWG, MRRIC, etc.) or public meetings, does not assure the incorporation of values into a CAM process. Additionally, it is important to capture needs, values, and preferences to get a sense of what concerns stakeholders may have, their attitudes, and potential CAM program-related behavior. As previously mentioned, an additional concern is that values, needs, and preferences expressed in a group do not mean that they are the same as those that will be expressed and acted on by an individual of the group privately (Barker 1978, Berkley 2009).
Attitudes and behavior do not necessarily operate at the same time scale as natural systems. Thus, when using tools to conduct multi-criteria decision analyses or SDM to optimize among resources and stakeholder associated values, one should take care the subjective indicator information is current. Otherwise, one may be optimizing something (needs, values, or preferences) that may no longer be relevant.
Additionally, stakeholder behavior, values, needs, and preferences can be quite dynamic and time dependent. An example from AMP is that in 2004 there was an energy crisis in California. Energy prices shot up practically overnight. Glen Canyon Dam provides power to California. Part of the dam reoperation for endangered species, and beach building downstream from the dam, required a decrease in ramping rates of the hydropower releases. This resulted in a decrease in potential power generation revenues. One can imagine that power marketers’ preferences, needs, and values shifted greatly overnight in relation to dam reoperations. Additionally, their perspective on the value of the AMP probably shifted also. Thus, they probably experienced a significant shift in their perspective on the metrics and magnitude of progress measurement. This is a dramatic example of how stakeholder attitudes and behavior can operate on dynamic time scales.
An example from the MRRP is a flood that occurred in 2011. The 70-member stakeholder group includes floodplain agricultural interests who work fields behind river levees. When the flood came, stakeholder attitudes about the restoration program shifted. For floodplain farmer stakeholders, the magnitude of importance of flood protection increased relative to other progress measures in the MRRP. There were many public meetings and frequent updates about the progress of the flood and dam releases. Although in a situation like this, some of the stakeholder values, needs, and preferences might appear obvious, it can nonetheless be useful to gather this kind of data. An obvious suggestion is to gather the information as far in advance of the anticipated event as possible, rather than under the pressure of emergency conditions. The usefulness of the information will last at least as long as the flood. However, gathering information about stakeholders’ attitudes towards other restoration-related measures at this time may also be quite useful in terms of maintaining support for the program in the future.
In the absence of such information, stakeholder behavior can be surprising; although collecting information on values, needs, and perceptions is not a guarantee against surprises. One of the results of the flooding and associated concerns was the discontinuation of work related to long-term planning for the ecosystem restoration component of MRRP. The integrated-behavior approach may have given advance warning that some stakeholders were not satisfied and might seek an avenue outside of the MRRP stakeholder process to get their needs met. Paying attention to these attitudes can help maintain support and progress for a CAM program. It also can help the facilitator of a process know where to focus efforts on bringing stakeholders together to solve problems.
The results of this work need to be further tested; it was completed for research purposes and no attempt was yet made to incorporate it into the AMP. The framework should be used through multiple cycles to determine whether the information gathered with this approach results in additional progress as compared with past approaches. In particular, it would be helpful to test whether gathering this information resulted in a decrease in stakeholders electing to go outside of the CAM process to get their needs met.
There are instances in which stakeholders are not interested in CAM progress. They may not view it as being in their interest to have a program make progress. In such instances, gathering information on attitudes and behavior may not provide much leverage for problem identification and solving.
MRRP future application
The integrated behavior progress framework has not been used on the MRRP. However, one of the advantages of applying it to this program would be to spot yet-unidentified stakeholder relevant system components and relationships in advance of them becoming problems and sources of barriers to MRRP progress. This framework could be useful in identifying the root of stakeholder concerns with respect to several scenarios. Examples of problems that have been cited as a concern in MRRP stakeholder meetings are the relationships between government land acquisition for habitat restoration, a decrease in the community tax base, and a decrease in the financial viability of river communities. Another example occasionally referred to as a concern is the relationship of degraded water quality, and the placement of excavated soil in the river as a part of restoration activities and the MRRP (National Research Council 2011). During the Missouri River flood of 2011, stakeholders expressed concern that there was a relationship between restoration, endangered species recovery activities, and flooding in the lower river. (See the 9 Jun 2011 article
that appeared in the Great Plains Examiner, for example.)
It is possible that the integrated behavior progress framework could be used to identify root causes associated with the stakeholder concerns above. It is also possible that the stakeholder concerns expressed are symptoms of a problem and not the root cause. The framework was used to conduct a root-cause analyses to determine four problem causes identified in the AMP example and could do the same for MRRP. As in the AMP study, interviews of MRRIC stakeholders about unmet needs related the MRRP AM would be needed as described for AMP.
Adger, N. W., K. Brown, J. Fairbrass, A. Jordan, J. Paavola, S. Rosendo, and G. Seyfang 2004. Governance for sustainability towards a ‘thick’ understanding of environmental decision making.
CSERGE Working Paper EDM 02-04:1–28. The Centre for Social and Economic Research on the Global Environment (CSERGE), School of Environmental Sciences University of East Anglia, Norwich UK. http://dx.doi.org/10.1068/a35289
Barker, R. G. 1978. Habitats, environments, and human behavior.
Jossey-Bass, San Francisco, California, USA.
Berkes, F., C. Folke, and J. Colding 2000. Linking social and ecological systems: management practices and social mechanisms for building resilience.
Cambridge University Press, Cambridge, UK.
Berkley, J. E. 2009. Stakeholder-relevant progress evaluation in adaptive management: Glen Canyon Dam and the Colorado River ecosystem.
Dissertation. University of Colorado, Denver, Colorado, USA.
Cash, D., W. Clark, F. Alcock, N. Dickson, N. Eckley, and J. Jäger. 2002. Salience, credibility, legitimacy and boundaries: linking research, assessment and decision making.
John F. Kennedy School of Government Faculty Research Working Paper RWP02-046. John F. Kennedy School of Government, Harvard University, Cambridge, Massachusetts, USA. http://dx.doi.org/10.2139/ssrn.372280
Gloss, S. P., J. Lovich, and T. Melis, editors. 2005. The state of the Colorado River ecosystem in Grand Canyon.
U.S. Geological Survey Circular 1282. U.S. Geological Survey (USGS), Reston, Virginia, USA.
Harwell, M. A., V. Myers, T. Young, A. Bartuska, N. Gassman, J Gentile, C Harwell, S. Appelbaum, J. Barko, B. Causey, C. Johnson, A. McLean, R. Smola, P. T. Gemplet, and S. Tosini. 1999. A framework for an ecosystem integrity report card. BioScience
Holling, C. S. 1978. Adaptive environmental assessment and management.
Wiley, New York, New York, USA.
Ludwig, D. 2001. The era of management is over. Ecosystems
National Research Council. 2011. Missouri River planning: recognizing and incorporating sediment management.
National Academies, Washington, D.C., USA.
Orenstein, S., S. Palmer, and P. Lewis. 2010. Assessment report: member and participant views about potential changes to the charter and operating procedures of the Glen Canyon Adaptive Management Workgroup.
U.S. Institute for Environmental Conflict Resolution, Tucson, Arizona, USA.
Rittel, H. 1984. Second-generation design methods. Pages 317–327 in
N. Cross, editor. Developments in design methodology.
Umi Research Press, Ann Arbor, Michigan, USA.
Susskind, L., A. Camacho, and T. Schenk. 2012. A critical assessment of collaborative adaptive management in practice. Journal of Applied Ecology
Wescoat, J. L., and G. F. White. 2003. Water for life: water management and environmental policy.
Cambridge University Press, Cambridge, UK.