Ecology and SocietyEcology and Society
 E&S Home > Vol. 22, No. 2 > Art. 39
The following is the established format for referencing this article:
Mavrommati, G., M. Borsuk, and R. B. Howarth. 2016. A novel deliberative multicriteria evaluation approach to ecosystem service valuation. Ecology and Society 22(2):39.
Research, part of a special feature on Ecosystems and Society: Interactions Among Climate, Land Use, Ecosystem Services, and Society

A novel deliberative multicriteria evaluation approach to ecosystem service valuation

1School for the Environment, University of Massachusetts Boston, 2Civil and Environmental Engineering, Duke University, 3Environmental Studies Program, Dartmouth College


Although efforts to address ecosystem services in decision making have advanced considerably in recent years, there remain challenges related to valuation. In particular, conventional economic approaches have been criticized for their inability to capture the collective nature of ecosystem services, for their emphasis on monetary metrics, and the difficulty of assessing the value of ecosystem services to future generations. We present a deliberative multicriteria evaluation (DMCE) method that combines the advantages of multicriteria decision analysis with a deliberation process that allows citizens and scientists to exchange knowledge and evaluate ecosystem services in a social context. Compared with previous applications we add the following: (i) a choice task that can be expected to lead to a more reliable assessment of trade-offs among ecosystem services, and (ii) an explicit consideration of the future by both presenting specific socioeconomic scenarios and asking participating citizens to serve as “trustees” for future generations. We implemented our DMCE framework with 11 panels of residents of the upper Merrimack River watershed in New Hampshire with the goal of assessing the relative value of 10 different ecosystem services in the form of trade-off weights. We found that after group deliberation and expert scientific input, all groups except one were able to reach internal consensus on the relative value of these ecosystem services. Additionally, the pattern of trade-off weights across groups was reasonably similar; there was no statistically significant effect of the specific future scenarios that were presented to the groups. Results of a survey given to participants after the deliberative process revealed that most felt that their opinion during the deliberation was heard by the others and that they were influential on the outcome. Further, the vast majority were satisfied with the outcome of the deliberation. We conclude by discussing the strengths and limitations of our framework at an operational level.
Key words: deliberative multicriteria evaluation; ecosystem services; informed decision making; public engagement; sustainability


The White House (2015) recently issued a memorandum “directing all Federal agencies to incorporate the value of natural, or ‘green,’ infrastructure and ecosystem services into federal planning and decision making.” Policy makers and researchers recognize that the ability to integrate ecosystem services into decision making has advanced considerably over the last several years (Schaefer et al. 2015). However, there are still challenges to overcome in using the ecosystem services concept to maximize social welfare. For example, the conventional economic valuation approaches typically used have been criticized for their inability to capture the collective character of ecosystem services (Sagoff 1990, Howarth and Wilson 2006, Wegner and Pascual 2011). Given that ecosystem services embody characteristics of public goods, evaluating ecosystem services in a socially constructed way is crucial for effective decision making (Dryzek 2000). The difficulty in assessing the value of protecting ecosystem services for future generations and the emphasis on monetary metrics are other common criticisms of conventional valuation methods (Wegner and Pascual 2011, Farley 2012).

Attempts to address these challenges have led to the development of alternative valuation techniques based on the principles of deliberative democracy (Habermas 1984, Dryzek 1994, Zografos and Howarth 2010). The idea behind deliberative democracy is that in democratic societies there is a need to engage the public more actively in decision making and to reach decisions through deliberation and discourse. Discourse-based valuation methods use a small group of citizens or stakeholders, acting as a focus group or a “citizens’ jury,” in which participants deliberate with the aim of reaching consensus about the value of public goods (Wilson and Howarth 2002).

The deliberative multicriteria evaluation (DMCE) method, in particular, combines the advantages of multicriteria decision analysis, i.e., structure and transparency, with local knowledge building and sharing through a process of deliberation (Proctor and Drechsler 2006). The DMCE method allows citizens and stakeholders to actively engage in decision making, resulting in a broader community understanding with respect to environmental issues and an increased probability of conflict resolution in the early stages of environmental planning (Lennox et al. 2011). The DMCE method has been applied to several problems in environmental resource management (Proctor and Drechsler 2006, Garmendia and Gamboa 2012, Karjalainen et al. 2013).

The DMCE method is flexible and provides a framework for experts to communicate social-ecological systems’ complexity and uncertainty to citizens and stakeholders (Liu et al. 2011). Especially with respect to ecosystem services valuation, the DMCE offers the basis for exploring issues that cannot be explored by conventional valuation methods (Sagoff 1990, Wegner and Pascual 2011, Farley 2012). For example, previous DMCE studies have integrated the concept of ecosystem services into environmental impact assessment, helping to identify ecosystem trade-offs, secure stakeholder participation, and communicate the science behind ecosystem service indicators (Oikonomou et al. 2011, Karjalainen et al. 2013). Yet, the DMCE method requires further investigation to explore and develop its full potential (Lennox et al. 2011).

With this in mind, we developed a novel valuation framework based on the DMCE method that embodies sustainability considerations, including uncertainty about future human preferences and the state of the social-ecological system. Given that future generations cannot participate in decision making today and there is no possible way to elicit their preferences, we need to explore novel ways for the current generation to address this challenging concern. Conventional valuation methods overcome this representation obstacle by assuming that the preferences of future generations will be similar to those of the present generation (Mavrommati et al. 2016). This assumption legitimatizes the actions of present generations that interfere with future generations’ choices and well-being. However, there are studies suggesting that values are dynamic in coupled human and natural systems (Winkler 2006a,b, Skourtos et al. 2010). Instead of simply assuming future generations’ preferences remain constant, in this paper we explore the potential of asking a group of citizens to serve as “trustees” for future generations.

We employed the DMCE method in four full-day workshops to evaluate the ecosystem services provided by the upper Merrimack River watershed, New Hampshire (NH). Multiple citizen juries were used in each workshop to provide for replicability of our approach. We focused on three innovations relative to conventional valuation methods:


Conventional methods to valuing the environment such as contingent valuation or multicriteria decision analysis have been subject to a variety criticisms (Gregory et al. 1993, Wegner and Pascual 2011). We discuss three arguments against such methods and in favor of discourse-based deliberation.

Individual vs social preferences

Many ecosystem services have the characteristics of common (rival and nonexcludable) or public goods (nonrival and nonexcludable) making them subject to the free rider problem (Muradian and Rival 2012). This implies that an individual can benefit from an ecosystem service without contributing to its management or conservation (Vatn 2009). This reveals the ethical dimension of environmental choices and challenges the idea that individual preferences adequately address ethical considerations (Vatn 2009). Thus, ecosystem service valuation is a concern that goes beyond self-interested utility and involves “social rights and wrongs” (Sagoff 1990, Wilson and Howarth 2002).

A related issue concerns the long time horizon of many ecosystem service considerations. Even though decisions taken today are likely to have an effect on future generations’ choices and well-being, conventional methods are typically based merely on the values of current generations. This concern is well described by O’Neill (2001) who points out that future generations (and nonhuman species) are at best misrepresented in conventional valuation methods.

With the above concerns in mind, Wilson and Howarth (2002) suggest that we need to explore new ways to assess the worth of ecosystem services that respects their collective character and at the same time treats individuals fairly within and across generations. Dryzek (2000) proposes that deliberative processes can lead to socially constructed values through mutual agreement. In other words, deliberative methods provide the basis for constructing social values through discourse-based interaction among citizens, stakeholders, and scientists. Further, if those participating in the deliberative process are asked to serve as trustees, then it may be possible through constructive and well-reasoned dialogue to make choices without assuming that future generations will have similar preferences as those of the current generation.

Role of social learning

Irrespective of the selected valuation method, the task of ecosystem service valuation typically employs one or more hypothetical scenarios as well as a particular valuation task. For example, in contingent valuation methods respondents are asked to express their willingness to pay for an ecosystem service given a particular scenario. Depending on the survey method, e.g. mail, internet, or phone, respondents may be confused about either the scenario description or the way in which the ecosystem services are characterized. This can result in protest votes or misconceptions leading to response errors. Accurate assessment thus requires the participants to fully understand the hypothetical scenario and the ecosystem attributes being used for the valuation task. Assuming that people hold perfect knowledge when they shape preferences for ecosystem services may result in misguided decisions. Also, the selected method to communicate scenario-related information to respondents or workshop participants may influence participants’ knowledge building and preference judgements. Apart from the traditional scientific PowerPoint presentation, scientists have been using alternative methods to communicate science to the public such as videos, games, illustrations, and theater (Sheppard et al. 2011, Wibeck et al. 2013, Carpenter et al. 2015). For example, Carpenter et al. (2015) used illustrations and videos to communicate their scenarios to the general public.

Deliberative valuation methods can help overcome the challenge of scenario communication and understanding by (i) bringing participants together so they can share information through dialogue and deliberation (Sagoff 1990, Howarth and Wilson 2006) and (ii) bringing scientific knowledge about the scenarios and attributes of ecosystem services into the process when it is required. More applications are necessary to gain a better understanding of how social knowledge is enhanced through deliberative processes.

Monetary vs nonmonetary values

Monetary techniques for ecosystem service valuation use economic gains and losses as a proxy for the utility derived from the environment. Revealed preference techniques, for example, use existing markets of normal economic goods, e.g., housing, to infer values of related ecosystem services, e.g., air quality. However, there are only a few ecosystem services for which relevant data exist for supporting revealed preference methods. Stated preference techniques, on the other hand, create a hypothetical market to elicit individuals’ willingness-to-pay (WTP) for the preservation of ecosystem services or their willingness-to-accept (WTA) payment for forsaking them. One argument against stated preferences techniques is that people may hold incommensurable values for ecosystem services such as water quality and endangered species (Gregory et al. 1993). Chan et al. (2012) argue that monetary valuation is ill-suited for a broad suite of ecosystem services, e.g., cultural services and nonuse values, because it cannot count for the multiple benefits that people may derive from the same ecosystem service. Another issue related to monetary valuation is the inability to capture characteristic ecosystem properties such as nonlinearity and irreversibility (Limburg et al. 2002, Winkler 2006a). For example, if the demand for an ecosystem service is inelastic, crossing critical ecological (or social) thresholds may result in extremely high marginal WTP or WTA at these thresholds.

Nonmonetary techniques such as multicriteria decision analysis (MCDA) provide an appropriate framework for valuing environmental attributes that have different measurement units, are intangible, and/or cannot be quantified in monetary terms (Belton and Stewart 2002). In MCDA, the multidimensionality of ecosystem services can be addressed in the valuation process, allowing for a representation of the trade-offs among them (Gregory et al. 1993, Borsuk et al. 2008, Chan et al. 2012). The ability to quantify the provision of diverse ecosystem services under alternative scenarios has been considerably improved in recent years (Samal, Wollheim, Zuidema, et al., unpublished manuscript). Yet, for application to MCDA, a remaining challenge is for social and natural scientists to express ecosystem services in terms that are salient and easily understandable to the public and stakeholders. Deliberative approaches assist in this process through mutual knowledge building in which ecosystem service indicators are subject to discussion and compromise and can be further explained by experts when required (Proctor and Drechsler 2006, Chan et al. 2012). Within the deliberative evaluation setting, researchers can convey information with respect to ecosystem processes that may result in nonlinear and irreversible responses and include this aspect in the evaluation task.


The DMCE method integrates MCDA and deliberation into a common framework for valuation of ecosystem services (Proctor and Drechsler 2006). MCDA techniques provide the appropriate structure and transparency, while deliberation assists in community involvement and knowledge building. Applying the method at an operational level involves several steps (choosing the jury, options, and overall objectives; selecting the criteria to gauge the objectives; weighing the criteria; assessing the options; aggregating the criteria for each option; sensitivity analysis; interaction and iteration) depending on the overall objective of the project (Proctor and Drechsler 2006). Compared to previous DMCE applications, we added (i) a choice task informed by conjoint analysis that can be expected to lead to a more reliable assessment of trade-offs and (ii) an explicit consideration of future generations by using socioeconomic scenarios and asking participants to serve as “trustees” for the future.

Description of MCDA and the role of deliberation

MCDA refers to the application of a structured approach to addressing trade-offs among multiple, often competing, objectives. A variety of specific methodologies for MCDA have been developed, including the analytic hierarchy process (AHP), outranking, e.g., ELECTRE or PROMETHEE, goal programming (GP), and multiattribute value theory (MAVT). We chose to employ MAVT because of its strong axiomatic foundations and because of our successful past experience using it in similar applications (Hostmann et al. 2005).

The goal of MAVT is to convert a particular state of the world, defined over a multidimensional space of attributes, to a real number, such that the ordering of such numbers across alternative states represents the preference ordering for those states that is consistent with individual or social value judgments (Fishburn 1967, Keeney and Raiffa 1993). In other words, MAVT employs a function (called the value function) that aggregates the degree to which a particular state of the world satisfies multiple objectives to provide an overall assessment of the relative desirability of alternative states of the world. One such function takes a weighted average, or additive, form:

Equation 1(1)

where V(x1, ... , xi, ... , xn) is the multidimensional value function, the vi(xi) are one-dimensional value functions that convert each of the n attributes xi to a common (typically 0 to 1) scale, and the wi are the trade-off weights for each attribute.

MAVT is inherently able to assess the sustainability of a particular state of the world because it allows for a simultaneous accounting of attributes characterizing the three dimensions of sustainability: economic, social, and environmental. However, the additive form of the value function given in Equation 1 assumes that there are no interactions among preferences for the levels of different attributes, a property known as mutual preference independence. Thus the additive multiattribute value function is compensatory, meaning that weak performance on one attribute can be offset by strong performance on another. This means that while the additive form is adequate for operationalizing the concept of weak sustainability, it is not appropriate for assessing strong sustainability.

A variety of methods have been developed to assist in the assessment of the trade-off weights, required for Equation 1 (Von Winterfeldt and Edwards 1986). However, there is no axiomatically satisfying method for mathematically aggregating the weights of multiple individuals. Averaging the weights, for example, may not lead to a multiattribute value function that describes any of the individuals’ preferences. Fortunately, as Proctor and Drechsler (2006) point out, a citizens’ jury may accomplish a satisfying aggregation of stakeholder weights by employing deliberation to achieve consensus. Proctor and Drechsler (2006), however, asked participating citizens to assign weights to attributes directly, a process prone to misrepresenting preferences if the range of possible attribute levels is not carefully considered (Clemen and Reilly 2013). Therefore, we employed a more reliable method called “swing” weighting, which is similar to conjoint analysis (Rao 2014) in that hypothetical, multiattribute states of the world are discussed and scored by participants, and the weights are then inferred from these scores.

Description of choice task

The Swing method typically uses a hypothetical reference state in which all attributes are set at their worst level. The person performing the valuation then scores (relative to this worst state) other hypothetical states in which one attribute at a time is “swung” to its best level. For the additive multiattribute value function, the weights on the swung attributes then turn out to be proportional to these scores. An advantage of swing weighting is that it only requires that the attribute ranges be known, i.e., the best and worst values of the attributes. It does not require knowing the shape of the underlying single attribute value functions nor the likelihood of intermediate attribute values (Eisenführ et al. 2010).

A drawback in applying the typical swing method in our study is that the hypothetical worst case state may be difficult for participants to imagine, given that, for many ecosystem attributes, these levels correspond to an anticipated future state rather than any recent past experience. Thus, we employed a “reversed swing” method (Schuwirth et al. 2012) in which the reference state sets all attributes to their best level and then the states to be scored each have one attribute “swung” to its worst level. For many attributes, the best-case reference state is likely to be similar to the actual current state, so participants can use it as a salient reference point.

The specific task that we asked each citizen jury to perform was to position cards, representing the best-case reference state and the hypothetical, multidimensional states, each with individual attributes swung to their worst level, along the length of a measurement stick scaled from 0 to 100. The card representing the most preferred state (presumably the best-case reference state) is to be placed at the 100 end of the stick, and the card representing the least preferred state (to be determined by the jury) is to be placed at the 0 end. The other cards are then to be placed anywhere between the two ends with position and spacing representing the jury’s shared preference ordering.

The weights, are then each calculated as:

Equation 2(2)

where si is the location (from 0 to 100) of the ith attribute along the measurement stick and n is the number of attributes being evaluated.

Considering the future using scenarios

A unique aspect of our study relative to previous applications of DMCE is that, rather than asking the participants in our citizen jury to focus on their own current preferences, we asked them to serve as “trustees” for future generations. To assist them in this process and to provide specificity, we presented them with a detailed hypothetical socioeconomic scenario for the year 2100 prior to their deliberations. In fact, to explicitly consider how their deliberations might be shaped by their perception of the future, we employed two sets of experimental “treatments”: one that varied the specific socioeconomic scenario that was presented and a second that varied the method of scenario presentation. Our hypotheses were that the trade-off weights derived from the choice task would systematically differ depending on the following: (1) the participants’ expectations regarding the socioeconomic conditions of future generations, and (2) the salience of the presented socioeconomic scenario to the participants.

Testing these two hypotheses required an experimental design involving four separate day-long workshops, each of which was defined by a unique combination of a future scenario and presentation style. Each workshop then involved two or three separate juries as “replicates.”


Study domain

We applied the DMCE method in the context of the upper Merrimack River watershed, New Hampshire, USA, defined by a point just south of the city of Manchester, New Hampshire. This subwatershed drains 8000 km² and is home to 410,000 people (Samal, Wollheim, Zuidema, et al., unpublished manuscript). The region is experiencing rapid population growth and land use change, leading to increased water use, nitrogen discharge, and other environmental impacts associated with development. Forests are the dominant land use and are thus critical influences on water supply, water quality, and aquatic habitat. The watershed sits at the boundary between strong and weak winters and is expected to shift to warmer temperatures and greater and more variable precipitation in the coming decades. Because the area is an important tourist destination for both winter and summer recreation, such changes in climate are likely to have important economic consequences. For all these reasons, the upper Merrimack watershed is an ideal location for assessing methods aimed at better understanding the value of ecosystem services to future generations.

Attribute selection

It is important to carefully consider the choice of attributes used to characterize the ecosystem services being assessed by a multiattribute value function. To the degree possible, the attributes should be mutually exclusive and collectively exhaustive so that they cover the range of key concerns without double-counting (Keeney and Raiffa 1993). Ideally, the process of selecting the attributes should involve both the ecosystem citizen stakeholders and the experts who measure or predict the attribute levels for current or future states of the world. However, because of limited time and resources for citizen workshops, we decided to rely on the expert judgement of our team to define the attributes. We organized four meetings of the Experimental Program to Stimulate Competitive Research (EPSCoR) scientists from diverse disciplines, e.g., aquatic ecology, forest ecology, ecological economics, and decision science, to iteratively consider the best way to represent key ecosystem services in a way that would be both salient and important to the members of the general public likely to compose our juries. It was also important that the attributes correspond to environmental indicators that could be measured or modeled for both present and future conditions (Samal, Wollheim, Zuidema, et al., unpublished manuscript). Ultimately, we settled on 10 attributes that could be classified into thee domains: land, climate, and water (Table 1). A detailed description of the environmental indicators is given by Samal, Wollheim, Zuidema, et al. (unpublished manuscript). The worst and best levels of each attribute correspond to two states of the world that bracket a broad range of potential futures as assessed by Thorn, Wake, Grimm, et al. (unpublished manuscript) and Samal, Wollheim, Zuidema, et al. (unpublished manuscript).

We applied the reversed swing method separately to each of the three domains (land, climate, water) and then across the three domains. Because we had participants evaluate the three domains separately, followed by an evaluation across the domains, our final multiattribute value function assumes a nested form:

Equation 3(3)

where wj are the m domain weights and wi are the nj attribute weights within each domain j. Figure 1 shows examples of the cards used in the assessment task.

Experimental treatments

For the scenario dimension of our study design (Fig. 2), we chose two scenarios referred to as the “backyard amenities” and “community amenities” scenario families by Thorn, Wake, Grimm, et al. (unpublished manuscript; Appendix 1). These scenarios depict alternate futures defined by either rapid suburban development and a lack of ecosystem service protection (backyard amenities) or concentrated mixed-use development and conservation of natural areas (community amenities). Each scenario included quantitative or qualitative information about population and demographics, land cover, patterns of development, public services, energy sources, and economic conditions.

The two scenarios were presented by two different methods on different days: a PowerPoint-style slide presentation and a theater performance. The slide presentation was given by a scientist who described the defining features of the scenarios using maps, graphs, bullet points, and tables. The theater performance was given by four actors who played the roles of representative citizens in the year 2100 and conveyed the key elements of the scenarios over the course of a reenacted conversation. Our hypothesis was that the scenario presented in the form of a theater performance might be more salient to the members of the jury than a slide presentation, especially to those who may be less scientifically or numerically adept. Details on the two presentation methods can be found in the Appendix 2.


Participant recruitment

We followed a two-step process to recruit participants for our workshops. First, we advertised in local media (radio and newspapers) and social media (Facebook) to request participation in an online survey (Appendix 3) which also invited respondents to participate in our full-day workshops in return for a $100 incentive, lunch, and travel costs. We had 286 respondents to our survey, 217 of whom lived within the upper Merrimack Watershed and were willing to participate in one of our workshops. Then, from among these 217 respondents, we invited 96 who, to the greatest degree possible, represented the demographics of the New Hampshire population by age, sex, income, and political affiliation. A total of 67 participants showed up for the four workshops, which were held on four separate dates in September 2015 (Table 2). Based on the number of attendees at each workshop, we had two juries on the first date and three on each of the other three dates. Each of the 11 citizen juries comprised between five and seven individuals with diverse demographic characteristics. We acknowledge that this self-selected sample has some limitations, but given the character of our experiment, we chose this recruitment and selection method as most appropriate. We did not share any workshop materials with the participants in advance because we were aiming to expose all participants to the same information at the same time. We recruited eight EPSCoR scientists (at least three for each workshop) to serve as expert “witnesses” during the citizen jury’s deliberation process.

Workshop structure

The goal of each workshop was for each citizen jury to assess how people living in the upper Merrimack River watershed in the year 2100 would value the benefits derived from the natural environment, given a specific future socioeconomic scenario. This time frame is appropriate for addressing research questions related to sustainability and for assigning the role of trustees for future generations because it represents a time horizon beyond their own lifetime. Participants were reminded that, like the members of a jury, they have the duty of promoting the common good. Specifically, in this context, they serve the role of trustees for future generations, to protect their rights and promote their interests like a parent does for his/her child or grandchild. Scientists, one for each of the land, climate, and water domains, served as “expert witnesses” to bring evidence to the jury in case they need more information during the deliberation process. We also employed one facilitator for each jury to serve the role of “judge” in a jury trial, keeping the jury on task but not advising them nor making the final decision. To accomplish our aims, we organized each full-day workshop into two parts:

Part I

During the morning session, we introduced participants to the process and their charge. This included a short presentation of the ecosystem services concept, the participants’ role in the process as well as the role of the scientists and facilitators. The specific socioeconomic scenario was then presented according to the specific presentation method assigned to that day (see Fig. 2). Finally, the attributes used to characterize the 10 selected ecosystem services were presented by the three scientists representing the three domains. Participants were given the opportunity to ask questions after each presentation.

Part 2

In the second session, participants were divided into two or three juries based on their demographic characteristics and brought to separate rooms. Each facilitator reminded the group of the process and the charge to ensure that the group understood the assigned task. Participants then introduced themselves, followed by a short “ice-breaker discussion” about their connection to the natural resources of the upper Merrimack River watershed. The juries then performed the choice tasks described in the Methods section above, one domain at a time. To start, the facilitator asked each participant to share with the rest of the group their thoughts and arguments for a particular ordering of the four or five cards representing the particular ecosystem service bundles needed to implement the reversed swing weight method. They then began to place the cards along the measurement stick with location and spacing representing their relative preferences for each bundle. An expert in each domain was available to answer questions as they arose. Discussion, deliberation, and debate continued until the jury reached a consensus, a compromise, or stalemate. Facilitators managed the process so that everyone participated and, in the case of consensus or compromise, each person explicitly agreed with the final ranking and rating of cards. Time was managed so that each domain was given approximately one hour of consideration. At the end of the workshops, participants were asked to complete a five-question survey about their experiences in the deliberative process.


All groups were able to reach consensus or compromise on the ratings of ecosystem service bundles, except for one (Group 10), which could not agree on the climate domain or the trade-offs across the three domains. For all other groups and domains, trade-off weights could thus be calculated for each ecosystem service attribute. Overall, the pattern of trade-off weights was reasonably similar across groups (Figure 3, Table 3). There was no statistically significant effect of scenario on the inferred weights in any one of the domains or across domains (MANOVA; p = 0.65 for Land, p = 0.25 for Water; p = 0.97 for Climate; p = 0.32 across domains). There was no statistically significant effect of presentation style on the weights in any of the domains except for Water (MANOVA; p = 0.38 for Land, p = 0.05 for Water; p = 0.52 for Climate; p = 0.86 across domains). For the Water domain, with the theater presentation Fish Habitat Loss received significantly lower weight and Water Shortage received significantly higher weight than with the slide presentation. Table 4 summarizes the arguments used in the deliberative groups in support of higher or lower weights for the various ecosystem service attributes. Most groups directed their discussion toward discussing the multiple ways that the ecosystem services contribute to ecosystem health and human well-being as well as the potential to restore or replace these ecosystem services if they are degraded in the future. Detailed results within each domain follow.

Land Domain

In the Land domain, Farm Land typically received the greatest weight and Forest Type the least, although these differences were not statistically significant (Table 3). The main arguments in favor of high weight on Farm Land were that local food may provide security and health benefits. Arguments supporting the importance of Forest Cover were the importance of linkages between forest cover, carbon sequestration, and ecosystem health. Also, many participants discussed the multiple ways that Forest Cover contributes to human’s connection to nature and environmental well-being, e.g., recreation. Participants in Groups 2, 8, and 11 agreed that so long as Forest Cover is sufficient, Forest Type is not a big concern. In particular, Group 8 discussed the limited ability to manage Forest Type compared to the other land indicators because of climate change impacts on forest composition. Members of Groups 2 and 4 also argued that forest resource exploitation provided economic benefits and thus protecting Forest Cover is not of critical importance.

Group 5, the only group that prioritized the land-related attributes in reverse order, agreed that less Farm Land may not necessarily reduce food production, given an historical trend of increased agricultural intensity, thus Farm Land could receive lower weight. This group also emphasized the maple forest’s importance to state identity and cultural heritage, thus supporting a greater weight to Forest Type.

Climate Domain

Hot Days were the most important of the Climate attributes for all groups, followed by Snow Cover, and these differences are statistically significant (Table 3). Nine groups weighed Hot Days highest and all groups weighed Recreation Days lowest. Most groups argued that the heat stress caused by hot days is critical to human health and labor productivity. In addition, Hot Days impose limits on recreational opportunities and increase energy use for cooling. A few participants argued that humans are adaptable to higher temperatures by presenting the case of other countries or states as well as various mitigation measures readily available, e.g., air-conditioning.

Winter recreation is a cultural ecosystem service inextricably linked to New Hampshire’s identity. Most groups discussed the recreational and economic benefits that arise from high Snow Cover and the couplings between snow cover and ecosystem health, e.g., wildlife population, as well as public health, e.g., Lyme disease. On these grounds, two groups (Group 2 and 5) gave Snow Cover greater weight than Hot Days. Only three groups (1, 8, and 10) emphasized the benefits that may arise from fewer days of snow resulting in the form of improved road safety and less salt usage. They also identified snowmaking as a possible substitute. There seemed to be a consensus among the groups that Recreation Days is the least important attribute given the low range between low and high values predicted.

Water Domain

In the Water domain, there was greater variability among groups. We believe that the systematic differences that seem to correspond to scenario presentation style actually result from a somewhat different description that different experts provided on different days of the various water-related attributes. Specifically, during the first two workshops in which the slide presentation was used, the water experts were specialists in water quality characteristics, while in the other two workshops the experts had greater specialization in water quantity aspects. Thus their description of the attributes and the way in which they answered participants’ questions could have influenced the juries’ deliberations. This is an issue requiring further investigation. Nevertheless, overall the trade-off weights indicate approximately equal importance of Water Shortage, Coastal Harm, and Fish Habitat Loss, while Flooding received a significantly lower weight (p = 0.002). Most groups believed that high Flooding might not present a problem because people have the capacity to mitigate flood risk through zoning and engineering solutions. Only a few groups (4, 5, 6, and 10) assigned relatively higher weight to Flooding mainly because of concerns regarding security and economic costs resulting from property loss.

Arguments by some participants in support of a low weight on Fish Habitat Loss (Table 4) were that other species may inhabit the river and replace the existing ones, as well as the potential for fish stocking to manage this problem. High weight on Fish Habitat Loss was argued by those who considered it an essential ecosystem service for supporting aquatic life, recreational activities, and basic human needs such as food. Group 5, the only group that weighed Fish Habitat Loss higher than the other ecosystem services, emphasized that the lack of technological fixes compared to other ecosystem services makes its protection imperative.

The main argument in support of the high importance of Coastal Harm was the impact of excess nitrogen on aquatic life, drinking water quality, tourism, and eventually on human health. Those arguing for lower importance of Coastal Harm raised the possibility of managing excess nutrients through engineering and/or green infrastructure and using excess nutrients for agriculture.

Six groups weighed Water Shortage highest and participants in those groups argued that water satisfies various human needs including basic ones such as drinking water. Some also mentioned the monetary implications for business. On the other hand, a few groups considered how lower water provision can be managed through human behavior and technological advances supporting efficient water use.

Across Domains

Comparing the three domains, the groups gave significantly greater weight to Water than to Climate or Land (p = 0.0002). All groups argued that water is the basis of life, and one participant stated, “You can’t live without water.” Also, most groups weighed Water higher compared to climate because it is more manageable; participants pointed out the global dimensions of Climate compared to the regional actions that can improve the attributes of Water. A few groups discussed the importance of Land for human well-being because the attributes of Land can “create comfort.” Overall, the capacity of policy and management tools at a regional level to maintain or restore the domain attributes along with the importance for human well-being drove the discussion and the trade-off weights.

Results of the survey concerning the participants’ experience in the deliberative process (Fig. 4) indicate that most felt their opinions on the relative importance of ecosystem services only changed “Slightly” or “Moderately.” Most participants felt that their opinion during the deliberation was heard “Considerably” or “Extremely” and that they were “Moderately” influential on the outcome of the deliberation. Opinion on the influence of the scientists testifying to the citizen jury was mixed, although all participants felt that the scientists had at least slight influence on the outcome. Overall, the vast majority of participants were at least "“Considerably” satisfied with the outcome of the deliberation.


Our method applies techniques from decision science and participatory planning to gauge public values concerning ecosystem services in a manner that addresses the potential weaknesses of monetary valuation. Our findings regarding the assessed trade-off weights suggest that, in each domain, participants assigned greater importance to the ecosystem services that satisfy basic human needs. For example, area of Farm Land is an indicator of potential local food production. Water Shortage and Coastal Harm are both aspects of water quantity and quality related to a healthy human condition. In the same vein, Hot Days was the major climate concern with participants emphasizing the effect of extreme heat on the human body and health as well as the repercussions for labor productivity and the economy. In general, we observed that the participants extensively discussed the ways that human well-being depends on ecosystem services, and they explicitly acknowledged the necessity of at least ensuring that basic human needs are satisfied in the future. The validity of these findings should be explored further with more applications at various spatial and temporal scales before utilizing in real world decision making.

Challenges in the application of the DMCE process

Designing and operationally applying DMCE involves several steps, requires the integration of disciplines and people, and thus demands the allocation of resources (time and funds). In our case study, scientists from various backgrounds, e.g., aquatic ecologists, forest ecologists, climate scientists, and economists, assisted in the process of selecting the appropriate ecosystem services and indicators that should be considered in our study. Even though the DMCE process provides the appropriate framework to include attributes that have various measurement units, it is critical that social and natural scientists work together to communicate these attributes to people who may not have any scientific background. This process was iterative and relied on the availability of relevant biophysical and climate models.

Even though we conducted our experiments with several parallel groups of citizens, the problem of representation and inclusiveness (Vatn 2009, O’Neill 2001) remains a challenge. First, citizens’ participation in a workshop is time consuming, resulting in a limited number of individuals who will devote their time. Even though we targeted the participation of 80 individuals, invited 96, and confirmed 85 to allow for no shows, only 67 ultimately attended the workshops. At the same time, limited funds devoted to this aspect of research constrained the number of people who could be invited. Second, overall participants were diverse in terms of political affiliation and income level, but most participants’ age was over 50 (Table 2). Given the time-consuming nature of the participation, we found it difficult to recruit younger people to this process. This is related to the tendency in New Hampshire of older people participating in collective decision making, such as government elections (File 2014), and is an issue that we need to find ways to address.

The replicability of the DMCE process is another challenge. We followed an identical overall process to organize and run the four workshops, and our results suggest that it is possible to replicate the process. However, as we learned in the case of the water-related attributes, it is important to have the same experts involved in every workshop so that the science behind the ecosystem service indicators is always communicated in the same way. This is supported by the finding that all the participants felt that the scientists had at least some influence on the eventual results of the deliberation (see Fig. 4d).

Opportunities in the application of the DMCE process

We found that knowledge-building during the deliberations was a two-way process; not only did participants advance their knowledge through their interactions with other participants and scientists, but also the scientists found new ways to communicate their work and learned how the public understands their science. The aid of professional facilitation enhanced this interaction by assuring inclusiveness and representation within each group.

The process of participants expressing their arguments and reasoning about ecosystem services, human well-being, and future socio-ecological state uncovered and improved their shared knowledge about ecosystem services in the Upper Merrimack watershed. In particular, we were able to identify some common themes of deliberation and to better understand some changes in opinion in response to other participants’ or experts’ assertions. Many participants explicitly mentioned that they changed their view in response to new evidence that an expert or another juror brought to the table. For example, one participant said “what [expert X] said caused me to change my thinking totally for water shortage. If it is only a possibility for three days a year, it’s not so severe. I wouldn’t see that as number one anymore.” Another participant mentioned “I changed my opinion because some of the people have given interesting arguments.”

The process also benefitted the scientists in understanding the information that citizens need to form beliefs and preferences about ecosystem services. Scientists found that participants could understand that, for practical reasons, some attributes had to serve as surrogates for multiple ecosystem services not explicitly identified. For example, participants were interested in forest products, recreation, carbon storage, and aesthetic value, all of which had to be represented by the single attribute Forest Cover. Scientists also gained experience in communicating their area of expertise in a comprehensible way and in lay terms. This was an interactive process in which scientists engaged with participants in an open-ended discussion without either predetermined questions or answers.


We presented a novel deliberative multicriteria evaluation method that allows citizens to assess ecosystem service trade-offs in a manner that accounts for the uncertainty that characterizes the future of social-ecological systems. The proposed method has its roots in the deliberative democracy in that it supports collective decision making through well-reasoned dialogue and debate that goes beyond self-interest and promotes the common good. In addition, the DMCE method provides a framework for including nonmonetary metrics into ecosystem service valuation and for building social learning in a structured way by integrating both scientists’ and participants’ knowledge. Analysis of our workshop results reveals the applicability of the proposed method at an operational level and the potential to explore issues that cannot be addressed by conventional valuation methods. In addition, the workshop participants felt that their opinions were heard, believed that they were personally influential, and were satisfied with the outcome. Therefore, we suggest that the proposed method may provide a solid basis for assessing ecosystem service trade-offs in a setting that builds social knowledge, generates consideration for the voiceless future generations, and creates satisfaction about the process.


Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link.


This work is supported by National Science Foundation's Research Infrastructure Improvement Award EPS1101245 and by the U.S. Environmental Protection Agency's Science to Achieve Results (STAR) Award Grant No. 836169. The citizens' workshops were approved by Dartmouth College IRB # STUDY00028709 and we appreciate the involvement of sixty seven citizens. We thank Stacie Smith, Matt Strassberg and Varun R. Mallampalli for facilitating, S. Rogers, D. Lutz, C. Wake, W. Wollheim, S. Zuidema, M. Mineau, N. Samal and A. Thorn for serving as experts, D. Kaye, CJ Lewis, J. Sears, S. Poulin and A. Desrosiers for the interactive theatre performance and L. Pyer for assisting in the preparation of the workshop, and the reviewers for providing comments on earlier drafts.


Belton, V., and T. J. Stewart. 2002. Multiple criteria decision analysis: an integrated approach. Springer Science & Business Media, Dordrecht, The Netherlands.

Borsuk, M. E., M. Maurer, J. Lienert, and T. A. Larsen. 2008. Charting a path for innovative toilet technology using multicriteria decision analysis. Environmental Science & Technology 42(6):1855-1862.

Carpenter, S. R., E. G. Booth, S. Gillon, C. J. Kucharik, S. Loheide, A. S. Mase, M. Motew, J. Qiu, A. R. Rissman, J. Seifert, E. Soylu, M. Turner, and C. B. Wardropper. 2015. Plausible futures of a social-ecological system: Yahara watershed, Wisconsin, USA. Ecology and Society 20(2):10.

Chan, K. M. A., T. Satterfield, and J. Goldstein. 2012. Rethinking ecosystem services to better address and navigate cultural values. Ecological Economics 74:8-18.

Clemen, R., and T. Reilly. 2013. Making hard decisions with DecisionTools. Cengage Learning, Boston, Massachusetts, USA.

Dryzek, J. S. 1994. Discursive democracy: politics, policy, and political science. Cambridge University Press, Cambridge, UK.

Dryzek, J. S. 2000. Deliberative democracy and beyond: liberals, critics, contestations. Oxford University Press, Oxford, UK.

Eisenführ, F., M. Weber, and T. Langer. 2010. Rational decision making. Springer, Berlin, Germany.

Farley, J. 2012. Ecosystem services: the economics debate. Ecosystem Services 1(1):40-49.

File, T. 2014. Young-adult voting: an analysis of presidential elections, 1964-2012. U.S. Census Bureau, Washington, D.C., USA. [online] URL:

Fishburn, P. C. 1967. Methods of estimating additive utilities. Management Science 13(7):435-453.

Garmendia, E., and G. Gamboa. 2012. Weighting social preferences in participatory multi-criteria evaluations: a case study on sustainable natural resource management. Ecological Economics 84:110-120.

Gregory, R., S. Lichtenstein, and P. Slovic. 1993. Valuing environmental resources: a constructive approach. Journal of Risk and Uncertainty 7(2):177-197.

Habermas, J. 1984. The theory of communicative action, volume I. Beacon, Boston, Massachusetts, USA.

Hostmann, M., M. Borsuk, P. Reichert, and B. Truffer. 2005. Stakeholder values in decision support for river rehabilitation. Archiv für Hydrobiologie. Supplementband. Large rivers 15(1-4):491-505.

Howarth, R. B., and M. A. Wilson. 2006. A theoretical approach to deliberative valuation: aggregation by mutual consent. Land Economics 82(1):1-16.

Karjalainen, T. P., M. Marttunen, S. Sarkki, and A.-M. Rytkönen. 2013. Integrating ecosystem services into environmental impact assessment: an analytic-deliberative approach. Environmental Impact Assessment Review 40:54-64.

Keeney, R. L., and H. Raiffa. 1993. Decisions with multiple objectives: preferences and value trade-offs. Cambridge University Press, Cambridge, UK.

Lennox, J., W. Proctor, and S. Russell. 2011. Structuring stakeholder participation in New Zealand’s water resource governance. Ecological Economics 70(7):1381-1394.

Limburg, K. E., R. V. O’Neill, R. Costanza, and S. Farber. 2002. Complex systems and valuation. Ecological Economics 41(3):409-420.

Liu, S., M. Hurley, K. E. Lowell, A.-B. M. Siddique, A. Diggle, and D. C. Cook. 2011. An integrated decision-support approach in prioritizing risks of non-indigenous species in the face of high uncertainty. Ecological Economics 70(11):1924-1930.

Mavrommati, G., K. Bithas, M. E. Borsuk, and R. B. Howarth. 2016. Integration of ecological-biological thresholds in conservation decision making. Conservation Biology 30:1173-1181.

Muradian, R., and L. Rival. 2012. Between markets and hierarchies: the challenge of governing ecosystem services. Ecosystem Services 1(1):93-100.

Oikonomou, V., P. G. Dimitrakopoulos, and A. Y. Troumbis. 2011. Incorporating ecosystem function concept in environmental planning and decision making by means of multi-criteria evaluation: the case-study of Kalloni, Lesbos, Greece. Environmental Management 47(1):77-92.

O’Neill, J. 2001. Representing people, representing nature, representing the world. Environment and Planning C 19(4):483-500.

Proctor, W., and M. Drechsler. 2006. Deliberative multicriteria evaluation. Environment and Planning C 24(2):169.

Rao, V. R. 2014. Applied conjoint analysis. Springer, Berlin, Germany.

Sagoff, M. 1990. The economy of the earth. Cambridge University Press, Cambridge, UK.

Schaefer, M., E. Goldman, A. M. Bartuska, A. Sutton-Grier, and J. Lubchenco. 2015. Nature as capital: advancing and incorporating ecosystem services in United States federal policies and programs. 2015. Proceedings of the National Academy of Sciences 112(24):7383-7389.

Schuwirth, N., P. Reichert, and J. Lienert. 2012. Methodological aspects of multi-criteria decision analysis for policy support: a case study on pharmaceutical removal from hospital wastewater. European Journal of Operational Research 220(2):472-483.

Sheppard, S. R. J., A. Shaw, D. Flanders, S. Burch, A. Wiek, J. Carmichael, J. Robinson, and S. Cohen. 2011. Future visioning of local climate change: a framework for community engagement and planning with scenarios and visualisation. Futures 43(4):400-412.

Skourtos, M., A. Kontogianni, and P. A. Harrison. 2010. Reviewing the dynamics of economic values and preferences for ecosystem goods and services. Biodiversity and Conservation 19(10):2855-2872.

The White House. 2015. Incorporating natural infrastructure and ecosystem services into federal decision-making. M-16-01. The White House, Washington, D.C., USA.

Vatn, A. 2009. An institutional analysis of methods for environmental appraisal. Ecological Economics 68(8):2207-2215.

Von Winterfeldt, D., and W. Edwards. 1986. Decision analysis and behavioral research. Cambridge University Press, Cambridge, UK.

Wegner, G., and U. Pascual. 2011. Cost-benefit analysis in the context of ecosystem services for human well-being: a multidisciplinary critique. Global Environmental Change 21(2):492-504.

Wibeck, V., T.-S. Neset, and B.-O. Linnér. 2013. Communicating climate change through ICT-based visualization: towards an analytical framework. Sustainability 5(11):4760-4777.

Wilson, M. A., and R. B. Howarth. 2002. Discourse-based valuation of ecosystem services: establishing fair outcomes through group deliberation. Ecological Economics 41(3):431-443.

Winkler, R. 2006a. Valuation of ecosystem goods and services: Part 1: an integrated dynamic approach. Ecological Economics 59(1):82-93.

Winkler, R. 2006b. Valuation of ecosystem goods and services: Part 2: implications of unpredictable novel change. Ecological Economics 59(1):94-105.

Zografos, C., and R. B. Howarth. 2010. Deliberative ecological economics for sustainability governance. Sustainability 2(11):3399-3417.

Address of Correspondent:
Georgia Mavrommati
100 Morrissey Blvd
Boston, MA 02125
Jump to top
Table1  | Table2  | Table3  | Table4  | Figure1  | Figure2  | Figure3  | Figure4  | Appendix1  | Appendix2  | Appendix3