Ecology and SocietyEcology and Society
 E&S Home > Vol. 24, No. 2 > Art. 7
The following is the established format for referencing this article:
Larson, K., E. Corley, R. Andrade, S. J. Hall, A. York, S. Meerow, P. Coseo, D. L. Childers, and D. Hondula. 2019. Subjective evaluations of ecosystem services and disservices: an approach to creating and analyzing robust survey scales. Ecology and Society 24(2):7.

Subjective evaluations of ecosystem services and disservices: an approach to creating and analyzing robust survey scales

1School of Geographical Sciences and Urban Planning, Arizona State University, 2School of Sustainability, Arizona State University, 3School of Public Affairs, Arizona State University, 4School of Life Sciences, Arizona State University, 5School of Human Evolution and Social Change, Arizona State University, 6The Design School, Arizona State University


Research on ecosystem services (ES) has largely focused on the ecological functions that produce services or the economic valuation of the benefits provided by ecosystems. Far less research has examined public perceptions of ES, and more so ecosystem disservices (EDS), despite evidence that ecosystem properties and functions can produce beneficial or detrimental outcomes for human well-being. To address this gap, we present a robust approach to measuring beliefs about ecosystem services and disservices. With various means to confirm the validity and reliability of ES and EDS measures, we demonstrate this approach with survey data that captures residents’ perceptions about whether their local neighborhood environment (as the ecosystem of focus) provides certain positive or negative impacts in metropolitan Phoenix, Arizona. The results highlight patterns in people’s views of: desirable and undesirable biota; benefits and risks pertaining to heat and stormwater; recreational and aesthetic values; and societal nuisances and problems. Composite survey scales for overall perceptions of services and disservices are presented, in addition to more distinctive dimensions of ES and EDS. To better understand and manage ecosystems for diverse benefits, the specific survey measures and the general methodological approach can be adapted to various ecosystems and contexts.
Key words: ecosystem services and disservices; environmental attitudes; risk perceptions; survey scales


The ecosystem services (ES) concept is useful for understanding the benefits that people derive from nature. As first conceived by ecologists and economists (Costanza et al. 1997, Daily 1997), and later presented in the Millennium Ecosystem Assessment (MEA), ES research has demonstrated the biophysical and monetary value of ecosystems (Chan et al. 2012, Martín-López et al. 2012, Larson et al. 2016); specifically, biophysical measures have been used to assess how ecosystem structure and function produce benefits (e.g., wildlife habitat, carbon storage) and valuation techniques (e.g., contingent valuation and replacement costs) have estimated the dollar value of services produced by ecosystems (Haase et al. 2014). Despite the abundance of studies that quantify nature’s benefits, the broader public may not perceive or value the services deemed important by researchers or decision makers (Larson et al. 2016). Yet planners and other professionals are charged with designing and managing landscapes and ecological infrastructure that provide multiple services (Lovell and Taylor 2013, Steiner 2014). This requires an understanding of people’s subjective evaluations of ecosystem services, so that decision makers can communicate, maintain, and enhance the benefits of ecosystems for human well-being and ecological conservation (Lovell and Johnston 2009, McPhearson et al. 2016).

Compared to ecological and economic evaluations, relatively little research has examined people’s subjective judgments about a broader array of ecosystem services (Wallace 2007, Kumar and Kumar 2008, Vihervaara et al. 2010, Chan et al. 2012, Gómez-Baggethun and Barton 2013, Larson et al. 2016). Although several survey studies have examined stated values, which are defined as the importance people assign to services (Scholte et al. 2015), we present a different approach to evaluating beliefs about the extent to which ecosystems deliver particular services or disservices. This focus is important because people may not perceive the services rendered by ecosystems and therefore might dismiss or undervalue them. As suggested by attitudinal theory, perceptions of ES affect their socio-cultural value along with people’s knowledge and use of ecosystems (Scholte et al. 2015). In addition, attributes of ecosystems and landscapes, along with personal characteristics and contextual factors, may affect the value people attach to services. In this paper, our primary goal is to demonstrate a robust approach to evaluating perceptions of ecosystem services and disservices (EDS). In doing so, we present a methodology for ensuring the reliability and validity of survey measures while also briefly showing how those measures can be analyzed to understand varied perceptions of ES and EDS.

Ecosystem disservices are often ignored in ES research (Lyytimäki and Sipilä 2009, Gómez-Baggethun and Barton 2013, Belaire et al. 2015, Shackleton et al. 2016). Though definitions vary, EDS are essentially the “perceived or actual negative impacts on human wellbeing” that result from ecosystem properties and functions (Shackleton et al. 2016:590). Disservices can include biological or geophysical hazards such as animal invasions or flooding, in addition to nuisances such as pests (Lyytimäki et al. 2008, von Döhren and Haase 2015). In some cases, disservices and services can be the “opposite sides of the same coin” (Shackleton et al. 2016:593). Regarding the regulation of microclimates, for example, trees and other vegetation can mitigate heat while the lack thereof can exacerbate it. Similarly, vegetative cover and other factors can mitigate water drainage or flooding. With regard to services such as biodiversity or aesthetics, ecological communities and landscape characteristics can be perceived as desirable or not. While we examined residents’ beliefs about these types of ES and EDS (Table 1), we also focused several survey questions on socio-cultural services. We intentionally use the term socio-cultural, rather than simply cultural, because some of the positive and negative impacts we evaluated go beyond the immaterial services commonly discussed in the ES literature (Milcu et al. 2013). For instance, we consider health problems such as allergies because vegetative communities can contribute to such ailments.

Although not exhaustive, our approach considers people’s beliefs about a variety of ES and EDS. The particular services and disservices we evaluated were chosen based on the interests of our collaborative research team. They also reflect salient issues in our study area, which encompasses the urban ecosystem of metropolitan Phoenix, Arizona. Although the data presented were collected in an arid desert city, the ES and EDS are relevant for diverse cities and beyond. The methodology could also be applied to agricultural and other contexts, though care should be taken to tailor survey questions to salient issues and stakeholders in specific study sites.


The Phoenix Area Social Survey (PASS) is conducted as a part of the Central Arizona-Phoenix Long-Term Ecological Research (CAP LTER) program (CAP LTER 2018). This household survey was implemented in 2016-2017 to better understand the causes, consequences, and feedbacks between people and a range of ecological patterns and processes. In this paper, we analyze a multi-item question that gauged how residents perceive the ES and EDS in their local environments, i.e., neighborhood. Specifically, we asked survey participants “to what extent do you agree or disagree that each statement describes the environment of your neighborhood?” Because we were interested in cognitive beliefs, i.e., perceptions, survey participants responded on a standard five-point Likert scale ranging from strongly disagree to strongly agree (with a neutral option in the middle). With reference to the “environment in your neighborhood,” we articulated consideration of “the grass, plants, and/or trees in the area, along with the streets, sidewalks, patios, porches, and built structures as well as parks and open spaces.” This description intentionally included both ecological and built infrastructure that affect the services and disservices in urban ecosystems (Shackleton et al. 2016).

Our collaborative team was primarily interested in examining patterns and processes linked to wildlife habitat and biodiversity, land and stormwater management, as well as to heat stress and mitigation. As a result, these topics were embodied in the descriptions for the 17 variables rated by respondents (see Table 1 for the verbatim wording of each survey item and Appendix 1 for Spanish wording). The variables captured an array of services (n = 9) and disservices (n = 8), with two or more survey statements/variables to capture primary services of interest: supporting services for wildlife and biodiversity; regulation of stormwater and microclimates; and socio-cultural benefits including aesthetics and recreation. Using multiple variables to capture distinct ES and EDS enhances the reliability of survey measures (Carmines and Zeller 1979, Spector 1992). For several items (see Table 1), we adopted or slightly modified survey statements published in past research (Brown et al. 2012, Larson et al. 2016). Most of the statements about disservices were newly developed because past survey work has focused on services.

The University of Wisconsin Survey Center administered the mail-only questionnaire between May and September of 2017, with four separate mailings. Incentives were offered, and the survey was available in English and Spanish (although no participants requested or completed the Spanish version). From demographically and geographically diverse neighborhoods, 1400 households were asked to participate. The sampling design was a random stratified sample of 12 targeted neighborhoods defined by census block groups. The neighborhoods varied in their location around the region, i.e., urban, suburban, fringe, and represent a range of low to high-income areas (Fig. 1). The final response rate was 39.4% (n = 496). For more details on the survey methods, please refer to (Larson et al. 2017).

Among the survey respondents, the average age (51 years) and household income ($80–10,000) were similar to the population of the study neighborhoods. However, the sample was slightly more educated, with 29% holding a Bachelor’s degree compared to 23% of the population. Although 37% of the study population was Hispanic or Latino, 20% of the survey sample identified as such. Given these differences as well as the neighborhood-based sample, caution must be used in generalizing the results.

We used three techniques to address the validity and reliability of the ES and EDS survey measures. First, to determine whether the wording of survey items logically reflected the intended constructs, we used expert judgments to confirm face validity (Bornstein 2004). For example, “offers a variety of plants” was designed to measure perceptions of plant diversity, whereas “it looks beautiful” was used to gauge aesthetic value (Table 1).

Second, to identify distinctive dimensions of perceived ES and EDS, based on the intercorrelations among variables, we used factor analysis with principal components extraction. This technique is commonly used with survey data to assess the validity of combining variables into composite scales. An oblimin rotation was used because dimensions of ecosystem services and disservices may be correlated to each other (Carman 1998, Larson 2009). Regarding the interpretation of results, Eigenvalues of 1 or greater (Tables 2 and 3) reflect statistically significant dimensions in factor analysis (Kim and Mueller 1978). Moreover, loadings of 0.5 and higher indicate variables that strongly load onto particular factors (Spector 1992). We focus our presentation of results on the rotated pattern matrices outputted from the factor analysis because these are the most clear and informative for highlighting the dimensions along which perceptions of ES and EDS varied among our sampled residents.

Third, to create reliable survey scales that reflect distinct dimensions of perceptions, we averaged responses to individual variables into composite measures for each survey participant (Table 4). The factor analysis principally informed this process. These commonly used techniques were coupled with additional tests for internal consistency (one form of reliability) to verify that the variables were highly correlated enough to reflect a particular construct (Spector 1992). Specifically, to ensure it was statistically appropriate to combine individual variables, Spearman correlations were calculated for bivariate pairs of variables, wherein a minimum correlation (rho) of 0.3 indicates a high enough correlation to justify the creation of a composite scale. For scales with three or more variables, the standard statistical test used was Cronbach’s alpha (Carmines and Zeller 1979), wherein values equal to or greater than 0.7 indicate internal consistency and, therefore, can be deemed reliable as composite survey scales (Nunnally 1967).

Last, we demonstrate how the ES/EDS scales can be analyzed to understand how perceptions vary across diverse people and places by (1) comparing the overall ES and EDS measures across the 12 study neighborhoods, and (2) presenting how specific ES and EDS scales vary in relation to ethnicity. First, we graphed the omnibus ES and EDS scales by the 12 study neighborhoods (Fig. 1). We also ran Analysis of Variance (ANOVA) tests to determine if perceptions varied among residents based on their location in urban, suburban, or fringe neighborhoods (see the notes in Fig. 1 for details). Second, we compared whether residents of Hispanic/Latino decent (as the dominant minority group in the study region) have different perceptions than other residents. Additionally, we ran Analysis of Covariance (ANCOVA) tests (Table 5) to assess the effect of Hispanic/Latino ethnicity on several ES/EDS scales (Fig. 2) controlling for income. The income variable measured household income on an 11-point scale ranging from “$20,000 or under” (1) to “$200,000 or more” (11) with $20,000 increments in-between.

All statistical analyses were run using the Statistical Package for Social Sciences (SPSS), version 24. The composite scales were also computed in SPSS.


We present (1) the patterns (based on descriptive statistics) in residents’ perceptions of various services and disservices; (2) the dimensions (from the factor analysis) along which perceptions vary to reflect distinctive dis/services; (3) the reliability (based on correlations and Cronbach’s alpha tests) of the composite scales established for particular ES and EDS; and last, (4) the trends and differences (from ANOVA and ANCOVA tests) in the ES and EDS scales across people who live in varied neighborhoods and are from different ethnic backgrounds and income levels.

(1) How do residents perceive local ecosystem services and disservices in an urban environment?

Overall, survey respondents evaluated their neighborhoods positively by largely agreeing that their local environments provide an array of services but not disservices (Table 1). Though generally positive, perceptions ranged widely among residents, with standard deviations ranging from 1.1–1.3 across the individual ES and EDS variables measured in our survey.

For each of the nine services evaluated, the median response was agree (4). On average (Table 1), respondents most highly rated aesthetics, i.e., looks beautiful, and biodiversity, i.e., provides habitat for birds. Rain-water drainage, natural-looking appearances, and the variety of plants were also strongly rated overall. The ES that residents perceived as least provided by their local environment was shade. Among the recreational services, opportunities for physical activity were most highly rated, followed by social activities and nature exploration. Residents varied most in their evaluation of the recreational services provided by their local environments.

Among the eight disservices evaluated, the median responses ranged from disagree (2) to agree (3). People were most negative about their local environment contributing to health problems (such as allergies), as well as attracting unwanted pests and making the summer heat worse (Table 1). In contrast, they tended to disagree that their neighborhood environment contributes to flooding, crime, and messy appearances. Weediness and environmental pollution were, by comparison, moderately perceived. Residents varied mostly widely in their perceptions of unwanted pests including animals and weeds.

(2) How do people’s perceptions reflect distinctive dimensions of ecosystem services and disservices?

We first ran the factor analysis on all 17 ES and EDS variables, which produced four factors (Appendix 2). The first factor dominated the others in explaining 39% of the variance in the data, compared to 6–9.5% for the subsequent factors. All but two variables loaded significantly on the primary factor; the exceptions were health problems and rainwater drainage. The second, third, and fourth factors (in the component matrix) were weak, with the majority of loadings for variables below 0.4. Moreover, this factor analysis did not produce a rotated pattern matrix. These results indicate that an omnibus scale could be created with all ES and EDS variables. In contrast, the factor analyses for the ES versus EDS variables suggest they could be treated as omnibus scales, or as distinctive perceptions about particular ES and EDS (as shown in the rotated matrix Tables 2 and 3).

The factor analysis of ES variables produced two distinctive factors (Table 2) that were identified as significant subdimensions of perceptions. The first, which accounted for 28% of the variance in the data, clearly underscored the benefits of a natural aesthetic that looks natural and beautiful while also offering a variety of plants. The second factor (11.5% of the variance) uniquely embodied the recreational value of local neighborhood environments, inclusive of opportunities for physical activity, nature exploration, and social activity.

For the perceived EDS, the factor analysis results were similar to those for services in producing an overall factor (in the component matrix) and two distinct factors (in the rotated, pattern matrix). The significant factor loadings in the latter (Table 3) highlighted two distinct subdimensions of perceived disservices. The first factor (45.1% of the variance explained) captured environmental risks including pollution, heat, and flooding. The second factor (12.8% of the variance in the data), labeled biotic nuisances, underscored weeds, messiness, and unwanted pests.

(3) How can the ES and EDS variables be combined to create reliable survey scales?

Based on the original intent and face validity of the individual variables, along with the findings from the factor analyses, we created a number of reliable scales to reflect distinct perceptions regarding particular types of ES and EDS (Table 4). First, two broad survey scales were created to reflect overall perceptions of ecosystem services and disservices. Not surprisingly, the ES and EDS scales were highly correlated (rho = -0.62, p < 0.01).

Regarding the socio-cultural ES and EDS, we created four scales to capture residents’ perceptions (Table 4). Based on the factor and reliability analyses, we created two scales for perceived services: recreational benefits and natural beauty (see Table 2 for the variables included in each). The third scale partly reflected aesthetic disservices and was labeled environmental nuisances (see Table 3 for the variables included in this scale). The last scale captured perceptions of societal problems including crime and health problems.

Regarding dis/services for supporting biodiversity, we created two scales. Labeled desirable biota (Table 4), the first one included the variables, provides habitat for birds and offers a variety of plants. The second one, labeled undesirable biota, reflected perceptions of unwanted pests/animals and weeds. Though some of the variables in these scales were included in other scales, i.e., natural beauty and environmental nuisances (see Table 4), we established these more focused scales because of our special interest in understanding residents’ views of flora and fauna in their local environments.

Finally, three scales were created for regulating disservices (Table 4). The first scale represented the environmental risks that emerged from the factor analysis (see Table 3 for the variables included). Given our research team’s special interests in heat mitigation and stormwater management, we created two more narrowly focused scales. Framed as perceived risks, these scales captured heat stress and stormwater problems. Heat stress included the variables, worsens summer heat and provides shade, wherein the shade variable was reverse coded such that high values amounted to perceived heat stress. Likewise, stormwater problems included the variables, causes flooding and the (reverse coded) drains rain water.

(4) How do perceived ecosystem services and services vary across different people and places?

We now demonstrate how the reliable ES/EDS scales might be analyzed in relation to diverse people and places. First, the neighborhood comparison (Fig. 1) clearly shows a trend wherein residents of relatively low-income areas perceived decreased services and increased disservices compared to residents in high-income areas. When neighborhoods were grouped by their location in the urban core versus suburban and fringe areas (see the notes in Fig. 1 for details), the ANOVA test indicated that residents of urban and suburban areas similarly report lower services (F = 18.2, p < 0.01) and higher disservices (F = 21.6, p < 0.01) compared to residents in fringe communities (p < 0.01 for pair-wise comparisons between fringe areas versus urban and suburban locations).

Second, the disparities across ethnicity were also clear. Specifically, Hispanic/Latino residents (Fig. 2) expressed neutral views of services and disservices in their local environments, while other, mostly White/Anglo residents perceived substantial services and few disservices in their neighborhoods. Even when controlling for income, Hispanic/Latino ethnicity significantly explained the disparities in perceived ES/EDS (Table 5).


We establish a multifaceted approach to measuring perceptions about the extent to which local urban environments contribute to ecosystem services and disservices. The methodology, including expert input on face validity and statistical tests of reliability, can be used broadly to establish robust survey scales that capture assorted beliefs as well as values and attitudes. This approach proved useful for understanding peoples’ beliefs about ES and EDS in their local communities, in addition to developing reliable composite scales that reflect distinctive views about particular dis/services. Moreover, the analyses of composite scales clearly revealed trends in perceived ES and EDS and how they vary in significant ways across diverse people, i.e., based on income and ethnicity, and places, i.e., neighborhoods in various locations throughout a large metropolitan area. Additional research is needed, however, to more fully understand subjective views about ecosystem services and disservices, both in general and in relation to personal characteristics, landscape features, and other contextual factors.

To apply our approach in other contexts, researchers should tailor the ES and EDS assessed to the biophysical and socio-cultural contexts of their study areas. Considering our data were collected in a hot desert urban ecosystem in the southwestern U.S., we focused significantly on aesthetic and recreation values as well as environmental risks such as heat stress. Yet in future assessments, modifications to the ES and EDS evaluated, along with their wording, will be necessary for capturing critical issues in other ecosystems and contexts. In agricultural or rural settings, for example, the provisioning of food, timber, and other resources that strongly impact local livelihoods should be included as essential services. Indeed, previous research reports that these provisioning services tend to be more important for traditional communities in less developed contexts compared to industrialized societies, where socio-cultural values such as aesthetics and recreation are relatively important (Milcu et al. 2013). Moreover, in less industrialized contexts, provisioning and other services are often central to survival as well as cultural identity. Thus, our survey approach should be modified to the local context of particular places.

In our study, we defined ES and EDS as respectively the positive and negative outcomes derived from ecosystems. In some cases, i.e., for heat stress/mitigation and flooding/stormwater drainage, the composite survey scales reflect the opposite sides of outcomes (Shackleton et al. 2016). These measures are akin to perceptions about exposure to environmental risks. Yet beliefs about other ES and EDS reflect attitudinal evaluations because some survey statements embodied inherently positive or negative adjectives, such as “beautiful” versus “messy” in evaluating aesthetics. For the biodiversity-related scales, one measure reflects biota that support wildlife habitat, which may be perceived as a service provided by the local environment even if an individual does not value or appreciate it. In other words, although we labeled this scale “desirable biota” from our vantage point as environmental researchers, people may not necessarily view wildlife habitat or plant diversity as beneficial to them or the well-being of their local community. By comparison, the “undesirable biota” scale represents innately more negative judgements given the terminology employed, e.g., “unwanted pests” and “too many weeds,” to frame these variables as disservices. These distinctions highlight different types of subjective evaluations based on the precise wording of survey questions, thereby demonstrating the need to carefully consider the face validity of survey measures in interpreting their meaning.

An important distinction should also be made across the socio-cultural ES/EDS compared to the other types of ES/EDS. That is, subjective judgments about socio-cultural services, especially immaterial ones, are essential to evaluating whether those services are provided by ecosystems because their value and meaning is fundamentally based on what people think and feel about them. In contrast, the more material benefits (or risks) embodied in ecosystem services (or disservices) may exist regardless of whether or not people care, know, and/or are directly impacted by them. Thus, we argue that the ultimate valuation (or devaluation) of any ecosystem service (or disservice) should consider resident or stakeholder views of them, in addition to how they might use or benefit from them directly (Milcu et al. 2013). This perspective is supported by Scholte et al. (2015), whose framework for determining the socio-cultural values of ES underscores the importance of interactions between services and their beneficiaries, specifically encompassing the information they have about ES as well as their perceptions and use of them.

Because perceptions of ES and EDS may not reflect the actual biophysical delivery of services or disservices (Sholte et al. 2015, Larson et al. 2016), assessments of perceptions and other subjective judgments, e.g., values, in relation to objective measures of ES and EDS are a critical direction for future research. Such research could answer the following questions: How accurate are people’s perceptions of the benefits (or risks) derived from ecosystems? To what extent do people value those ecosystem services, i.e., in assigning importance to them or viewing them as beneficial? How does human behavior, e.g., recreational activities, reflect the use-derived benefits from ecosystems, and how do those directly and indirectly affect other perceived ED/EDS? These questions would be most effectively explored in relation to landscape configurations, vegetation composition, and other ecosystem services, which will affect and be affected by the people who interact with them (Sholte et al. 2015). In addition to biophysical measurements and surveys of ES and EDS, other valuable techniques to evaluating them include field observations, document research, and qualitative interviews or focus groups.

Overall, interdisciplinary research with mixed methods will be central to advancing knowledge about how the structure and function of ecosystems affect ecosystem services and disservices in particular places or contexts (Raudsepp-Hearne et al. 2010, Martín-López et al. 2012, Mouchet et al. 2014). Here, special consideration should be given to how human activities, including management regimes, have impacted ecosystems, and by extension, the services and disservices derived from them. This information can help planners, designers, and other decision makers meet multiple environmental and social objectives while minimizing negative outcomes or trade-offs among them.


We present an approach to assessing ecosystem services and disservices that emphasizes public perceptions, or in other words, beliefs about the extent to which ES and EDS are provided by local environments. The conceptual and methodological approach complements existing ES research by examining beliefs about ecosystem services and disservices. Although we evaluated perceptions in the urban ecosystem of Phoenix, Arizona, the approach can be modified for other ecosystems and places. Although the inclusion and wording of specific survey items should be adapted to fit other study contexts, the methods used to ensure the validity and reliability of the survey measures can be applied broadly to other research. Integrating social survey methods into interdisciplinary research will improve understanding of what people think, feel, and do in relation to ecosystems and their outcomes. This knowledge will, in turn, advance knowledge of social-ecological dynamics while also informing decision making about how to enhance ecosystem services while diminishing the disservices among them.


Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link.


Thanks to Barbara Trapido-Lurie for assistance with graphics. Also, this material is based upon work supported by the National Science Foundation (NSF) under the grants BCS-1026865, DEB-1637590, EF-1638725, and, SES-1462086. Any opinions, findings, conclusions, or recommendation expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.


Belaire, J. A., L. M. Westphal, C. J. Whelan, and E. S. Minor. 2015. Urban residents’ perceptions of birds in the neighborhood: biodiversity, cultural ecosystem services, and disservices. Condor 117(2):192-202.

Bornstein, R. F. 2004. Face validity. Page 368 in M. S. Lewis-Beck, A. Bryman, and T. F. Liao, editors. The SAGE Encyclopedia of Social Science Research Methods. Sage, Newbury Park, California, USA.

Brown, G., J. M. Montag, and K. Lyon. 2012. Public participation GIS: a method for identifying ecosystem services. Society and Natural Resources 25(7):633-651.

Carman, C. J., 1998. Dimensions of environmental policy support in the United States. Social Science Quarterly 79(4):717-733.

Carmines, E. G., and R. A. Zeller. 1979. Reliability and validity assessment. Sage University Paper Series on Quantitative Applications in the Social Sciences, 07-017. Sage, Newbury Park, California, USA.

Central Arizona-Phoenix Long-Term Ecological Research (CAP LTER). 2018. Phoenix area social survey. Global Institute of Sustainability, Arizona State University, Tempe, Arizona, USA. [online] URL:

Chan, K. M. A., T. Satterfield, and J. Goldstein. 2012. Rethinking ecosystem services to better address and navigate cultural values. Ecological Economics 74:8-18.

Costanza, R., R. d'Arge, R. DeGroot, S. Farber, M. Grasso, B. Hannon, K. Limburg, S. Naeem, R. V. O'Neill, J. Paruelo, R. G. Raskin, P. Sutton, and M. van den Belt. 1997. The value of the world’s ecosystem services and natural capital. Nature 387:253-260.

Daily, G. C. 1997. Nature’s services: societal dependence of natural ecosystems. Island Press, Washington, D.C., USA.

Gómez-Baggethun, E., and D. N. Barton. 2013. Classifying and valuing ecosystem services for urban planning. Ecological Economics 86:235-245.

Haase, D., N. Larondelle, E. Andersson, M. Artmann, S. Borgström, J. Breuste, E. Gomez-Baggethun, Å Gren, Z. Hamstead, R. Hansen, N. Kabisch, P. Kremer, J. Langemeyer, E. Lorance Rall, T. McPhearson, S. Pauleit, S. Qureshi, N. Schwarz, A. Voigt, D. Wurster, and T. Elmqvist. 2014. A quantitative review of urban ecosystem service assessments: concepts, models, and implementation. AMBIO 43(4):413-433.

Kim, J.-O., and C. W. Mueller. 1978. Introduction to factor analysis: What it is and how to do it. Sage University Paper Series on Quantitative Applications in the Social Sciences, 07-013. Sage, Newbury Park, California, USA.

Kumar, M., and P. Kumar. 2008. Valuation of the ecosystem services: a psycho-cultural perspective. Ecological Economics 64(4):808-819.

Larson, K. L. 2009. Social acceptability of water resource management: a conceptual approach and empirical findings. Journal of the American Water Resources Association 45(4):879-893.

Larson, K. L., R. Andrade, A. York, D. Childers, P. Coseo, S. Earl, M. Ehlenz, N. Grimm, S. Hall, S. Harlan, D. Hondula, S. Lerman, D. Pfeiffer, J. Ripplinger, K. Smith, P. Warren, M. Watkins, D. White, S. Wittlinger, M. Wright, and A. Wutich. 2017. The Phoenix area social survey IV: linking social and biophysical dynamics in urban neighborhoods. A report from the Central Arizona-Phoenix Long-Term Ecological Research (CAP LTER) project. The Global Institute of Sustainability, Tempe, Arizona, USA. [online] URL:

Larson, K. L., K. C. Nelson, S. R. Samples, S. J. Hall, N. Bettez, J. Cavender-Bares, P. M. Groffman, M. Grove, J. B. Heffernan, S. E. Hobbie, J. Learned, J. L. Morse, C. Neill, L. A. Ogden, J. O'Neil-Dunne, D. E. Pataki, C. Polsky, R. R. Chowdhury, M. Steele, and T. L. E. Trammell. 2016. Ecosystem services in managing residential landscapes: value priorities, dimensions, and cross-regional patterns. Urban Ecosystems 19(1):95-113.

Lovell, S. T., and D. M. Johnston. 2009. Designing landscapes for performance based on emerging principles in landscape ecology. Ecology and Society 14(1):44.

Lovell, S. T., and J. R. Taylor. 2013. Supplying urban ecosystem services through multifunctional green infrastructure in the United States. Landscape Ecology 28(8):1447-1463.

Lyytimäki, J., L. K. Petersen, B. Normander, and P. Bezák. 2008. Nature as a nuisance? Ecosystem services and disservices to urban lifestyle. Environmental Sciences 5(3):161-172.

Lyytimäki, J., and M. Sipilä. 2009. Hopping on one leg - the challenge of ecosystem disservices for urban green management. Urban Forestry and Urban Greening 8(4):309-315.

Martín-López, B., I. Iniesta-Arandia, M. García-Llorente, I. Palomo, I. Casado-Arzuaga, D. G. Del Amo, E. Gómez-Baggethun, E. Oteros-Rozas, I. Palacios-Agundez, B. Willaarts, J. A. González, F. Santos-Martín, M. Onaindia, C. López-Santiago, and C. Montes. 2012. Uncovering ecosystem service bundles through social preferences. PLoS ONE 7(6):e38970.

McPhearson, T., S. T. Pickett, N. B. Grimm, J. Niemelä, M. Alberti, T. Elmqvist, C. Weber, D. Haase, J. Breuste, and S. Qureshi. 2016. Advancing urban ecology toward a science of cities. BioScience 66(3):198-212.

Milcu, A. I., J. Hanspach, D. Abson, and J. Fischer. 2013. Cultural ecosystem services: a literature review and prospects for future research. Ecology and Society 18(3):44.

Mouchet, M. A., P. Lamarque, B. Martín-López, E. Crouzat, P. Gos, C. Byczek, and S. Lavorel. 2014. An interdisciplinary methodological guide for quantifying associations between ecosystem services. Global Environmental Change 28(1):298-308.

Nunnally, J. 1967. Psychometric theory. McGraw Hill, New York, New York, USA.

Pataki, D. E., M. M. Carreiro, J. Cherrier, N. E. Grulke, V. Jennings, S. Pincetl, R. V. Pouyat, T. H. Whitlow, and W. C. Zipperer 2011. Coupling biogeochemical cycles in urban environments: ecosystem services, green solutions, and misconceptions. Frontiers in Ecology and the Environment 9:27-36.

Raudsepp-Hearne, C., G. D. Peterson, and E. M. Bennett. 2010. Ecosystem service bundles for analyzing tradeoffs in diverse landscapes. Proceedings of the National Academy of Sciences 107(11):5242-5247.

Scholte, S. S. K., A. J. A. van Teeffelen, and P. H. Verburg. 2015. Integrating socio-cultural perspectives into ecosystem service valuation: a review of concepts and methods. Ecological Economics 114:67-78.

Shackleton, C. M., S. Ruwanza, G. K. Sinasson Sanni, S. Bennett, P. De Lacy, R. Modipa, N. Mtati, M. Sachikonye, and G. Thondhlana. 2016. Unpacking Pandora’s Box: understanding and categorising ecosystem disservices for environmental management and human wellbeing. Ecosystems 19(4):587-600.

Spector, P. E. 1992. Summated rating scale construction: an introduction. Quantitative Applications in the Social Sciences, 07-082. Sage, Newbury Park, California, USA.

Steiner, F. 2014. Frontiers in urban ecological design and planning research. Landscape and Urban Planning 125:304-311.

Vihervaara, P., M. Rönkä, and M. Walls. 2010. Trends in ecosystem service research: early steps and current drivers. AMBIO: A Journal of the Human Environment 39(4):314-324.

von Döhren, P., and D. Haase. 2015. Ecosystem disservices research: a review of the state of the art with a focus on cities. Ecological Indicators 52:490-497.

Wallace, K. J. 2007. Classification of ecosystem services: problems and solutions. Biological Conservation 139:235-246.

Address of Correspondent:
Kelli L. Larson
Arizona State University
School of Geographical Sciences and Urban Planning
Mail Code 5302
Tempe, AZ 85287-5302
Jump to top
Table1  | Table2  | Table3  | Table4  | Table5  | Figure1  | Figure2  | Appendix1  | Appendix2