Human populations will reach 9 billion and require 70% more food by 2050 (FAO 2009). Increased agricultural production is already threatened by shortage of water, severe land degradation in existing agricultural landscapes, land-use intensification, invasive species, prospects of energy price hikes, loss of biodiversity and ecosystem services, and climate change (Millennium Ecosystem Assessment 2005, Pretty et al. 2010). This alignment of current and rapidly emerging problems represents a truly major set of coupled environmental, economic, and social challenges for humanity. One answer to these challenges is the development of more sustainable food systems. Learning and transformation requires a broadly accepted, trusted, and accountable method of assessing the food system’s sustainability and resilience.
There is growing emphasis on the need to make assessments based on multiple and more varied criteria and underlying values. The term “multicriteria assessment” originates from the use of multicriteria analysis as an alternative to cost-benefit analysis in economics for decision problems characterized by multiple, noncommensurate, and possibly conflicting criteria (Bogetoft and Pruzan 1991). Prime examples of multicriteria assessment are the various forms of “sustainability assessment” made with reference to multiple criteria of sustainability, including environmental, social, governance, and economic aspects (see, e.g., Pope et al. 2004 for a discussion of the concept and the overview of tools by Ness et al. 2007). However, multicriteria assessment is a broader term that also applies, for instance, to overall assessments of organic food systems made with reference to the ethical principles of organic agriculture (Alrøe and Noe 2011). There are many challenges, as well as opportunities, in making such overall multicriteria assessments that are explored in this Special Feature of Ecology and Society.
Sustainability assessment of food systems is challenging because food systems are complex and comprise many different farming systems, chains, networks, and actors with conflicting goals (Binder et al. 2012), which work across very different spatial and temporal scales (Darnhofer et al. 2010); stakeholders have different values and ideas about what constitutes better food systems (Bond et al. 2011); stakeholders have different vested interests (some are actively involved in producing food or shaping food systems while others are influenced by their various effects; Bond and Morrison-Saunders 2011, Herrmann et al. 2011); and sustainability itself is a multifaceted concept that relies on many different areas of scientific and practitioners’ expertise (Noe and Alrøe 2015).
In the last decades, many methods to do overall sustainability assessment have been developed, including integrated, holistic, and multicriteria tools. However, the methods involve different logics and value institutions (cf. Vatn 2009), they produce different assessments, and none of them can claim to have the right answer seen from other perspectives. Furthermore, for such overall multicriteria assessments to help generate sustainable solutions, there is also a need to be able to effectively communicate the multidimensional assessments and how they are founded in sets of knowledge and values to engender collaboration between stakeholders and actors at local, regional, national, and global scales.
Together, these challenges pose a range of problematic issues for how to make and use multicriteria assessments of food systems to improve food system sustainability. Accordingly, the 10 papers in this Special Feature show different perspectives on the assessment and communication of food system sustainability, both across the articles and within articles that compare different assessment tools or scientific perspectives. The papers contribute insights of two main types: (1) the choice and development of appropriate tools for making sustainability assessments, and (2) reflections on the role of multicriteria assessments in the transformation to more sustainable food systems. First we summarize findings within these two main themes, before turning to some overarching opportunities and challenges for multicriteria assessments of food system sustainability in the future.
The first theme in this Special feature focuses on appropriate assessment tools. Choice of the right assessment tool for the job is contested and if done uncritically, could have unintended consequences (cf. Gasparatos and Scolobig 2012). Peano et al. (2014) provide a case study of how to assess sustainability of an alternative food system, and three other papers discuss methodological conflicts in choosing and developing assessment tools. The choice of method is based on built-in methodological differences and trade-offs between different objectives. For instance there may be a trade-off between the scope of the assessment and the precision and validity of the results (Schader et al. 2014); between complex, expert-based full assessments and participatory rapid assessments (Marchand et al. 2014); and between measures of sustainability performance and management and development of agricultural enterprises (Triste et al. 2014).
Peano et al. (2014)’s test of sustainability within the Slow Food Presidia project in Italy evaluated a farm-to-market system for local, high-quality, sustainable products. They developed an indicator-based tool to monitor the sustainability of the agri-food systems considering quality as well as economic, ecological, social, and cultural aspects. Indicators and criteria were weighted either equally or based on their importance to surveyed stakeholders, i.e., consumers, producers, and scientists/experts. Formal weighting of the relevance of different aspects of sustainability offers a method for tuning the assessment process to local context and thereby building customization and ownership of the assessment process for key stakeholders. Plurality is preserved by including a wide range of sustainability criteria, while weighting provides a way of locally grounding the process. The very act of measuring weights also starts to explicitly reveal the values deployed in a given assessment. Peano et al. (2014)’s approach included an interactive process with the stakeholders helping them to improve all dimensions of sustainability. In particular the socioeconomic and cultural capital increased by preserving the environmental and quality aspects of the food products.
The paper by Schader et al. (2014) analyses 35 sustainability assessment approaches and makes detailed comparisons of 6 of them. They find considerable differences between them in terms of scope, the level of assessment, and the precision of indicators used for impact assessment. In particular, three different types of methodological trade-offs were recognized: between different kinds of scope in terms of area, level and comprehensiveness; between the different criteria for precision; and between scope and precision. Thus one-size-fits-all solutions, with respect to tool selection, are rarely feasible. Furthermore, these trade-offs can lead to contradictory assessment results that may not be comparable. The authors recommend that assessment tools should include a precise definition of “sustainability,” e.g., whether from a farm or society perspective, along with a description of the methodological approach and the indicator sets; and that they should aim for harmonization of indicators and assumptions. Schader et al. (2014)’s formal survey of the variation between assessments underscores the challenge and need for more research of how the balances and scope of the entire set of indicators, rather than the nature of individual measures, can effect learning and trust in the use of an overall assessment framework.
Some tools are designed for practical decision making and learning on individual farms and others for more aggregated and performance-focused rather than learning-focused agendas. Marchand et al. (2014) discuss the appropriate choice of assessment tools to support decision making on farms. They analyze characteristics of different tools, with a detailed comparison of MOTIFS (Monitoring Tool for Integrated Farm Sustainability) and PGT (Public Goods Tool), to determine criteria for tool choice. Based on the key characteristics, they define two types of indicator-based tools: full sustainability assessment and rapid sustainability assessment. Rapid sustainability assessment tools are oriented toward communicating and learning, and can help to raise awareness and highlight areas of good or bad performance for later more intensive investment to find solutions for problems or capture opportunities for transformation. They are suitable for most farmers. Full sustainability assessment tools can provide additional insight and help monitor particular aspects of sustainability. But they are expert-based, complex tools that require more time and money, and are best suited for small groups of motivated farmers. The authors suggest modular construction of tools and complementary use of full and rapid assessment tools as topics for future research.
Triste et al. (2014) note that the adoption of sustainability assessment tools in agricultural practice is often disappointing. The paper therefore aims to foster scientific debate on how these tools are developed rather than the end product itself, with particular emphasis on how stakeholders are involved. This is done through rigorous self-reflexive research on the development of the sustainability assessment tool MOTIFS. The tool failed to become widely adopted because (i) there was not sufficient time for reflection and decision making to ensure shared process visions and objectives in the research team during the development process, (ii) there was insufficient active involvement of stakeholders and end users in development, and (iii) the tool had multiple functions, such as monitoring, communication, and decision support, that required different specifications depending on end users and their needs. For future tool development the authors recommend learning from stakeholders and end users, coaching for appropriate tool use, and structuring the development of different tool types and exploring spin-offs from existing tools.
The papers in this theme underscore the value of linking diverse methods and tools, or nesting and stepping their deployment, to help build resilience and sustainability. The built-in methodological trade-offs mean that there is no one tool, one framework, or one indicator set that is appropriate for the different purposes and contexts of sustainability assessment. A diversity of linked tools can capture the best features of each and assist the different “layers and players” to combine efforts. The papers emphasize an overall conclusion and expectation: if the key stakeholders are not given a responsible and full role in the development of any assessment tool, it is less likely to be fit for their purpose and they are unlikely to take ownership or have confidence in it. Ultimately they are less likely to use the tool or heed its signals. This is not to say that sustainability scientists and other distant decision makers are not important—they have potentially vital roles of detecting, interpreting, and underscoring impending opportunities and threats for the stakeholders involved in the practice of producing food, fiber, or biofuels. A combination of both emic (insider) and etic (outsider) perspectives can build resilience by proffering early warning of distant drivers of change and opportunity, identifying a wider set of choices for local decision makers, and sharing potential tools for picking the best choice for local actors.
The second theme in this Special Feature includes deeper reflections on how multicriteria assessments can contribute to transformations toward more sustainable food systems, mostly inspired by organic agriculture as an alternative food system. This theme involves fundamental considerations on the conception of values in different assessment perspectives (Thorsøe et al. 2014), the role of complementarity in sustainability assessment (Alrøe and Noe 2016), and the role of motivation (Læssøe et al. 2014), communication (Kastberg 2015), and trust (Freyer et al. 2014, Rittenhofer and Povlsen 2015) in organic food systems. The benefits of bringing in different research perspectives, and the problems and methodological issues in doing so, characterize most of the papers in this theme.
Thorsøe et al. (2014) highlight that assessment of the overall effects of organic food systems cannot be done from any single research perspective. Different research perspectives (here: food science, discourse analysis, phenomenology, neoclassical welfare economics, and actor-network theory) ascribe very different meanings to values and their role in assessments. Therefore, a strategy is needed to ensure that such differences are included and balanced in overall assessments. Including a combination of multiple perspectives is a radical break with the tradition of conducting multicriteria assessment from one hegemonic perspective. To function as tools for the continual development of food system sustainability, multicriteria assessments must be able to combine and balance knowledge from different perspectives and deal with multiple understandings of values. Four courses of action will help form such a strategy: (i) elucidate and make explicit the values of each research assessment; (ii) openly discuss the decisive choice of perspective; (iii) formulate common goals that can be translated into the different perspectives; and (iv) consider assessment of food system sustainability as a learning process and design it as such.
Alrøe and Noe (2016) further analyze the problems of integrating multiple incompatible perspectives into overall assessments. Based on a generalization of Niels Bohr’s complementarity from quantum mechanics, they identify two forms of complementarity in sustainability assessment, observer stance complementarity (e.g., monitoring vs. development in assessment tools) and value complementarity (e.g., naturalness vs. care in animal welfare). Unlike many other problems of sustainability assessment, complementarity is of a fundamental character connected to the very conditions for observation. Therefore complementarity cannot be overcome methodologically, only handled better or worse. Complementarity plays an important role in understanding the two key problems of integrating different assessments and indicators and implementing assessments in practice to instigate the transformation to more sustainable food systems. Participation in itself is impotent. To assist participation and enable sustainability transformation there is a need for awareness of the importance of different perspectives and values and the complementarities connected to these differences.
Læssøe et al. (2014) consider motivation to be a key issue in determining how a multicriteria assessment tool should be designed to facilitate reflections, communication, and decision making in relation to organic food systems. They compare three research perspectives on motivation, economic, psychosocial, and relational, using the example of a consumer assessing and choosing products in the supermarket. From an economic perspective motivation is located in the buying situation and consumers’ need to choose between products, focusing on decision-making processes. From a psychosocial perspective individual experiences and lifeworld strategies are important, focusing on the influence of cognitive structures and emotional drivers. From a relational perspective motivation is a matter of how value relations are created and influenced, focusing on the diversity of buying relations and social interactions. The three perspectives only partially agree on the purpose of using a multicriteria assessment tool and its scope, strategic focus, constraints, and potential, and emphasize the importance of an overview of the options, influence on the criteria and values, and dialogue between the agents in the food chain, respectively. The paper thus demonstrates the value of a multiple-perspective approach in exploring what aspects of motivation are relevant when designing multicriteria assessment tools for organic food systems.
The approach taken when communicating about organic foods critically affects the extent of consumer participation and learning (Kastberg 2015). Communication forms that encourage dialogue or cooperative action are preferred by the organic producers, but coactional communication is crucially dependent on highly motivated participants and is difficult to control and encourage. The paper presents an Internet-based coactional platform for promoting communication, participation, and learning about organic foods. Each stakeholder harbors different criteria for determining what good organic food is and makes decisions based on multiple criteria. The tool represents an alternative route to handling problems of sustainability assessment of food systems by allowing stakeholders to communicate on the criteria behind their actions and in this way cooperate in coconstructing understanding, trust, etc., and, in effect, coconstruct multicriteria assessment.
Trust and credibility are crucial for the success of organic labels as a decision guide for promoting more sustainable agriculture. Rittenhofer and Povlsen (2015) critically scrutinize the construction of trust and credibility and how this influences the performance of organics within both a management and media research perspective. They conclude that belief in organic labels’ direct impact on consumer choices cannot be supported, that the concept of trust in the relations between organic products, labels, and consumers is still poorly understood, and that the explanatory value of trust for the success of organic production remains unproven. They suggest that future research on the relevance of credibility and trust for the success of organic production should be interdisciplinary, focus on the emergence of trust, and consider other social factors than trust and credibility.
Rittenhofer and Povlsen’s analysis therefore emphasizes the importance of Freyer et al. (2014)’s conceptual framework to analyze how consumers might build and practice trust in the organic agri-food chain, with particular regard to the role of multicriteria assessment tools for trust building. They identify three consumer trust types: (i) uninformed and unreflective trust in labels; (ii) informed and reflective trust based on extensive information, control, and certification; and (iii) informed, engaged, and self-reflective trust based on close farmer-consumer relationships. The authors conclude that multicriteria assessment tools are not relevant for the uninformed consumer to build trust, while the informed consumer would like to apply a predefined multicriteria assessment tool that helps to prove whether they can trust. The informed and engaged consumer mostly would not be interested in predefined multicriteria assessment tools, but in some cases might develop their own tool together with their partners in the organic agri-food chain.
Sustainability is a paradoxical perspective because it strives to comprise the whole to sustain life and well-being, but it must rely on a multitude of specialized perspectives to do so. Similarly, organic agriculture does not have its own holistic perspective, from which to observe the development of organic food systems compared to the vision and goals laid down in the principles of organic agriculture; it always depends on other perspectives. The papers on this theme highlight issues of working with multiple perspectives in sustainability assessments that have hitherto not received sufficient attention. This comprises both the inclusion of different research perspectives—perspectives that may be complementary and therefore mutually exclude each other—and the role of stakeholder perspectives and values in sustainability assessments. The papers also highlight the role of trust and credibility, something that is at the fore in research on organic food systems, but that is equally relevant to sustainability assessments in general.
Alternative food systems, especially those that are developed with the aim to be more sustainable, offer particular lessons for wider agriculture, including how the sustainability assessment itself could be conducted. This Special Feature of Ecology and Society includes many examples from organic agriculture, but we believe the opportunities and challenges demonstrated by contributing researchers are pertinent to all systems of agriculture, including conventional and Integrated Management farming. Close examination of organic farming is particularly instructive because it is guided by a plurality of societal objectives and by ethical guidelines (Luttikholt 2007) and it enjoys widespread acceptance among its customers as being more sustainable. The success of these alternative developments depends on whether they can continue to improve in relation to their goals, and thereby maintain credibility and trust. The emergence of a plethora of broadly holistic Integrated Management sustainability assessment criteria to deliver multifunctional agriculture underscores that organic agriculture must demonstrate its performance against an extremely broad range of benefits and costs for society, economy, environment, and nature (Merfield et al. 2015). The key question of this Special Feature is how best can such assessments be designed and conducted?
Three pivotal and interconnected challenges have emerged from the Special Feature and the allied, recently completed, MultiTrust project (Alrøe and Noe 2014a) about how to develop overall assessments of food systems to improve food system sustainability (Fig. 1). The first challenge is how to balance different types of knowledge, and to avoid that what is most well-known, precise, or easiest to measure gets the most weight (cf. Rotmans 1998). Some of the effects of food systems can be measured in quantitative terms, for others, qualitative assessments are more suitable; some are well known, others little known; some are easy to measure, others difficult and costly to assess; and some effects are indirect because of linkage and feedback within coupled social, economic, and ecological systems that underpin agricultural production (cf. Wiek and Binders 2005, Ewert et al. 2009). This challenge concerns both foundational, theoretical issues and more practical, methodological issues. Choices are needed, but how they are done and shown becomes crucial. For instance, Schader et al. (2014) shows the trade-off between different aspects of knowledge, such as scope vs. precision, in assessment tools, and Marchand et al. (2014) investigate the different forms of assessment tools and types of knowledge needed for scientific monitoring vs. farm development. Different target groups differ in what form of knowledge they expect and use, and the informed consumer identified by Freyer et al. (2014) is the most likely to be influenced by multicriteria assessments. Measuring weights given to different disciplines (Peano et al. 2014) quantifies differences in emphasis of different types of knowledge and provides a method to reframe the same sustainability data in different ways for different audiences, but it does not reconcile the underlying perspectives and debates about which should take overall precedence in how food and fibre are produced.
The second challenge is how to expose the values in assessment tools and choices, and how these built-in values relate to the ethical principles of sustainable food systems, to societal goals, and to the interests of different stakeholders (cf. Gregory 2000). Assessments are inherently value-based; they are always judgments of better and worse in some sense, but often these values are more or less hidden. For example, indicators are chosen as measures for certain aspects of food systems because they are “important to us” in certain ways, and Hartmut Bossel (2001) has suggested the term “orientor” for the way an indicator is important. There is a need for normative transparency to truly enable user-driven assessments, because values point out the direction in which to move and the values in different research and stakeholder perspectives often differ. An important question is how normative criteria are built into a multicriteria assessment framework in the selections and condensations made (cf. Gasparatos 2010). Many of the papers in this Special Feature touch upon the essential role of values in multicriteria assessments and some of the authors go further by emphasizing the “locked in” nature of values from different disciplines and the need for normative consistency (Thorsøe et al. 2014, Alrøe and Noe 2016).
The third challenge is how to enable communication in such a way that the assessments can effectively contribute to the development of more sustainable food systems; something that requires a mutual learning process between researchers and stakeholders. Reduction of complexity of multicriteria assessments is necessary, but where and how is decisive. Sustainability assessments will be of little use if they cannot be understood, accepted, and utilized by the many different actors and stakeholders in food systems (cf. Binder et al. 2010, 2012). The ability to handle complex information differs, there are multiple expert and stakeholder perspectives involved, media play an important role, and ownership is a key concept (cf. Stoeglehner et al. 2009). Therefore definitions and assessment tools must remain pluralistic, open-ended, and adaptable to embrace new concerns that arise continuously in society (Darnhofer et al. 2010, Noe and Alrøe 2015). According to Niklas Luhmann the function of trust is precisely “the reduction of complexity” in social systems (as cited in Lewis and Weigert 1985:968), and trust thereby becomes a key concept in relation to this challenge. Kastberg (2015) in this Special Feature concludes that communication in the form of coaction enables the coconstruction of understanding, trust, etc., but that this emancipation of the consumer comes at the price of loss of control of the communication processes.
A wider question pervades the two main groups of papers in the Special Feature: How can we best get from sustainability assessment to sustainability transformation? The end goal of instigating transformations to better and more sustainable food systems is inherent in the three challenges described above. Food systems are social, communicative systems (cf. Noe and Alrøe 2015), and from the challenge of enabling communication comes a focus on the food system as the hub of future prospects for sustainability transformation. Food systems cannot be transformed piecemeal, but need to be transformed in a coordinated, synchronized way. Changing global and market-driven food systems will be particularly difficult and will require innovative new tools that reconnect producers, regulators, policy makers, marketers, and consumers across the entire value chain. Peano et al. (2014) focuses on how to assess a food system, the Slow Food Presidia, as a whole, with a tool that monitors networking and communication, but does not as such facilitate communication. Two other papers in this Special Feature discuss a tool that focuses entirely on facilitating communication in the food system (Kastberg 2015, Alrøe and Noe 2016). This tool was proposed by the MultiTrust project as a cooperative communicational platform for developing more sustainable food chains. The tool works by revealing and communicating the value-laden criteria used by different agents, e.g., producers, processors, retailers, consumers, etc., in the food chain for selecting goods and taking new development initiatives (Alrøe and Noe 2014b; see also an animated sketch of this tool at https://youtu.be/UF15_4knPUA). Another tool that stretches across the food system is the New Zealand Sustainability Dashboard, which uses internet technology and smart reporting and benchmarking, in essence, primarily to build a social network for sharing information on sustainability performance among producers at the start of the supply chain (Merfield et al. 2015). However, the same information simultaneously informs industry strategists and marketers, land regulators and customers making purchase choices. This type of cross-scale linking tool can potentially bridge the insider/outsider stances and viewpoints by communicating and rewarding ethical and sustainable farming practices by reconnecting actors on opposites sides of the planet.
The wider question of how to instigate sustainability transformation also speaks to the two other challenges of balancing types of knowledge and exposing values. The papers in this Special Feature differ in the weight they put on these two challenges, from papers that emphasize the role of assessments in enhancing the knowledge basis for making decisions toward sustainability transformation through better tools and tool choices (Marchand et al. 2014, Peano et al. 2014, Schader et al. 2014, Triste et al. 2014), to papers that emphasize different approaches to exposing the value basis (Thorsøe et al. 2014, Alrøe and Noe 2016) and working with the value basis for taking actions toward sustainability transformation in form of value communication (Kastberg 2015), motivation (Læssøe et al. 2014), and trust (Freyer et al. 2014, Rittenhofer and Povlsen 2015). However, none of the papers explicate this difference between working with better knowledge or stronger values in the development of methods and tools for sustainability transformation.
The act or intent of assessment itself implies a “target-based” or “performance-based” approach, the notion that having some measures of progress toward some goal of sustainability or resilience will help bring about a transition to more sustainable systems. Experience tells that this is not always the case. The values-based approach opens a whole new set of choices that potentially circumvents the type of knowledge-based transformations assumed by many of our authors. But the role of assessment is still unclear in such approaches. We think this distinction between performance-based and values-based approaches is of key importance for the prospects for future sustainability assessments in bringing about real transformation toward more sustainable food systems. And we strongly recommend research on the strengths, weaknesses, and complementarities of taking a values-based rather than a performance-based approach to promoting the resilience and sustainability of coupled ecological, economic, and social systems for ensuring food security and agroecosystem health in the coming millennium.
We encourage readers to use the discussion page set up by Ecology and Society for this guest editorial to comment on the Special Feature and continue the discussion begun here. All types of input are welcome, from simple comments and queries to more formal commentaries. We think the themes raised here are really important for the development of more sustainable food systems, and we would warmly welcome all who want to engage in the discussion.
The papers in this Special Feature derive from a workshop entitled “Balancing and communicating overall assessments of food systems,” held at the 10th International Farming Systems Symposium (IFSA 2012) “Producing and reproducing farming systems: New modes of organisation for sustainable food systems of tomorrow” 1-4 July 2012 at Aarhus University, Denmark. The workshop was organized by the transdisciplinary research project MultiTrust in collaboration with its international partners. As guest editors we wish to thank the organizers of IFSA 2012 and the participants in the original workshop, and especially the contributing authors and the reviewers from Ecology and Society for their time and efforts in helping to put together this Special Feature. Special tanks are due to Managing Editor Jennifer Miner for support and patience in bringing this Special Feature to its conclusion.
We acknowledge the financial support from the Danish Government for the MultiTrust project, “Multicriteria assessment and communication of the effects of organic food systems” (http://www.multitrust.org), including support for the original workshop and parts of the editorial work. We also wish to acknowledge the contributions from the partners in the MultiTrust project, including partners from university departments across the human, social, and natural sciences, farmers associations and advisory services, public regions and municipalities, and private companies. The MultiTrust project ran 2011-2014 with Hugo Alrøe as the project leader. It was funded by the Organic Research, Development and Demonstration program under the Danish Ministry of Food, Agriculture and Fisheries; a program that is coordinated by the International Centre for Research in Organic Food Systems (ICROFS). Furthermore, the OECD Co-operative Research Programme granted Hugo Alrøe a three month research fellowship in 2011 at the Centre for Sustainability (CSAFE), University of Otago to collaborate with Henrik Moller on developing overall assessments of organic and sustainable agriculture. Henrik Moller’s participation was funded by the New Zealand Sustainability Dashboard project (http://www.nzdashboard.org.nz), funded by New Zealand’s Ministry of Business, Innovation & Employment (Contract AGRB1201).
Alrøe, H. F., and E. Noe. 2011. A cross-disciplinary approach to multicriteria assessment and communication of the effects of organic food systems. Pages 313-316 in D. Neuhoff, N. Halberg, I. A. Rasmussen, J. E. Hermansen, C. Ssekyewa, S. Ssekyewaand, and S. Mok, editors Organic is life - knowledge of tomorrow. ISOFAR, Bonn, Germany, Vol. 2. [online] URL: http://orgprints.org/19951/
Alrøe, H. F., and E. Noe. 2014a. How can we know if organics becomes better? A perspectivist view on multicriteria assessment. Pages 191-194 in G. Rahmann and U. Aksoy, editors. Building organic bridges. Thuenen Report no. 20, Johann Heinrich von Thünen-Institut, Braunschweig, Germany. [online] URL: http://orgprints.org/23904/
Alrøe, H. F., and E. Noe. 2014b. A prototype tool for multicriteria assessments to develop organic food chains. Pages 195-198 in G. Rahmann and U. Aksoy, editors. Building organic bridges. Thuenen Report no. 20, Johann Heinrich von Thünen-Institut, Braunschweig, Germany. [online] URL: http://orgprints.org/23914/
Alrøe, H. F., and E. Noe. 2016. Sustainability assessment and complementarity. Ecology and Society 21(1):30. http://dx.doi.org/10.5751/es-08220-210130
Binder, C. R., G. Feola, and J. K. Steinberger. 2010. Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture. Environmental Impact Assessment Review 30:71-81. http://dx.doi.org/10.1016/j.eiar.2009.06.002
Binder, C. R., A. Schmid, and J. K. Steinberger. 2012. Ecological sustainability solution space of the Swiss milk value added chain. Ecological Economics 83:210-220. http://dx.doi.org/10.1016/j.ecolecon.2012.06.022
Bogetoft, P., and P. Pruzan. 1991. Planning with multiple criteria: investigation, communication, choice. North-Holland, Amsterdam, The Netherlamnds.
Bond, A. J., T. Dockerty, A. Lovett, A. B. Riche, A. J. Haughton, D. A. Bohan, R. B. Sage, I. F. Shield, J. W. Finch, M. M. Turner, and A. Karp. 2011. Learning how to deal with values, frames and governance in sustainability appraisal. Regional Studies 45(8):1157-1170. http://dx.doi.org/10.1080/00343404.2010.485181
Bond, A. J., and A. Morrison-Saunders. 2011. Re-evaluating sustainability assessment: aligning the vision and the practice. Environmental Impact Assessment Review 31(1):1-7. http://dx.doi.org/10.1016/j.eiar.2010.01.007
Bossel, H. 2001. Assessing viability and sustainability: a systems-based approach for deriving comprehensive indicator sets. Conservation Ecology 5(2):12. [online] URL: http://www.consecol.org/vol5/iss2/art12/
Darnhofer, I., J. Fairweather, and H. Moller. 2010. Assessing a farm’s sustainability: insights from resilience thinking. International Journal of Agricultural Sustainability 8:186-198. http://dx.doi.org/10.3763/ijas.2010.0480
Ewert, F., M. K. van Ittersum, I. Bezlepkina, O. Therond, E. Andersen, H. Belhouchette, C. Bockstaller, F. Brouwer, T. Heckelei, S. Janssen, R. Knapen, M. Kuiper, K. Louhichi, J. A. Olsson, N. Turpin, J. Wery, J. E. Wien, and J. Wolf. 2009. A methodology for enhanced flexibility of integrated assessment in agriculture. Environmental Science & Policy 12(5):546-561. http://dx.doi.org/10.1016/j.envsci.2009.02.005
Food and Agriculture Organization of the United Nations (FAO). 2009. How to feed the world 2050. Issues briefs from the High level expert forum 12-13 October 2009. FAO, Rome, Italy. [online] URL: http://www.fao.org/wsfs/forum2050/wsfs-forum/en/
Freyer, B., J. Bingen, and R. Paxton. 2014. Can multicriteria assessment tools help build trust into organic products? Ecology and Society 19(4):5. http://dx.doi.org/10.5751/ES-06793-190405
Herrmann, S., E. van de Fliert, and J. Alkan-Olsson. 2011. Editorial: Integrated assessment of agricultural sustainability: exploring the use of models in stakeholder processes. International Journal of Agricultural Sustainability 9(2):293-296.
Gasparatos, A. 2010. Embedded value systems in sustainability assessment tools and their implications. Journal of Environmental Management 91:1613-1622. http://dx.doi.org/10.1016/j.jenvman.2010.03.014
Gasparatos, A., and A. Scolobig. 2012. Choosing the most appropriate sustainability assessment tool. Ecological Economics 80:1-7. http://dx.doi.org/10.1016/j.ecolecon.2012.05.005
Gregory, R. 2000. Using stakeholder values to make smarter environmental decisions. Environment: Science and Policy for Sustainable Development 42(5):34-44. http://dx.doi.org/10.1080/00139150009604888
Kastberg, P. 2015. Promoting communication, participation, and learning with regard to organic food products: a communication theoretical approach. Ecology and Society 20(1):3. http://dx.doi.org/10.5751/ES-07139-200103
Læssøe, J., A. Kruse Ljungdalh, H. F. Alrøe, E. Noe, T. Christensen, A. Dubgaard, S. Bøye Olsen, N. Kærgård, and P. Kastberg. 2014. Three perspectives on motivation and multicriteria assessment of organic food systems. Ecology and Society 19(3):7. http://dx.doi.org/10.5751/ES-06397-190307
Lewis, J. D., and A. Weigert. 1985. Trust as a social reality. Social Forces 63(4):967-985. http://dx.doi.org/10.1093/sf/63.4.967
Luttikholt, L. W. M. 2007. Principles of organic agriculture as formulated by the International Federation of Organic Agriculture Movements. NJAS - Wageningen Journal of Life Sciences 54(4):347-360. http://dx.doi.org/10.1016/s1573-5214(07)80008-x
Marchand, F., L. Debruyne, L. Triste, C. Gerrard, S. Padel, and L. Lauwers. 2014. Key characteristics for tool choice in indicator-based sustainability assessment at farm level. Ecology and Society 19(3):46. http://dx.doi.org/10.5751/ES-06876-190346
Merfield, C., H. Moller, J. Manhire, C. Rosin, S. Norton, P. Carey, L. Hunt, J. Reid, J. Fairweather, J. Benge, I. Le Quellec, H. Campbell, D. Lucock, C. Saunders, C. MacLeod, A. Barber, and A. McCarthy. 2015. Are organic standards sufficient to ensure sustainable agriculture? Lessons from New Zealand’s ARGOS and Sustainability Dashboard projects. Sustainable Agriculture Research 4:158-172. http://dx.doi.org/10.5539/sar.v4n3p158
Millennium Ecosystem Assessment. 2005. Ecosystems and human well-being: synthesis. Island Press, Washington, D.C., USA.
Ness, B., E. Urbel-Piirsalua, S. Anderberg, and L. Olsson. 2007. Categorising tools for sustainability assessment. Ecological Economics 60:498-508. http://dx.doi.org/10.1016/j.ecolecon.2006.07.023
Noe, E., and H. F. Alrøe. 2015. Sustainable agriculture issues explained by differentiation and structural coupling using social systems analysis. Agronomy for Sustainable Development 35(1):133-144. http://dx.doi.org/10.1007/s13593-014-0243-4
Peano, C., P. Migliorini, and F. Sottile. 2014. A methodology for the sustainability assessment of agri-food systems: an application to the Slow Food Presidia project. Ecology and Society 19(4):24. http://dx.doi.org/10.5751/ES-06972-190424
Pope, J., D. Annandale, and A. Morrison-Saunders. 2004. Conceptualising sustainability assessment. Environmental Impact Assessment Review 24:595-616. http://dx.doi.org/10.1016/j.eiar.2004.03.001
Pretty, J., W. J. Sutherland, J. Ashby, J. Auburn, D. Baulcombe, M. Bell, J. Bentley, S. Bickersteth, K. Brown, J. Burke, et al. 2010. The top 100 questions of importance to the future of global agriculture. International Journal of Agricultural Sustainability 8:219-236. http://dx.doi.org/10.3763/ijas.2010.0534
Rittenhofer, I., and K. K. Povlsen. 2015. Organics, trust, and credibility: a management and media research perspective. Ecology and Society 20(1):6. http://dx.doi.org/10.5751/ES-07169-200106
Rotmans, J. 1998. Methods for IA: the challenges and opportunities ahead. Environmental Modeling & Assessment 3(3):155-179. http://dx.doi.org/10.1023/A:1019019024003
Schader, C., J. Grenz, M. S. Meier, and M. Stolze. 2014. Scope and precision of sustainability assessment approaches to food systems. Ecology and Society 19(3):42. http://dx.doi.org/10.5751/ES-06866-190342
Stoeglehner, G., A. L. Brown, and L. B. Kørnøv. 2009. SEA and planning: ‘ownership’ of strategic environmental assessment by the planners is the key to its effectiveness. Impact Assessment and Project Appraisal 27(2):111-120. http://dx.doi.org/10.3152/146155109x438742
Thorsøe, M. H., H. F. Alrøe, and E. Noe. 2014. Observing the observers: uncovering the role of values in research assessments of organic food systems. Ecology and Society 19(2):46. http://dx.doi.org/10.5751/ES-06347-190246
Triste, L., F. Marchand, L. Debruyne, M. Meul, and L. Lauwers. 2014. Reflection on the development process of a sustainability assessment tool: learning from a Flemish case. Ecology and Society 19(3):47. http://dx.doi.org/10.5751/ES-06789-190347
Vatn, A. 2009. An institutional analysis of methods for environmental appraisal. Ecological Economics 68(8-9):2207-2215. http://dx.doi.org/10.1016/j.ecolecon.2009.04.005
Wiek, A., and C. Binder. 2005. Solution spaces for decision-making—a sustainability assessment tool for city-regions. Environmental Impact Assessment Review 25:589-608. http://dx.doi.org/10.1016/j.eiar.2004.09.009