Observing the observers : uncovering the role of values in research assessments of organic food systems

Assessing the overall effects of organic food systems is important, but also a challenge because organic food systems cannot be fully assessed from one single research perspective. The aim of our research was to determine the role of values in assessments of organic food systems as a basis for discussing the implications of combining multiple perspectives in overall sustainability assessments of the food system. We explored how values were embedded in five research perspectives: (1) food science, (2) discourse analysis, (3) phenomenology, (4) neoclassical welfare economics, and (5) actor-network theory. Value has various meanings according to different scientific perspectives. A strategy for including and balancing different forms of knowledge in overall assessments of the effects of food systems is needed. Based on the analysis, we recommend four courses of action: (1) elucidate values as a necessary foundation for research assessment across perspectives; (2) openly discuss the choice of perspective, because it is decisive; (3) formulate common goals that can be translated into the different perspectives; and (4) consider assessment of food system sustainability a learning process and design it as such.


INTRODUCTION
Assessments of the overall effects of organic food systems are important for the continuous development of these systems.Such assessments influence how consumers, producers, and policy makers view and in turn act in relation to the food systems.Organic food systems are complex because they are based on the organic principles of health, ecology, fairness, and care, which are a diverse and sometimes incommensurable set of ethical norms (International Federation of Organic Agriculture Movements 2009).Many different research perspectives describe specific aspects of organic food systems.However, because of their complexity, organic food systems cannot be fully assessed from one perspective, and the organic principles do not detail how the food systems should be assessed.
Facts and values become entangled during the production of knowledge (Putnam 2002), and assessments are fundamentally founded on values (Bossel 1999, Binder et al. 2010).Therefore, we explored how values are embedded within the different research assessments of organic food systems.Because those values result in distinctly different assessments, we discuss the implications of combining multiple perspectives in overall sustainability assessments of the food system.
We based this article on a perspectivist understanding of scientific knowledge production (Giere 2006, Alrøe andNoe 2011).A perspective entails observing from a certain position and rejection of a privileged or absolute observation point (Bourdieu 2004).In this understanding, phenomena are coupled with the apparatus applied in the observation of the phenomena.Knowledge production depends on humanly produced artifacts, both material and abstract, such as laboratory equipment, models, methods, and concepts.These all incorporate a built-in perspective on the world and consequently, the scientific practice embeds the perspective into the knowledge produced (Giere 2006).A perspective only describes a particular aspect of the world, and observations and scientific claims only apply to this particular aspect.Perspectives can also be considered a mental model, a cognitive structure upon which reasoning, decision making, and behavior are based (Lynam and Brown 2011).

METHODS
Our methodology was to "observe the observers," i.e., to describe in neutral terms and without commenting how organic food systems are assessed from five different perspectives.We chose that methodology because we did not want to project our understanding of value onto each perspective, but rather to understand how value was understood and practiced in different contexts.We focused on five different perspectives: (1) food science, (2) discourse analysis, (3) phenomenology, (4) neoclassical welfare economics, and (5) actor-network theory (ANT).
To guide our inquiry and have a comparable basis for discussions, we observed how each of the five perspectives related to three central questions.(1) How was value fundamentally understood?Following Pirsig (1999), we distinguished between three different understandings of value: value found with the objects under study, within the subjects studying the objects, or in the relation between subject and object.(2) How was value measured?We determined the basis for the assessments and what scales were used to assess the food system.(3) How was the term "organic" understood?We addressed how the term was used and what was included in its definition.
The three questions were addressed separately for each perspective by looking into the core concepts and researchers' reflections about the perspectives.For every perspective, we selected two cases to illustrate and identify the core concepts and rationale of each perspective.The aim was not to provide a comprehensive overview of the field, but to explore the types of insights provided by the different perspectives.

RESULTS
The basic features of the five perspectives, i.e, food science, discourse analysis, phenomenology, neoclassical welfare economics, and ANT, are summarized in Table 1.

Food science
Food science is grounded in a logical-empirical, scientific tradition and is an umbrella for researchers within fields such as biology, chemistry, microbiology, and food engineering who  (Potter and Hotchkiss 1998).Scientists within the food science field are interested in the physical, chemical, and microbiological properties of food products, as well as the processes that the food undergoes during production and processing.Food science can, therefore, be seen as a perspective for which value is found within the objects, because the objects and object properties are the only aspects considered.Two recent publications illustrate this approach: . "Qualitative and nutritional differences in processing tomatoes grown under commercial organic and conventional production systems" (Barrett et al. 2007).Samples of tomatoes grown in different production systems were analyzed for water content, color, acidity, and so forth.
Barrett and colleagues concluded that there was a large variation between the producers but that the organic products had a higher quality in relation to the selected indicators.
. "A review of the nutrition claims made by proponents of organic food" (Rosen 2010).Rosen reviewed the research behind the claim that organic products have more nutrients than conventionally produced products.According to this review, it is not valid to claim that organic products have more nutrients, because this claim is backed only by studies that are not peer reviewed or that have statistically insignificant results.
Within food science, value is measured as food quality, meaning a higher or lower content of specific substances that affect the human organism.Human health is important for the research in the field, and human health is seen as something that can be improved by altering the physical properties of the food products, i.e., food equals nutrition.Actions are then assessed based on whether or not they modify the products in a desired way.
Examples include increasing the content of beneficial substances and reducing the harmful ones, enhancing the longevity of the products, and protecting the products from being contaminated by pathogens.
Food science assesses the food system by examining the food items, thereby reducing the production process to the food itself.
In the study by Barrett et al. (2010), focus was on the concept of quality, which was understood as the sensory qualities and factors related to the processing of tomatoes.In the study by Rosen (2010), the focus also was on quality, but quality was understood to be nutritional content.The food science perspective provides a description of the product in terms of certain indicators.Interestingly, neither of the two studies discusses the quality indicators in any great detail; it is implicit that they are important indicators of quality.
In both articles, the authors discuss whether or not the organic production process has an influence on the products.Assessment of the organic products was done by comparing them with conventionally produced products.It is only relevant to know that a product has a specific amount of substance if that product is compared with other products, or if you know the effect it has on the human organism.Otherwise, the amount is just an insignificant number; indicators become meaningful only in a comparison.Consequently, a huge amount of statistically significant and peer-reviewed research is required before conclusions can be drawn.

Discourse analysis
Rooted in a poststructural tradition, discourse analysis is focused on the symbolic representation of food and how linguistic http://www.ecologyandsociety.org/vol19/iss2/art46/structures influence how meaning is constructed for the subjects enrolled in the system.In this perspective, discourses are important in understanding why we act the way we do and are closely related to questions of power, governance, and how understanding of the world is constructed; exposing these linguistic structures is an important undertaking in this perspective (Fairclough 1992, Hajer 1995).Because discourse analysis entirely focuses on the social structures, it can be seen as a perspective in which value is found with the subjects.Discourse analysis has been applied to the contested concept of organic, what constitutes organic food and how the understanding is produced, as illustrated by the following selected two studies: . "Naming organics: understanding organic standards in New Zealand as a discursive field" (Campbell and Liepins 2001).
Campbell and Liepins analyzed the evolution of organic standards in New Zealand as a discursive field using qualitative stakeholder interviews and incorporating social movements, consumers, food scares, and regulatory politics in their analysis.They concluded that initially local organic pioneers, i.e., organizations with links to organic agriculture in Europe, were influential in initiating a request for standards, but that the formulation of standards was dominated by corporate exporters and producer associations.This has produced two ways of performing organic farming, one that is certified and export oriented and another that is trust based and focused on the local market.
. "Italesaettelser af økologisk mad" (Larsen 2006).Larsen analyzed the values ascribed to organic food in Denmark by looking at the discourses about organic food in the media and concluded that the term organic is a floating signifier, a concept to which different meanings are ascribed.The meaning ascribed to organic varies with time.In the 1980s the organic discourse was associated with environmentally friendly production and alternative lifestyles; whereas from the middle of the 1990s onward, these discourses have gradually been supplemented or replaced by organic signifying animal welfare, health, and gastronomy.The discourses become not only arguments for justification of behavior, but also the underlying basis for assessing the standards for organic farming.
Within discourse analysis, value is measured using qualitative social science methods such as analysis of texts, policy documents, and interviews.The objective is to understand who or what governs the food system and how a specific understanding of organic has come to dominate.Therefore, the analysis is done on a macro level.
In this perspective, the food system is assessed based on how the subjects perceive organic food and by the meaning they ascribe to the food, and no attention is given to the material products.
The food system is thus assessed as discursive structures that govern how food is produced and consumed, rather than for its material qualities or the effect it has on people.It is not enough to focus on the actors that directly handle the food in the system; it is also necessary to include the actors that influence the discourses governing food production and consumption (Campbell and Liepins 2001).
Discourse analysis researchers understand organic as a contested concept and provide a perspective on how to understand the evolution of organic as a concept, or as Campbell and Liepins (2001:36) put it: "applying Discourse Analysis to a specific region or country the explanatory outcomes are not entirely idiosyncratic.There are broad commonalities and explicit linkages-the global organic social movement, harmonizing export standards, global food scares and world market demand -between New Zealand and other regional spaces constructing organics."How we perceive the meaning of organic and what we choose to eat are the results of a complicated process by which certain meaning becomes associated with organic, a process to which many actors on many different scales contribute.

Phenomenology
Phenomenology investigates how objects are represented in consciousness and how phenomena appear to subjects (Moran 2000).Research inspired by this tradition is generally focused on social interactions and situations that appear in the lifeworld of subjects, how meaning is ascribed to these situations, and how identities are created and maintained in the social practice.Therefore, value within phenomenology is found with the subjects.In relation to the assessment of the organic food systems, research is focused on the role of production and consumption in identity formation and how this identity is maintained through practices in the social system.Two selected studies illustrate this approach: . "Values about nature in organic farming practice and knowledge" (Kaltoft 1999).Kaltoft analyzed values about nature in organic farming practice and how the values resulted in different farming practices.The methodology was qualitative interviews with six farmers, ranging from biodynamic producers to family farmers to rationalistic and academically trained large-scale producers.Kaltoft concluded that ideology and institutionalization are determining what constitutes organic farming practice and that farmers evaluate their practice by using four incompatible paradigms of knowledge.Thus, organic farming is not a singular phenomenon; rather, it is a variety of different practices existing simultaneously.
. "Consumers' purchase of organic food products.A matter of convenience and reflexive practices" (Hjelmar 2011).
Hjelmar analyzed how consumers develop a meaningful shopping practice concerning organic food and how attitudes toward organic food are formed by social interactions within the household, using qualitative interviews focused on motivation in the shopping process.
Hjelmar concluded that the decision to purchase organic produce is influenced by factors such as availability, price, perceived quality, family considerations, political/ethical concerns, and health concerns.The resulting shopping practice is, therefore, the outcome of a complex reflexive process balancing different and sometimes conflicting concerns.http://www.ecologyandsociety.org/vol19/iss2/art46/Value is measured using qualitative social science methods such as in-depth interviews on a micro scale to gain a holistic understanding of how the different elements of the food system interact with and appear to the subjects.In-depth interviews are used to gain an understanding of the subjects' individual concerns and the lifeworlds, practices, and rituals established to give meaning to their daily lives, or as Halkier (1998:25) puts it: "Consumption is part of the social space in which people participate in creating and reproducing meanings about the occurrences of everyday life by attempting to knit together the experiences and roles they encounter daily" [our translation from the Danish].Centered on individuals and the situations they take part in, the assessment of organic aims to gain a holistic understanding of the motivations that drive the individuals in their daily lives and the complexities surrounding how they interact with the food system.In this perspective, the values function as guiding principles for the individuals and in the strategic decisions they have to make regarding how to organize their daily lives.
Phenomenological analysis assesses the food system based on how it appears to a single individual and on the individual considerations, values, and meanings that are used to form a meaningful practice.The formation of a practice is complicated; often the individuals must choose between many different and conflicting considerations.The food system cannot be assessed outside of its context for the individuals.
As in discourse analysis, perception of the meaning of organic is at the heart of the analysis.However, unlike the structural focus of discourse analysis, phenomenology focuses on the individuals and how they perceive that meaning.Organic products are here understood as a prop in the practice of daily life, and individuals' attitudes toward them reflect how they perceive themselves and their surroundings.

Neoclassical welfare economics
Neoclassical welfare economics provides a perspective on the exchange of goods and services in a market influenced by prices, output, and income (Mäler and Vincent 2005).The marketplace is seen as the meeting place of the producers who supply the goods and the consumers who buy the goods; there is no market without one or the other.Thus, neoclassical welfare economics applies a relational perspective to values.Research conducted in this tradition focuses on understanding how consumers act in the market for organic products and what influences their behavior.Two selected studies illustrate the perspective: . "The character of demand in mature organic food markets: Great Britain and Denmark compared" (Wier et al. 2008).Wier et al. studied the character of demand in the mature organic markets of Denmark and Great Britain by conducting qualitative surveys of the stated preferences of the consumers and their registered purchasing behavior.The organic market was sustained by labeling schemes and mainly organized around large supermarket chains, which secured effectiveness, abundant supply, and low prices.They concluded that there is a discrepancy in stated behavior and registered behavior; people stated that public good attributes mattered the most, but acted according to private good attributes.
. "Comparison of consumer perceptions and preference toward organic versus conventionally produced foods: a review and update of the literature" (Yiridoe et al. 2005).These researchers reviewed the international literature on welfare economics regarding consumer perceptions and preferences.They concluded that consumer preference for organic food is based on the general perception that organic food has more desirable attributes than the conventionally grown alternatives.At the same time, studies point to inconsistencies regarding the understanding of what organic food actually is.There was also a large variation across countries in the valuation of organic products.North American consumers preferred organic products because of better sensory qualities, whereas European consumers preferred organic products because of safety and environmental concerns.
Value is measured as preferences, which are translated within welfare economics into willingness to pay, a monetary indicator for how much a person is willing to pay to meet the preference, thus also indicating the intensity of the preference.Willingness to pay is generally measured by a multitude of different quantitative methods either directly, by registering purchasing behavior, or indirectly, through questionnaires and surveys.The goals of welfare economics research are to produce conclusions that are valid for a larger population and to understand which attributes are preferred by consumers.
Within welfare economics, the market is the fundamental unit for assessing the food system.In the market, people display preferences for certain food attributes over others, and these preferences are the main focus of welfare economics.It is assumed that each person has limited resources and needs to prioritize choices among a range of different attributes (Mäler and Vincent 2005).The preferences are influenced by producers, consumers, the state, media, and so forth.Therefore, welfare economics provides a perspective that captures a whole array of factors influencing the food system.
In this perspective, organic is understood as an aggregate of different food attributes that are valued by the consumers.This methodology only includes components of the food system that are valued by the market; other properties are not considered.
Externalities from the organic production practice are included in the consumers' willingness to pay if that information is supplied and if the complexities of the food production are comprehended by the consumers.

Actor-network theory
ANT is a constructivist approach to social theory, often described as "material semiotic," because both human and nonhuman actors can contribute to the formation of the network.Analysis within ANT can both include material actors, e.g., physical properties, technology, or infrastructure, and social actors, e.g., persons or organizations.In ANT, agency is located neither in the actors nor the objects, but in the relationships between them.A central objective of ANT research is to explain how actors come together in a network and act as one (Latour 2005).In the assessment of organic farming, ANT has been applied to understand the agency of the actors and the networks that organic food production is linked with.Although able to account for the http://www.ecologyandsociety.org/vol19/iss2/art46/material dimension, ANT is not always applied to elucidate this dimension.Two selected studies illustrate the approach: . "Things becoming food and the embodied, material practices of an organic food consumer" (Roe 2006).Roe explored how things become organic food.She attempted to map the stages in the transformation from vital materiality to organic food and reflected on the actors that are mobilized in the transformation.The methods used were observation of participants in a workshop, a focus group interview, and analysis of a video diary with a consumer preparing and consuming potatoes.The findings contribute to a debate about how quality and consumption practices are embodied in an intimate connection between human and nonhuman actors.
. "Conversion to organic dairy production in the Netherlands: opportunities and constraints" (Smit et al. 2009).Smit and colleageus assessed the opportunities for and constraints against conversion to organic dairy farming in the Netherlands.The methods used were analyses of statistics, policies, documents, and interviews.Barriers to conversion were found on different levels.For example, at the farm level, where the market is still fairly small, actors must form new relations with suppliers and buyers, and face possible negative financial consequences.At the chain level, actors do not see a great environmental impact of a conversion, and structural reforms may result in lower prices for conventional produce, thus stronger competition.
ANT assesses the food system based on how actors are mobilized to be a part of the system and the relations that uphold the system.What constitutes and acts as an actor at one level might unfold a network upon more thorough examination; in that sense, ANT is a very descriptive account of the food system, but also one that integrates many different elements of the food system.The study by Smit et al. provides an illustrative example, because all actors along the commodity chain were conditioning the farmers' decision not to convert to an organic farming practice.Actors are all interlinked, and if one farmer changes practice, his relations with all other actors in the network must be renegotiated.The network thus provides opportunities for and constraints against certain actions.
A plethora of methods such as interviews, observations, and personal accounts are used in the assessment of the actors' relations.As illustrated by the studies, the actors at both micro and macro levels are integrated in the analysis.Within ANT, each actor negotiates its position with all the other actors in the network when enrolling, and value within an ANT analysis is measured in terms of how an actor is related to other actors and influences their actions.This means that every organic food item has value in a negotiation with the other actors in the organic food actor-network.
From the ANT perspective, organic is seen as an actor network in which meaning and agency are relationally negotiated with the other actors in the food system.The network can be composed of both social and material actors; the ANT perspective provides an understanding of how different actors in the food system influence how organic acquires meaning.Organic cannot be defined a priori, but is something that food becomes as a property of the network it belongs to.

DISCUSSION
Observing the organic food system As we have demonstrated, value is not a singular phenomenon, but can be understood as something found either with the subjects, the objects, or in the relations between subject and object.This fundamental difference in perception is important because it directs the inquiry to different locations in the food system, e.g., looking at food or looking at actors.A consequence of the different definitions of value is different approaches to measuring value.Each perspective involves observation only of the aspects considered valuable and excludes all others.Food science, for example, includes the physical properties of the objects, but excludes the social practices associated with the objects.Within the limitations of a single perspective, the researcher can observe a specific part of the food system while developing blind spots for other parts.Therefore, the underlying value and measurement of the value in the assessments have practical implications because the perspective implies a certain way of observing and analyzing the food system.The differences between perspectives explain why it is difficult to combine insights from one perspective with another.Conclusions are fundamentally grounded in different understandings of value and cannot be transferred across perspectives without losing some of the meaning.
Organic is understood in multiple ways in the five perspectives, as physical properties, a social phenomenon influenced by discursive structures, identity, meaning, and economy, respectively.Within each perspective, an understanding of organic is embedded as a concrete outcome of the underlying understanding of value.Neoclassical welfare economics, for instance, understands organic as product attributes, which are relational and can be measured using willingness to pay.It would be convenient if there was one correct understanding of organic, but there are multiple ways in which the term can be understood.Each perspective contributes aspects relevant for understanding the organic food system.In relation to the development of organic food systems, it is important to include the different qualities of organic food and not optimize the systems according to only one understanding of value.

The challenge of combining assessments
Food systems are complex phenomena that can be perceived from many different perspectives, each providing different insights.Problems that occur in the food system are not necessarily associated with one perspective, but will transgress the traditional disciplinary boundaries and are interpreted differently across perspectives.This situation, of course, accentuates the need to combine and balance assessments.
Observing the observers provides a framework for qualifying how the assessments differ and which view on the subject matter is associated with each perspective.This knowledge is fundamental for understanding the assessments.(Giere 2006).Consequently, perspectives are necessary for science in developing questions and producing answers.However, the perspectives cannot be reduced to each other or combined in one perspective without the loss of meaning (Alrøe and Noe 2011).A prominent task in doing multicriteria assessments is to organize multiple sources of information about a complex subject like organic food to provide knowledge and support for decision making (Belton andStewart 2002, Recchia et al. 2011).This is difficult because of the different understandings of values.If multicriteria assessments are to function as decision support tools for the continual development of food system sustainability, then they must be able to deal with multiple understandings of values.If these multiple understandings are not taken into consideration, multicriteria assessments will become a power struggle over the assessment criteria.A way forward would be to abandon the turf wars so common between scientific perspectives and instead focus on formulating common research goals (Youngblood 2007).The common goals should be formulated so they can be covered by different perspectives, thereby ensuring that the multicriteria assessments become a complementary dialogue rather than competing monologues.
Viewing multicriteria assessments as a combination of multiple perspectives requires a radical break with the tradition of conducting multicriteria assessment from one hegemonic perspective.Instead of insisting that multicriteria assessments should provide us with one answer, we should instead embrace the possibility that they could provide us multiple answers to different questions.This, of course, increases the complexity of decision making, but also guards against brash actions by reminding us of the complexity of a sustainable food system and facilitates participation because it requires different stakeholders to be part of the decision-making process (Norgaard 1989).In particular, scientists need to reflect on how values are embedded within the perspective they are representing, acknowledging that everything cannot be explained from one perspective (Lélé and Norgaard 2005).The multicriteria assessment needs to be designed as a mutual learning process for the researchers involved in the process of conducting it.
According to Dodgson et al. (2009), multicriteria assessments involve the exercise of judgment.Therefore, it is important to note that the choice of assessments itself requires exercise of judgment.Choosing a perspective is a complex process and requires a thorough understanding of the information that each perspective provides and how the different perspectives supplement each other in the decision-making process.It should also be recognized that the production and selection of knowledge are related to power (Flyvbjerg 1998).The selection of perspectives also needs to be acknowledged as a part of the political process and included in the decision making.Decisions on sustainable development will in many cases involve trade-offs between several desirable outcomes in the spheres of the environment, society, and economy (World Commission on Environment and Development 1987).Consequently, the division of labor between researchers and decision makers needs to be clear and well considered not only to ensure that research is not used merely as justification for policy, but also to ensure that decision makers are supplied with a sound scientific basis for their decisions that details the consequences of different courses of action.Multicriteria assessments should be seen as one step in the decision-making process.Democratic ideals such as transparency, inclusion, and reflectivity must be prominent in the assessment process.

CONCLUSION
We observed the observers of organic food systems in an attempt to uncover the role of values in research assessments of these systems.Values are central for understanding how food systems are assessed.The five perspectives we discuss are based on different understandings of value, with focus placed on different aspects of the organic food system.A researcher's particular perspective enables him or her to observe specific parts of the food system, but it also results in blind spots for other parts and in different perspectives being incompatible.
For multicriteria assessments to function as decision support tools for the continual development of food system sustainability, they must be able to deal with multiple understandings of values.There is consequently a need for approaches that are able to combine and balance knowledge from different perspectives on both human and ecological systems.We propose four recommendations for coping with the challenges of working across perspectives in the assessments of food system sustainability: (1) elucidate values as a necessary foundation for research assessment across perspectives; (2) openly discuss the choice of perspective, because it is decisive; (3) formulate common goals that can be translated into the different perspectives; and (4) consider assessment of food system sustainability a learning process and design it as such.
Responses to this article can be read online at: http://www.ecologyandsociety.org/issues/responses.php/6347

Table 1 .
Main features of the five perspectives presented in the article.
However, a claim made from one perspective might have no validity in another.Elucidating the underlying values does not secure a good combination of assessments; it merely clarifies where and how perspectives are incompatible.http://www.ecologyandsociety.org/vol19/iss2/art46/Perspectives enable communication among scientists within perspectives and rigorous safeguarding and development of research methodologies