|Home | Archives | About | Login | Submissions | Notify | Contact | Search|
Copyright © 2001 by the author(s). Published here under license by The Resilience Alliance.
The following is the established format for referencing this article:
Bossel, H. 2001. Assessing viability and sustainability: a systems-based approach for deriving comprehensive indicator sets. Conservation Ecology 5(2): 12. [online] URL: http://www.consecol.org/vol5/iss2/art12/
To print the entire article (including any tables, figures, or appendices), use this link to the unified version of the article.
To print separate parts of the article, click in the frame containing that part (text, figure, table, or appendix) before choosing File, Print.
Synthesis, part of Special Feature on Integrated Natural Resource Management Assessing Viability and Sustainability: a Systems-based Approach for Deriving Comprehensive Indicator Sets Hartmut Bossel
Sustainable Systems Research
Performance assessment in holistic approaches such as integrated natural resource management has to deal with a complex set of interacting and self-organizing natural and human systems and agents, all pursuing their own "interests" while also contributing to the development of the total system. Performance indicators must therefore reflect the viability of essential component systems as well as their contributions to the viability and performance of other component systems and the total system under study. A systems-based derivation of a comprehensive set of performance indicators first requires the identification of essential component systems, their mutual (often hierarchical or reciprocal) relationships, and their contributions to the performance of other component systems and the total system. The second step consists of identifying the indicators that represent the viability states of the component systems and the contributions of these component systems to the performance of the total system. The search for performance indicators is guided by the realization that essential interests (orientations or orientors) of systems and actors are shaped by both their characteristic functions and the fundamental and general properties of their system environments (e.g., normal environmental state, scarcity of resources, variety, variability, change, other coexisting systems). To be viable, a system must devote an essential minimum amount of attention to satisfying the "basic orientors" that respond to the properties of its environment. This fact can be used to define comprehensive and system-specific sets of performance indicators that reflect all important concerns. Often, qualitative indicators and the study of qualitative systems are sufficient for reliable performance assessments. However, this approach can also be formalized for quantitative computer-assisted assessment. Examples are presented of indicator sets for the sustainable development of regions, including the computer-based, time-dependent assessment of system performance using time-series data. Because of its systems-theoretical foundation, this approach avoids the problems of incompleteness and double-counting common in ad hoc methods of indicator selection.
KEY WORDS: indicators of sustainability, integrated natural resources management, orientors, performance indicators, sustainability assessment, systems approach, viability.
Competent management of real-world problems must recognize and account for real-world system complexity. Holistic approaches such as integrated natural resource management (INRM) and system dynamics are therefore spreading in a wide range of disciplines (Holling 1978, Sterman 2000). A crucial aspect of this development is the search for appropriate indicators of system performance to condense vital information into a compact set of reliable signals for management. The need for comprehensive indicator sets that assess system viability, performance, and sustainability is especially urgent in management for sustainable development at all levels, from the global to the village.
The search for appropriate indicators of sustainable development has been going on for many years at many different levels of society: small communities, cities, regions, countries, and the world as a whole. There seems to be general agreement that it is impossible to define only a single indicator of sustainable development, and that a substantial number of indicators are necessary to capture all the important aspects of sustainable development in a particular application (Becker 1997, Hardi and Zdan 1997, Moldan and Billharz 1997, Meadows 1998, Bossel 1999). However, defining an appropriate set of indicators for sustainable development is a difficult task. If too few indicators are monitored, crucially important developments may escape attention. If a large number of indicators have to be examined, data acquisition and data analysis may become prohibitively expensive and time-consuming. Obviously, practical schemes cannot include indicators for everything. It is therefore essential to define a set of representative indicators that provide a comprehensive description, or as many as are essential, but no more. But what are the "essential" indicators?
In the past, this problem has been solved mostly by the intuitive assessment of experts familiar with a particular discipline such as economics, ecology, sociology, or engineering. Corresponding indicator sets are usually characterized by specific disciplinary biases, with gaping oversights in some critical areas and overly dense indicator specifications in others.
This paper describes a different system-based approach based on new developments in ecological and general systems theory (Müller and Leupelt 1998). Development is seen as a coevolutionary process involving interacting systems in a common environment in which each system follows its own path of self-organization in response to the challenges of its particular environmental circumstances. The complex web of interacting systems can then be broken down recursively into a network of individual systems, each of them determining its own fate and affecting that of one or more other systems. Indicators then have to be found to describe the performance of the individual system and its contribution to the performance of the other system(s). A first task in the search for a proper indicator set therefore consists of identifying the essential component systems and analyzing and defining the relevant system structure. Obviously, a considerable amount of aggregation and condensation is required at this point to keep the project within manageable dimensions.
The next step requires that essential indicators be found for the performance of each "affecting" system and its contribution to each "affected" system. Based on orientation theory (Bossel 1977, 1999), it is argued that the essential indicators are those that provide a complete description of the state of satisfaction of the fundamental interests of each system, i.e., its basic orientors: existence, effectiveness, freedom of action, security, adaptability, coexistence, and psychological needs (for humans and for systems with humans as components). This leads to the selection of a comprehensive, but minimum, set of indicators that provide information about all essential aspects of viability, sustainability, and performance. In this context, "viability" means the ability to survive and develop, and "performance" refers to functions extending beyond mere viability requirements.
Integrated (holistic) management typically has to deal with complex systems (Fig. 1). The viability and performance of the total system of concern (e.g., a watershed region) depends on the viability and performance of each of several component systems (e.g., settlements, farming, the forest, the water system). Each of these is again dependent on the viability and performance of several subsystems (e.g., the farming system depends on the livestock, cropland, and family subsystems).
For the following discussion, it is essential to remember that a system is more than its parts, i.e., its function is not merely the sum of the functions of its component systems. Rather, its function and viability emerge from the interactions of the component systems within the specific system structure. Component systems contribute to the whole by being individually viable and by contributing to the viability of other component systems and/or the total system. Individual component systems may be affected by other component systems as well as by system variables outside the boundaries of component systems.
Depending on the purpose and depth of a particular study, component systems may have to be further broken down into their component subsystems in a recursive manner to track the "viability chain" to some possible cause of performance deficiency in the total system. A system study is therefore essential. A key element of such a study is the recursively repeated relationship of affecting system vs. affected system. A system study will usually reveal many relationships and component systems that are important to the operation and viability of the total system even though they may not be immediately obvious. It may also reveal deficiencies on different levels of the system hierarchy, i.e., on different scales. It is therefore often necessary and entirely appropriate to develop indicator systems that consist of crucial indicators on very different scales, as in the Seattle example discussed below. Moreover, indicator sets should allow for dynamic change, because other indicators on different scales may become more appropriate during the course of dynamic changes in the system, in particular, during Schumpeter-Holling cycles.
Different kinds of relationships have to be considered when examining systems. All systems depend to some degree on the resource-providing and waste-absorbing capacities of their environments. Most systems interact with other systems that are essential to their viability. Many interactions are hierarchical, with systems controlling subsystems and subsystems contributing to the functioning of the systems that contribute to the functioning of a suprasystem, etc. The viability and performance of the total system depend on the viability and performance of many, but not necessarily all, of its subsystems.
At the level of the total system, the task is to find indicators that provide reliable information about its viability and performance, much as a thermometer is used to determine the health of a person (Fig. 2). In the case of the nested systems we usually encounter in the real world (Fig. 1), two sets of indicators are required for every system-subsystem relationship (Fig. 3). One set is required for determining subsystem viability and performance, whereas a second set is needed to assess the contribution of the subsystem to the viability and performance of the system as a whole. This duality of indicators is repeated at every level of the system hierarchy.
This paper deals with a practical procedure for finding appropriate indicators. It has several distinct steps:
At all levels of analysis, systems operate and must be viable in a particular, and variable, system environment. This system environment usually contains other systems that affect the particular system and/or are affected by it. For example, a human society depends on the natural environment and the systems in it for vital resources, which society in turn affects through its actions and waste streams.
The viability and performance of a particular system are therefore determined by (1) its characteristic system functions and (2) the characteristic properties of its particular system environment and of the systems in this environment. Viability implies that the two sets of characteristics must match in some way, because a system can survive and develop only in an environment to which it is adapted, or which has adapted to it. Fish have adapted their form and function for viability in their aquatic environment, but this particular adaptation is deadly in a terrestrial environment. The key to understanding viability and performance therefore lies in understanding the challenges of a particular environment.
There is obviously an immense variety of particular system environments, just as there is an immense variety of systems. However, the analysis is greatly simplified by the observation that all system environments have certain fundamental properties. System environments are characterized by these six fundamental environmental properties (Bossel 1998, 1999):
Applied in the context of human society, these fundamental environmental properties constrain development possibilities and limit management opportunities on all spatial scales. The normal environmental state in which humankind must develop is characterized by laws of nature and logic that cannot be broken and that limit the spectrum of possible physical, technical, and biological processes. The possibilities are further limited by the resource constraints of the global environment: available space; the waste-absorption capacity of soils, rivers, oceans, and atmosphere; the availability of renewable and nonrenewable resources; soil fertility; and climate. Some of these are state limitations (e.g., the amount of nonrenewable resources); others are rate limitations (e.g., the maximum rate of waste absorption). Variety is introduced by a wide spectrum of geological and climatic conditions, ecosystems, animal and plant species, cultures, languages, organizations, political systems, and technical solutions. Variability is introduced by unpredictable weather and other natural phenomena, random events, the spontaneous behavior of humans, or the population dynamics of organisms. Permanent changes in the normal environmental state result from the irreversibility of many developments; the coevolution of systems, organisms, and the environment; the modification of climate and the natural environment by humans, their technology, and their wastes; the depletion of resources; and changing human organizations, values, and aspirations. The many processes of the real world, which operate at different time scales ranging from seconds to centuries, often dictate the pace of development and determine the dynamics of change. Finally, human society is a composite of interacting systems that depend on the dynamic functions of more or less autonomous natural systems. Each of these systems is in some way dependent on, interacts with, or affects other systems. As the behavior of one system affects the "interests" of another, it often provokes an active response that negatively influences its own interests. This type of reaction is qualitatively different from the passive compliance of the environment.
These fundamental properties of the environment are each unique, i.e., no fundamental property can be expressed by any combination of other fundamental properties. If we want to describe a system's environment fully, we have to say something about each of these properties. What is the normal environment? What resources are available in the environment? How rare are they? How are they distributed? How much diversity and variety exist in the environment? How variable is it? What are the trends toward change in the environment? What other actor systems and interests have to be respected in one way or another?
A system can be viable and sustainable only if the constraints imposed by the fundamental environmental properties are respected. This requirement imposes certain orientations or interests on systems in the course of their self-organizing or coevolutionary development. Systems fail when they do not respect the constraints of their environments. This is true for species shaped by thousands of generations of evolution as well as for the technologies or organizations developed by humans. The terms "orientation" and "interest" as used here do not imply any consciousness on the part of the system. A system is said to have an interest or orientation if it can be observed to express a preference (e.g., a plant growing toward light).
The six fundamental environmental properties cause six respective interests or orientations in self-organizing systems such as the organisms and organizations that constitute and support human society) (Fig. 4). For such systems to be viable and sustainable, they must pay sufficient attention to and satisfy each of their basic orientors (Bossel 1977, 1999):
System orientation for viability and performance is a two-phase assessment process. Each of the basic orientors stands for a unique requirement. First, a certain minimum satisfaction must be obtained separately for each of the orientors to ensure viability. A deficit in even one of the orientors threatens the viability of the whole system. The system will have to focus its attention on this deficit. Compensation for deficits by the overfulfillment of other basic orientors is not possible. For example, a surplus of security cannot compensate for a deficit of freedom in a society. Second, only if the required minimum satisfaction of all the basic orientors is guaranteed, is it permissible to try to raise system performance by further improving the satisfaction of individual orientors.
Characteristic differences in the behavior ("life strategies") of organisms or of humans or human systems (organizations, political or cultural groups) can often be explained by differences in the relative importance attached to different orientors (i.e., the emphasis on freedom or security or effectiveness or adaptability) after the minimum requirements for all the basic orientors have been satisfied (Krebs and Bossel 1997). Such an emphasis may improve performance and provide a competitive advantage. Development, and in particular sustainable development, is therefore possible along many different paths, provided that minimum viability requirements are met.Using basic orientors to guide indicator selection
Because it is obviously better adapted to its environment, the system that most effectively satisfies all its basic orientors will have better fitness and better performance (Krebs and Bossel 1997). Assessment of orientor satisfaction therefore provides a measure of system viability and performance in a given environment. This can be done by identifying the indicators that provide information about how well each of the orientors is being fulfilled at a given time. In this context, the basic orientors can serve as a checklist for asking a set of questions whose answers provide an assessment of viability and system performance.
The application of these concepts is complicated by the fact that reality consists of webs of linked and interacting systems. The general relationship between two systems is always based on the fact that System A affects System B. Hierarchical dependencies (individuals support community schools, community schools supply qualified labor, qualified labor increases the gross national product, corresponding tax income pays for the health care system) and feedback loops (better health care benefits individual families) are possible. To assess sustainability, we have to find indicators for each essential system within the total system in each orientor category that can answer two sets of questions. First, how viable is each system, i.e., how satisfied is each basic orientor of that system? Second, how does a given system contribute to the viability (the basic orientors) of another system or the total system? For chains of dependencies, this scheme must be applied in a recursive fashion (Fig. 5). We find the same duality of indicators and assessment in all walks of life: the farmer is interested in the health of his cow and the amount of milk it produces for him, the forester checks the vitality of the forest and its timber production, the operator of a power plant keeps pressures and temperatures within a safe range and measures power output, the pilot has instruments that help him fly safely and indicate the direction to and distance from his destination, a minister of economics checks the health of the economy and its contribution to the GNP and government revenues.
The first set of questions that define relevant indicators of sustainability concerns the viability and performance of the (sub)system under consideration (Bossel 1999):
The second set of questions defines the indicators that can be used to assess the contributions of a given system to the viability and sustainability of an affected system. These questions examine the contribution of the given system to the satisfaction of basic orientor x (existence, effectiveness, freedom of action, security, adaptability, coexistence, psychological needs) of the affected system (see Bossel 1999).
The number of indicators to be observed depends on the number of systems recognized as essential for the viability and performance of the total system and on the level of permissible aggregation. Using a functional approach, the essential component systems of human development are commonly distinguished by the capital assets (state variables, stocks) that supply human livelihoods. The widely used "livelihoods framework" recognizes five essential capitals: human, natural, financial, social, and physical (Carney 1998, Bebbington 1999, Campbell et al. 2001). Other authors add "organizational capital" as a sixth essential capital that has a particular relevance in development problems (Bossel 1998, 1999). The distinction between these component systems is made on system-functional grounds (in Bossel 1998) and does not coincide with traditional disciplinary boundaries. Each component system is defined by its relative autonomy and the much greater connectivity of variables within this system than across its boundaries (more details concerning these system aspects are available in Bossel 1998:100 ff).
For the sustainable development of human society on any spatial scale from village to global, it is necessary to observe indicators that represent all six essential systems or capitals. For many purposes, it is possible to aggregate to three essential subsystems (Fig. 6): the human system (the human, social, and organizational aspects), the support system (the physical and financial aspects, i.e., the infrastructure and economy), and the natural system (the environment and resources). To determine how sustainable each of these systems is and its respective contribution to the total system requires 7 x 3 x 2 = 42 indicators. This number is probably near the upper limit for practical applications.
A detailed analysis based on this scheme will, in most cases, produce a large number of component systems, long "viability chains" of nested systems, and many potential indicators. A typical example is presented in Fig. 5. Furthermore, there will generally be several, perhaps many, appropriate indicators that correspond to each of the assessment questions. It is therefore essential to condense the systems analysis and the indicator set as much as possible without losing essential information. There are several ways to do this:
Note that each indicator has been chosen to represent only one particular aspect of orientor assessment. It must therefore be understood as representing certain orientor-relevant trends and judged under that particular aspect only, not under others to which it may also be related. Note further that the application of these rules for generating an efficient indicator set may result in a collection of indicators that represent very different scales (e.g., local waste disposal, the national judicial system). The important criterion is whether the indicator correctly represents current system viability.
Even with a solid scientific approach based on physical facts as well as systems theory and analysis, indicator sets cannot be defined without a significant amount of subjective choice. We therefore should not be surprised if researchers using the same data and scientific method produce different indicator sets. Although on the surface this may appear to be another of those cases where scientists cannot agree, these indicator sets are far from arbitrary selections. If indicators have been selected to represent basic orientor satisfaction of essential systems, sustainability assessments should produce comparable results even when the indicator sets are completely different.
It is useful to recall the areas in which subjective choice is inevitable in the process of determining an indicator set. Some of this subjectivity is due to differences in objective knowledge, whereas some is the result of divergent normative concepts, i.e., ethical preferences.
Ethics and normative preferences thus enter into the selection of indicators in at least three ways: (1) in the selection of the systems to be included in the assessment, including ourselves, our descendants (and even ancestors), and other species, organisms, and systems (human, political, cultural, technical, and ecological); (2) in the relative weights (importance) we assign to ourselves and to all the other systems; and (3) in the relative emphasis we put on the different interests corresponding to the basic orientors, particularly our own.
Although subjective qualitative assessment of viability and performance using the basic orientor scheme suffices in many cases, a more formalized quantitative procedure is required if results are to be reproducible or assessments compared. Quantification of the indicator/orientor relationships (the impact functions) is essential, particularly if the assessment procedure is computerized and a time series of indicators is used to produce a dynamic assessment. Impact functions capture the subjective assessment of how a particular indicator state contributes to the orientor satisfaction of a system. Using these functions, a set of indicators reflecting the time-dependent system state can be mapped on the basic orientors to produce information about the dynamics of viability and performance, as described below.
Relative (nondimensional) quantification of orientor satisfaction is simple if two numerical indicators can be meaningfully related. To stay viable, a system must be able to respond or adapt to threats before they do serious damage. This suggests that it is advisable to concentrate on indicators that relate the rates of viability threats (threats to a system's basic orientors) to the rates of evasive system response, or their respective inverse, the respite time to the response time (Biesiot 1997, Bossel 1999). These two quantitative measures will often be available from system observations. They can be combined in a nondimensional indicator by taking the ratio of the rate of system response to the rate of system or environmental change caused by a particular threat. If this measure is greater than one, the system is viable (with respect to that particular orientor); if it is less than one, the system's viability is threatened. Viability means that the system can cope with challenges and will not be overwhelmed by them, i.e., that its responses can outpace the threats to it. For example, the regeneration rate vs. the harvesting rate can be used to measure the "existence" of a basic orientor such as a vital resource.
As a general rule, indicators of sustainable development will be region-specific, with "region" defined using an ecoregional scale of analysis. These indicators will express the specific characteristics of the systems under study and their particular environments. Comprehensive indicator sets for sustainability assessment have been developed for many cities and regions (Hardi and Zdan 1997). Examples of recent applications of orientation theory to the development of comprehensive indicator sets of sustainable development are found in Bossel (1999) and Ömer (2000) for Austria, Mothibi (1999) for Botswana, and Peet and Bossel (2000) for New Zealand. In earlier applications of orientation theory, indicator sets were derived for the energy supply system, the health care system, information and communication technologies, and agricultural production, among others (Bossel et al. 1989).
A famous and often cited example of indicators of sustainable development is the set derived for the city of Seattle. This set was the result of a long process of discussion and development that involved intensive citizen participation (Hardi and Zdan 1997). The Seattle indicators are all quantitative, on a local scale, and based on locally available data. Their relative changes are reported at regular intervals, pinpointing corresponding improvements or declines in the viability and performance of the Seattle system.
This set of indicators (short names only) is presented in an orientor-based scheme (Fig. 7) that corresponds to the three essential, functionally defined subsystems, i.e., the human, support, and natural systems seen in Fig. 6. In Fig. 7, the three sets of indicators on the left provide information about the viability and performance of each individual subsystem. Consider, for example, the indicators for basic orientor adaptability (A), in which the indicator "adult literacy" is used as a measure of the adaptability of "human capital" because it correlates with level of education and the ability to adapt to new challenges. The indicator "vehicle miles traveled and fuel consumption" is used to represent the adaptability of "built capital," where the dominance of road transportation indicates a limited ability to adapt the support system to, say, a fuel shortage. The indicator "biodiversity" measures the adaptability of "natural capital," i.e., species and genetic diversity, when it comes to maintaining the functions of the natural environment under changing conditions.
The three sets of indicators on the right express the contributions of each of the three subsystems to the viability and performance of the total system, i.e., the anthroposphere composed of the three essential subsystems: human, support, and natural. For example, the indicator "youth involvement in community service" is chosen to reflect the contribution of the human system to the adaptability of the total system, and a more caring, active, flexible, and future-oriented attitude can be expected if this value is high. The indicator "library and community center usage" measures the quality and quantity of the contribution of the support system to the adaptability of the total system, i.e., the potential for ongoing education, innovation, and social (re)organization. The indicator "wetlands" is a measure of the contribution of the natural system to the adaptability of the total system: wild, uncultivated areas represent adaptive potential for the total system and for its human, built, and natural capitals.
There is excellent correspondence between the Seattle set and the orientor-based scheme, although the original indicators were derived without reference to orientation theory. This seems to indicate that the Seattle indicators are indeed comprehensive, covering all important aspects of basic orientor fulfillment for viability and sustainability.Formalized quantitative performance assessment
In Bossel (1999), the formalized method of dynamic orientor assessment is demonstrated using empirical time-series data for selected indicators from the database of the Worldwatch Institute (Worldwatch 1999) for 1950–2000. A set of 21 indicators is used (seven basic orientors and three component systems: human, support, and natural). The 21 indicators from the Worldwatch database were chosen for their ability to provide answers to the corresponding 21 orientor assessment questions.
What each of the indicators means for the respective orientor must be expressed in terms of the impact function. Except for simple cases and Biesiot-type indicators, there is currently no method to objectively measure indicator impact on orientor satisfaction (e.g., how does a particular groundwater level contribute to the "security" of the "natural system?"). These impact functions must therefore be generated by subjective assessments. To capture the spectrum of subjectivity, it will often be essential to incorporate the viewpoints of different stakeholders in competing assessments and to compare the results, which will often not differ substantially.
Orientor star or "radar" diagrams can be used to reflect orientor satisfaction for each of the component systems and the total system (see Fig. 8 for orientor stars for the natural system). This time sequence clearly brings out the dynamics of stresses and threats to a system and makes it possible to trace their origins. In another approach (Peet and Bossel 2000) that is particularly well-suited for spreadsheet assessments, the degree of basic orientor satisfaction is color-coded: green for good or excellent, yellow for satisfactory, red for unsatisfactory or bad.
The systematic and theory-based nature of the orientor method of determining indicators is important. We are not simply asking people to find and agree on a set of indicators; we are asking them to find answers (indicators) to very specific questions concerning all the vital aspects of viability and system performance, i.e., the basic orientors. In this structured approach based on systems theory and empirical evidence, we can be reasonably confident of obtaining a comprehensive set of indicators that cover all important aspects of systems viability and performance. The method avoids both unnecessary "bunching" of redundant indicators in some areas and gaping holes resulting from oversight and neglect in others.
Although it is advisable to choose indicators that allow unambiguous quantification, and hence comparison with conditions at other points in time or in other regions, this is not a necessity with the method proposed here. The important point is that the indicators chosen provide us with reliable answers to the different orientor assessment questions. If satisfactory qualitative answers are obtained in all categories for all the subsystems and the total system, we can conclude that the system is (currently) viable and performing satisfactorily. Even if only one of the categories is in an unsatisfactory state, this could indicate the existence of a problem with the potential to endanger viability and development.
From the many applications to date, it can be concluded that a systems approach using orientor concepts can be a very useful tool, not only for defining comprehensive indicator sets for system viability and performance, but also for checking existing indicator sets for completeness (in the mathematical sense of covering all essential aspects) and possible redundancy. It provides systematic guidance for a comprehensive indicator search, thus minimizing the danger of overlooking essential areas or overemphasizing others.
In a broader context, regarding indicators as reflections of the fundamental interests (basic orientors) of all participants and affected systems puts a solid foundation under the search for indicator sets and removes much of the arbitrariness implicit in current and proposed indicator sets. It turns the focus from an uncertain ad hoc search and bargaining process to a much more systematic procedure with a clear goal: to find indicators that represent all the important aspects of viability, sustainability, and performance.
Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a comment, follow this link. To read comments already accepted, follow this link.
The paper was prepared for the Workshop on Integrated Natural Resource Management in the CGIAR, 21-25 August 2000, Penang, Malaysia.
Bebbington, A. 1999. Capitals and capabilities: a framework for analyzing peasant viability, rural livelihoods and poverty. World Development 27:2021-2044.
Becker, B. 1997. Sustainability assessment: a review of values, concepts, and methodological approaches. Consultative Group on International Agricultural Research, The World Bank, Washington, D.C., USA.
Biesiot, W. 1997. Respite time and response time. Balaton Bulletin (Fall 1997):12-13.
Bossel, H. 1977. Orientors of non-routine behavior. Pages 227-265 in H. Bossel, editor. Concepts and tools of computer-assisted policy analysis. Birkhäuser, Basel, Switzerland.
Bossel, H. 1998. Earth at a crossroads—paths to a sustainable future. Cambridge University Press, Cambridge, UK.
Bossel, H. 1999. Indicators for sustainable development: theory, method, applications; a report to the Balaton Group. International Institute for Sustainable Development, Winnipeg, Manitoba, Canada. Available online at http://iisd.ca/about/prodcat/perfrep.htm#balaton.
Bossel, H., B. R. Hornung, and K.-F. Müller-Reissmann. 1989. Wissensdynamik mit DEDUC—Grundlagen und Methoden dynamischer Wissensverarbeitung: Wirkungsanalyse, Folgenabschätzung und Konsequenzenbewertung. Vieweg, Braunschweig, Germany.
Campbell, B., J. Sayer, P. Frost, S. Vermeulen, M. Ruiz Pérez, A. Cunningham, R. Prabhu, S. Waddington, and E. Schuma. 2001. Evaluating the impacts of integrated natural resource management (INRM) research. INRM/CGIAR Workshop Proceedings, CGIAR, Penang, Malaysia, in press.
Carney, D. 1998. Sustainable rural livelihoods: what contribution can we make? Department for International Development, London, UK.
Hardi, P., and T. Zdan, editors. 1997. Assessing sustainable development: principles in practice. International Institute for Sustainable Development, Winnipeg, Manitoba, Canada.
Holling, C. S., editor. 1978. Adaptive environmental assessment and management. John Wiley, Chichester, UK.
Krebs, F., and H. Bossel. 1997. Emergent value orientation in self-organization of an animat. Ecological Modelling 96:143-164.
Meadows, D. H. 1998. Indicators and information systems for sustainable development: a report to the Balaton Group. Sustainability Institute, Hartland Four Corners, Vermont, USA.
Moldan, B., and S. Billharz, editors. 1997. Sustainability indicators—a report on the Project on Indicators of Sustainable Development (SCOPE 58). John Wiley, Chichester, UK.
Mothibi, J. 1999. A pragmatic-evolutionary based systems approach for science and technology policy formulation for sustainable development, with an application to Botswana. Dissertation, University of Cape Town, Cape Town, South Africa.
Müller, F., and M. Leupelt, editors. 1998. Eco targets, goal functions, and orientors. Springer, NewYork, New York, USA.
Ömer, B. 2000. Ökologische Leitplanken einer Nachhaltigen Entwicklung. Austrian Institute for Sustainable Development (OINE), Vienna, Austria.
Peet, J., and H. Bossel. 2000. An ethics-based systems approach to indicators of sustainable development. International Journal of Sustainable Development 3:221-238.
Sterman, J. D. 2000. Business dynamics—systems thinking and modeling for a complex world. Irwin McGraw-Hill, Boston, Massachusetts, USA.
Worldwatch Institute. 1999. Worldwatch database disk. Worldwatch Institute, Washington, D.C., USA.
Address of Correspondent:
Sustainable Systems Research
D34289 Zierenberg, Germany
|Home | Archives | About | Login | Submissions | Notify | Contact | Search|