Collaborative adaptive management (CAM) has shown great promise for addressing complex natural resource management problems but the transition from promising theory to effective practice has been challenging (Mitroff and Featheringham 1974, Catton 1989, George 1995, Miller 1999, Susskind et al. 2010, Spradlin 2012, Chevallier 2016). The difficulty in implementation is largely because of the great complexity of the problem situations considered suitable for CAM. Such situations do not consist of isolated decision processes through which a single actor (or a limited set of actors) addresses a single problem. Instead, they are evolving situations in which many interconnected actors are engaged simultaneously in many overlapping and interrelated but commonly poorly interconnected decision processes that address different sets of problematic conditions at various organizational and governmental levels, and that are constantly being adjusted in response to events. In short, a common characteristic is that both the systems being managed and the management systems themselves are complex adaptive systems. In addition, such systems continually change so interventions meant to influence them must also change continually, and practitioners must frequently adapt and redevelop their plans in response to systems change (van Bokhoven et al. 2003, Shiell et al. 2008, Sugihara et al. 2012).
The complexity of the management system presents particular challenges for practitioners responsible for project planning. Development of an effective action plan requires identification of potential interventions that are both likely to have an impact and be practical in that particular social-political context. Because human behavior change is almost always needed to resolve complex natural resource management problems, human behavior needs to be an intervention target. This is not easy for most natural resource management practitioners; the majority come from biophysical science backgrounds with training in how to work with those elements of the system of interest, i.e., the ecological parts of social-ecological systems. They typically lack background and experience in addressing human behavior, i.e., the social parts of these systems, and few resource management professionals have access to training in how to understand and influence the problem-relevant behavior of individuals and groups. There is a need to translate promising research findings into practical guidelines that are written and organized in ways that make it relatively simple for practitioners to integrate into their work. For CAM concepts to have real impact in the world, practitioners have to be both willing and able to apply those concepts in real-world settings.
The results of descriptive-analytical research (Weik and Lang 2016) associated with projects is not typically available to participating resource management practitioners early enough to influence project planning. When available, the information is often not in a form that is usable or useful to those responsible for actions that have a direct impact on the problem situation. The academic literature currently provides little guidance for practitioners, and there has been only limited translation of theory-driven guidelines into practical and readily useable recommendations for project managers (Allen and Garmestani 2015). This guidance gap has not gone unnoticed. For example, it is been noted that existing human dimensions research has not been fully utilized in recreational fisheries management, and this utilization gap appears to be widening (Hunt et al. 2013).
Similar translation and implementation challenges are shared by many applied social science and transdisciplinary fields of research that focus on interventions aimed at changing individual and collective human behavior. It has been acknowledged that standard approaches to project planning and management are not well-suited to situations characterized by uncertainty, complexity, and the need for innovation (Kapsali 2013, Mahmoud-Jouini et al. 2016). In response, new approaches and tools have been developed in several applied fields for strategic planning in such situations. I suggest that these can be of value to natural resource management practice.
In a recent paper (Beratan 2019), I discussed how natural resource managers setting up a CAM project can enhance that project’s likelihood of success by committing more time and attention to the very first steps in getting a project underway. That paper introduced an exploratory problem assessment (EA) approach designed to help managers learn about a problem situation quickly to usefully inform project planning. Exploratory problem assessment assists managers to quickly obtain information needed to develop an appropriate initial problem definition without antagonizing potential project participants, and then integrate and present that information in a readily understandable conceptual model that is useful for strategic planning.
In this paper, I delve deeper into one aspect of the EA process, i.e., how to organize information about human behavior that is specifically relevant to the problem situation so that it can be integrated into the EA conceptual model and inform project planning. I base my approach on insights emerging from applied social science fields that focus on fostering changes in human behavior, including public health and health promotion, social work, sustainability science, and transition management, along with methods and approaches from intervention design and social marketing. Areas of overlap among these fields provide opportunities for exchanging practical lessons for achieving directed system change.
I focus particularly on the subfield of intervention research, which was developed to assist with program development in applied social science fields. I suggest that intervention design strategies and tools can help natural resource management practitioners increase their practical understanding of the specific human factors shaping their particular problematic situation, thereby enabling more realistic and effective change theories that can improve project outcomes. They can also assist in developing leading indicator metrics capable of tracking a program’s progress in a time frame permissive of adjusting the management plan.
“The planning process is about changing the expected future by changing the links between the present and the future” (Abbott 2005:245).
The purpose of a collaborative adaptive management process is directed system change, i.e., altering the dynamics of a particular social-ecological system in ways that shift it away from a problematic trajectory toward a more desirable one. Accomplishing this requires that individuals and organizations take actions that are capable of altering one or more factors shaping the system’s dynamic behavior. The design of an effective response to a problematic situation thus requires an accurate general understanding of relevant determinants of change that explain how a particular action or event would cause change (van Bokhoven et al. 2003, Craig et al. 2008, Fraser and Galinsky 2010, Funnell and Rogers 2011). In other words, it requires a change model or theory of change (Maxwell 2004, Patton 2011, Gilgun and Sands 2012).
The term “theory of change” is used to describe the sequence of outcomes expected to occur as a result of an intervention (Weiss 1995). Theories of change are derived from assumptions about how the system of interest responds to interventions, actions, or perturbations (Margoluis et al. 2013, Qiu et al. 2018). Logic models and results chains are the most common representations of change theories (Margoluis et al. 2013; Fig. 1). Logic models are required by many funding agencies to show the relationships among an applicant’s resources, planned activities, and expected results. However, logic models are not particularly appropriate for complex problem situations because they generally consider only a limited set of factors while leaving out important contextual information, and they do not adequately show how specific actions are expected to lead to expected outcomes (Rogers 2008, Margoluis et al. 2013). Other change theory representations such as results chains have been developed to address these limitations. A results chain is a diagram depicting a series of “if...then” relationships, the assumed causal linkages between an intervention and the expected impacts, through a series of expected intermediate results (Foundations of Success 2009). Results chains have been used effectively in natural resource management projects. However, effective results chains require significant up-front work to identify targets, threats, and driving factors and to figure out appropriate intervention strategies (Margoluis et al. 2013). Practitioners need assistance with this up-front work; intervention design strategies offer some practical guidance.
Human behavior is a major (if not “the” major) determinant shaping change in social-ecological systems, and so changes in human behavior are almost always needed to direct system change toward a particular trajectory. This means that change theories for CAM projects need to focus on factors shaping the behavior of relevant human actors and how those link to the resource system.
Insights into how to better integrate human behavior into causal models can be drawn from research and practice in applied social science fields such as social work, public health, and health promotion that focus on fostering changes in human behavior. Researchers in these fields recognized by the early 1990s that problems such as drug addiction have multiple causes, pathways, and correlates, and so efforts to address such problems must involve multiple program components that address dynamic risk factors across multiple groups and levels (Cázares 1994, Hawe 2015). The subfield of intervention research was developed to explore ways to tease out the complex factors at multiple levels that shape specific behaviors, and to learn how practical actions with the potential to foster positive changes can be identified.
An intervention has been defined as “a specified strategy or set of strategies designed to change the knowledge, perceptions, skills, and/or behavior of individuals, groups, or organizations, with the goal of improving...outcomes” (Clauser et al. 2012:127). More generally, interventions can be thought of as efforts to introduce planned change into social systems (Schensul 2009) or as purposively implemented change strategies (Fraser and Galinsky 2010). In this sense, rather than being simply a package of actions, an intervention is a critical event in a system’s history from which new structures of interaction and new shared meanings evolve (Hawe et al. 2009).
Until recently, the term “complex intervention” was applied to any intervention built up from a number of components that may act interdependently (MRC 2000). Many intervention design and evaluation researchers have shifted toward thinking of complexity as a property of the systems being targeted by the intervention rather than of the intervention itself (Shiell et al. 2008). Informed by complexity science and ecosystem studies, this new approach recognizes that interventions in complex adaptive systems need to target multiple levels because determinants at multiple scalar, temporal, and governance levels interact to influence human behavior and health outcomes (Eoyang 2006, Hawe et al. 2009, Clauser et al. 2012, Weiner et al. 2012). In such a context, simple consideration of the merits of individual interventions is insufficient. Instead, sound causal reasoning about the likely interactions among the combined interventions is needed (Weiner et al. 2012). Given the difficulty in determining causality in complex adaptive systems, the intervention design process needs to be iterative and adaptive (Gilgun and Sands 2012).
Intervention design usually involves consideration of three basic elements: interventions, targets, and settings. The intervention is the set of actions to be taken. An intervention target is a determinant (causal factor) that the intervention is intended to modify. The targets of interventions are human behaviors, characterized in terms of scope or level of influence. Levels of influence shaping human behavior include intrapersonal (biological, psychological), interpersonal (social, cultural), organizational, community, and policy (e.g., McLeroy et al. 1988). Bronfenbrenner’s classic model of social systems describes several subsystems among levels (Table 1): a microsystem (the immediate environment such as the home, school, and neighborhood), the mesosystem (the interrelationships between various settings within the microsystem), the exosystem (mass media, extended family, social services, etc.), and the macrosystem (cultural ideologies and attitudes; Bronfenbrenner 1979, 1986, Bronfenbrenner and Crouter 1983). Corresponding levels can be identified within ecological systems.
Behaviors at one level can be influenced by factors at multiple levels, so a critical part of multilevel intervention work is consideration of the levels that are to be targeted by the intervention (Natasi and Hitchcock 2009). Environmental or policy factors at the organizational, community, state, and national levels are often important to specify as mediators and moderators of behavioral or system-level interventions (Clauser et al. 2012). An intervention setting is the social system in which the intervention target is reached; like targets, intervention settings can range in scope (Weiner et al. 2012).
The basic organizing structure of intervention design can be summarized as: intervening actors take actions intended to cause modification of the behavior of targeted actors leading to improvement in the conditions of interest. In other words, the actions making up the intervention are intended to cause (e.g, enable, encourage, trigger) a shift in the particular behaviors of individuals, groups, and organizations that are hypothesized to drive the key processes shaping the emergent conditions of interest.
Actors are social entities (individuals, groups, organizations, communities) that have some ability to influence the system of interest, either directly or indirectly. Actors can be considered a subset of stakeholders, a category that also includes social entities that have an interest in the system but lack ability to influence the system. For CAM applications, I recognize two basic categories of actors; intervening actors, i.e., individuals, groups, and organizations who need to be actively involved in planning and/or implementing the intervention, and targeted actors, i.e., individuals, groups, and organizations whose behavior is the focus of the intervention (Table 2). Because complex, multilevel interventions include multiple actions, there is likely to be considerable overlap in these two categories, with a given actor being among the targets of one action while also needing to be involved in implementing another action. It is important to keep in mind that the goals of an intervention project will not exactly match those of key actors, and the goals of organizational actors will not exactly match those of the individuals representing them. The intervention design must balance among various and potentially competing goals to obtain and maintain actors’ productive engagement with the intervention project.
Three basic categories of roles are available for intervening actors in CAM applications. Project planners are actively engaged in project initiation and planning and will help design the intervention. Project implementers carry out the actions specified in the intervention’s action plan. Project influencers influence the planning process through provision of important guidance and oversight or through their likely response to the planned action, but do not play an active role in intervention design or implementation.
Because individual and collective human behavior is the most important factor shaping system dynamics in CAM and other integrative research applications, the determinants (causal factors) of most concern to intervention designers should be factors that influence (mediate or moderate) the behavior of targeted individuals, groups, and organizations, and the paths through which those behaviors shape the system’s emergent structure and dynamics.
Individual and collective behaviors are influenced by a very wide range of determinants. Among the factors reported in the literature as needing to be taken into account when designing interventions are:
Plummer (2009) presented an analytical framework that can assist in exploring the social processes involved in adaptive comanagement. Two categories of determinant variables are distinguished: exogenous, originating outside the CAM actor network, and endogenous, originating within the CAM actor network (Table 3). Endogenous variables can be altered by actions taken by members of the network and thus are particularly important for intervention design. Exogenous variables influence the real or perceived circumstances of the actors involved and thus provide important context for understanding the dynamic interactions among endogenous variables and the consequent impacts on the system. These variables appear to be especially powerful in catalyzing the process early on (Pinkerton 1989, Plummer and FitzGibbon 2004). Intervention designers need to distinguish between these two types of factors so that they can focus resources on the influenceable determinants and treat unalterable contextual determinants as design constraints. For example, factors such as regional weather patterns and federal regulations can constrain the options available to regional and local actors. Leverage points to consider as intervention targets would be endogenous variables for which a relatively small action (input) is likely to produce a disproportionately large response (output).
The existence and importance of particular determinant-behavior relationships must be inferred on the basis of assumptions, as are the expected impacts of those behaviors on the system. Critical assumptions may be explicit or implicit; an important task for intervention researchers in a given situation is to “daylight” actors’ implicit assumptions so that they can be explored and tested. It is useful to differentiate between necessary, sufficient, and contributory causes (Rouse and Serban 2011). In addition, it is useful to distinguish between factors that can and cannot be influenced through actions within the scope of the project.
Some methods for interrogation of intervention logic have been developed (Hawe et al. 2009). For example, intervention mapping was developed as a systematic method of linking problem analysis, program design and evaluation, and integrating theories and scientific evidence in the design in health promotion interventions (Bartholomew et al. 1998, 2001). Hawe et al. (2009:269) observed that intervention mapping and other similar approaches are potentially productive but may simply represent “more meticulous ways of doing the same thing” by focusing over simplistically on the “package” of activities and/or their educational messages, just like conventional thinking about preventative interventions. As an alternative, they suggested focusing on the dynamic properties of the context into which the intervention is introduced, including the nature and diversity of activity settings in the system and the strategic positions occupied by key actors. Their approach draws attention to the networks of social relationships that make up the system, the variety of roles that exist or can be created within those networks, and the meaning that different actors might attach to an intervention event. This information can be more clearly and usefully presented in graphic form as the diagrammatic results chain than in a textual description.
These various approaches and methods highlight important considerations and show promise as tools for gaining useful understandings of a range of complex problems and potential solutions. However, they have had only limited influence on real-world applications beyond their original social work and health promotion contexts. Implementation challenges are greater for sustainability and natural resource management applications, which tend to have vaguer and more contentious goals, more complex and subtle causal relationships with more cross-scale and cross-sector links, greater misalignment of scale and timeframe between key dynamics and available governance mechanisms, and a kaleidoscope of influences from a wide array of past and current interactions that a new program must find a place within.
I have adapted the concepts described into an organizing structure for deriving and displaying contextualized causal relationships within a problem situation using readily understandable conceptual models of the problem space. These diagrams show relevant system elements (polygons) and critical linking relationships (arrows) that correspond to the categories listed in Tables 2 and 3. Categories of elements and linkages are differentiated by differences in object shape, line style/width, and color of elements and arrows (Fig. 2). These graphic elements function as a form of grammar that can be used to capture and convey information about complex causal relationships (Beratan 2019). These diagrams can be considered a variant of results chains, distinguished by the more structured graphic grammar used to enhance interpretability. The resulting conceptual models, such as the example described below, are designed to be of practical use for the selection and design of interventions in CAM projects.
An example application of the organizing structure described comes from a 2011 land-use planning effort in eastern North Carolina. The Governor’s Land Compatibility Task Force was charged with developing recommendations for “maintaining and enhancing the military presence in North Carolina through preservation of land uses that are compatible with the military mission and that also preserve and sustain economic development and natural and cultural resources” (NCDENR 2012:1). In other words, they conducted strategic planning to design a collaborative, integrative, and complex intervention. The task force comprised representatives of the major actors with responsibility for preservation and conservation of undeveloped land (Table 4) including state agencies, local governments, and military installations. The military is one of the two top economic sectors in the state, with the other being agribusiness (food, fiber, and forestry industries). The state’s natural systems are the basis for economically important tourism/outdoor recreation and fisheries industries and provide irreplaceable ecosystem services. Development is a common threat to these three economic engines; development pressure has been growing in eastern North Carolina in recent years largely in response to expansion of the military missions of Camp Lejeune and Fort Bragg,
As is typical of sustainability projects, this planning effort was just one small step in a larger, longer term process. This was clearly understood by the process participants and in fact the particular goal of this process was to set the stage for and shape a longer term process that would directly address the problem. Also typical was the relatively short time period allotted for this planning process.
I assisted the project manager and served as technical writer of the final report, functioning in the roles of knowledge intermediary and process facilitator. I applied the intervention design organizing structure summarized in Table 1 and Figure 2 to develop a conceptual system model (Appendix 1) to assist with strategic planning. I obtained information about the project from three main sources: unpublished written reports, informal interviews with project leaders, and task force meetings during which information was exchanged among participants.
In my note taking, I followed the organizing structure presented in this and my previous paper (Beratan 2019) to represent assumed or observed causal linkages in the form of short graphic sentences. These formed the building blocks for conceptual models of the group’s collective change theories. Interim and final diagrams were checked for reasonableness and relevance by having them assessed by the project leader and two other leading participants. I used the conceptual model summarized in the model to help identify feasible mutual benefits options, suggest appropriate metrics for tracking progress in implementation, and to guide preparation of the final report submitted to the governor at the conclusion of the intervention design process. Companion graphic representations of aspects of the recommended plan of action were also created to assist with outreach, implementation, and evaluation when it was time for the process to move forward.
Two major benefits were realized through this intervention design process. First, the organizing structure guided my information search, serving as a filter that permitted me to sort through a large amount of information and identify the most important elements and relationships in a limited time period. This allowed me to integrate relevant information from all sources at a level of detail that was appropriate to the project and to focus information I provided to the task force on elements critical to the project. Second, the process was useful in helping me organize the final report to make a strong case for the interventions selected and to lay out a clear path forward from this planning effort to implementation.
The most valuable outcome of the project as a whole was more open communication among the project partners, which led to a greater understanding of each other’s missions and constraints. This allowed the task force to successfully developed a joint-benefits strategy and implementation plan for state-wide prioritization of land-use protection targets that was strongly supported by primary decision makers in all of the partner organizations. The final report outlining the plan, the first of its kind in the United States, was presented to Governor Beverly Perdue in May 2012. Unfortunately, the plan has not yet been adopted by the governor and general assembly as a consequence of changes in political priorities following the 2010 and 2012 elections. Despite the disappointing political outcome of the process, the project succeeded in achieving enhanced cooperation among the partners on this issue for several years.
The exploratory problem assessment approach presented here and in Beratan (2019) was developed from a transformational perspective with the practical goal of assisting project managers to increase the effectiveness of their CAM project planning. The intervention design concepts focus on integrating specific individual and collective human behaviors into change theories in ways that are directly useful for project planning and evaluation. The approach begins with the recognition that the behavior of specific actors who can affect outcomes needs to be factored into change theories. Based on my experience, the particular network of relationships between people, power, and influence in a given problem situation are the critical determinants of what gets done and how it gets done.
The multilevel approach presented provides an integrative modeling process that can be done by or for a CAM project manager with a minimum of facilitation. It provides information directly relevant to project planning: conceptual models based on this integrative approach can help natural resource managers gain a more complete overview of the complex and dynamic causal pathways responsible for problematic conditions. These models can help daylight assumptions and highlight potential leverage points. In addition, these causal models can guide the development of evaluation metrics that can usefully inform partners, stakeholders, and funders about the project’s progress and impacts quickly enough to permit modification of the plan if needed.
The EA process can serve as an aide to sensemaking (Weick et al. 2005), which “refers to how we structure the unknown so as to be able to act upon it” (Ancona 2012:3). The EA process can enhance articulation (Benner 1994, Winter 1987), a particularly important aspect of sensemaking, which is defined as “the social process by which tacit knowledge is made more explicit or usable” (Weick et al. 2005:413).
The EA process is highly flexible and can be used in many ways. For example, in my experience it is rare for the complete conceptual model to be widely circulated. Instead, I have found that project leaders make use of it for their strategic planning and then request specific communication products based on the diagram as needed to meet the specific needs of a particular audience and purpose.
The academic intervention terminology adopted is technical and can be off-putting for stakeholders. Nonetheless, the basic concepts are readily understandable to nonspecialists both because humans are predisposed to perceive causal relationships and, as noted in Beratan (2019), a diagrammatic presentation using information design principles can readily convey quite complex relationships. A conceptual model constructed using the exploratory problem assessment’s graphic grammar can be particularly effective in communicating causal hypotheses to diverse stakeholders and can serve as a change theory, guiding project planning and evaluation in complex problem situations.
In conclusion, I show how CAM practice can benefit from insights derived from intervention research and other tools from the applied social sciences. I focus particularly on how to organize information about human factors that is specifically relevant to the problem situation so that it can be integrated into the exploratory problem assessment conceptual model. The resulting graphic representation, codified in the graphic grammar presented here and in Beratan (2019), is designed to be a practical and useful tool that can help diverse participants share information and gain a more complete and shared understanding of the problem situation. The EA approach affords the positive features of results chains while further enhancing the usability and timeliness of the results for facilitation of strategic planning and project evaluation. The approach and objectives draw from Weik and Lang’s (2016:32) “transformational” research stream, which focuses on “providing evidence for how successfully to intervene in sustainability problems in order to resolve or at least mitigate them.”
The organizational structure I present here and in Beratan (2019) can be successfully used by any natural resource practitioner as a guide for gathering and filtering information. Because the approach is structured around causal relationships, it can help individuals and groups focus on specific and feasible pathways to desired outcomes rather than on information that is not relevant to the problem being addressed. As noted in Beratan (2019), greater value can be obtained by engaging a process facilitator to conduct a full exploratory problem assessment at the start of a planning process. A suitably trained process facilitator can conduct the assessment process quickly and cost-effectively.
Much of the work presented in this article was done while the author was a member of the Research Faculty at North Carolina State University. Special thanks to Jim Berkley, Jennifer Pratt Miles, and other members of the Collaborative Adaptive Management Network (CAMNet) Core Group for their encouragement and many thought-provoking conversations. Thanks also to two anonymous reviewers who provided thoughtful comments that greatly improved the paper.
Abbott, J. 2005. Understanding and managing the unknown: the nature of uncertainty in planning. Journal of Planning Education and Research 24:237-241. https://doi.org/10.1177/0739456X04267710
Allen, C. R., and A. S. Garmestani. 2015. Adaptive management. Pages 1-10 in C. R. Allen and A. S. Garmestani, editors. Adaptive management of social-ecological systems. Springer, Dordrecht, The Netherlands. https://doi.org/10.1007/978-94-017-9682-8_1
Ancona, D. 2012. Sensemaking: framing and acting in the unknown. Pages 3-19 in S. Snook, N. Nohria, and R. Jhurana, editors. The handbook for teaching leadership: knowing, doing, and being. Sage, Los Angeles, California, USA. [online] URL: https://www.sagepub.com/sites/default/files/upm-binaries/42924_1.pdf
Bartholomew, L. K., G. S. Parcel, and G. Kok. 1998. Intervention mapping: a process for developing theory and evidence-based health education programs. Health Education and Behavior 25(5):545-563. https://doi.org/10.1177/109019819802500502
Bartholomew, L. K., G. S. Parcel, G. Kok, and N. H. Gottlieb. 2001. Intervention mapping: designing theory- and evidence-based health promotion programs. Mayfield, Mountain View, California, USA.
Benner, P. 1994. The role of articulation in understanding practices and experience as sources of knowledge in clinical nursing. Pages 136-156 in J. Tully, editor. Philosophy in an age of pluralism: the philosophy of Charles Taylor in question. Cambridge University Press, New York, New York, USA. https://doi.org/10.1017/CBO9780511621970.011
Beratan, K. K. 2019. Improving problem definition and project planning in complex natural resource management problem situations using knowledge brokers and visual design principles. Ecology and Society 24(2):31. https://doi.org/10.5751/ES-10815-240231
Bronfenbrenner, U. 1979. The ecology of human development. Harvard University Press, Cambridge, Massachusetts, USA.
Bronfenbrenner, U. 1986. Recent advances in research on human development. Pages 287-309 in R. K. Silbereisen, K. Eyferth, and G. Rudinger, editors. Development as action in context: problem behavior and normal youth development. Springer, New York, New York, USA. https://doi.org/10.1007/978-3-662-02475-1_15
Bronfenbrenner, U., and A. C. Crouter. 1983. The evolution of environmental models in development research. Pages 357-414 in W. Kessen, editor. History, theories, and methods: handbook of child psychology. Wiley, New York, New York, USA.
Catton, W. R. 1989. Choosing which danger to risk. Society 27(1):6-8. https://doi.org/10.1007/BF02695443
Cázares, A. 1994. Prevention intervention research: focus and perspective. Pages 5-35 in A. Cázares and L. A. Beatty, editors. Scientific methods for prevention intervention research. NIDA Research Monograph 139, National Institutes of Health, Rockville, Maryland, USA. https://doi.org/10.1037/e495892006-003
Chevallier, A. 2016. Strategic thinking in complex problem solving. Oxford University Press, New York, New York, USA. https://doi.org/10.1093/acprof:oso/9780190463908.001.0001
Clauser, S. B., S. H. Taplin, M. K. Foster, P. Fagan, and A. D. Kaluzny. 2012. Multilevel intervention research: lessons learned and pathways forward. Journal National Cancer Institute Mongraphs 2012(44):127-133. https://doi.org/10.1093/jncimonographs/lgs019
Craig, P., P. Dieppe, S. Macintyre, S. Michie, I. Nazareth, and M. Petticrew. 2008. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337:a959. https://doi.org/10.1136/bmj.a959
Eoyang, G. H. 2006. Human systems dynamics: complexity approach to a complex evaluation. Pages 123-140 in B. Williams and I. Imam, editors. Systems concepts in evaluation: an expert anthology. American Evaluation Association, Edge Press, Point Reyes, California, USA.
Foundations of Success. 2009. Using results chains to improve strategy effectiveness: an FOS how-to guide. Foundations of Success, Bethesda, Maryland, USA. [online] URL: http://www.fosonline.org/resource/using-results-chains
Fraser, M. W., and M. J. Galinsky. 2010. Steps in intervention research: designing and developing social programs. Research on Social Work Practice 20(5):459-466. https://doi.org/10.1177/1049731509358424
Funnell, S. C., and P. J. Rogers. 2011. Purposeful program theory: effective use of theories of change and logic models. Jossey-Bass, San Francisco, California, USA.
George, R. V. 1995. Expert support for problem formulation in urban planning. Environment and Planning B: Planning and Design 22(2):175-190. https://doi.org/10.1068/b220175
Gilgun, J. F., and R. G. Sands. 2012. The contribution of qualitative approaches to developmental intervention research. Qualitative Social Work 11(4):349-361. https://doi.org/10.1177/1473325012439737
Hawe, P. 2015. Lessons from complex interventions to improve health. Annual Review of Public Health 36:307-323. https://doi.org/10.1146/annurev-publhealth-031912-114421
Hawe, P., A. Shiell, and T. Riley. 2009. Theorizing interventions as events in systems. American Journal of Community Psychology 43:267-276. https://doi.org/10.1007/s10464-009-9229-9
Hunt, L. M., S. G. Sutton, and R. Arlinghaus. 2013. Illustrating the critical role of human dimensions research for understanding and managing recreational fisheries within a social-ecological system framework. Fisheries Management and Ecology 20:111-124. https://doi.org/10.1111/j.1365-2400.2012.00870.x
Kapsali, M. 2013. Equifinality in project management exploring causal complexity in projects. Systems Research and Behavioral Science 30:2-14. https://doi.org/10.1002/sres.2128
Mahmoud-Jouini, S. B., C. Midler, and P. Silberzahn. 2016. Contributions of design thinking to project management in an innovation context. Project Management Journal 47(2):144-156. https://doi.org/10.1002/pmj.21577
Margoluis, R., C. Stem, V. Swaminathan, M. Brown, A. Johnson, G. Placci, N. Salafsky, and I. Tilders. 2013. Results chains: a tool for conservation action design, management, and evaluation. Ecology and Society 18(3):22. https://doi.org/10.5751/ES-05610-180322
Maxwell, J. A. 2004. Using qualitative methods for causal explanation. Field Methods 16(3):243-264. https://doi.org/10.1177/1525822X04266831
McLeroy, K. R., D. Bibeau, A. Steckler, and K. Glanz. 1988. An ecological perspective on health promotion programs. Health Education Quarterly 15:351-377. https://doi.org/10.1177/109019818801500401
Medical Research Council (MCR). 2000. A framework for the development and evaluation of randomized controlled trials for complex interventions to improve health. MRC, London, UK. [online] URL: https://mrc.ukri.org/documents/pdf/rcts-for-complex-interventions-to-improve-health/
Miller, A. 1999. Environmental problem solving: psychosocial barriers to adaptive change. Springer, New York, New York, USA.
Mitroff, I. I., and T. R. Featheringham. 1974. On systemic problem solving and the error of the third kind. Behavioral Science 19(6):383-393. https://doi.org/10.1002/bs.3830190605
Natasi, B. K., and J. Hitchcock. 2009. Challenges of evaluating multilevel interventions. American Journal of Community Psychology 43(3-4):360-376. https://doi.org/10.1007/s10464-009-9239-7
North Carolina Department of Environment and Natural Resources (NCDENR). 2012. Governor’s Land Compatibility Task Force report. State Library of North Carolina, NC Digital Documents Collection, Raleigh, North Carolina, USA. [online] URL: http://cdm16062.contentdm.oclc.org/cdm/ref/collection/p16062coll9/id/22139
Patton, M. Q. 2011. Developmental evaluation: applying complexity concepts to enhance innovation and use. Guilford, New York, New York, USA.
Pinkerton, E. 1989. Attaining better fisheries management through co-management prospects, problems and propositions. Pages 3-33 in E. Pinkerton, editor. Co-operative management of local fisheries: new direction in improved management and community development. University of British Columbia Press, Vancouver, British Columbia, Canada. https://doi.org/10.2307/1242652
Plummer, R. 2009. The adaptive co-management process: an initial synthesis of representative models and influential variables. Ecology and Society 14(2):24. https://doi.org/10.5751/ES-03130-140224
Plummer, R., and J. FitzGibbon. 2004. Co-management of natural resources: a proposed framework. Environmental Management 33(6):876-885. https://doi.org/10.1007/s00267-003-3038-y
Qiu, J., E. T. Game, H. Tallis, L. P. Olander, L. Glew, J. S. Kagan, E. L. Kalies, D. Michanowicz, J. Phelan, S. Polansky, J. Reed, E. O. Sills, D. Urban, and S. K. Weaver. 2018. Evidence-based causal chains for linking health, development, and conservation actions. Bioscience 68(3):182-193. https://doi.org/10.1093/biosci/bix167
Richard, L., L. Gauvin, and K. Raine. 2011. Ecological models revisited: their uses and evolution in health promotion over two decades. Annual Review of Public Health 32:307-326. https://doi.org/10.1146/annurev-publhealth-031210-101141
Rogers, P. J. 2008. Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation 14(1):29-48. https://doi.org/10.1177/1356389007084674
Rouse, W. B., and N. Serban. 2011. Understanding change in complex socio-technical systems. Information, Knowledge, Systems Management 10:25-49. https://doi.org/10.3233/IKS-2012-0184
Schensul, J. J. 2009. Community, culture and sustainability in multilevel dynamic systems intervention science. American Journal of Community Psychology 43:241-256. https://doi.org/10.1007/s10464-009-9228-x
Shiell, A., P. Hawe, and L. Gold. 2008. Complex interventions or complex systems? Implications for health economic evaluation. British Medical Journal 336:1281-1283. https://doi.org/10.1136/bmj.39569.510521.AD
Spradlin, D. 2012. Are you solving the right problem? Asking the right questions is crucial. Harvard Business Review 90(9):84-93. [online] URL: https://hbr.org/2012/09/are-you-solving-the-right-problem
Sugihara, G., R. May, H. Ye, C.-H. Hsieh, E. Deyle, M. Fogarty, and S. Munch. 2012. Detecting causality in complex ecosystems. Science 338:496-500. https://doi.org/10.1126/science.1227079
Susskind, L., A. E. Camacho, and T. Schenk. 2010. Collaborative planning and adaptive management in Glen Canyon: a cautionary tale. Colombia Journal of Environmental Law 35:1-55. [online] URL: https://scienceimpact.mit.edu/sites/default/files/documents/Collaborative%20Planning%20and%20Adaptive%20Management%20in%20Glen%20Canyon-%20A%20Cautionary%20Tale.pdf
Taylor, A., C. Cocklin, R. Brown, and E. Wilson-Evered. 2011. An investigation of champion-driven leadership processes. Leadership Quarterly 22:412-433. https://doi.org/10.1016/j.leaqua.2011.02.014
van Bokhoven, M. A., G. Kok, and T. van der Weijden. 2003. Designing a quality improvement intervention: a systematic approach. Quality and Safety in Health Care 12(3):215-220. https://doi.org/10.1136/qhc.12.3.215
van den Belt, M., J. R. Kenyan, E. Krueger, A. Maynard, M. G. Roy, and I. Raphael. 2010. Public sector administration of ecological economics systems using mediated modeling. Annals of the New York Academy of Sciences 1185:196-210. https://doi.org/10.1111/j.1749-6632.2009.05164.x
Weick, K. E., K. M Sutcliff, and D. Obstfeld. 2005. Organizing and the process of sensemaking. Organization Science 16(4):409-421. https://doi.org/10.1287/orsc.1050.0133
Wiek A., and D. J. Lang. 2016. Transformational sustainability research methodology. Pages 31-41 in H. Heinrichs, P. Martens, G. Michelsen, and A. Wiek, editors. Sustainability Science. Springer, Dordrecht, The Netherlands. https://doi.org/10.1007/978-94-017-7242-6_3
Weiner, B. J., M. A. Lewis, S. B. Clauser, and K. B. Stitzenberg. 2012. In search of synergy: strategies for combining interventions at multiple levels. Journal of the National Cancer Institute Monographs 44:34-41. https://doi.org/10.1093/jncimonographs/lgs001
Weiss, C. H. 1995. Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community-based initiatives for children and families. Pages 65-92 in J. P. Connell, A. C. Kubisch, L. B. Schorr, and C. H. Weiss, editors. New approaches to evaluating community initiatives: concepts, methods and contexts. Vol. 1. Aspen Institute, Washington, D.C., USA. [online] URL: https://pdfs.semanticscholar.org/ff5a/3eea8d1d4e07a768ded6b426b425efde8f7c.pdf
Wilder, D. 1999. Network exchange theory: issues and directions. Pages 1-21 in D. Wilder, editor. Network exchange theory. Praeger, Westport, Connecticut, USA.
Winter, S. 1987. Knowledge and competence as strategic assets. Pages 159-184 in D. J. Teece, editor. The competitive challenge? Strategies for industrial innovation and renewal. Bollinger, Cambridge, Massachusetts, USA.
W. K. Kellogg Foundation. 2001. Logic model development guide: using logic models to bring together planning, evaluation, and action. W. K. Kellogg Foundation. Battle Creek, Michigan, USA. [online] URL: https://www.bttop.org/sites/default/files/public/W.K.%20Kellogg%20LogicModel.pdf
Yano, E. M., L. W. Green, K. Glanz, J. Z. Ayanian, B. S. Mittman, V. Chollette, and L. V. Rubenstein. 2012. Implementation and spread of interventions into the multilevel context of routine practice and policy: implications for the cancer care continuum. Journal of the National Cancer Institute Monographs 44:86-99. https://doi.org/10.1093/jncimonographs/lgs004