There has been an explosion of interest in the role of public participation in natural resource management and environmental decision making. Participation is increasingly embedded in national and international policy-making processes and supported by an array of legal requirements (Bryson et al. 2012), such as the Rio Conventions on Biodiversity, Climate Change and Desertification, freedom of information acts, and a range of statutory obligations governing planning processes and natural resource management. Given the legal importance of public participation, it is crucial to understand the factors that determine its success in policy outcomes. Consistent with statutory expectations, we focus on participation in public policy making, understood as the involvement of those outside the formal governmental apparatus, such as citizens or nongovernmental organizations, in public policy decision making (Verba 1967, van Tatenhove and Leroy 2003). A thorough focus on democratic participation would require investigation of relations between individuals and authorities in the families, schools, organizations, and other nongovernmental institutions to which individuals belong. It is beyond our scope to address citizen activism and engagement, which relate to the involvement of individual and community groups in activities that take place independent of state association.
Two categories of arguments are advanced in support of participation. An instrumental, pragmatic rationale holds that participation enhances public input into, and support for, policy decisions, leading to effective and efficient implementation (Grimble and Wellard 1997, Burgess et al. 1998, Bulkeley and Mol 2003). From an instrumental perspective, participation gives decision makers two vital resources: information and support. Through regular contact with citizens, administrators can gain information that supports the policy process and can learn which policies are likely to be explosively unpopular and how to avoid policy failures (Irvin and Stansbury 2004). Dialogue between policy makers and citizens also informs the public about the intent and context of individual policies and may enhance buy-in to policy decisions.
A second rationale, which is normative, is based on claims that participation supports democratic values by fostering a more inclusive and deliberative form of decision making (Dryzek 1990, Fischer 2003). Normative goals are frequently a component of broader discussions on developing community and deepening democracy. This stems from a vision of citizenship as obligation and responsibility (Dobson 2003) and the belief that an engaged citizenry supports and advances new forms of democratic practices (Fiorino 1990, Putnam 1995). The rise of new social movements, such as the civil rights, environment, and women’s movements (della Porta and Diani 2006) raised public demand for greater involvement in public policy decisions. Over time, this led to the emergence of more inclusive arrangements within institutions and to innovative political practices (van Tatenhove and Leroy 2003, Parkins and Mitchell 2005).
However, the growing interest in public participation, including its deliberative forms, is premised on a number of reasonable but largely untested assumptions, and there remain deep doubts about its practicality, political significance, and even appropriateness as a core feature of a vibrant democracy (Delli Carpini et al. 2004). There is a strong and persistent suspicion that participatory practices are so infrequent, unrepresentative, subject to conscious manipulation and unconscious bias, and disconnected from actual decision making, that it is at best an impractical mechanism for determining the public will and at worst misleading or dangerous (Delli Carpini et al. 2004, Innis and Booher 2004). In addition, success of devolved commons management in achieving positive ecological and social outcomes is highly variable (Gurney et al. 2016). Furthermore, in the current period of economic austerity and fiscal retrenchment, there is growing suspicion that participation provides a forum for state retreat from public policy provision and a guise for the privatization of public interests (Cooke and Kothari 2001, Guarneros-Meza and Geddes 2010).
In light of the complex mix of positive and negative outcomes of public participation (Delli Carpini et al. 2004, Innis and Booher 2004, Brooks et al. 2012, Cinner et al. 2012), recent syntheses of the literature frequently call for research on the contexts in which participation yields positive versus negative outcomes (Rowe and Frewer 2000, Delli Carpini et al. 2004, Abelson and Gauvin 2006, Dietz and Stern 2008, Newig and Fritsch 2009, O’Faircheallaigh 2010, Bryson et al. 2012). However, simply acknowledging that context exists (“it depends”) in the messy real world of resource management (Mccool and Guthrie 2001) provides no useful guidance to citizens and policy makers on appropriate forms of participation. Although the importance of context is often recognized and described qualitatively in empirical studies, overall understanding of how contextual factors shape the outcomes of participation remains limited (Restall and Kaufert 2011, Gurney et al. 2016). We explore the ways in which the outcomes of participatory practices are shaped by the interplay of three types of context: participatory rationales, social structures, and institutional settings. This provides understanding of how context dependence shapes participatory outcomes. Although there is considerable social-science literature on each of these contextual dimensions, none of them, by themselves, adequately explains the outcomes of participation. We therefore focus particular attention on the interactions among these contextual dimensions, taking account of how multidimensional dynamics shape participatory outcomes. Examples are drawn primarily from natural resource management.
Rationales for public participation, advocated in the literature, i.e., the reasons why a particular course of action is taken, range from instrumental to normative (Table 1), although others classify the rationale for participation in slightly different ways (Stirling 2006, Turnhout et al. 2010, Cini 2011). Because our primary aim is not to detail participatory matrices, we remain with the instrumental-to-normative distinction. This spectrum of rationales shapes the framing of the problem to be addressed and desired outcomes, providing a foundation for discussing how interactions among contextual factors influence outcomes of public participation. Specific rationales shape the ways that participation might contribute to solving problems (Table 1).
Some goals of participation are primarily instrumental (Table 1, 1). Public participation is often required by law (Table 1, 1a), for example as mandated in the USA National Environmental Policy Act, 1969. Although one goal of participation is to increase the knowledge base for policy making (Table1, 1b), this may not occur if local groups (including scientific experts) are not repositories of knowledge that is credible, relevant, legitimate, and accountable (Cash et al. 2003) or if agencies are not receptive to the information that they receive.
The extent of “openness” in the policy process is a key, understood to involve five core dimensions: (1) the scope of participation, referring to those allowed to participate in the process; (2) access to information, namely what information is made available to participants; (3) timing, that is, when participants are allowed to participate, (4) scope of contribution, referring to which aspects participants are allowed to contribute to, and (5) impact of contribution, namely, the extent to which participant contributions influence outcomes, that is, how much weight is given to participant contributions (Pohjola and Tuomisto 2011). No system is fully open and there is no guarantee that the information provided by participants increases the options available to policy makers (Table 1, 1c). After all, public agencies exercise considerable control over the ways that public input is sought, received, and used (Hoover and Stern 2014a). Therefore, information derived from participation does not necessarily lead to better policy outcomes (Baker and Eckerberg 2008). The values held by those working within the system of public administration are an important determinant of policy outcomes (Meier and O’Toole 2006). Openness is linked, in turn, to agency culture, encompassing values, priorities, commitments, leadership style, and ways of conducting business within an organization. These shape the extent to which an organization is flexible and supportive of change, and facilitating and encouraging learning and innovation (Laurian et al. 2017). Evidence from stakeholder engagement in public waste management in Ireland, for example, showed that groups espousing zero waste strategies were systematically disregarded in policy decisions, not least because their values conflicted with administrators’ perceptions that waste management could be a source of privatized and profitable business (Connaughton at al. 2008). Similarly, the US National Environmental Policy Act directs agency personnel to focus on “substantive” comments, allowing them to disregard comments that are conjectural or opinion-based (see Predmore et al. 2011a). In this case, the values held by agency actors were shown to be a critical factor shaping both the nature of public involvement and the associated outcomes (Yang and Callahan 2007, Stern and Predmore 2012).
Disregard for stakeholder interests can also occur when public administrators see participation as a burden (Table 1, 1d), as a distraction from scientific management, or as a barrier to hitting performance targets (Stern et al. 2010). There are many instances where the demand for societal responsiveness and representativeness compete with other organizational goals, such as efficiency requirements, as evidenced by conflict over organizational norms in the forestry sector in the USA (Tipple and Wellman, 1991). In response to competing demands, administrators may make communications to the public scientifically and technically complex (Predmore et al. 2011a), resulting in further societal estrangement and lost opportunities for information sharing (Table 1, 1e). Finally, shaping public policy to suit the needs of multiple participants can bring loss of coherence and consistency of implementation compared to centrally designed plans (Verba 1967). This can lead to a patchwork of management directives that may or may not meet regulatory requirements or other stated objectives (Hoover and Stern 2014b). Collectively, these institutional factors, including administrators’ perceptions as to the risks and costs of participation, play a key role in determining whether the instrumental goals of public participation are realized (March and Olsen 2006). In summary, instrumental rationales and goals are not shared by all actors, even those state actors that are expected to gain the most.
Deeper benefits of participation draw on a mix of instrumental and normative rationales (Table 1, 2). Participation may open meaningful dialogue that enables marginalized groups to convey concerns otherwise excluded from the policy process (Gouldson and Bebbington 2007; Table 1, 2a). Cogeneration of policy solutions, particularly at the local level through collective action (i.e., action taken by a group), provides opportunities to incorporate a diversity of values and needs (Reed 2008; Table 1, 2c, 2d). However, local administrators may retain control over some issues that they deem strategic and restrict participatory opportunities to more local forums (Newman et al. 2004). This may lead to conflict between actors in the strategic center of organizations and those involved in local forums (Newman et al. 2004).
In addition, outcomes can become skewed by unequal distribution of power among participants (Table 1, 2a), including education and financial resources (Raik et al. 2008). In general, the less organized components of any society are less likely to participate effectively (Innes and Booher 2004). For marginalized groups, large entry barriers and transaction costs (Irvin and Stansbury 2004) restrict participation, and there is thus a tendency for high participation costs to lead to over-representation of groups with more resources (Lynham et al. 2017). Participation therefore has the potential to ratify decisions that favor the better-resourced members of society, potentially reinforcing rather than mitigating social inequities (Kenney 2011). A common problem discussed in the literature is that of consultation fatigue, often related to disillusionment when the views of participants are not taken into account (Yaffee and Wondolleck 2000).
Participation can provide governments with learning opportunities (Table 1, 2b) that allow them to be more relevant and responsive (Table 1, 2e). If successful, such learning can reduce adversarial dynamics (Table 1, 2c) and widen the sense of collective responsibility (Table 1, 2d). In this way, participation advantages may spill over to the rest of society (Beierle 2002; Table 1, 2f). However, collective action can also lead to unintended consequences by increasing the risk that policy will be captured by interest groups to serve their private agendas. In those cases, participation may benefit organized stakeholder groups and established interests, rather than fostering the development of deliberative democratic processes (Arnstein 1969, Dasgupta and Beard 2007, Gurney et al. 2016).
Thus, it is important not to oversimplify the nature of the public (Predmore et al. 2011b), especially if this involves a convenient construction of community that ignores the heterogeneity of social structures and norms (Agrawal and Gibson 1999; Table 1, 2a, 2c). There are competing publics, such that understanding public policy making becomes one of trying to explain why some interests dominate over others. From an administrator’s perspective, such heterogeneity may make the involvement of the public in natural resource comanagement highly unpredictable (Carlsson and Berkes 2005). Concern about such risks can make administrators reluctant to open up policy-making processes to meaningful participatory practices, thus weakening the ability of policy makers to reflect wider, collective interests.
In addition, although participation holds potential for positive reinforcement of political legitimacy, participatory practices are often weak in terms of political accountability (Table 1, 2e). The democratic requirement for transparency becomes less assured when public policy is made and implemented in dense networks of institutional actors and private interests (Black 2008). Finally, participation can be used to combine state retreat and promote a neoliberal agenda of privatizing the delivery of public services (Swyngedouw 2005, Leal 2007). This allows the state to delegate responsibility for societal well-being to others, often without providing them with the resources necessary to avoid detrimental social consequences.
An understanding of democracy that sees legitimacy depend upon the active and enduring participation of ordinary citizens in political life is closely tied to expectations about procedural fairness (Table 1, 3a). Procedural fairness requires equal opportunity for all affected to acquire the knowledge and skills to contribute meaningfully to the decision-making process (Zuhair and Kurian 2016). People’s perception of the procedural fairness of public policy making is linked to their ability to participate and this, in turn, shapes their support for participation practices and subsequent acceptance of policy outcomes (Dobson 2014).
Although there are several conceptual differences among authors working in this field (Cini 2011), participation is seen as more than merely ensuring the legitimacy of existing structures and processes (Table 1, 3g). Normative goals for participation are frequently a component of broader discussions on developing community and deepening democracy (Table 1, 3). Benefits include improved promotion of the common good over individual interests (Table 1, 3a); reduced marginalization, i.e., (Table 1, 3b) the building of social capital, which underpins continued societal engagement (Table 1, 3c); and improved sense of community and belonging (Table 1, 3d). Underpinning this is the belief that an engaged citizenry is better than a passive citizenry in supporting and advancing new forms of democratic practices (Fiorino 1990, Putnam 1995; Table 1, 3e). In this view, participation promotes a vision of citizenship as obligation and responsibility (Dobson 2003). For many, participation can also be seen an alternative to lives centred on material consumption (Kemp et al. 2005).
Many of these variables can be captured by the concept of opportunity structure. In its classical formulation, Tarrow (1994:85) defines political opportunity structures as “dimensions of the political environment that provide incentives for people to undertake collective action by affecting their expectations for success or failure.” This speaks to the structural conditions that influence access, including institutional possibilities for participation. More recently, participation has become an essential component of new processes of governance (Table 1, 3e). These new processes acknowledge that central government lacks the capacity to deal with the growing array of “wicked” policy, in which complexity, diversity, and uncertainty are key features, and in which simple regulatory control is insufficient (Rittel and Webber 1973; see also Kooiman 2000, Berkes et al. 2003, Head and Alford 2015). Partly influenced by the rise of neoliberalism, this has allowed participating actors to play a greater role in shaping the rules and objectives of governing, including within natural resource management (Lange et al. 2013), rather than simply responding to goals and priorities circumscribed in advance (Table 1, 3f). However, realizing these benefits requires that meaningful participation be institutionalized for sustained periods. Nancy Fraser’s (1997) analysis of processes of inclusion and exclusion in the public sphere also points to the democratic weaknesses of governance systems that fail to engage with what she terms counter-publics, i.e., groups or networks that have the capacity to challenge official norms and ways of doing.
Although participation is often driven by instrumental rationales, it has the potential to promote normative change. Three potential beneficiaries from participatory processes are government, citizens, and special interest groups. However, none of their interests is assured a priori. The instrumental rationales for public participation are relatively short-term and primarily address the needs of government (Table 1). In contrast, normative rationales emphasize the benefits to citizens and society over the longer term. Intermediate rationales that have both instrumental and normative dimensions link the short-term benefits to government with the longer-term benefits to society.
Strategies based on these intermediate rationales often govern the balance of benefits to government, citizens, and special interest groups. Participation can support radical, new forms of democracy that are based on active citizen engagement, on new levels of trust between the state and citizen arising from mutual learning, and on enhanced capacity to coproduce more effective policy solutions to complex societal challenges. However, it can also enhance the power of vested interest, be cost inefficient, disrupt the ongoing business of governing, and further alienate those who are already socially marginalized. Whether participatory practices meet their desired goals depends heavily upon several contextual variables, including the nature of those goals; the societal, structural factors that shape the capacity of groups to engage and the types of formal access given; and the institutional constraints that are placed on them. In other words, whether or not the benefits of participation are realized or constrained is shaped by the particular configuration of community goals, resources and capacity, institutional arrangements, and historic precedents existing within the polity in question. Institutions themselves can lack capacity to engage, including the necessary resources of staff, finance, and time.
In short, the relationship between state and nonstate actors, as well as the distinctive exercise of power, shapes how preferences are translated into policy choices and how different social interests are reflected in participatory outcomes. This points to the importance of institutional factors, such as the norms and procedural settings, i.e., the institutional architecture in which policy making takes place, and the system of formal and informal rules of the game (Lange et al. 2013) in shaping the patterns of interaction among actors and their outcomes. However, less attention has been given to an elaboration on the social and cultural context of participation. The general tendency in the literature is to acknowledge that social and cultural context matters, but to treat this context only in the most general of terms.
Although instrumental and normative rationales for participation describe the motivations of different actors to foster participation, social and cultural contexts strongly influence the extent to which participation meets these goals. These structural factors can be seen as operating at three, albeit interrelated, levels.
Lack of good governance reduces both the willingness and opportunities of the public to participate. Participation is grounded on the belief that groups or local community can in principle influence decisions (Tsang et al. 2009). Trust becomes important in lubricating cooperation (Pretty and Smith 2004), although the relationship is best seen as reciprocal: participation tends to enhance public trust, whereas participatory processes depend, in turn, upon that trust for success. Trust in a broad sense “refers to public belief that officials are bearing and sustaining their moral, societal and fiduciary obligations” (Wang and Wart 2007:266). It denotes the public's confidence in the integrity of public officials to be fair and uphold the public's interest, and well as public confidence in the competence of government to carry out its assigned duties. On the other hand, many participate because they do not in fact trust government and wish to have greater oversight or say in what public decision makers are doing (Wang and Wart 2007). An administrator’s trust in citizens is also important (Yang 2005).
Given that participation can have both positive and negative outcomes, how can it be designed to maximize the likelihood of good outcomes? We discuss the fit between types of participation and opportunity structures (Kitschelt 1986) within the policy processes that influence policy making. Prescriptively, the concept of institutional fit holds that institutional arrangements should match “the defining features of the problems they address” (Young 2008:20). The attributes of a problem are used to identify the governance arrangements that might best address them (see Cox 2012).
Many typologies of public participation have been proposed, based, for example, on levels of citizen participation (Arnstein 1969, Biggs 1989, Lawrence 2006, van Zeijl-Rozema et al. 2008, O’Faircheallaigh 2010), rationale (Beierle 2002, Parkins and Mitchell 2005), institutional settings (Newig and Fritsch 2009, Sandström 2009, Turnhout et al. 2010, DeCaro and Stokes 2013), direction of communication flow (Rowe and Frewer 2000), types of problems (Turnhout et al. 2010, Hurlbert and Gupta 2015), participatory practices (Rowe and Frewer 2000, Mccool and Guthrie 2001), outcomes (Abelson and Gauvin 2006, Turnhout et al. 2010), and objectives of participation (Lynam et al. 2007, Bryson et al. 2012). These typologies overlap substantially with one another. We therefore take one of these typologies that has been widely used (Arnstein 1969) and focus on the contexts that influence the fit of particular types of participation with various institutional settings to maximize the likelihood of favorable outcomes. This provides guidance to managers, policy makers, community groups, and other social actors interested in fostering public participation to achieve particular outcomes.
Arnstein (1969) proposed eight levels of public participation that might occur. Although numerous alternative terms have been proposed for the different rungs of this ladder (see Biggs 1989; and Reed 2008 for a review), and an alternative metaphor of a “wheel of participation” suggested (Davidson 1998), the original typology proposed by Arnstein describing a continuum from passive dissemination of information to active citizen engagement remains central to discussions. Arnstein’s two lowest levels of participation (manipulation and therapy) are mechanisms by which policy makers seek to “prevent” public participation, and her highest level of participation (citizen control) has no state involvement. We therefore omit these forms of nonparticipation and focus on Arnstein’s five intermediate forms that represent meaningful interactions between citizens and the state, ranging from informing to delegating power to the public (Table 2). These levels of participation suggest a set of pragmatic strategies for fostering participation in response to the particular rationales and social-cultural contexts that we discussed earlier.
These are the forms most widely stipulated in legislation and are therefore the most easily achieved fit to the instrumental rationales of most agencies (Table 2). This is particularly true at national and international scales, in which public involvement in policies relating to natural resource management occurs primarily by providing information to citizens (informing) and opportunities for them to provide input (advising and consultation). At these scales, more active participation is generally prohibitively time-consuming and expensive (Stringer et al. 2006). This relatively weak (advising and consultation) role provides information that might not otherwise be considered and communicates international accords downwards (informing), in which local implementation strategies have the potential to be developed (Lambin et al. 2002, Stringer et al. 2007). However, if public administrators see participation as a mere procedural hoop, this may lead to long-term problems with agency credibility, a loss of public trust, deteriorating agency effectiveness, and in some cases, active opposition to plans (Predmore et al. 2011b). The biggest shortcoming of these forms is the lack of public involvement in decision making, which tends to maintain existing power structures and inequalities and breed public resentment (Innis and Booher 2004, Predmore et al. 2011b).
When there is some balance of power between stakeholders and policy makers, partnership is strongly advocated by proponents of deliberative democracy. More deliberative engagement draws on public knowledge to improve resource-management decisions and garner public support (Parkins and Mitchell 2005; Table 2). In turn, such engagement enables groups to contribute to problem definition and solutions and to provide information and feedback (the more conventional consultative role; van Tatenhove and Leroy 2003). For example, in Canada, the Forest Stewardship Councils provided a venue for dialogue among laypeople, research managers, and scientists to address forestry standards (Auld and Bull 2003), as witnessed also in the MacKenzie Valley Pipeline inquiry (Parkins and Mitchell 2005). However, from the perspective of the implementing agency, partnership may be laden with risk, including financial risk, demands on time and staffing, and may be seen to provide a platform for increased expectations and conflicts that undermine administrative authority (Hoover and Stern 2014b). Nevertheless, a dynamic relationship may build between one and the other, such that the political environment, especially the openness of its political institutions, affects the emergence, strategies, and forms of social mobilization that occur; and in turn, that mobilization can feed back to shape the policy outcomes, the institutional context in which policy is made, and, over time, the political environment itself. In short, outcomes of partnerships depend on the interplay between the capacity of groups to engage fully, related in turn to issues of social inequality and power distribution, and the nature of institutional gatekeeping. Partnership can be implemented through a wealth of mechanisms, including citizen juries, public consultations, town hall meetings, internet conferencing, scenario workshops, informal roundtables, deliberative opinion polls, more formal and less open public hearings and inquiries, and consultative process restricted to a limited set of designated stakeholders. Typical products include agreed public statements, policy plans, forms of joint implementation, and public/private partnership arrangements for policy delivery, often at the project level. The contexts in which specific platforms, tools, and products are most effective are beyond our scope. Nonetheless, different levels of participatory engagement are likely to be appropriate in different situations, depending, for example, on the objectives of the policy and the circumstances (Davidson 1998, Richards et al. 2004).
Delegated power, in which citizens exert primary control over policy formation and implementation, occurs within relatively restricted domains of democratic societies, for example, through delegation of technical decisions by school governance to a school board. Delegation functions most effectively when legally mandated (for example, delegation of certain powers from the federal government to states or provinces) or when power holders trust those to whom they have delegated power. Sometimes delegation occurs through abdication of government responsibility (such as in the privatization of health care). This abdication of responsibility can lead to severe problems of coordination and control and thus risks of policy paralysis (O’Faircheallaigh 2010), as in current climate negotiations. However, participation at this level holds the potential for deliberative processes that bring together different forms of knowledge (expert systems, local knowledge, and everyday life perspectives) that can help natural resource management by generating new perspectives on the interrelationship between nature and society (Vasstrøm 2014). This provides opportunities for fresh management strategies.
In summary, analysis of institutional fit can identify participation strategies that are most likely to address particular rationales and their social and cultural contexts. This clarifies how the particular configuration of institutional arrangements and community capacity shape whether the benefits of participation are to be realized or constrained in practice. It details how key variables, namely: (1) the nature of the problem at hand and the geographic scale at which it is addressed; (2) institutional factors related to organizational culture, such as degrees of openness, attitude to risk, and historic precedents, alongside existing statutory obligations; and (3) the strength of organized interests, seen in the context of societal equality gaps, all serve to shape whether participatory processes deliver on their potentials.
The literature explores context in only the most general of ways. We develop the notion of context around three key dimensions, detailing instrumental and normative rationales, social and cultural determinants, and institutional fit. Through a detailed elaboration of the concept, we contribute to a deeper understanding of the ways in which context shapes participatory practices and outcomes. We conclude that public participation is inherently neither good nor bad but provides a mix of benefits and pitfalls, whose balance depends strongly on goals (rationales) and institutional and social contexts. Identification of specific factors that are important within each of these contextual dimensions provides a basis for identifying plausible goals (instrumental to normative), institutional or social structural elements (such as agency culture or social stratification) that are likely to influence the feasibility of achieving goals, and appropriate types of participation (from informing to delegation) that meet these goals within particular social contexts. Armed with this understanding, strategies can be developed to frame dialogues about how to foster effective public participation.
Although community capacity building is a well-known strategy for ensuring that natural resource management strategies address local needs, our contribution lies in drawing upon an array of factors simultaneously to facilitate a more holistic understanding of the factors that shape participatory outcomes. The contextual dependence of public participation is key to understanding the role of civil society in social-ecological systems. In the absence of public participation, government creates top-down policies to implement administrative goals. Active and effective public participation creates feedbacks from civil society back to policy steering by government. Rationales, social-cultural factors, and institutional fit are categories of contextual contingencies that influence whether these feedbacks are effective.
Seeing participation as driven by mixed goals of instrumentality and normativity provides a useful starting point for understanding participatory practices. However, participation, rather than being construed as driven by dichotomous rationales, is best seen as a process of interacting goal formation. Understanding participation in this dynamic way also turns attention to another dynamic, i.e., the ways in which the benefits and pitfalls of participation play out in institutional settings. Using a policy-analysis lens, we see how these settings, including legislative and regulatory requirements, institutional values and norms, and perceptions of risk, shape gatekeeping actions by administrative agencies and their actors. Societal demands have also influenced the emergence of new and often innovative institutional arrangements, which have, in turn, deep political significance. However, structural determinants from within society, including social inequities, continue to constrain participatory processes. In short, whether participation realizes its benefits or succumbs to its pitfalls is shaped by the dynamic interactions between participatory rationales, institutional factors, and power distribution within society.
This research was funded by the Swedish Research Council for Environment, Agricultural Sciences and Spatial Planning, Formas, Umea University, and the Swedish University of Agricultural Science, for the project “Ecosystem restoration in policy and practice: restore, develop, adapt“ (RESTORE), Grant number 2009-450 http://www.restore-project.org.
Abelson, J., and F. P. Gauvin. 2006. Assessing the impacts of public participation: concepts, evidence, and policy implications. Canadian Policy Research Networks, Ottawa, Ontario, Canada. [online] URL: http://www.ipea.gov.br/participacao/images/pdfs/abelson%20and%20gauvin_assessing%20pp%20impacts_2006.pdf
Agrawal, A., and C. C. Gibson. 1999. Enchantment and disenchantment: the role of community in natural resource conservation. World Development 27(4):629-649. http://dx.doi.org/10.1016/S0305-750X(98)00161-2
Agrawal, A., and K. Gupta. 2005. Decentralisation and participation: the governance of common pool resources in Nepal’s Terai. World Development 33:1101-1114. http://dx.doi.org/10.1016/j.worlddev.2005.04.009
Ansell, C., and A. Gash. 2008. Collaborative governance in theory and practice. Journal of Public Administration Research and Theory 18:543-571. http://dx.doi.org/10.1093/jopart/mum032
Arnstein, S. R. 1969. A ladder of citizen participation. Journal of the American Institute of Planners 35:216-224. http://dx.doi.org/10.1080/01944366908977225
Auld, G., and G. Q. Bull. 2003. The institutional design of forest certification standards initiatives and its influence on the role of science: the case of forest genetic resources. Journal of Environmental Management 69:47-62. http://dx.doi.org/10.1016/S0301-4797(03)00118-X
Baker, S., and K. Eckerberg, editors. 2008. In pursuit of sustainable development: new governance practices at the sub-national level in Europe. Routledge, New York, New York, USA.
Baral, N., and J. Heinen. 2007. Decentralisation and peoples’s participation in conservation: a comparative study of Western Terai of Nepal. Sustainable Development and World Ecology 14:520-531. http://dx.doi.org/10.1080/13504500709469751
Beierle, T. C. 2002. The quality of stakeholder-based decisions. Risk Analysis 22:739-749. http://dx.doi.org/10.1111/0272-4332.00065
Berkes, F., J. Colding, and C. Folke, editors. 2003. Navigating social-ecological systems: building resilience for complexity and change. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/CBO9780511541957
Biggs, S. D. 1989. Resource-poor farmer participation in research: a synthesis of experiences from national agricultural research systems, OFCOR, comparative Study No. 3. International Service for National Agricultural Research, The Hague, The Netherlands.
Bisset R. 2000. Methods of consultation and public participation. Pages 150-160 in N. Lee, and C. George, Editors. Environmental assessment in developing and transitional countries: principles, methods, and practice. Wiley, New York, New York, USA.
Black, J. 2008. Constructing and contesting legitimacy and accountability in polycentric regulatory regimes. Regulation and Governance 2:137-164. http://dx.doi.org/10.1111/j.1748-5991.2008.00034.x
Brooks, J. S., K. A. Waylen, and M. Borgerhoff Mulder. 2012. How national context, project design, and local community character influence success in community-based conservation projects. Proceedings National Academy of Sciences 109(52):21265-21270. http://dx.doi.org/10.1073/pnas.1207141110
Bryson, J. M., K. S. Quick, C. S. Slotterback, and B. C. Crosby. 2012. Designing public participation processes: theory to practice. Public Administration Review 73:23-34. http://dx.doi.org/10.1111/j.1540-6210.2012.02678.x
Bulkeley, H., and A. P. J. Mol. 2003. Participation and environmental governance: consensus, ambivalence and debate. Environmental Values 12:143-154. http://dx.doi.org/10.3197/096327103129341261
Burby, R. J. 2003. Making plans that matter: citizen involvement and government action. Journal of the American Planning Association 69(1):33-49. http://dx.doi.org/10.1080/01944360308976292
Burgess, J., C. M. Harrison, and P. Filius. 1998. Environmental communication and the cultural politics of environmental citizenship. Environment and Planning A. 30(8):1445-1460. http://dx.doi.org/10.1068/a301445
Carlsson, L., and F. Berkes. 2005. Co-management; concepts and methodological implications. Journal of Environmental Management 75:65-76. http://dx.doi.org/10.1016/j.jenvman.2004.11.008
Cash, D. W., W. C. Clark, F. Alcock, N. M. Dickson, N. Eckley, D. H. Guston, J. Jäger, and R. B. Mitchell. 2003. Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences 100:8086-8091. http://dx.doi.org/10.1073/pnas.1231332100
Chen, H., T. Zhu, M. Krott, and D. Maddox. 2013. Community forestry management and livelihood development in northwest China: integration of governance, project design, and community participation. Regional Environmental Change 13(1):67-75. http://dx.doi.org/10.1007/s10113-012-0316-3
Cini, L. 2011. Between participation and deliberation: toward a new standard for assessing democracy? Pavia Graduate Conference in Political Philosophy, Pavia, Italy. [online] URL: https://www.academia.edu/1369778/Deliberative_and_Participatory_Democracy_Towards_a_New_Model_of_Radical_Democracy
Cinner, J. E., T. R. McClanahan, M. A. MacNeil, N. A. J. Graham, T. M. Daw, A. Mukminin, D. A. Feary, A. L. Rabearisoa, A. Wamukota, N. Jiddawi, S. J. Campbell, A. H. Baird, F. A. Januchowski-Hartley, S. Hamed, R. Lahari , T. Morove, and J. Kuange. 2012. Comanagement of coral reef social-ecological systems. Proceedings National Academy of Sciences 109(14):5219-5222. http://dx.doi.org/10.1073/pnas.1121215109
Connaughton, B., B. Quinn, and N. Rees. 2008. Rhetoric or reality: responding to the challenge of sustainable development and new governance patterns in Ireland. Pages 145-168 in S. Baker and K. Eckerberg, editors. Pursuit of sustainable development: new governance practices at the sub-national Level in Europe. Routledge, New York, New York, USA.
Cooke, B., and U. Kothari. 2001. The case for participation as tyranny. Zed, London, UK.
Cox, M. 2012. Diagnosing institutional fit: a formal perspective. Ecology and Society 17(4):54. http://dx.doi.org/10.5751/ES-05173-170454
Dalton, T., G. Forrester, and R. Pollnac. 2012. Participation, process quality, and performance of marine protected areas in the wider Caribbean. Environmental Management 49(6):1224-1237. http://dx.doi.org/10.1007/s00267-012-9855-0
Dasgupta, A., and V. A. Beard. 2007. Community driven development, collective action and elite capture in Indonesia. Development and Change. 38(2):229-249. http://dx.doi.org/10.1111/j.1467-7660.2007.00410.x
Davidson, S. 1998. Spinning the wheel of empowerment. Planning 3:14-15.
DeCaro, D. A., and M. K. Stokes. 2013. Public participation and institutional fit: a-social-psychological perspective. Ecology and Society 18(4):40. http://dx.doi.org/10.5751/ES-05837-180440
della Porta, D., and M. Diani. 2006. Social movements: an introduction. Second edition. Blackwell, Oxford. UK.
Delli Carpini, M., F. M. Cook, and L. R. Jacobs. 2004. Public deliberation, discursive participation, and citizen engagement: a review of the empirical literature. Annual Review Political Science 7:315-344. http://dx.doi.org/10.1146/annurev.polisci.7.121003.091630
Dietz, T., and P. C. Stern, editors. 2008. Public participation in environmental assessment and decision making. National Academies Press, Washington, D.C., USA. https://doi.org/10.17226/12434
Dobson, A. 2003. Citizenship and the environment. Oxford University Press, Oxford, UK. http://dx.doi.org/10.1093/0199258449.001.0001
Dobson, A. 2014. Listening for democracy: recognition, representation, reconciliation. Oxford University Press, Oxford, UK. http://dx.doi.org/10.1093/acprof:oso/9780199682447.001.0001
Dryzek, J. S. 1990. Discursive democracy: politics, policy, and political science. Cambridge University Press, Cambridge, UK.
Emerson, K., T. Nabatchi, and S. Balogh. 2012. An integrative framework for collaborative governance. Journal of Public Administration Research and Theory 22(1):1-29. http://dx.doi.org/10.1093/jopart/mur011
Feudenberg, N., M. Pastor, and B. Israel. 2011. Strengthening community capacity to participate in making decisions to reduce disproportionate environmental exposures. American Journal of Public Health 101:123-130. http://dx.doi.org/10.2105/AJPH.2011.300265
Fiorino, D. J. 1990. Citizen participation and environmental risk: a survey of institutional mechanisms. Science, Technology, and Human Values 15(2):226-243. http://dx.doi.org/10.1177/016224399001500204
Fischer, F. 2003. Reframing public policy: discursive politics and deliberative practices. Oxford University Press, Oxford, UK. http://dx.doi.org/10.1093/019924264X.001.0001
Fraser, N. 1990. Rethinking the public sphere: a contribution to the critique of actually existing democracy. Social Text 25/26:56-80. http://dx.doi.org/10.2307/466240
Fraser, N. 1997. Justice interruptus: critical reflections on the “postsocialist” condition. Routledge, London, UK.
Garnett, S. T., J. Sayer, and J. T. du Toit. 2007. Improving the effectiveness of interventions to balance conservation and development: a conceptual framework. Ecology and Society 12(1):2. http://dx.doi.org/10.5751/ES-01963-120102
Gouldson, A., and J. Bebbington. 2007. Corporations and the governance of environmental risk. Environment and Planning C: Government and Policy 25:4-20. http://dx.doi.org/10.1068/c0614j
Grimble, R., and K. Wellard. 1997. Stakeholder methodologies in natural resource management: a review of principles, contexts, experiences and opportunities. Agricultural Systems 55(2):173-193. http://dx.doi.org/10.1016/S0308-521X(97)00006-1
Guarneros-Meza, V., and M. Geddes. 2010. Local governance and participation under neoliberalism: comparative perspectives. International Journal of Urban and Regional Research 34(1):115-129. http://dx.doi.org/10.1111/j.1468-2427.2010.00952.x
Gurney, G. G., J. E. Cinner, J. Sartin, R. L. Pressey, N. C. Ban, N. A. Marshall, and D. Prabuning. 2016. Participation in devolved commons management: multiscale socioeconomic factors related to individual’s participation in community-based management of marine protected areas in Indonesia. Environmental Science and Policy 6:212-220. http://dx.doi.org/10.1016/j.envsci.2016.04.015
Gutiérrez, N. L, R. Hilborn, and O. Defeo. 2011. Leadership, social capital and incentives promote successful fisheries. Nature 470(7334):386-389. http://dx.doi.org/10.1038/nature09689
Head, B. W., and J. Alford. 2015. Wicked problems: implications for public policy and management. Administration and Society 47(6):711-739. http://dx.doi.org/10.1177/0095399713481601
Hoover, K., and M. J. Stern. 2014a. Constraints to public influence in US Forest Service NEPA processes. Journal of Environmental Planning and Management 57:173-189. http://dx.doi.org/10.1080/09640568.2013.849232
Hoover, K., and M. J. Stern. 2014b. Team leaders’ perception of public influence in the US Forest Service: exploring the difference between doing and using public involvement. Journal of Environmental Planning and Management 57(2):157-172. http://dx.doi.org/10.1080/09640568.2012.756807
Hurlbert, M., and J. Gupta. 2015. The split ladder of participation: a diagnostic, strategic, and evaluation tool to assess when participation is necessary. Environmental Science and Policy 30(50):100-113. http://dx.doi.org/10.1016/j.envsci.2015.01.011
Innes, J. E., and D. E. Booher. 2004. Reframing public participation: strategies for the 21st century. Planning Theory and Practice 5:419-436. http://dx.doi.org/10.1080/1464935042000293170
Irvin, R. A., and J. Stansbury. 2004. Citizen participation in decision making: is it worth the effort? Public Administration Review 64:55-65. http://dx.doi.org/10.1111/j.1540-6210.2004.00346.x
Kemp, R., S. Parto, and R. B. Gibson. 2005. Governance for sustainable development: moving from theory to practice. International Journal of Sustainable Development 8:13-30. http://dx.doi.org/10.1504/IJSD.2005.007372
Kenney, S. 2011. Towards unsettling community development. Community Development Journal 46:i7-i19. http://dx.doi.org/10.1093/cdj/bsq050
Kitschelt, H. P. 1986. Political opportunity structures and political protest: anti-nuclear movements in four democracies. British Journal of Political Science 16:57-85. http://dx.doi.org/10.1017/S000712340000380X
Kooiman, J. 2000. Societal governance: levels, modes, and orders of social-political interaction. Pages 138-166 in J. Pierre, editor. Debating governance: authority, steering, and democracy. Oxford University Press, Oxford, UK.
Lambin, E. F., P. S. Chasek, T. E. Downing, C. Kerven, A. Kleidon, R. Leemans, M. Ludeke, S. D. Prince, and Y. Xue. 2002. The interplay between international and local processes affecting desertification. Pages 387-401 in J. F. Reynolds and D. M. Stafford-Smith, editors. Global desertification: do humans cause deserts? Dahlem University Press, Berlin, Germany.
Lange, P., P. P. J. Driessen, A. Sauer, B. Bornemann, and P. Burger. 2013. Governing towards sustainability-conceptualising models of governance. Journal of Environmental Policy and Planning 15(3):403-425. http://dx.doi.org/10.1080/1523908X.2013.769414
Laurian, L., M. Walker, and J. Crawford. 2017. Implementing environmental sustainability in local government: the impacts of framing, agency culture, and structure in US cities and counties. International Journal of Public Administration 40(3):270-283. http://dx.doi.org/10.1080/01900692.2015.1107738
Lawrence, A. 2006. ‘No personal motive?’ Volunteers, biodiversity, and the false dichotomies of participation. Ethics, Place and Environment 9:279-298. http://dx.doi.org/10.1080/13668790600893319
Leal, P. A. 2007. Participation: the ascendancy of a buzzword in the neo-liberal era. Development in Practice 17(4-5):539-548.
Lynam, T., W. De Jong, D. Sheil, T. Kusumanto, and K. Evans. 2007. A review of tools for incorporating community knowledge, preferences, and values into decision making in natural resources management. Ecology and Society 12(1):5. http://dx.doi.org/10.5751/ES-01987-120105
Lynham, J., B. S. Halpern, T. Blenckner, T. Essington, J. Estes, M. Hunsicker, C. Kappel, A. K. Salomon, C. Scarborough, K. A. Selkoe, and A. Stier. 2017. Costly stakeholder participation creates inertia in marine ecosystems. Marine Policy 76:122-129. http://dx.doi.org/10.1016/j.marpol.2016.11.011
March, J. G., and J. P. Olsen. 2006. Elaborating the “new institutionalism.” Pages 3-20 in S. A. Binder, R. A. W. Rhodes, and B. A. Rockman, editors. The Oxford handbook of political institutions. Oxford University Press, Oxford, UK.
Mccool, S. F., and K. Guthrie. 2001. Mapping the dimensions of successful public participation in messy natural resources management situations. Society and Natural Resources 14(4):309-323. http://dx.doi.org/10.1080/713847694
Meier, K. J., and L. J. O’Toole, Jr. 2006. Political control versus bureaucratic values: reframing the debate. Public Administration Review 66(2):177-192. http://dx.doi.org/10.1111/j.1540-6210.2006.00571.x
Nadeem, O., and T. B. Fischer. 2011. An evaluation framework for effective public participation in EIA in Pakistan. Environmental Impact Assessment Review 31:36-47. http://dx.doi.org/10.1016/j.eiar.2010.01.003
Newig, J., and O. Fritsch. 2009. Environmental governance: participatory, multi-level - and effective? Environmental Policy and Governance 19(3):197-214. http://dx.doi.org/10.1002/eet.509
Newman, J., M. Barnes, H. Sullivan, and A. Knops. 2004. Public participation and collaborative governance. Journal of Social Policy 33(2):203-223. http://dx.doi.org/10.1017/S0047279403007499
O’Faircheallaigh, C. 2010 Public participation and environmental impact assessment: purposes, implications, and lessons for public policy making. Environmental Impact Assessment Review 30(1):19-27. http://dx.doi.org/10.1016/j.eiar.2009.05.001
Ostrom, E. 1990. Governing the commons. Cambridge University Press, Cambridge, UK.
Padgee, A., Y.-S. Kim, and P. J. Daugherty. 2006. What makes community forestry management successful: a meta-study from community forests throughout the world. Society and Natural Resources 19(1):33-52. http://dx.doi.org/10.1080/08941920500323260
Parkins, J. R., and R. E. Mitchell. 2005. Public participation as public debate: a deliberative turn in natural resource management. Society and Natural Resources 18:529-540. http://dx.doi.org/10.1080/08941920590947977
Pohjola, M. V., and J. T. Tuomisto. 2011. Openness in participation, assessment, and policy making upon issues of environment and environmental health: a review of literature and recent project results. Environmental Health 10:58. http://dx.doi.org/10.1186/1476-069X-10-58
Predmore, S. A., M. J. Stern, and M. J. Mortimer. 2011a. Constructing the public; the ‘substantive sieve’ and personal norms in US Forest Service Planning. Journal of Environmental Planning and Management 54(3):403-419. http://dx.doi.org/10.1080/09640568.2010.507981
Predmore, S. A., M. J. Stern, M. J. Mortimer, and D. N. Seesholtz. 2011b. Perceptions of legally mandated public involvement processes in the U. S. Forest Service. Society and Natural Resources 24(12):1286-1303. http://dx.doi.org/10.1080/08941920.2011.559617
Pretty, J., and D. Smith. 2004. Social capital in biodiversity conservation and management. Conservation Biology 18(3):631-638. http://dx.doi.org/10.1111/j.1523-1739.2004.00126.x
Putnam, R. D. 1993. Making democracy work. Princeton University Press, Princeton, New Jersey, USA.
Putnam, R. D. 1995. Bowling alone: America’s declining social capital. Journal of Democracy 6:65-78. http://dx.doi.org/10.1353/jod.1995.0002
Raik, D. B., A. L. Wilson, and D. J. Decker. 2008. Power in natural resources management: an application of theory. Society and Natural Resources 21(8):729-739. http://dx.doi.org/10.1080/08941920801905195
Reed, M. S. 2008. Stakeholder participation for environmental management: a literature review. Biological Conservation 141:2417-2431. http://dx.doi.org/10.1016/j.biocon.2008.07.014
Restall, G., and J. Kaufert. 2011. Understanding how context shapes citizen-user involvement in policy making. Healthcare Policy 7(2): 68-82. http://dx.doi.org/10.12927/hcpol.2011.22664
Richards, C., K. L. Blackstock, and C. E. Carter. 2004. Practical approaches to participation. SERG Policy Brief No. 1. Macauley Land Use Research Institute, Aberdeen, Scotland, UK. http://www.macaulay.ac.uk/ruralsustainability/SERG%20PB1%20final.pdf
Rittel. H. W. J., and M. M. Webber. 1973. Dilemmas in a general theory of planning. Policy Sciences 4:155-169. http://dx.doi.org/10.1007/BF01405730
Rowe, G., and L. J. Frewer. 2000. Public participation methods: a framework for evaluation. Science, Technology, and Human Values 25:3-29. http://dx.doi.org/10.1177/016224390002500101
Sandström, C. 2009. Institutional dimensions of comanagement: participation, power, and process. Society and Natural Resources 22(3):230-244. http://dx.doi.org/10.1080/08941920802183354
Stern. M. J., and S. A. Predmore. 2012. The importance of team functioning to natural resource planning outcomes. Journal of Environmental Management 106(15):30-39. http://dx.doi.org/10.1016/j.jenvman.2012.03.049
Stern, M. J., S. A Predmore, M. J. Mortimer, and D. N. Seesholtz. 2010. The meaning of the National Environmental Policy Act in the U.S. Forest Service. Journal of Environmental Management 91(6):1371-1379. http://dx.doi.org/10.1016/j.jenvman.2010.02.019
Stirling, A. 2006. Analysis, participation and power: justification and closure in participatory multi-criteria analysis. Land Use Policy 23:95-107. http://dx.doi.org/10.1016/j.landusepol.2004.08.010
Stringer, L. C., A. J. Dougill, E. Fraser, K. Hubacek, C. Prell, and M. S. Reed. 2006. Unpacking “participation” in the adaptive management of social-ecological systems: a critical review. Ecology and Society 11(2):39. http://dx.doi.org/10.5751/ES-01896-110239
Stringer, L. C., M. S. Reed, A. J. Dougill, M. K. Seely, and M. Rokitzki. 2007. Enhancing participation in the implementation of the United Nations Convention to Combat Desertification. Natural Resources Forum 31:198-211. http://dx.doi.org/10.1111/j.1477-8947.2007.00154.x
Swyngedouw. E. 2005. Governance innovation and the citizen: the Janus face of governance-beyond-the-state. Urban Studies 42(11):1991-2006. http://dx.doi.org/10.1080/00420980500279869
Tarrow, S. 1994. Power in movement: social movements, collective action and mass politics in the modern state. Cambridge University Press, Cambridge, UK.
Tipple, T. J., and J. D. Wellman. 1991. Herbert Kaufman’s forest ranger thirty years later: from simplicity and homogeneity to complexity and diversity. Public Administration Review 51(5):421-428. http://dx.doi.org/10.2307/976411
Tsang, S., M. Burnett, P. Hills, and R. Welford. 2009. Trust, public participation and environmental governance in Hong Kong. Environmental Policy and Governance. 19(2):99-114. http://dx.doi.org/10.1002/eet.502
Turnhout, E., S. Van Bommel, and N. Aarts. 2010. How participation creates citizens: participatory governance as performative practice. Ecology and Society 15(4):26 http://dx.doi.org/10.5751/ES-03701-150426
van Tatenhove, J. P. M., and P. Leroy. 2003. Environment and participation in a context of political modernisation. Environmental Values 12:155-173. http://dx.doi.org/10.3197/096327103129341270
van Zeijl-Rozema, A., R. Cörvers, R. Kemp, and P. Martens. 2008. Governance for sustainable development: a framework. Sustainable Development 16:410-421. http://dx.doi.org/10.1002/sd.367
Vasstrom, M. 2014. Rediscovering nature as commons in environmental planning: new understandings through dialogue. International Journal of the Commons 8(2):493-512. http://doi.org/10.18352/ijc.459
Verba, S. 1967. Democratic participation. Annals of the American Academy of Political and Social Science 373:53-78. http://dx.doi.org/10.1177/000271626737300103
Wang. X., and M. Wan Wart. 2007. When public participation in administration leads to trust: an empirical assessment of managers’ perceptions. Public Administration Review 67(2):265-278. http://dx.doi.org/10.1111/j.1540-6210.2007.00712.x
Waylen, K. A., A. Fischer, P. J. K. McGowan, S. J. Thirgood, and E. J. Milner-Gulland. 2010. Effect of local cultural context on the success of community-based conservation interventions. Conservation Biology 24(4):1119-1129. http://dx.doi.org/10.1111/j.1523-1739.2010.01446.x
Yaffee, S. L., and J. M. Wondolleck. 2000. Making collaboration work: Lessons from a comprehensive assessment of over 200 wideranging cases of collaboration in environmental management. Conservation 1(1):17-24. http://dx.doi.org/10.1111/j.1526-4629.2000.tb00156.x
Yang, K. 2005. Public administrators’ trust in citizens: a missing link in citizen involvement efforts. Public Administration Review 65(3):273-285. http://dx.doi.org/10.1111/j.1540-6210.2005.00453.x
Yang, K., and K. Callahan. 2007. Citizen involvement efforts and bureaucratic responsiveness: participatory values, stakeholder pressures, and administrative practicality. Public Administration Review 67(2):249-264. http://dx.doi.org/10.1111/j.1540-6210.2007.00711.x
Young, O. R. 2008. The architecture of global environmental governance: bringing science to bear on policy. Global Environmental Politics 8(1):14-32. http://dx.doi.org/10.1162/glep.2008.8.1.14
Zuhair, M. H., and P. A. Kurian. 2016. Socio-economic and political barriers to public participation in EIA: implications for sustainable development in the Maldives. Impact Assessment and Project Appraisal 34(2):129-142. http://dx.doi.org/10.1080/14615517.2016.1176404