|Home | Archives | About | Login | Submissions | Subscribe | Contact | Search|
Copyright © 2010 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf version of this article The following is the established format for referencing this article:
Huitema, D., C. Cornelisse, and B. Ottow. 2010. Is the jury still out? Toward greater insight in policy learning in participatory decision processes—the case of Dutch citizens' juries on water management in the Rhine Basin. Ecology and Society 15(1): 16. [online] URL: http://www.ecologyandsociety.org/vol15/iss1/art16/
Research, part of Special Feature on Implementing Participatory Water Management: Recent Advances in Theory, Practice and Evaluation Is the Jury Still Out? Toward Greater Insight in Policy Learning in Participatory Decision Processes—the Case of Dutch Citizens' Juries on Water Management in the Rhine Basin
1VU University Amsterdam - Institute for Environmental Studies (IVM), 2Deltares
This article discusses the potential for policy learning offered by participatory processes, specifically so-called citizens' juries. We establish the need for policy learning by pointing to the increased complexity of water management tasks and challenges. A conceptual discussion subsequently distinguishes between cognitive, normative, and relational learning. The public participation literature suggests that participatory processes will contribute to various forms of learning. We assess the truth of this assumption on the basis of three case studies: citizens' juries on water management in the Dutch part of Rhine basin. We analyze whether the three forms of learning have occurred among jurors, and among policy makers. We find high levels of cognitive, normative, and relational levels of learning for the jurors, but relatively low levels of learning for policy makers. We analyze the reason for this divergence.
Key words: citizen participation; citizens' jury; policy learning; Rhine river; water management
The field of water management is in flux. Climate change is making itself felt, as an increase in the occurrence of extreme water events and rising sea levels are expected (see, e.g., Easterling et al. 2000, Cabanes et al. 2001, Gleick at al. 2001, Alley et al. 2005). Serious flaws in the traditional engineering approach to water management have become clear in recent decades—including the massive social and ecological damage caused by dams (see, e.g., World Commission on Dams 2000, Gleick 2003, Stone 2008). There has been a great deal of poorly planned development in arid and semi-arid areas, which has led to a large demand for water and considerable—and often unexpected—environmental impacts (see, e.g., Genxu and Guodong 1999, Turner et al. 2007).
The relative contributions to water problems and the burden of their consequences are often apportioned in an asymmetrical way among actors, creating a significant collective action problem (Conca et al. 2006, Meinzen-Dick 2007). Institutions able to overcome this are often still in a formative stage. One complication is that water problems are often interrelated with many other issues, including political tensions between countries and groups within countries (Burchi and Spreij 2003, Blomquist et al. 2005, Kemper at al. 2005). Therefore, water managers are urged to act by protagonists who have widely different values, worldviews, and understandings of the problem at hand (Falkenmark et al. 2004).
In developing responses, water managers are faced with relatively high levels of uncertainty surrounding the consequences of their actions as they are dealing with social–ecological systems that exhibit, among other things, complexity, non-reducibility, spontaneity, variability, and a collective quality (Dryzek 1987:28–33). The implication is that water managers do not and will not completely know the social–ecological systems they are intervening in. Yet, they often have little room for error as so many depend on them to find the right answers. Water issues can thus be portrayed as yet another example of a “wicked problem” (Rittel and Webber 1973).
In response, “soft solutions,” such as flood retention areas, community-scale infrastructure, decentralized and open decision-making systems, and their like, are increasingly advocated and turned into practice (Gleick 2003; Huitema and Meijerink (2009) offer a global perspective). These solutions are characterized by an emphasis on flexibility and reversibility, but more importantly perhaps, by a strong orientation toward learning. Learning our way out of environmental problems has become the standard recommendation in the debate on sustainability (see, e.g., Dryzek 1987, Pahl-Wostl 2002, Tàbara and Pahl-Wostl 2007).
The literature on “adaptive (co-)management” (Olsson et al. 2004, Armitage et al. 2007, Huitema et al. 2009) and “social learning” (see, e.g., Tàbara and Pahl-Wostl 2007) suggests that the greatest opportunities for learning arise in settings where multiple actors, with varying understandings of the problems at hand, collectively discuss problems, potential solutions, and the effects of measures taken. It is argued that managing water requires the involvement of all interests. They can, and must, “learn together to manage together” (Ridder et al. 2005), which implies that high levels of public participation are required.
In this article, we wish to assess empirically the connection between public participation and learning, while acknowledging that learning may not be the only goal of public participation exercises and that other goals may have greater relevance in some circumstances (for an overview of possible goals and evaluation criteria, see Renn et al. 1995, Coenen et al. 1998, Rowe and Frewer 2000, Rowe et al. 2004, 2008, van de Kerkhof and Huitema 2004). We define public participation as the taking part, by ordinary citizens or their collectives, in the processes of government and/or governance; this refers to situations in which a (substantial) number of citizens play a part in the process by which leaders are chosen and policies are shaped and implemented (Birch 1993). Typical advantages associated with public participation are almost all—directly or indirectly—associated with various forms of learning. Public participation is expected to:
The potential obstacles en route to meaningful public participation, including the desired learning effects, have not gone unnoticed in the literature on adaptive management and adaptive (co-)management. Armitage et al. (2007:6–10), for instance, mention the need for more insight on enabling policy environments for adaptive (co-)management. They suggest that “partnerships and power sharing” and “learning, knowledge use, and social capital” should be among the priorities for additional research. They indicate that questions about the conditions for partnerships that “really” share power, and about the ways to move from instrumental learning to learning about appropriate goals are key for the adaptive (co-)management literature. This article seeks to contribute to our understanding of the relationship between public participation and learning by an in-depth analysis of three so-called citizens’ juries that were organized in the Netherlands. All three juries took place in the Dutch part of the Rhine basin, each addressed questions about water management, and each resulted in recommendations for public policies. We will not report on our full evaluation of the three cases, but instead our key question is whether “policy learning” occurred in these three cases and what were the contributing or inhibiting circumstances for this.
In the next and third sections, we describe further the participatory methodology applied in our three case studies. We show that the mechanisms for learning envisioned in this method are partly related to deliberation between members of the jury, partly related to the presentation of competing “truth claims” by various witnesses, representing a diversity of viewpoints and interests. In the fourth section, we describe the topics addressed, the background of the three juries, their composition, their location, etc. We note that, although we applied the same methodology, the status of the various juries was somewhat different. Especially noteworthy is the fact that two out of the three juries described here essentially had an experimental status, whereas one jury was connected to a real-life policy-making process.
In the fifth section, we conceptualize the term “policy learning” and suggest that policy learning can occur at several levels, and that one can analyze “who learns,” “what is learned,” and “why” learning takes place. Our article focuses on the “what is learned” question, distinguishing three potential forms of learning: cognitive, normative, and relational. In the sixth section, we describe the range of methods we have applied to assess the degree of policy learning that has occurred in the three case studies. The methods include pre- and post-jury “cognitive maps,” standardized questionnaires for jurors, close observation of the interactions within the jury and between the jury and witnesses and elected representatives, and several ex post interviews with those involved.
In the seventh section, we discuss whether the three juries resulted in new factual knowledge. On the basis of an analysis of ex ante and ex post cognitive maps, we conclude that members of the jury, in particular, learned a great deal during the jury. Learning on the part of the water managers was limited, however. In the eighth section, we analyze whether the jurors came to evaluate the issues at hand differently after having taken part in the jury. We conclude that the jurors did change their norms. Normative learning on the part of the water managers was again limited, however. Based on the questionnaires (before and following jury participation, with the control group), section nine describes how the jurors became much more involved in water issues. Interestingly, however, this overall increase in the level of involvement was mainly thanks to an increase in self-confidence and interest in water issues. At the same time, cynicism about the policy makers increased. There was moderate learning on the part of the water managers, especially in the sphere of communication with “ordinary” citizens.
In essence, we have found that the members of the juries learned a lot, but water managers not so much. In the tenth section, we describe a range of possible explanations, building on Mostert et al. (2007), and examine whether they apply to our cases or not. The final section sketches the implications of our analysis and suggests some further research priorities.
The citizens’ jury and the related “planning cell” were pioneered in the early 1970s by Ned Crosby in the United States and Peter Dienel in Germany (see Stewart et al. 1994). In the United States (USA), the United Kingdom (UK), and Germany, citizens’ juries have been organized for decades. In other countries, such as Belgium and Australia, the method has been introduced more recently, and it is now also being applied at the level of the European Union.
A citizens’ jury usually consists of a group of 12 to 24 randomly selected citizens who attend a series of meetings in order to learn about and discuss a specific issue or “charge” (as the question up for debate is commonly termed) and make their recommendations public (Crosby 1995). A citizens’ jury looks somewhat like the criminal-court juries in the UK and the USA. An important difference is that citizens’ juries do not pass judgment on criminal matters but on policy issues. Furthermore, a citizens’ jury does not elicit a simple choice such as a “guilty” or “not guilty” verdict, but rather an informed policy recommendation.
The jury receives its information from a range of so-called witnesses, all representing specific interests and viewpoints. The task of each witness is to present their own viewpoint to the jurors; the jury’s decision is the result of their own (local and other) knowledge and the open competition between, and balancing of, the different, conflicting interests and viewpoints of the witnesses. The witnesses are selected on the basis of their expertise and/or on the grounds that they represent affected interests (Kuper 1996, Smith and Wales 2000). The neutrality of the jury outcomes can be improved by having the jurors interrogate a variety of witnesses representing different, even opposing, opinions. With trained moderators ensuring fair proceedings, the jurors hear evidence, question witnesses, discuss the issues that are raised, and make an informed judgment about the charge.
Normally, agreements are reached between the organizers of juries and the commissioning body about the status of the jury recommendations. In some cases, juries have provided binding recommendations (Stewart et al. 1994), but it is far more common that the commissioning bodies commit themselves only to careful consideration of the jury’s recommendations. This implies that they need to engage with the jurors and discuss the recommendations with them; it also means that they have to indicate how the recommendations will translate into action, and if that is not the case, why not.
How does the citizens’ jury literature approach learning, and what learning effects are to be expected? The starting point for the learning process in juries is deliberation (for the concept of deliberation, see, e.g., Bohman and Rehg 1997, Burkhalter et al. 2002, Delli Carpini et al. 2004, Levine et al. 2005). According to Smith and Wales (2000), central to the deliberative conceptualization of citizens’ juries is that every member of the jury is fundamentally equal. This equality could be enforced by drawing up “rules of conduct, which typically emphasize the need to respect and listen to the arguments of others” (Smith and Wales 2000:58). The moderators of the jury are given the important role of facilitating equality—a difficult task requiring considerable social skills.
In the context of a citizens’ jury, much information on problems, solutions, and possible criteria for weighting solutions is exchanged so that issues can be considered holistically and bounded rationality may be overcome. New interpretative frames emerge during the deliberative process, which shed a new light on existing information. Therefore, the learning process that evolves in a citizens’ jury may alter the perceptions of the participants and enhance their readiness to change attitudes and opinions (Burkhalter et al. 2002). This is not supposed to occur through a process of bargaining and persuasion, but through a process of discussion, the mutual exchange of arguments and reflections, and the weighting of these arguments in an atmosphere that is “egalitarian, uncoerced, competent, and free from delusion, deception, power and strategy” (Smith and Wales 2000:53). In some cases, it will be necessary to develop new ways of talking and reasoning so that participants become able to understand each other despite cultural and other differences (Burkhalter et al. 2002). Segall (2005:371) suggests that participating in a citizens’ jury may contribute to expanding participants’ moral and intellectual horizon and may enable people to get insight into the workings of the political process and thus increase “political sophistication,” and finally that jurors tend to develop more “solidaristic” attitudes toward the subject that they were charged with.
Obviously, the jurors themselves are the main beneficiaries of the learning that occurs. Indeed, the citizens’ jury is often put forward as a way to achieve a more informed citizenry (Crosby 1995). It is, however, not only the jurors who learn. Smith and Wales suggest that policy makers should also be able to learn from citizens’ juries as they obtain an insight into what ordinary citizens think and are provided with an opportunity to establish closer relations with a part of the population: “It is commonly argued that citizens’ juries afford the opportunity for informed deliberation and active citizenship and are a potential mechanism for overcoming cleavage between the ‘privileged’ decision maker and the ‘administrees’, the majority of the population” (Smith and Wales 2000:55).
We can conclude that the literature on citizens’ juries sees learning as an important goal; the literature is rife with high expectations about such positive effects. There have, however, also been critical discussions of citizens’ juries in general and learning in particular (see, e.g., Huitema et al. 2007). Consensus building is associated with the deliberative processes of interaction in juries. Ward et al. (2003) express concern about the opportunity for jurors to express differences, and they warn citizens’ jury practitioners not to move too quickly to a consensus as this may keep certain viewpoints and (value) issues that stand in the way of reaching a consensus out of the discussion, thus limiting the learning effect. Another potential problem is that, despite being a very intensive process, citizens’ juries are disbanded after a period of 2 to 6 mos., thus limiting the possibilities for continued engagement and deliberation and, possibly, follow-up of the implementation process of the recommendations. Finally, as with participatory processes more generally, the interaction between jury and commissioning body will not always be characterized by learning; non-use, strategic use, or purely symbolic use are also possible. Part of the explanation for that is that the citizens’ jury will quite often sit somewhat uncomfortably in the environment of decision making by elected representatives.
The authors of this article were involved in the organization of three citizens’ juries on water management in the Dutch part of the Rhine basin; these were among the first ever to be held in that country. The first jury took place at the end of 2003 and over the first few months of 2004 in the city of Lelystad (see Fig. 1). This jury consisted of 14 members and it discussed the priorities that government should set for water quality in the Markermeer, a lake in the central Netherlands. The European Commission funded the jury as an experiment to assess the usefulness of the method in the drafting of water-basin management plans as required by 2008 legislation of the European Union (the Water Framework Directive, 2000/60/EC). The experiment was not connected to a real-life policy process, although policy makers at the provincial level and the National Water Management Authority enthusiastically collaborated.
The second jury took place at the end of 2004 and during the first half of 2005. It was organized in the same manner, with the notable difference that its recommendations fed into the regional land-use planning process for the province of Flevoland (see Fig. 1). This is a legally required plan, which also addresses water management. The jury was commissioned by the provincial Parliament that formally had to adopt the plan. Whereas the first jury consisted of 14 residents from the city of Lelystad only, the second jury was subdivided into three separate sub-juries, each consisting of 12 to 14 residents from the three zones of the province of Flevoland. The second jury process was also more extensive, taking place in three different rounds over 7 mos. rather than the 6 weeks allotted for the first jury.
The third jury, finally, took place in the city of Utrecht (see Fig. 1) in the summer of 2007, and addressed priorities in managing urban water streams. This jury was initiated under a EU-funded study of adaptive water management (see http://www.newater.info/), and was supported by the regional waterboard and the municipality of Utrecht. The jury consisted of 15 residents from the city of Utrecht who were active over a period of 4 mos.
In all three cases, the organizing team followed a standard jury format as described by Huitema (2003). There is no full-fledged international standard for organizing citizens’ juries, although various guidelines and indications for organizers are available (see, e.g., the website of the Jefferson Center, http://www.jefferson-center.org/). The format laid out by Huitema (2003) is based on a meta-analysis of the then available guidance documents on how to organize a citizens’ jury and, therefore, provided an explicitly reasoned and systematic approach.
The approach implies that the jury must resemble the public in terms of gender, age, and education level, and this drove our efforts in selecting from the pool of potential jurors. The public authorities involved in all cases pointed to the importance of ethnicity as an additional criterion for selecting jurors. We defined the jury as a place sheltered from advocacy by selecting jurors who had an open mind and who did not pursue a specific interest. Talks among the organizing team of the first jury led us to keep the topic of the juries rather general so that the jury selection process would not attract advocates (this was obviously not possible for the second and third juries as the general topic was clear). In addition, we decided to phone all volunteers and ask them a standard list of questions to assess their motivation, both in terms of their resolve to follow the entire program and their reasons for participating.
The juries were presented with expert knowledge during several preparation meetings, and we invited witnesses with rather different expertise, interests, and viewpoints with regard to the issue. During the first jury, the organizing team took on the task of finding witnesses. We set out to find witnesses who were advocates of a particular interest (e.g., fisheries) and who would enthusiastically defend it. Interestingly, the jurors did not appreciate that approach and required us to invite government witnesses, who they felt to be more neutral. In the second and third juries, the jurors were able to play a greater role in naming various relevant witnesses. In all cases, the organizing team phoned the witnesses and stressed the importance of sending a representative with appropriate communication skills, who would be open to different kinds of questions from the citizens.
We also made a list of publications that provided relevant background information and we screened these for neutrality and factuality. With regard to the decision-making process in the jury, we told the jurors that they should signal and discuss instances where, in their opinion, other jurors and also people of the organizing team were biased toward certain positions. We stressed the fact that everyone should feel free to express themselves, and the moderators involved made an effort to draw the less-vocal people into the discussion. As organizers, we told the jurors that it was not necessary for them to reach a consensus on every topic. It was emphasized that not only the opinion of the majority, but also the minority had to be taken into account. Therefore, the proposal was that if the discussion tended to become unproductive, the jury could terminate the debate by voting. In all three juries, such instances were very rare, however, as jurors preferred to develop common positions.
There is a rich literature on learning in public policy and management. Here, we refrain from an exhaustive treatment of this literature but focus on certain often-used distinctions instead. At a general level, policy learning is mostly conceived as implying a change in thought in a policy community, emanating from the assimilation of new knowledge or from past experience. The policy-learning literature is characterized by considerable definitional ambiguity and different concepts underlying the same or similar terms. Lacking space to delve into the specifics of this debate, we simply borrow one of the most quoted definitions of policy learning, which defines the term as “relatively enduring alterations of thought or behavioral intentions that result from experience and that are concerned with the attainment (or revision) of public policy” (Sabatier 1998).
Bennett and Howlett (1992) surveyed the literature on policy learning and found several rather different perspectives on the questions “who learns,” “what is learned,” and “to what effect.” As to the question about who learns, the basic distinction is between policy makers and societal actors. Some authors, such as Hall (1993), largely focus on the lessons that policy makers draw from their experiences, whereas others have shown greater interest in the way in which (groups of) societal actors, such as “advocacy coalitions” (Sabatier 1988) or “epistemic communities” (Haas 1992), learn, whether in interaction with policy makers or not. Obviously, the way the general public learns about policies is also relevant, but this is the topic of a different literature, the literature on agenda formation and agenda setting (see, e.g., Wanta 1997, McCombs 2005).
As for “what is being learned,” most writings on policy learning distinguish between different types and different degrees of learning (see Swartling and Nilsson 2007). Regarding the types of lessons learned, we can mention Webler et al. (1995), who suggest that there is a difference between the “cognitive enhancement” of participants—i.e., the acquisition of knowledge—and their “moral development”—how individuals come to be able to make judgments about right and wrong. Others have pointed to the importance of what we may refer to as “relational learning.” This type of learning relates to issues such as trust building, changes in the ability to collaborate, and changes in the ability to understand another party’s goals and preferences (see, e.g., Imperial and Hennessey 1999, Imperial 2005).
Apparently, the distinction between different degrees of learning is a useful device for carving out a more precise tableau in empirical studies on policy learning, as many authors suggest typologies on this basis. Most authors distinguish between a technical level and one or two “conceptual” levels at which learning can take place (cf. Foil and Lyles 1985, Hall 1993, Argyris and Schön 1996). Relevant here is the conceptualization of Argyris and Schön, for whom single-loop learning is “when a mismatch is corrected without changing the underlying values and status quo that govern the behaviours” (see Argyris 2003). Double-loop learning, by contrast, implies the mismatch being “corrected by first changing the underlying values and other features of the status quo” (Ibid.). The similarity between this type of learning and “moral development” as just discussed is obvious. Deutero-learning, finally, reflects on the institutional context for learning within an organization, and pertains, among other things, to the awareness that the organization needs to learn in the first place (Ibid.).
As to “what effect” policy learning is intended for, the overview of Bennett and Howlett (1992) suggests that most authors associate policy learning with policy change, in the sense that they only want to speak of policy learning in cases where policies have been modified or new policies have been adopted. In our opinion, this is a dubious assumption, for two reasons. The first is that policy change is often a result of other factors than policy learning. One can think of changes in government, bargaining between parties in the policy process, the emergence of powerful lobby groups, etc. Secondly, even if policy learning does occur, it does not always express itself in the form of policy change, but may equally well result in a better foundation for existing policies. This could also be seen as a form of policy learning, as the evidence base for the current policy would have increased in such a case.
All this means that policy learning may not always be intended for policy change. In fact, we would argue that a look at the different possible goals of participatory processes provides better answers to the question “to what effect” learning occurs from the perspective of the current article. Participatory processes can be initiated for several possible reasons, including raising awareness, improving decision quality, enhancing legitimacy and gaining acceptance, improving transparency, and improving collaboration (van de Kerkhof and Huitema 2004; also see Fiorino 1990, Laird 1993, Rowe and Frewer 2000, Burkhalter et al. 2002, Delli Carpini et al. 2004). The specific learning goal chosen for any participatory exercise is likely to affect the other elements (who needs to learn, what needs to be learned) as well. If participatory exercises are intended to improve collaboration, both water managers and societal actors will need to learn, and possibly some level of relational learning may be required. This is less so when the goal is to gain acceptance of decisions already made (for a broader discussion of the connection between design choices and possible goals, see Fung (2003)).
In our empirical analysis of the three cases, we decided to focus exclusively on questions about “what” was learned as such questions had relevance for all three cases. Table 1 provides an overview of the types of learning we decided to measure.
Obviously, the questions about “who learns” and “to what effect” have relevance for our cases too. Because citizens’ juries are intended as an exchange between a representative sample of the general population and a set of policy makers, we decided to focus on the learning of these two parties (jury members and policy makers). As the various forms of learning itself were an important objective for each of the three juries discussed here, the “to what effect” question largely overlaps with questions about “what” is learned.
The findings presented in this article stem from the application of various research methods. The following methods were used:
In this section, we discuss the indications of cognitive learning we found in our empirical data. We are looking for factual learning without changing underlying norms, values, or belief systems, and we analyze whether such learning has occurred among the jurors and among the policy makers.
We start with cognitive learning among the jurors. In the first jury, we collected data about the jurors’ factual knowledge by asking them to sketch their cognitive map of the issue that was under consideration in the jury, namely water quality in the Markermeer. In this map, they were asked to list important actors and substances that affect water quality in the lake. Figures 2 and 3 provide an example of the cognitive maps that were drawn before and after the jury, respectively, by one of the jurors. One can glean from these maps that at the start of the jury process this specific juror had very few ideas about the issue of water quality, but that he increased his understanding of the ecological relationships with regard to water quality considerably by the end of the process.
Figures 2 and 3 provide only one example of how the underhstanding of water-quality issues was enriched by the jury process. A justifiable question, therefore, is whether this is applicable across the entire jury. To determine this, we counted the number of items drawn on all 13 cognitive maps before (T0) and after the jury (T1). By adding the total number of items (usually actors and substances) listed in the maps and comparing the total numbers before and after the jury, we got an impression of the cognitive learning that occurred in the group. The analysis by one of our team members (Terweij 2004) showed that the total number of items mentioned in connection with water quality rose by 77% over the period of the jury, tentatively suggesting that the jurors’ awareness of factors affecting water quality had substantially increased. We have to treat the findings on the increased number of items with due care. First of all, learning may express itself not only in the increase in the number of items mentioned by jurors, but also in a decrease. This relates to the second issue we should mention, which is the fact that not all items mentioned by jurors may really be connected to water quality, or not in the way they envisioned. The organizing team considered having the cognitive maps reviewed by experts (of whatever nature), but we decided against it for a number of reasons. One is the fact that “water quality” in itself is a contested concept, which was interpreted very differently by the various witnesses for the jury (for details, see Huitema et al. 2004). The second is that the relationships between various items and the issue of water quality are not always clear and are the subject of scientific uncertainty. Finally, there was an objection of a more philosophical nature that having experts review the cognitive maps would imply that expert knowledge was valued more highly than ordinary knowledge.
Fortunately, there is further supporting evidence of cognitive learning among the jurors. This can be found in the answers to the pre- and post-jury questionnaires, which contained the question “which substances does the Markermeer contain and which properties does it have at this moment?” The jurors were given a list of several polluting substances (e.g., phosphates and heavy metals) and properties (e.g., mosquitoes and fish); they were asked to tick one or more of these items if they thought they were present in the lake. Before the jury, half the jury members could not indicate one specific property or substance (they answered “I do not know”). After the jury process, all jurors except one were able to indicate one (or more) specific property or substance. In addition, the jury recommendations were sent to the various witnesses and policy makers, and they expressed strong satisfaction with the underlying rationale of the jury recommendations.
Did the juries also result in cognitive learning on the part of the policy makers? Here, we need to base ourselves on the ex-post interviews we had with them (for all juries), and the comparison of jury recommendations with the land-use plan for the province (second jury). Most of our interviewees said they had witnessed a high level of cognitive learning among the jurors, but also indicated that, from their perspective, the jury recommendations did not contain much new evidence or many new policy ideas. Our impression is that this may be because the juries stayed relatively close to the information they received from the witnesses. And because the jurors tended to favor “neutral” witnesses, relatively few new ideas emerged in the process. It is thus no surprise that policy makers saw the recommendations as “relatively safe” and “nothing wild.” Most of the elected representatives we interviewed suggested that the jury recommendations were mainly regarded as a support for the draft policies made by the provincial executive. This is confirmed by Leever’s (2007) analysis of the debate on the provincial land-use plan in Parliament. He concluded that politicians made relatively little use of recommendations, save for some references to some of the creative slogans that the juries came up with, such as the idea to promote Flevoland as the “weekend province” and thus increase the level of recreation. However, an extensive comparison between the jury recommendations and the eventual provincial land-use plan (Edwards 2007) suggests that there was not a big discrepancy between the jury recommendations and the final version of the plan either.
Earlier we defined normative learning as learning encompassing a change in norms, values, and belief systems. As with cognitive learning, we assess to what extent this type of learning is observable in the three cases. As for normative learning within the juries, we relied heavily on a comparison of pre- and post-jury questionnaires filled out by the jurors in the first jury. The assessment of normative learning among the policy makers draws on the comparison of jury recommendations with the content of Parliamentary debates and the eventual policy plan that was adopted after the second jury. Our interviews with the policy makers involved (all three juries) also form a source of information for us.
Regarding normative learning among the jurors who participated in the first jury, we used a set of normatively laden questions in our pre-jury questionnaire that was repeated directly after the jury (for a sample of the questions, see Appendix 1). The normative questions inquired about the importance the jurors attached to the various users of the lake. Eight user groups—including the most important users of the lake in economic terms—were included in the research (Huitema et al. 2004). Six of these user groups also played an active role in the jury as witnesses. Only representation from the industry and shipping sectors was absent in the jury although the jury did receive written materials on these two user groups. The jurors were asked to rank the importance of the various groups on a one-to-ten scale, thereby indicating their normative preferences in terms of policy development. We then measured the standard deviation in the answers per sector, and calculated an average standard deviation for all sectors. Changes in the average standard deviations were interpreted as indications of normative convergence or divergence within the jury, and thus as normative learning.
Figure 4 shows that the priorities of the jurors changed, and that there was greater normative agreement after the jury (T1) than before (T0). The figure shows that for seven out of eight sectors, the standard deviation of juror answers declined, and the average standard deviation as indicated by the horizontal lines declined quite substantially. A T-test suggested that this difference is significant (t = 2.564, p = 0.019). Therefore, we can conclude that convergence of opinions, and subsequently normative learning, did occur among the jurors. The prioritization of sectors itself is interesting because it shows that the jury members, after taking part in the jury process, came to attach less priority to sectors like fisheries and agriculture and more priority to nature conservation and recreation.
Which indications did we see for normative learning on the part of the policy makers? As with cognitive learning, our interviewees largely pointed to a lack of learning in this area, as they interpreted the jury recommendations as a support for their own political positions. This is a remarkable finding, as the elected politicians we interviewed hailed from various political parties, and thus take different views of the issues at stake. This remarkable tendency can be explained by the “cherry picking” by the various Members of Parliament in the jury recommendations. In the formal presentations of the jury findings, several political parties emphasized the similarities between their own positions and those of the jury, while asking critical questions about some of the recommendations that their party did not agree with (Edwards 2007, Leever 2007). This made the interactions between the jury and Parliament relatively uncomfortable, a problem that was aggravated by the bureaucratic language used by the Members of Parliament, and the tendency of some of the Members to refer to existing (detailed) policy documents, which the jurors were not always aware of. What also complicated the interaction (Edwards 2007) is the fact that the timing of the jury was several months before the debate on the land-use plan in Parliament and thus before the various parties had formed their opinions internally. As long as they did not have backing from their respective political parties, the Members of Parliament were not inclined to take positions and have a meaningful debate with the jury. Edwards (2007: 19) suggests that the chair of the Parliamentary Committee that received the jury recommendations could have forced such positions by defining a set of “decision points” for the committee after the interaction with the jury.
Here, we have defined relational learning as enhanced trust and improved understanding of the mindsets of others. Again, our empirical observations for both the members of the jury and the policy makers are discussed.
As for the members of the second jury, we surveyed their opinion through a written questionnaire, before and after the jury. In this questionnaire (available upon request), we incorporated a set of questions from the Dutch National Elections Research Program. These questions are intended to measure the level of “political involvement” of the Dutch population, which refers to the way citizens relate to the political system. Political involvement consists of five items, including interest in politics/policy, satisfaction and cynicism about the political system, and the belief that one can or will become involved in politics. Obviously, this is merely one indicator for relational learning, and there are various other elements that we could have looked at, such as the change in relationships between the jurors, or between the jurors and witnesses, and the relationship between the various policy makers involved. The level of political involvement of jurors was the most interesting aspect for us because of the lively debate in the Netherlands about the distance between the ordinary public and policy makers, and because sustained co-management is dependent on the willingness of ordinary citizens (and others) to engage with policy makers in a debate on environmental issues. Political involvement is thus a condition sine qua non for adaptive (co-)management, and participatory methods that reduce the level of political involvement should preferably not be used.
As it is well known that those willing to engage in participatory exercises are often those who already have a high level of political involvement, and because of methodological reasons, we decided to create a control group for our measurements. The control group consisted of a random sample of those people who had been invited to join the jury, but had refused to do so. Methodologically, the control group is important because of the “Hawthorne effect,” which refers to the fact that observation itself has an effect on behavior. In this case, asking questions on the political system may affect the level of political involvement that one finds.
Figure 5 shows the levels of political involvement of both the jurors (G1) and the control group (G2) before (T0) and after the jury proceedings (T1).
Some issues stand out. First is the fact that the citizens’ jury members were more politically involved than the control group from the start, as denoted by their greater level of interest, confidence, satisfaction, and readiness and their lower level of cynicism at T0. Their overall level of political involvement is 0.40 higher on a five-point scale than the level for the control group (Lavrijsen 2005). This suggests potential problems with the representativeness of the jurors in the sense of political–social resemblance to the general public (for further analysis, see Huitema et al. 2007). Another interesting finding is that both the jury members and the control group grew in their levels of political involvement in the period before and after the jury (from 3.82 to 4.34 and from 3.42 to 3.66, respectively), suggesting the potential occurrence of the Hawthorne effect. However, the level of political involvement of the jurors grew more substantially and the growth was found to be statistically significant (T-test, p = <0.10) only for the jury group. Finally, it is noteworthy that the overall level of political involvement—determined as an average of the scores on the five components of political involvement—for the jury grew despite the fact that cynicism actually rose, effectively depressing the rise in political involvement. Cynicism is measured through the jurors’ response to questions such as “politicians only care for their own interest” and “politicians will never listen to someone like me.” In this sense, an increase in cynicism is most likely to be caused by the failed interaction between jury and Provincial Parliament, which we just discussed. The measured increase was not significant (0.173) in this case, but it does signal a warning for organizers. Lavrijsen (2005) suggests that the greatest increase in cynicism was located in the most active contingent of jurors in terms of number of interventions in the debate. The greatest risk of disillusionment is probably in that group.
How was the relational learning for the policy makers? Here, we lack statistical evidence, but we can lean on our ex-post interviews with the various policy makers for all three juries and the evaluations of the second jury (Huitema 2006, Edwards 2007, Leever 2007). One thing that stands out in these evaluations is the fact that the first and third juries took place in a relatively relaxed environment, as these juries were more or less seen as experiments, without high expectations regarding follow-up in policy terms. In both cases, a high-ranking politician (member of the executive) was greatly supportive of the idea, and this implied that the juries received much collaboration from the province and municipality, respectively (locations for meetings, officials were told to make themselves available on evenings and weekends, etc.). These politicians also had great communication skills, which they used to motivate the juries at the outset, and give accurate feedback at the end. At the end of the first jury, the member of the provincial executive gave a presentation indicating what she had learned and how she would incorporate these lessons into policy actions. This was concerning, in particular, the jury recommendation that greater collaboration between public authorities was necessary.
The second jury, which was part of real policy process, took place in a far less relaxed atmosphere, largely because several of the political parties in the provincial Parliament saw the jury as potentially threatening to their own legitimacy. The idea to institute the jury emerged from the team of bureaucrats who managed the decision process of the regional land-use plan under the guidance of an independent process facilitator. Although the secretariat of the parliament was involved in the decision, and the leaders of the larger parties had been consulted before the idea was presented to the Parliamentary Committee that dealt with the land-use plan, the decision to institute the jury resulted in many questions from the opposition. The debate focused on the legitimacy of the jury and the relationship between the jury and Parliament itself. Several members of Parliament were concerned that the jury recommendations would effectively bind them to certain decisions, which they did not want. By decision of Parliament, the chairman of the provincial committee communicated this to the jury, but he did so in such a way that some of the jurors received the impression that their work would be in vain.
Our interviews with policy makers suggest that one form of relational learning was consistently present for all three juries. The juries provided a great opportunity to learn how “ordinary citizens” perceive water issues and water management. In the interactions with the jurors, jargon needs to be clarified, built-in assumptions about problems and policies need to be explained, and issues related to the collaboration between public authorities become clear. All these aspects need to be clarified and the policy makers are confronted with the perspective of the ordinary citizen, who tends to have little patience with petty turf wars, may reject the jargon, and see problems very differently. For example, during the first jury, the abysmal relations between the professional fisheries and the Ministry of Agriculture, Nature Management and Fisheries were clearly evident during the hearings of the jury. This aspect drew the attention of the jury, and the jurors quite forcefully suggested that the Ministry should work out a financial compensation package for the fishermen so that they would end their protests. Although the fishermen and the Ministry were not on speaking terms, they did meet in the shadows of the jury proceedings. Although we do not wish to imply that we can attribute this to the jury proceedings, the fact is that the fishermen and the Ministry did reach an agreement about financial compensation about a year later. During our ex-post interviews, policy makers involved in the third jury indicated that their views on citizens had positively changed during the jury process. They were quite surprised that the jury members were able to discuss such complicated and abstract notions and ideas concerning water. They were also amazed about the number of citizens who had responded to the invitation to participate in the jury as they did not expect that level of involvement from citizens in water issues.
Roughly summarizing, we can conclude that high levels of policy learning did occur in the three juries analyzed here, but that such learning was greatly concentrated in the group of jurors. There was only a moderate level of relational learning among policy makers, and hardly any cognitive or normative learning there. Mostert et al. (2007) have suggested eight “general themes” related to the occurrence of learning in river basin management in this journal. Below, we relate our analysis of the policy learning effects we find in our case studies to these eight themes. Two preliminary points are worth making. The first is that these “themes” are here treated as independent variables that affect learning. However, they could equally be treated as evaluation criteria in their own right (the themes are very much comparable to the evaluation criteria of Rowe and Frewer (2000)). The second remark is that our current focus on learning as a dependent variable should not be taken to imply that learning is the only possible goal for citizens’ juries. There are other possible goals, which could make citizens’ juries very attractive even if the learning effect is limited or one sided. The eight themes suggested by Mostert et al. (2007) are as follows:
We see the strength of our article residing in several points. First, we have introduced and discussed one particular participatory methodology, which had not previously been discussed in connection with the need for adaptive (co-)management. Second, we think that our methodology for measuring learning effects is potentially useful in other situations too, which is relevant given the paucity of evaluation research on participatory methods (Rowe et al. 2008). Especially the relatively simple idea of “cognitive mapping,” but also the measurement of normative and relational learning through pre- and post-jury questionnaires (with a control group) seem to have generated useful insights in policy learning. Third, we have shed further light on the dynamics of “partnerships and power sharing” (Armitage et al. 2007: 6–10) by describing the interaction between our juries and elected representatives. The citizens’ jury is not intended as a vehicle for long-term power sharing, in the sense that the jury is disbanded after having delivered its recommendations. But, at a more abstract level, it can be a vehicle for involving ordinary citizens in the management of ecological systems. As such, the method creates high levels of learning, but this learning is almost completely concentrated in the circle of jurors, showing the importance of “politics and institutions” in the interplay between participatory forms and elected bodies. This is, to some degree, a tension that will always be present, but our analysis also shows that good political leadership and communications can go some way to ameliorating the problem. Fourth, we have shed some light on ways to move from instrumental learning to learning about appropriate goals (ibid.). Here, the presentation of a diversity of normative perspectives seems key.
Obviously, our analysis has limitations, and by mentioning these, we suggest further avenues for research. We mention the main caveats of our article, without aiming for exhaustiveness. First, our shifting interests have meant that, with each new jury, we decided to observe different aspects, which has meant that potentially valuable opportunities for comparison between the juries went out the window. This has affected the external validity of our findings, but also implies that our analysis of the potential explanations for the different degrees of learning is relatively shallow and speculative. For instance, it would have been very interesting to analyze the effect of different facilitation styles on learning, or the effect of diversity in the jury composition on learning. Second, we have chosen relatively specific and narrow focuses for our analyses. For example, when studying relational learning, we decided to focus on the way the jurors related to the political system, which is only one of the possible forms of relational learning that could have been measured here, and is probably not a form of learning that everyone assigns great priority to. And regarding normative learning, we decided to look at the importance that the jurors assigned to the various user groups of the lake. This is only a starting point for those who would like to know whether participation in a jury provides the jurors with a greener perspective than before. Third, we reiterate the point that learning is but one goal organizers may wish to achieve with citizens’ juries. Evaluation from other goals may generate different insights and conclusions about effectiveness. Fourth, our measurements are concentrated around the start and the end of the juries, and we have, therefore, not observed the effects of the juries over a longer timeframe. Some of the learning effects we have observed, are they lasting or not? Have the jury recommendations impacted policy in other ways than just a reference in an established plan or not? How is the network of parties involved in water management or even the general public affected by the jury, will former jury members become involved in water management, will they become ambassadors for smarter management?
Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link.
ACKNOWLEDGMENTSThe authors of this article wish to express their gratitude to the special feature editors for giving us the opportunity to participate. We thank the anonymous reviewers for their excellent and useful comments. The juries described here were organized with the support of a range of organizations. We thank the European Commission for its support for some of the research described here. The first jury was part of the “River Dialogue” project, financed under the EU Fifth Framework Program, contract RPAM-2002-00057, and we received additional help from Rijkswaterstaat and the province of Flevoland. The second jury described here was made possible by the province of Flevoland. The third jury was part of the Newater project (www.newater.info), financed under the EU Sixth Framework Program, contract 511179 (GOCE). The juries were organized by different teams with particpation from Jan Achterbergh, Leontien Bos, Ron Lavrijsen, Robert Leever, Rienk Terweij, Maria van Tilburg, Marleen van de Kerkhof, Susan van ‘t Klooster, and Femke Winsemius. We gratefully acknowledge their input and comradery during the juries. We thank Paula van Asperen for designing some of the figures in this article. We shall fondly remember her forever.
Alley, R. B., P. U. Clark, P. Huybrechts, and I. Joughin. 2005. Ice-sheet and sea-level changes. Science 310(5747):456–460.
Argyris, C. 2003. A life full of learning. Organization Studies 24(7):1178–1192.
Argyris, C., and D. A. Schön. 1996. Organizational learning 2. Theory, method, and practice. Addison-Wesley, Reading, Massachusetts, USA.
Armitage, D., F. Berkes, and N. Doubleday. 2007. Adaptive co-management. Collaboration, learning and multilevel governance. UBC Press, Vancouver, British Columbia, Canada.
Bennett, C. J., and M. Howlett. 1992. The lessons of learning. Reconciling theories of policy learning and policy change. Policy Sciences 25(3):275–294.
Birch, A. H. 1993. The concepts and theories of modern democracy. Routledge, London, UK.
Blomquist, W., A. Dinar, and K. Kemper. 2005. Comparison of institutional arrangements for river basin management in eight basins. World Bank (Working paper series 3636), Washington, D.C., USA.
Bohman. J., and W. Rehg, editors. 1997. Deliberative democracy. Essays on reason and politics. MIT Press, Cambridge, Massachusetts, USA.
Burchi, S., and M. Spreij. 2003. Institutions for international freshwater management. UNESCO, Paris, France.
Burkhalter, S., J. Gastil, and T. Kelshaw. 2002. A conceptual definition and theoretical model of public deliberation in small face-to-face groups. Communication Theory 12(4):398–422.
Cabanes, C., A. Cazenave, and C. Le Provost. 2001. Sea level rise during past 40 years determined from satellite and in situ observations. Science 294(5543):840–842.
Coenen, F. H. J. M., D. Huitema, and L. J. O’Toole, editors. 1998. Participation and the quality of environmental decision making. Kluwer Academic Publishers, Dordrecht, The Netherlands.
Conca, K., F. Wu, and C. Mei. 2006. Global regime formation or complex institution building? The principled content of international river agreements. International Studies Quarterly 50:263–285.
Crosby, N. 1995. Citizens’ juries: one solution for difficult environmental questions. Pages 157–174 in O. Renn, T. Webler, and P. Wiedemann, editors. Fairness and competence in citizen participation. Evaluating new models for environmental discourse. Kluwer Academic Publishers, Dordrecht, The Netherlands.
Delli Carpini, M. X., F. L. Cook, and L. R. Jacobs. 2004. Public deliberation, discursive participation and citizen engagement. A review of the empirical literature. Annual Review of Political Science (7):315–344.
Dryzek, J. 1987. Ecological rationality. Environment and political economy. Basil Blackwell, New York, New York, USA.
Easterling, D. R., G. A. Meehl, C. Parmesan, S. A. Changnon, T. R. Karl, and L. O. Mearns. 2000. Climate extremes: observations, modeling, and impacts. Science 289(5487):2068–2074.
Edwards, A. 2007. Jury’s achter Lely’s meesterwerk. Erasmus University, Rotterdam, The Netherlands. [online] URL: http://publishing.eur.nl/ir/repub/asset/11505/BSK-2007-002.pdf.
Falkenmark, M., L. Gottschalk, J. Lundqvist, and P. Wouters. 2004. Towards integrated catchment management: increasing the dialogue between scientists, policy-makers and stakeholders. Water Resources Development 20(3):297–309.
Fiorino, D. J. 1990. Citizen participation and environmental risk. A survey of institutional mechanisms. Science, Technology and Human Values 15(2):226–243.
Foil, C. M., and M. A. Lyles. 1985. Organizational learning. Academy of Management Review 10:80–83.
Fung, A. 2003. Recipes for public spheres: eight institutional design choices and their consequences. The Journal of Political Philosophy 11:338–367.
Genxu, W., and C. Guodong. 1999. Water resource development and its influence on the environment in arid areas of China—the case of the Hei River basin. Journal of Arid Environments 43(2):121–131.
Gleick, P. 2003. Global freshwater resources: soft-path solutions for the 21st century. Science 302:1524–1528.
Gleick, P., A. Sing, and H. Shi. 2001. Threats to the world’s freshwater resources. Pacific Institute, Oakland, California, USA.
Haas, P. M. 1992. Epistemic communities and international policy coordination. International Organization 46(1):1–35.
Hall, P. A. 1993. Policy paradigms, social learning, and the state. Comparative Politics 25(3):275–296.
Huitema, D. 2003. Organizing a citizens’ jury. IVM Report (W-03/34). Vrije Universiteit, Amsterdam, The Netherlands. [online] URL: http://www.ctc.ee/riverdialogue/uploads/citizen_jury_organization_by_dave.pdf.
Huitema, D., and S. Meijerink, editors. 2009. Water policy entrepreneurs. A research companion to water transitions around the globe. Edward Elgar, Cheltenham, UK.
Huitema, D., E. Mostert, W. Egas, S. Moellenkamp, C. Pahl-Wostl, and R. Yalcin. 2009. Adaptive water governance: assessing the institutional prescriptions of adaptive (co-)management from a governance perspective and defining a research agenda. Ecology and Society 14(1): 26. [online] URL: http://www.ecologyandsociety.org/vol14/iss1/art26/.
Huitema, D., M. van de Kerkhof, and U. Pesch. 2007. The nature of the beast. Are citizens’ juries deliberative or pluralist. Policy Sciences 40(4):287–311.
Huitema, D., M. van de Kerkhof, R. Terweij, M. van Tilburg, and F. Winsemius. 2004. Exploring the future of the IJsselmeer. Report to the River Dialogue project on the Dutch Citizens’ Jury. IVM Report (W-04/09). Vrije Universiteit, Amsterdam, The Netherlands.
Imperial, M. T. 2005. Using collaboration as a governance strategy—lessons from six watershed management programs. Administration and Society 37(3):281–320.
Imperial, M. T., and T. Hennessey. 1999. Environmental governance in watersheds. Collaboration, public value and accountability. Paper presented at the 21st Annual research conference of the Association for Public Policy Analysis and Management, 4–6 November 1999, Washington D.C., USA. [online] URL: http://people.uncw.edu/imperialm/Instructor/papers/Imperial_APPAM_99.pdf.
Kemper, K., A. Dinar, and W. Blomquist, editors. 2005. Institutional and policy analysis of river basin management decentralization. The principle of managing water resources at the lowest appropriate level – when and why does it (not) work in practice? World Bank, Washington D.C., USA.
Kuper, R.1996. Citizens’ juries: the Hertfordshire experience. University of Hertfordshire (Business School Working Paper Series), Hatfield, UK.
Laird, F. N. 1993. Participatory analysis, democracy, and technological decision making. Science, Technology and Human Values 18(3):341–361
Lavrijsen, R. 2005. Burgerparticipatie en betrokkenheid. Een evaluatie van de burgerjury in Flevoland. Thesis, University of Utrecht, Utrecht, The Netherlands.
Leach, W. D., and N. W. Pelkey. 2001. Making watershed partnerships work: a review of the empirical literature. Journal of Water Resources Planning and Management 127:378–385.
Leever, R. 2007. Doorwerking van burgerparticipatie in ruimtelijke planvorming. Een verkennend onderzoek naar de doorwerking van burgerjury’s op het beleidsvoorbereidend traject van het Omgevingsplan Flevoland 2006. Thesis, University of Utrecht, Utrecht, The Netherlands.
Levine, P., A. Fung, and J. Gastil. 2005. Future directions for public deliberation. Journal of Public Deliberation 1(1): 1003.
McCombs, M. 2005. A look at agenda-setting: past, present and future. Journalism Studies 6(4):543–557.
Meinzen-Dick, R. 2007. Beyond panaceas in water institutions. Proceedings of the National Academy of Sciences 104(39):15200–15205.
Mostert, E., C. Pahl-Wostl, Y. Rees, B. Searle, D. Tàbara, and J. Tippett. 2007. Social learning in European river-basin management: barriers and fostering mechanisms from 10 river basins. Ecology and Society 12(1): 19. [online] URL: http://www.ecologyandsociety.org/vol12/iss1/art19/.
Olsson, P., C. Folke, and F. Berkes. 2004. Adaptive co-management for building resilience in social–ecological systems. Environmental Management 34(1):75–90.
Pahl-Wostl, C. 2002. Towards sustainability in the water. The importance of human actors and processes of social learning. Aquatic Sciences 64:394–411
Renn, O., T. Webler, and P. Wiedemann, editors. 1995. Fairness and competence in citizen participation. Evaluating new models for environmental discourse. Kluwer Academic Publishers, Dordrecht, The Netherlands.
Ridder, D. A., E. Mostert, and H. A. Wolters, editors. 2005. Learning together to manage together. Improving participation in water management. University of Osnabrück, Institute of Environmental Systems Research, Osnabrück, Germany. [online] URL: http://www.seecon.org/de/uploads/downloads/hCOPmanual.pdf.
Rittel, H., and M. Webber. 1973. Dilemmas in a general theory of planning. Policy Sciences 4:155–169.
Rowe, G., and L. J. Frewer. 2000. Public participation methods. A framework for evaluation. Science, Technology and Human Values 25(1):3–29
Rowe, G., T. Horlick-Jones, J. Walls, W. Poortinga, and N. E. Pidgeon. 2008. Analysis of a normative framework for evaluating public engagement exercises: reliability, validity and limitations. Public Understanding of Science 17:419–441.
Rowe, G., R. Marsh, and L. J. Frewer. 2004. Evaluation of a deliberative conference. Science, Technology and Human Values 29(1):88–121.
Sabatier, P. A. 1998. The advocacy coalition framework: revisions and relevance for Europe. Journal of European Public Policy 5:98–130.
Sabatier, P. A., W. Focht, M. Lubell, Z. Trachtenberg, A. Veditz, and M. Matlock, editors. 2005. Swimming upstream. Collaborative approaches to watershed management. MIT Press, Cambridge, Massachusetts, USA.
Segall, S. 2005. Political participation as an engine of social solidarity: a skeptical view. Political Studies 53:362–387
Smith, G., and C. Wales. 1999. Citizenship and locality. The theory and practice of citizens’ juries. Policy and Politics 3:295–308.
Smith, G. and C. Wales. 2000. Citizens’ juries and deliberative democracy. Political Studies 48:51–65.
Stewart, J., E. Kendall, and A. Coote. 1994. Citizens’ juries. Institute for Public Policy Research, London, UK.
Stone, R. 2008. China’s environmental challenges. Three Gorges dam: into the unknown. Science 321(5889):628–632.
Swartling, A. G., and M. Nilsson. 2007. Social learning and EPI: communicative governance in Swedish climate policy formation. Ecologic, Berlin, Germany.
Tàbara, D., and C. Pahl-Wostl. 2007. Sustainability learning in natural resource use and management. Ecology and Society 12(2): 3. [online] URL: http://www.ecologyandsociety.org/vol12/iss2/art3/.
Terweij, R. 2004. De burgerjury een convergerend democratisch instituut? Een onderzoek naar de meningverandering over een beleidskwestie van individuele personen uit een Burger Jury. Thesis, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.
Turner, B. L., E. F. Lambin, and A. Reenberg. 2007. The emergence of land change science for global environmental change and sustainability. Proceedings of the National Academy of Science 104(52):20666–20671.
van der Kerkhof, M., and D. Huitema. 2004. Public participation in river basin management. A methodological perspective. Pages 141–148 in J. G. Timmerman, H. W. A. Behrens, F. Bernardini, D. Daler, P. Ross, K. J. M. van Ruiten, and R. C. Ward, editors. Information to support sustainable water management. From local to global levels. RIZA, Lelystad, The Netherlands.
Wanta, W. 1997. The public and the national agenda: how people learn about important issues. Lawrence Erlbaum Associates, Mahwah Publishers, New Jersey, USA.
Ward, H., A. Norval, R. Landman, and J. Pretty. 2003. Open citizens’ juries and the politics of sustainability. Political Studies 51:282–299.
Webler, T., H. Kastenholz, and O. Renn. 1995. Public participation in impact assessment: a social learning perspective. Environmental Impact Assessment Review 15(5):443–464.
World Commission on Dams. 2000. Dams and developments. A new framework for decision making. Earthscan, London, UK.
|Home | Archives | About | Login | Submissions | Subscribe | Contact | Search|