Toward Successful Joint Knowledge Production for Climate Change Adaptation: Lessons from Six Regional Projects in the Netherlands

In the domain of climate change adaptation, joint knowledge production (JKP) through intensive cooperation between scientists, policy-makers, and other actors is often proposed as a means to reconcile supply and demand for knowledge. Regional adaptation projects in the Netherlands form prominent examples of this. However, there is a lack of systematic empirical studies on how JKP can be done successfully. Here, we take the next step toward generating design principles for JKP. We do so by carrying out a comparative analysis of six Dutch adaptation projects using a previously developed assessment framework. Project documents were studied, and 30 semi-structured interviews were held with researchers, policy-makers, and financiers in the projects. Based on project comparisons, we derive and elaborate on two design principles for JKP. First, the most successful projects managed to create what we term a protected space for knowledge development while establishing connections with ongoing policy processes. Successful JKP seems to be more likely in cases in which actors make a conscious decision for the institutional location of the project on the research–policy nexus, whereby the coordinating entity has some characteristics of a boundary organization. Second, specific resources, including facilities, boundary objects, and specific competencies increase the chance for success. INTRODUCTION Scientific knowledge is indispensable for governing climate change adaptation. However, as the introduction paper of this special issue rightly states, the relationship between climate science and adaptation policy is highly complex. Implicit in much literature and even more in practice is the assumption that science and policy are located in two separate worlds with a gap between them to be bridged. Literature on science brokers (Pielke 2007), the science–policy interface (van den Hove 2007), and boundary organizations (Guston 2001), however, emphasizes that there is always some degree of interaction between science and society, and that efforts to improve this interaction are underway, although experiences are mixed. Generally, this literature emphasizes the hybrid character (Miller 2001) of science–policy interactions and acknowledges that science and society continuously co-produce each other (Jasanoff 2004). Science-society co-production is also influenced by and influences long-term changes in culture and macro-scale institutions.


INTRODUCTION
Scientific knowledge is indispensable for governing climate change adaptation.However, as the introduction paper of this special issue rightly states, the relationship between climate science and adaptation policy is highly complex.Implicit in much literature and even more in practice is the assumption that science and policy are located in two separate worlds with a gap between them to be bridged.Literature on science brokers (Pielke 2007), the science-policy interface (van den Hove 2007), and boundary organizations (Guston 2001), however, emphasizes that there is always some degree of interaction between science and society, and that efforts to improve this interaction are underway, although experiences are mixed.Generally, this literature emphasizes the hybrid character (Miller 2001) of science-policy interactions and acknowledges that science and society continuously co-produce each other (Jasanoff 2004).Sciencesociety co-production is also influenced by and influences longterm changes in culture and macro-scale institutions.
Here, we focus on a purposeful form of such science-policy coproduction: joint knowledge production (JKP).This term has recently been introduced to refer to projects in which scientists and policy-makers cooperate directly (van Buuren and Edelenbos 2004, Regeer and Bunders 2009, Pohl et al. 2010, Edelenbos et al. 2011, Hegger et al. 2012a).Several climate change adaptation programs in Western Europe (e.g., Climate Changes Spatial Planning, Living with Water, and Knowledge for Climate in the Netherlands; Klimzug in Germany) include such projects.JKP is claimed to lead to better, more policy-relevant or more socially robust knowledge (Climate Changes Spatial Planning and Knowledge for Climate 2009, de Pater et al. 2010).
Literature suggests that JKP can lead to knowledge that actors could only gain easily through direct cooperation (Climate Changes Spatial Planning and Knowledge for Climate 2009, Hegger et al. 2012b).There is also some evidence that JKP can enhance the role of scientific knowledge in the policy process, for example, if scientists act as co-developers of policy concepts (Kemp and Rotmans 2009).However, one can logically assume JKP to go wrong from time to time because well-known barriers against connecting science and policy may also occur in JKP projects.Research-based knowledge might fail to match expectations of policy-makers, it may be used differently than was expected or intended and suffer from fragmentation across disciplines.The interaction between science and policy is also complex because of differences in timeframes, reward structures, goals, process cycles, and epistemologies (McNie 2007, Sarewitz and Pielke 2007, van den Hove 2007, Weichselgartner and Kasperson 2010).Furthermore, the issue of climate change adaptation is complex because of value pluralities and uncertainties (Kemp and Martens 2007).JKP projects may even risk becoming strategic or symbolic processes (Edelenbos et al. 2011).Therefore, some JKP projects will be less successful than others in meeting actors' demands for credible and salient knowledge that has been produced through a legitimate process (Cash et al. 2003).
There is a lack of comparative empirical assessments on how to do JKP (Hoppe 2011, Hegger et al. 2012a).This is understandable because researchers assessing JKP practices have to address complicated issues such as choosing normative criteria for determining the success of science-policy cooperation and actually measuring this degree of success and associated success conditions.However, such empirical assessments are needed to be able to identify design principles for JKP.With this we mean, roughly in line with Ostrom (1990), concrete recommendations that have an action perspective and are based on empirical evidence.
Here, we aim to take the next step in deriving such design principles for JKP by conducting a comparative analysis of six Dutch adaptation projects that were part of two recently finalized research programs: Climate Changes Spatial Planning (CCSP) and Living with Water (LWW).Our analysis focuses on Ecology and Society 19(2): 34 http://www.ecologyandsociety.org/vol19/iss2/art34/similarities and differences in both the degree of success of the project and the degree to which potential success conditions were met.By comparing several projects, we attempt to look for patterns, asking questions such as: Are projects in which a certain success condition was present consistently more successful than projects in which the condition was absent?
For the analysis, we used a framework developed and refined by Hegger et al. (2012a,b).These authors have made a case for assessing JKP processes in a constructivist way by putting the opinions of actors participating in projects at the center of the analysis.Based on a literature review, Hegger et al. (2012a,b) conceptualized successful JKP and derived seven success conditions.As we will argue, this framework provides a good starting point for determining what to look at when analyzing, explaining, and evaluating the processes taking place in JKP projects.Some first steps in operationalizing the framework were taken by using the framework for an in-depth analysis of a Dutch climate change adaptation project called Hotspot Zuidplaspolder (Hegger et al. 2012b), which is also one of the six cases included here.Hegger et al. (2012b) derived some plausible lessons on how to do JKP, but because these are based on a single case study, a logical next step is to carry out a more comparative analysis using multiple cases.
We first present a conceptual clarification followed by the main features of Hegger et al.'s (2012a) framework, including their understanding of successful JKP as well as the seven success conditions they derived from the literature.We then describe our research methods.Thereafter, we briefly introduce the six programs and projects.Next, we rank the projects according to their degree of success.We then assess whether, to what extent, and how, the seven success conditions were met in each project, and subsequently explore the degree to which the success conditions might explain success.We conclude with a discussion and conclusion in which we define two design principles for JKP as well as possible next research steps.

Broader debates on more participatory forms of knowledge (co-) production
The term JKP refers to direct cooperation between scientists, policy-makers, and other actors in projects (van Buuren and Edelenbos 2004, Edelenbos et al. 2011, Hegger et al. 2012a).As is discussed by Hegger et al. (2012a), the discussion on JKP links up with broader discourses on what is often termed "new forms of knowledge production" (Funtowicz and Ravetz 1993, Gibbons et al. 1994, Nowotny et al. 2001, Bäckstrand et al. 2010).Space limits do not allow us to discuss these broader debates in great detail here.A common characteristic of these debates, however, is that a shift toward forms of knowledge production in which scientists engage more actively with other actors in society is seen to be both occurring and necessary.
A related term is knowledge co-production.We have chosen not to adopt this term because its use in literature is more diverse than the use of the term JKP and therefore potentially confusing.Some authors (Pohl et al. 2010, Armitage et al. 2011) use the term in a similar way as we use the term JKP.Armitage et al. (2011:996) define knowledge co-production as "the collaborative process of bringing a plurality of knowledge sources and types together to address a defined problem and build an integrated or systemsoriented understanding of that problem."Scholars from the field of science and technology studies, however, use the term to indicate that science and society continuously co-evolve or coproduce one another, often in indirect and intractable ways (Jasanoff 2004, Latour 1987).Our notion of JKP in projects refers to a direct and more easily recognizable form of such coproduction.

Dialog between different forms of knowledge
We assume that in JKP projects, a dialog between scientific and other forms of knowledge occurs, where these other forms might include tacit, lay, and practical knowledge, among others (Enengel et al. 2012).It is challenging to define "scientific" knowledge in comparison to these other forms of knowledge.Sociologists of science have taught us that boundaries between "science" and "nonscience" are not fixed, but on the contrary, that actors may have certain interests in creating boundaries, demarcations, and other divisions between domains (Gieryn 1983).Science can thus be seen as an essentially social activity (van Opstal and Hugé 2013).Nevertheless, there seems to be some consensus that "scientific knowledge" can be distinguished from "nonscientific knowledge" because the former is based on "empirical evidence or scientifically acknowledged theories" (Enengel et al. 2012:4).It is also "systematic, formalized, and explicit" (Enengel et al. 2012:4).Finally, scientific knowledge is seen to be general in nature, expressed in a systematic way, and its results should be controllable as well as controlled (e.g., via peer-review procedures).It can be argued that scientific knowledge, albeit different from other forms of knowledge, is not to be seen as more pure, exemplary, or truthful than other types of knowledge (Smith 2009:698).

Joint knowledge production as part of long-term societal processes
Although we focus on concrete dynamics in projects, we keep in mind that JKP projects are part of long-term societal processes.Several authors believe that the potential contribution of JKP to such processes is positive.Armitage et al. (2011Armitage et al. ( :1003) ) hypothesized it to be conducive to "positive social and ecological outcomes in the face of environmental change".In line with this, Shaw et al. (2009:461) concluded, based on a study of the synthesizing, downscaling, and visualizing of climate change scenarios, "that addressing climate change in a participatory way, with credible but easily accessible visuals, and at a scale that matters to people, may be critical in building capacity for climate change action."It is our assumption that once actors have been brought together in projects, various dynamics can take place that affect the collaboration process positively or negatively.In line with Armitage et al. (2011), we expect that positive experiences in projects may lead to sustained cooperation and commitment of actors, and vice versa.

Evaluating the success of joint knowledge production
Two reasons can be given for analyzing JKP through a constructivist approach (Edelenbos et al. 2012, Hegger et al.Table 1.Seven expected success conditions for joint knowledge production (JKP) projects.Adapted from Hegger et al. (2012a:56-61).The chance that JKP is successful is enhanced if actors decide, reflectively, which role to pursue in a project and how to define their identity in relation to the other actors, and they make these choices known 5. Role of researchers and their knowledge is clear The chance that JKP is successful is enhanced in cases in which the role of researchers and their knowledge is clear 6. Innovations in reward structures are present The chance that JKP is successful is enhanced through novel forms of reward structure Resources 7. Specific resources such as boundary objects, facilities, organizational forms, and competencies are present The chance that JKP is successful is enhanced through the availability of specific resources (boundary objects, organizational forms, and competencies) 2012a).First, actors involved in JKP projects will have different interests due to differences in affiliation (university department; public policy body) and personal background.Second, JKP projects generally involve value pluralities and uncertainties.This makes a process evaluation relevant because one can assume that a successful process positively contributes to project outcomes (Hegger et al. 2012a).Successful JKP can be defined as "a process in which the actors involved have managed to maximize synergy and minimize tradeoffs between the salience and credibility of the knowledge produced as well as the legitimacy of the process" (Hegger et al. 2012a:54).
The notions of credibility, salience, and legitimacy were originally coined by Cash et al. (2003), who found that science-policy collaboration is likely effective if those three criteria can be met simultaneously for all actors involved.Credibility relates to the scientific adequacy of technical evidence and arguments.Salience refers to the relevance of knowledge to the needs of decisionmakers.Legitimacy entails that knowledge development has been "respectful of stakeholders' divergent values and beliefs, unbiased in its conduct and fair in its treatment of opposing views and interests" (Cash et al. 2003:14).Hegger et al. (2012a) treat the three criteria in an actor-specific way, assuming that all actors have ideas on how credible and salient knowledge produced has been for them, and how legitimate a JKP project has been according to them.We follow this constructivist way of using the concepts of credibility, salience, and legitimacy.The concepts are treated as indicators of the extent to which actors' interests were met in the studied JKP projects.We see the latter as a measure for the projects' success.Having set out how we attempt to measure the success of JKP projects, i.e., the dependent variable, we now turn to a discussion of seven success conditions for JKP, i.e., the independent variables.Hegger et al. (2012a) derived seven success conditions for JKP projects from the literature.Inspired by Arts et al. (2006), Liefferink (2006), andvan Tatenhove et al. (2000), they classified success conditions into actors, discourses, rules, and resources (Table 1).The expectation is that the presence of a certain condition increases the chance for successful JKP.Each success condition has an expected relationship to credibility, salience, and legitimacy (Table 2).

From success conditions to design principles
The seven success conditions were derived from a comprehensive study of literature from the sociology of knowledge and the field of environmental governance (Hegger et al. 2012a).Their relevance was roughly confirmed by Hegger et al. (2012b).They can therefore be seen as a good starting point for more comparative empirical analyses.To be able to translate them into design principles for JKP, however, the success conditions derived from literature should be further operationalized with empirical research.

Case selection
Our units of analysis comprise six JKP projects that we analyzed by carrying out a document analysis combined with interviews with relevant actors.The six projects form a small sub-set of the programs CCSP (consisting of 51 projects in total; Climate Changes Spatial Planning 2008) and LWW (consisting of > 100 projects; www.levenmetwater.nl).Both programs involved a relatively homogeneous community of climate scholars and practitioners.By selecting cases from these two programs, we wanted to maximize the chance that differences between the projects would be attributable to the seven success conditions, the independent variables in our framework, and not to the substantive issues on which the projects are focused.Based on an initial analysis of project documents, including project plans and final reports, we selected projects that differed in terms of budget, duration, and the actual participants, although the type of participants was somewhat comparable, always including actors from science and from regional and local governments.The projects also differed in terms of how the programs in which they were embedded were organized, and in the type of coordinating actor (one research institute, two universities, two provincial entities, and one freelance coordinator; Table 3).

Operationalization
Credibility, salience, and legitimacy as perceived by the participating actors were seen as indicators of the degree of success of the projects, which was the dependent variable.The seven success conditions were the independent variables.Some preliminary steps in operationalizing the concepts have been taken (Hegger et al. 2012b), but all 10 concepts are still to be seen as sensitizing concepts (van den Hoonaard 1997), as opposed to definitive concepts.Sensitizing concepts have no fixed operational definition, but provide the researchers a frame on "where to look".The goal of our empirical research was therefore twofold.First, we aimed to further operationalize the concepts to make the next step toward the formulation of definitive concepts.Second, we attempted to compare the scores of the variables among the projects to look for patterns.

Scoring credibility, salience, and legitimacy
Our interviews with actors in the projects (see below) included open questions on their interests in the projects and the degree to which these were met, their passion for the project, and whether they considered the science-policy cooperation in the project to be a fruitful one.From the responses, we tried to deduce what their criteria for credibility, salience, and legitimacy were, and to what extent these were met.We assumed that these interests were met to a greater extent if the interviewees made few or no negative comments regarding the degree to which their interests were met.Accordingly, we gave each project a score for each of the three criteria: + meaning no serious criticism was encountered; ± meaning some remarks or comments that seemed minor were encountered; or −− meaning criticism was encountered.To be able to rank the projects according to their degree of success we calculated an overall score that ranged between +3 (three plusses) and −3 (three minuses).

Scoring success conditions
In all cases we attributed a score based on the interview findings (++ clearly present, + seemingly present, ± neutral, − seemingly absent, −− clearly absent) for the following seven success conditions.

Broadest possible actor coalition is present:
We attributed a lower score in cases with clear missing links in the actor network, for instance, the nonparticipation of a regional government entity with a stake in the issues addressed.A + was given in cases in which all the "usual suspects" were involved.A ++ was given in cases in which the network was broader than the usual participants.We argue that the crucial actors have to be involved and should be willing to participate, and that in some cases, it may be instrumental to include dissidents in the project, and that in any collaborative process, the number of actors needs to be limited to keep the process manageable (Hegger et al. 2012a).In all cases, we approached the issue of actor participation from an instrumental viewpoint, asking to what extent the actor coalition would be capable of bringing different types of knowledge together to deal with sustainability problems.We did not assess the extent to which the projects were inclusive in the sense that all actors that wanted to be involved in an issue were actually involved in the project.On the contrary, we deem it realistic to expect that in the process of defining programs and projects, something took place that Few et al. (2007:55) would term "selective inclusion, cooptation and/or exclusion of stakeholders".

Shared understanding of goals and problem definitions:
A higher score (++) was given in cases in which extensive deliberation took place during the project (e.g., via reflection workshops).If the project seemed to be clearly demarcated before its start, but no such deliberation during the project took place, we gave a +.If both conditions were absent or clearly absent, a negative score was given.

Recognition of differences in actor perspectives takes place:
In a particular case, we attributed a positive score if we found examples of actions (to be defined during the interviewing process) that could contribute to recognition of differences in actor perspectives.

Organized reflection on division of tasks by participating actors takes place:
In a particular case, we attributed a positive score if we found examples of actions (to be defined during the interviewing process) that could contribute to organized reflection.

Role of researchers and their knowledge is clear:
We attributed a positive score in cases in which we were able to clearly describe these roles because the statements of the http://www.ecologyandsociety.org/vol19/iss2/art34/interviewees and the information found in project documents were consistent with one another.

6.
Innovations in reward structures are present: In a particular case, we attributed a positive score if we found examples of innovations in reward structures (to be defined during the interviewing process).
7. Specific resources are present: In a particular case, we attributed a positive score if we found (many) examples of innovations in reward structures (to be defined during the interviewing process).
With this procedure, we expected to be able to score projects and draw comparisons among them.However, in most cases, our criteria for attributing a positive or a negative score were initially a black box.For example, we decided to give a positive score if we could find innovations in reward structures, but did not determine a priori when something would qualify as an innovation in reward structures.

Conducting and analyzing interviews
Thirty semi-structured interviews were conducted with involved actors (four to seven interviewees per project).In each of the six projects, we interviewed at least one person with a predominantly researcher role, one policy-maker, and in all but one case, the project leader.In one case, it was not practically possible to plan an interview with the project leader.Project leaders generally played a double role in that they were formally a researcher or policy-maker, but in practice played the role of bringing parties together and managing the project, rather than carrying out research or making policy recommendations.Among the interviewees related to each project, we also included one person at the level of the program in which the project was embedded.This person was interviewed to obtain insight into the extent to which, and the way in which, the project was steered by this program level.Generally, the interviewees gave us suggestions for follow-up interviews, some of which we have followed.It is our impression that all interviewees were interested in how to do JKP and that they genuinely wanted to help us to get a complete picture.
The types of issues addressed were similar in all interviews.These included the interviewee's opinion on the main outputs and outcomes of the project, e.g., knowledge development, new networks or communities of practice, indirect or direct policy impact; the interviewee's personal and professional interest in the project; how the interviewee sees him/herself, e.g., as a scientist, policy-maker, project/program manager; the interviewee's initial expectations regarding the project and the degree to which these were met or how perceptions changed; whether the interviewee considered the science-policy cooperation in the project to be a fruitful one; and how the project relates to the interviewee's other professional activities, e.g., similar, completely different, etc.
All interviews were carried out between February and September 2011 and lasted on average 1.5 h (range: 45 min-2 h).Extensive summaries of the interviews were written (~1 page A4 for every 15 min of interviewing) and sent to the interviewees for feedback and approval.These were analyzed using Atlas.ti.Data were coded according to the three notions of credibility, salience, and legitimacy, as well as the seven expected success conditions.In the following sections, we sometimes provide quotations from these interviews.These particular quotations were selected because they are a potentially fruitful way to further operationalize the concepts.Actors from science and public policy participated in all projects but in different roles (Table 3).The projects also differ in the dynamics through which they were initiated.The projects were more often initiated by the demand side than by the supply side, contrary to the observation of Talwar et al. ( 2011), who found that, in Swiss sustainability research, virtually all transdisciplinary projects are science-driven.Nevertheless, issues were put on the agenda and projects were planned via various mutual interactions among scientists, policy-makers, and program managers.Project Routeplanner was the only purely policy-driven project.The establishment of Project What's the Future of Low-lying Peat Land? by applied researchers was a reaction to knowledge needs articulated by national and regional policy-makers.Project CCSP Hotspots was set up and coordinated by provinces, but their participation was a reaction to the research program's funding opportunities.Two of the projects were science-driven projects.

SIX EMPIRICAL CASES
In Project Transitions SUW, scientists initiated research and Advice for provincial government on regional climate resilience; agenda-setting function for regional actors "Some space was left in the provincial plan to allow for using some of the project results.At a certain moment, however, the timing of the Hotspot lagged behind that of the regional planning process, making it more complicated to actually influence the plan."(project supervisor) Routeplanner Various reports (climate resilience, climate effects, knowledge gaps, evaluation of adaptation options); some journal articles and book chapters Providing insight to policymakers on the consequences of climate change; introduction of concepts (e.g., robustness, resilience) to policy-makers "Routeplanner and ARK [national adaptation program] were conducted in parallel.Sometimes knowledge exchange took place.You saw a clear distinction between those demanding and those supplying knowledge.[the two coordinators] often had to act as a knowledge broker, explaining things in a specific way, or act as spokesperson."(researcher) sought collaboration with two municipalities (Rotterdam and Heerhugowaard), who provided the case studies.Both municipalities initially saw their role as facilitators of research.At least in the case of Rotterdam, this changed when it was discovered that the researched concepts could provide economic opportunities.Project Co-valuation of Water was initiated by the Dutch applied research organization TNO and Erasmus University Rotterdam.These institutes sought collaboration with the local stakeholders and applied for funding from LWW. Example responses from the interviewees give an idea of what actors in the projects deemed salient knowledge (Table 4).

COMPARING THE SUCCESS OF THE PROJECTS
The projects were scored for credibility, salience, and legitimacy based on the interview results.An indicative overall score was also assigned so that we could rank the success of the projects (Table 5).

Credibility
In most projects, credibility did not seem to be an issue of great concern (e.g., Hotspot ZPP, What's the Future of Low-lying Peat Land?, Transitions SUW, and Routeplanner).Actors' remarks on credibility were general in nature.For instance, it was frequently mentioned that "practical knowledge enables researchers to do more credible research".In Co-valuation of Water and Hotspot Groningen, however, serious criticisms were raised, which can be interpreted as a lack of credibility.In Co-valuation of Water, an interviewee mentions that "the developed visions were unrealistic and not well-supported".In Hotspot Groningen, some interviewees described the project leader as a visionary person who, although he was officially a policy officer, was seen as a "representative of science".Apparently, credibility only became an issue in cases in which there were dissidents in the projects (value pluralities).Actors sometimes coupled credibility of knowledge to the credibility of persons.

Salience
Actors had different criteria for the salience of knowledge, and these were met in only four of the six analyzed projects.The two projects in which they were not met were Co-valuation of Water and Hotspot Groningen.In these projects, actors involved perceived the quantity of relevant knowledge produced to be relatively low and, moreover, did not manage to reconcile their different views on relevance.Actors in Co-valuation of Water had different views on whether implementing the developed visions was desirable and whether this was a goal of the project.In Hotspot Groningen, we see a distinction between people who found that input should be given to the provincial plan, including the board of CCSP (which was said to be initially dissatisfied about the project outcomes, but approved the project later on) and others who emphasized "awareness raising" and "having scenarios available for future use".In these two projects, some actors' thresholds for salience were not met.In the four projects in which the salience criteria were met, we observed some differences between the interviewees in the elements that contributed to the salience of the projects.The interviewed researchers experienced a tension.On the one hand, they generally liked being involved in a practice-oriented project and deemed the implementation of sustainable concepts and visions important.However, they also felt the need to publish and the extent to which they could derive enough publishable materials from their participation in the projects varied.Most researchers were at the start of their career.For them, it was important that the project provided job opportunities.In two projects, we found examples of the generation of such job opportunities.Although it was not their primary interest, researchers in several projects (What's the Future of Low-lying Peat Land?, Hotspot ZPP, and Routeplanner) indicated that they valued the acquisition of practical knowledge.They learned about terminologies and about how policy-making works.One interviewee (Routeplanner) who had a natural science background also learned "to think in terms of actors rather than processes".
Policy-makers and program managers deemed the applicability of the knowledge most important.The projects differed in this respect.A first type of knowledge production observed was agenda-setting knowledge.Policy-makers in Hotspot ZPP became more aware of the importance of desiccation for the area.
Various policy-makers in Hotspot Groningen started to think about the regional consequences of climate change.A reported result of What's the Future of Low-lying Peat Land? was that "it is no longer possible to deny the existence of soil subsidence".
Second, in some projects, policy concepts were developed.Policymakers portrayed Routeplanner as a knowledge dissemination project, familiarizing national-level policy-makers with such concepts as climate-proofing, climate scenarios, uncertainties, and resilience.The project also provided policy-makers with stateof-the-art climate change knowledge.
Third, some policy-makers referred to the generation of insights and ideas.A policy-maker in Transitions SUW claimed to have learned most from the project's transitions part, which made him familiar with the role of actors in transitions, long-term thinking, and thinking in terms of opportunities.According to several interviewees, What's the Future of Low-lying Peat Land? contributed to the development of a nuanced and pragmatic plan for implementing a new policy concept that translates as "function follows water level", in which land-use functions depend on the water level in certain areas rather than the other way round.
Among other things, knowledge was developed on the strengths and weaknesses of underwater drainage, a mitigation technology.
Fourth, several projects provided arguments supporting and legitimizing ongoing planning processes.In Hotspot ZPP, scientific underpinning of existing plans and approaches (the socalled layer approach, a relatively new Dutch spatial planning principle) was generated, reducing controversies.The project also legitimized the plans for building in Zuidplaspolder.
Fifth, policymakers referred to the identification of economic opportunities.This was the case in Transitions SUW (Rotterdam municipality).
Finally, actors, particularly at the program-level, deem the development of process-related knowledge important (e.g., Hotspot ZPP, CCSP's first hotspot).

Legitimacy
In the projects Co-valuation of Water and Hotspot Groningen, actors referred to a perceived lack of legitimacy.In the former project, a civil servant believed that "the local population was fooled".Inhabitants were asked to participate in the development of plans, although "it was clear from the outset that these would not be executed".Also, several interviewees claimed that the position of experts in the project was problematic.They were not familiar with the subordinate role they were expected to play, i.e., providing feedback rather than making plans (see also Edelenbos et al. 2011).A researcher in Co-valuation of Water learned that commitment of organizations is largely dependent on individuals.After a civil servant and the responsible alderman left, the municipality turned out to be no longer committed.The same researcher mentioned that "scientists are wrong in assuming that practitioners know everything about integrated water management, stakeholder involvement, and making room for water".The fact that the project leader of Hotspot Groningen, a policy officer, was seen by some as "a representative of science" was claimed to have complicated the internal acceptance of his work within the province.An employee of one of the participating water authorities said that he believed he was "merely facilitating science".This employee indicated that "if the local water authority had been the principal, the current project results would not have been sufficient".
In some of the other projects, statements were made that can be linked to a lack of legitimacy, although the issues addressed seem to be less serious ones.The PhD researcher within Transitions SUW found working at the intersection of science, policy, and practice exiting and instructive.It resulted, however, in a high http://www.ecologyandsociety.org/vol19/iss2/art34/workload because policy-makers were interested in easily accessible reports, whereas his supervisors were interested in journal articles.This could be interpreted as a lack of legitimacy in the eyes of the PhD researcher, although this researcher did not use that term.Some interviewees in What's the Future of Lowlying Peat Land? referred to the, according to them, theoretical possibility that actors would prematurely use intermediary products.Farmers could have an interest in claiming that "underwater drainage is a solution for continuing agricultural activities in low-lying peat areas".Claiming more than science justifies can be interpreted as a lack of legitimacy.However, at the time of writing, such premature use had not taken place, so legitimacy seems to have been uncompromised.

IDENTIFYING SUCCESS CONDITIONS IN THE PROJECTS
Having determined indicative scores for the degree of success of the projects, we next explore the seven success conditions.Each project was provided as score as to whether, to what extent, and how the theoretical success conditions were met (Table 6).We compare the differences in scores for each success condition.

Broadest possible actor coalition
We did not find any obvious gaps in actor participation.All projects included the actors that we would expect given the stated goal of the project as well as our knowledge of the actors involved in the community associated with climate change adaptation in the Netherlands.The projects differed somewhat in this respect.
In the case of Co-valuation of Water (−), some interviewees found that execution of the developed visions was a project goal.However, the parties that should play a role in the execution, i.e., building contractors and project developers, were not involved.
The projects What's the Future of Low-lying Peat Land? (++) and Hotspot ZPP (++) managed to include a rather large proportion of the policy actors, and some societal actors, related to the geographical area under study.They did so in different ways, but both managed to create both distance and connections between knowledge development and the policy process (see also Hegger et al. 2012b).

Deliberation on goals and problem definitions
Deliberation on goals and problem definitions took place in all projects, at least to some extent.In all cases, at some point before the official start or at the beginning of the project, decisions were made on how to delineate the project.However, we found no examples of extensive deliberations during the project in any of the projects.

Recognition of differences in actor perspectives
All projects did undertook actions that could enable recognition of differences in actor perspectives (Table 6), but based on the interviews, we cannot determine if these differences were actually recognized.In What's the Future of Low-lying Peat Land?, boundary objects were a prominent means to do so.In Covaluation of Water, there was reflection by parties.In Hotspot ZPP and Transitions SUW, there was physical proximity between the people that had to collaborate most closely.In Hotspot Groningen, there were meetings at and site visits to interesting locations.In the Routeplanner project, exchange took place via knowledge brokering.

Organized reflection on division of tasks
We found no examples of extensive reflection on the division of tasks (e.g., reflection workshops, retreat days) in any of the projects.Only in the case of Co-valuation of Water did we find examples of some bilateral reflection on the role of individuals.

Role of researchers and their knowledge is clear
The projects differed in terms of the roles played by researchers.In all projects, this included roles as scientific or applied researchers.In What's the Future of Low-lying Peat Land? and Transitions SUW, researchers provided direct input to ongoing policy processes.In all cases, the role of researchers and their knowledge seemed to be clear.

Presence of innovations in reward structures
All projects had in common that the Dutch national government made funding for science-policy cooperation available in the first place.Besides that, we found only one more specific example of innovations in reward structures at project level.In What's the Future of Low-lying Peat Land? actors were allowed to use 5% of the project budget for the right to extract knowledge from the project; for example, Water Boards could ask researchers to give a presentation for their board without being charged for it.In the other cases, we found examples of people who thought that they had to cope with the perceived shortcomings of conventional reward structures.For instance, researchers are expected to publish, and there were differences in the extent to which these researchers succeeded in doing so.

Presence of specific resources
We found many different specific resources.Some of these are contingent and project specific, for instance, whether there is initial support for a project in case-study municipalities (Transitions SUW).In addition, we also came across some replicable resources.First, we saw examples of actors investing much time and energy in the project and doing more than one would expect based on a stereotypical view of their role.Both the PhD researcher within Transitions SUW and the project leader of Hotspot ZPP contributed labor and competencies to the project.Second, and related to the previous point, in all projects we saw clearly identifiable persons, often including the project leader, playing a role as broker between the worlds of science and policy (competencies).Third, in Hotspot ZPP and What's the Future of Low-lying Peat Land? we saw examples of physical boundary objects (specific material arrangements) as well as the establishment of specific entities (Xplorelab).As the differences in scores indicate, there seem to be some differences in the quantity of measures taken.

EXPLORING THE RELATIONSHIP BETWEEN SUCCESS CONDITIONS AND SUCCESS
The six projects were ranked in the order of their degree of success (Table 7).The resulting pattern seems to be in line with what we expected based on our causal model: the more successful projects score higher on the success conditions than do the less successful projects, suggesting that the success conditions have a stimulating influence on project success.As expected, the more successful projects Hotspot ZPP and What's the Future of Low-lying Peat Land? scored relatively high on the theoretical success conditions, Co-valuation of Water and Hotspot Groningen scored relatively low, and Transitions SUW was intermediate.Routeplanner scored lower on the success conditions than expected.This may be explained by the fact that, contrary to the other projects, this project focused on bringing together existing knowledge, rather than developing new knowledge.Also, our interviews suggest that the actors involved in this project saw their project less as a joint effort than did actors in the other projects, possibly reducing the relative importance of the success conditions, which are success conditions for JKP.
The success conditions "broadest possible actor coalition," "presence of innovations in reward structures," and "presence of specific resources" are more distinguishing than the other four conditions.These conditions seem to be better met in the more successful projects.We could not find clear differences in the degree to which the other success conditions were met, but, as discussed previously, we did find differences in how they were met.
We might argue, first, that the three distinguishing conditions are more important than the others in enhancing the success of a project.According to this line of reasoning, involving as many actors as possible in a smart way (broadest possible actor coalition) is one of the crucial factors influencing the course of a project.The methodology used in Hotspot ZPP and What's the Future of Low-lying Peat Land?, in which researchers were connected to a specific policy issue but with some distance between the research process and the policy process, can then be considered good practice.It is also plausible that the mobilization of specific resources (presence of specific resources) increased the chance for success.The empirical material seems to indicate that the projects differed in terms of the amount of specific resources.
Interviewees in What's the Future of Low-lying Peat Land? and Hotspot ZPP could easily point to a multitude of such resources, whereas in other projects such as Hotspot Groningen, actors referred to a lack of resources, for example, a lack of secretarial support for the project leader.Both of these two success conditions, furthermore, can be expected to enable and constrain the extent to which the other success conditions can be met, suggesting that these two conditions form the main leverage points for setting up successful JKP projects.
A second reason why these three success conditions were more distinguishing for explaining project success than the others is that we were better able to measure those distinguishing conditions.We might have missed some probably subtle ways to achieve deliberation on goals and problem definitions (deliberation of goals and problem definitions takes place).Also, we only measured actions that could lead to recognition of actor perspectives (recognition of differences in actor perspectives).We neither measured the perspectives themselves nor the degree to which they were recognized.However, we may have researched a specific group of uncontroversial JKP projects in which the need for deliberation and reflection was minimized.
Third, the framework deliberately excludes some explanatory factors for the success of JKP.The framework aims at the identification of actions that increase the chance for successful http://www.ecologyandsociety.org/vol19/iss2/art34/JKP (Hegger et al. 2012a) and hence strongly focuses on action perspectives at the project and program levels.Contingent and contextual factors are not included in the framework.In the case of Co-valuation of Water, for instance, interviewees referred to set-backs resulting from the fact that the contact persons within the involved municipality (a civil servant and the alderman) had to resign from their project activities.At the same time, some interviewees in the case of What's the Future of Low-lying Peat Land? told us that the fact that funding became available for a project of its size was because there was a political window of opportunity at the time of setting up the project: actors at ministries, provinces, and water boards perceived problems of soil subsidence in low-lying peat areas as urgent.

DISCUSSION
Our research reconfirms that actually measuring knowledge production dynamics is challenging.For the purpose of the current research, we deem legitimate our choice to focus on documents and interviews, predominantly because of our diagnosis that a further operationalization of the concepts in the assessment framework was still necessary.A strength of retrospective, qualitative, semi-structured interviews is that the interviewer can ask follow-up questions in a safe environment.This makes it possible to arrive at some in-depth understanding of issues and to collect data from a substantial number of interviewees without the risk of positively or negatively affecting the projects studied because they had already been finalized.An obvious shortcoming of the method chosen is that some issues may not be retrievable through such interviews.For instance, the precise social dynamics in a project, i.e., who initiated something and why and how it influenced the course of affairs in a project, can only be assessed to the extent that the interviewees can remember, and it is furthermore colored by the interviewee's interpretation.It may also be more difficult to find out about the precise settings in which interactions took place, although we could gain some insight into this because these settings were often the offices of the interviewees, and this is where most of the interviews were carried out.Certainly, in follow-up research, the inclusion of additional data collection methods, including participant observation, would be advisable.We invite other scholars to join us in this endeavor.
Reflecting on the external validity of our analysis, we researched a limited set of projects from a specific homogeneous community of Dutch climate researchers, policy-makers, and practitioners.We tentatively argue that the conclusions derived from these projects can be expected to be valid for other projects from within this same community.We deem it also plausible that these conclusions are relevant for the phenomenon of JKP more generally.At least, we do not see a reason to assume a priori that they are not.However, we must note that we researched a specific type of project in which deliberation and joint reflection was less essential than in cases in which the underlying problems have become much more politicized.In this respect, the analyzed projects seem to differ from other projects in which there appears to be more tension between the interests of researchers and those of policy-makers (e.g., Kemp andRotmans 2009, Lamers et al. 2010).It would be an interesting point for further research to assess the origin of such differences.

CONCLUSION
The aim of this study was to take the next step in deriving design principles for JKP by conducting a comparative analysis of the success of six Dutch adaptation projects within two recently finalized research programs: CCSP and LWW.To do this, we further specified an existing assessment framework (Hegger et al. 2012a), denominating seven theoretical success conditions for JKP.
Our empirical material allows us to translate and further specify two of our seven success conditions into design principles for JKP: presence of broadest possible actor coalition, and presence of specific resources.There is no need to dismiss the other five success conditions, but additional research is needed to determine their role more precisely.
We found that what constitutes the broadest possible actor coalition depends not only on which actors are involved, but also on the mechanisms through which they are involved.Obviously, in our study, there has been an inherent bias in the types of actors involved because all cases come from a small, homogeneous, centrally located, tightly regulated, engaged society.Still, we identified differences between the projects, that is, within this society.It therefore seems imperative to make a conscious decision for the institutional location of the project on the science-policy nexus, whereby the coordinating entity functions as a boundary organization, e.g., an applied research institute, a knowledge broker, or a transdisciplinary innovation lab within a province (Guston 2001).Most successful projects managed to build a large network of actors, including actors from science and policy.These projects created what might be termed a protected space for knowledge development (see also Boon and Horlings 2013).As a consequence, the researchers and policy-makers in the projects had some space to carry out the research that they deemed relevant while the research was still seen as policy relevant.
Our analysis also indicates that specific resources, including facilities, boundary objects, and specific competencies, should be employed to increase the chance for success.The range of specific resources employed in the analyzed projects was very heterogeneous and included specific material arrangements (GIS maps, places to meet), labor, competencies, and finances.Clearly, resources to enable successful JKP should include more than finances alone.As we noted in Identifying Success Conditions in the Projects, examples included actors investing much time and energy in the project and doing more than one would expect based on a stereotypical view of their role, people playing a role as broker between the worlds of science and policy, physical boundary objects (specific material arrangements), and special entities.Hence, successful JKP is demanding and does not occur automatically.One can logically assume that this is easily underestimated, although we can only speculate if this aspect was actually underestimated in the two less successful projects and that this was the main cause for their lack of success.
The empirical research shows that the success or failure of JKP projects lies not solely in the hands of actors at the project and program levels.Set-backs in projects cannot always be avoided and, similarly, windows of opportunity cannot always be created.These windows of opportunity are dependent on, for instance, the political will to engage in the issue of climate change http://www.ecologyandsociety.org/vol19/iss2/art34/adaptation or the presence of a knowledge need on the part of governing actors.
In conclusion, our two design principles of presence of broadest possible actor coalition and presence of specific resources, as well as our warning that JKP is contingent seem to suggest that, in theory, certain actions would have increased the chance of success of the two least successful projects.We cannot know, however, if this success would have materialized.For instance, in Covaluation of Water, it might have been possible to try to attribute a more important role to the experts in a substantive or symbolic way.In Hotspot Groningen, perhaps more support, in terms of labor, for the project leader would have been helpful.It would not be fair, however, to hold these projects accountable for these deficiencies.First, with hindsight, one can only speculate about what would have been possible in the projects.It is more difficult to carry out a project "on the fly" than it is to reflect on it afterwards as a relative outsider.Second, in the more successful projects, it will also certainly be possible to find areas for improvement, and there are also reasons to assume that at least some of the more successful projects also had their deal of good luck.
Our design principles and any future elaborations and refinements should be seen not as a framework for ex-post summative evaluation of projects, but primarily as a framework for ex-ante evaluation.The design principles may offer project participants and funding bodies a tool to make a best educated guess as to how a JKP project can be set up in a good way, as well as inspiration for what could be done to improve JKP practices.They are necessary, but not sufficient preconditions for successful JKP.
Failure should be tolerated to some extent, and space is needed for making and learning from mistakes.
Responses to this article can be read online at: http://www.ecologyandsociety.org/issues/responses.php/6453

Table 2 .
Expected relationship between success conditions of joint knowledge production (JKP) projects and the perceived credibility, salience, and legitimacy of the knowledge produced.†AdaptedfromHegger et al. (2012a:61).
Both CCSP and LWW were two large national research programs financed by the Dutch government.The projects were mainly executed between 2005 and 2009, a period in which in The Netherlands, the notion of climate adaptation was still relatively new.In many cases, climate adaptation had not entered the agendas of policy-makers and practitioners, and there were still many questions as to what "climate-proofing" might entail and how one could operationalize it in a specific geographical area.The projects have in common that researchers (both natural scientists and social scientists), policy-makers, and sometimes also other actors looked together at specific geographical areas and made a first step in climate-proofing these areas, for instance, by evaluating existing plans, developing visions, or determining, together, how specific policy concepts could be implemented.The projects differed in terms of duration, budget, participants, stated goals, and coordinating actor (Table3).

Table 4 .
Some reported results of the analyzed projects.

Table 6 .
Presence of success conditions in the projects.† the table also repeats the overall scores for success of table 5.

Table 7 .
Projects ranked according to the presence of success conditions in the projects.†