Synthesis Studying the implementation of the Water Framework Directive in Europe : a meta-analysis of 89 journal articles

The Water Framework Directive (WFD) is arguably the most ambitious piece of European Union (EU) legislation in the field of water. The directive defines a general framework for integrated river basin management in Europe with a view to achieving “good water status” by 2015. Institutional novelties include, among others, water management at hydrological scales, the involvement of nonstate actors in water planning, and various economic principles, as well as a common strategy to support EU member states during the implementation of the directive. More than 15 years after the adoption of the WFD, and with the passing of an important milestone, 2015, we believe it is time for an interim assessment. This article provides a systematic review of existing scholarship on WFD implementation. We identify well-documented areas of research, describe largely unchartered territories, and suggest avenues for future studies. Methodologically, we relied on a meta-analysis. Based on a codebook of more than 35 items, we analyzed 89 journal articles reporting on the implementation of the directive in EU member states. Our review is organized around three major themes. The first is “who, when, and where”; we explore publication patterns, thereby looking into authors, timelines, and target journals. The second is “what”; we analyze the object of study in our source articles with a particular focus on case study countries, policy levels, the temporal stage of WFD implementation, and if the directive was not studied in its entirety, the aspect of the WFD that received scholarly attention. The third is “how,” i.e., theoretical and methodological choices made when studying the WFD.


INTRODUCTION
The Water Framework Directive (WFD), adopted in 2000, is arguably the most ambitious piece of European Union (EU) legislation in the field of water.The directive defines a general framework for integrated river basin management in Europe with a view to achieving "good water status" by 2015.Institutional novelties include, among others, water planning at hydrological rather than administrative scales, the involvement of nonstate actors, various economic principles as reflected in tools such as cost-effectiveness analysis, and a common strategy to support EU member states implementing the directive (Kallis and Butler 2001, Kaika 2003, Adshead 2004, Grimeaud 2004).
Good surface water status as well as good groundwater status were the key objectives to be achieved by 2015.Additionally, member states are required to protect existing water bodies from deterioration.For surface waters, the assessment of the status is based on a measurement scale that rates biological and hydromorphological characteristics as high, good, moderate, poor, and bad, and chemical characteristics as good and fail.The directive thus breaks new ground by complementing chemical water quality assessments with the more general assessment of ecological quality.In particular, a surface water body is of good quality if there are only minor departures from the quality of pristine water bodies with minimal anthropogenic impact.Groundwater is classified as good or poor, based on its chemical and its quantitative status.Artificial or heavily modified water bodies such as canals are to achieve at least good ecological potential, which is as close as possible to good status.The achievement of the 2015 water policy goals may be delayed up to 2027 or even lowered to a less stringent objective under reference to natural conditions, technical feasibility, or disproportionate costs.
Taking into account that the existing body of EU water legislation already consists of far-reaching substantive measures, the WFD puts a high premium on the procedural side of water management.Five novelties are crucial: River basin districts: Recognizing that water is not static and that water bodies exist across political and administrative boundaries, member states are required to manage water at hydrological scales.To this end, River Basin Districts (RBDs) shall be established, respective management bodies shall be created, or if member states elect to remain within their existing administrative structures, collaboration shall be ensured between jurisdictions to manage river basins.
River basin management planning: The WFD planning process consists of eight steps: assessment of water status, characterization of physical and societal pressures on water bodies, designation of artificial and heavily modified water bodies, determination of water bodies at risk, revision of an existing River Basin Management Plan, adoption of a Programme of Measures to specify concrete actions, implementation of those two documents, monitoring, and review.This sequence of activities is to be repeated every six years.
Public participation: Engagement activities involve three components: information, consultation, and active involvement.Information requirements mainly include obligations to make status and risk assessments, background information, and maps publicly available.In terms of consultation, member states must organize three rounds of public comment during the preparation of River Basin Management Plans.Active involvement describes a more intense mode of participation and may include planning in small groups and face-to-face.

Economic analysis:
The directive encourages decision makers to consider economic principles at various stages of the planning process.This may involve cost-benefit analysis to justify exemptions, cost-effectiveness tests and other analyses to identify suitable management options, and pricing and cost-recovery mechanisms to change water consumption patterns.
Policy integration: To achieve the directive's policy goals, member state authorities are required to ensure policy integration not only within the water sector (for instance, integration of surface water and groundwater), but also within adjacent fields, such as flood control, forestry, or climate change.Rather than being a specific governance tool, policy integration represents a guiding principle of WFD water management.
These five components are held together by a strict timetable.Adopted in 2000, the directive required EU member states to transpose it into national law within three years.Until 2009, member states were then obliged to establish RBDs, to designate or create new management authorities at the river basin level, to identify and map water bodies, to establish monitoring networks, and to adopt a first River Basin Management Plan and Programme of Measures for each RBD.From 2009 onwards, member states would then reinitiate the above-mentioned sequence of management activities every six years.http://www.ecologyandsociety.org/vol21/iss2/art19/All member states engage, though in varying degrees, in the Common Implementation Strategy (CIS).Established by the European Commission and the member states in 2001, the CIS brings together domestic water practitioners, regulators, and experts so as to report best practices and encourage mutual learning.In terms of organization, CIS participants collaborate in working groups that reflect key challenges to WFD implementation, for instance, on monitoring, groundwater, heavily modified water bodies, or economics.CIS outputs include more than 30 legally nonbinding implementation guidelines that provide best-practice cases, advice for specific water management problems, and benchmarks for good water governance.The CIRCABC website (Communication and Information Resource Centre for Administrations, Businesses and Citizens; https:// circabc.europa.eu/faces/jsp/extension/wai/navigation/container.jsp)serves as an important coordination and communication device.
Who implements the directive?The WFD is a legally binding EU policy, which is why all member states are obliged to transpose the directive and put it into practice.EU candidate states may implement parts of the directive as part of their accession agreements; likewise, associated countries such as Norway and Switzerland may adopt key features of WFD water management.
To sum up, all EU member states implement the WFD.In the past nine years, those countries may have engaged in at least six overlapping, yet conceptually distinct, activities: water management at hydrological scales, the preparation of River Basin Management Plans and Programmes of Measures, public participation, economic analysis, and policy integration, plus participation in CIS working groups at the EU level.However, there is also a temporal dimension: We distinguish the preparation period between 2003 and 2009, characterized by institution building and the preparation of the first set of River Basin Management Plans and Programmes of Measures, from the first management cycle, from 2009 to 2015.This perspective leaves aside the transposition into national law (up until 2003).
Let us now undertake a thought experiment and describe WFD implementation as a 3-dimensional space, consisting of 28 (countries) x 6 (activities) x 2 (time periods) = at least 336 cells that could be filled with exciting data.Importantly, these 336 cells are not mere data points, but spaces that, each for itself, may tell complex stories about actors, ideologies, and institutions.To illustrate, one cell would contain information on public participation in France during the first management cycle.We appreciate that this thought experiment is somewhat crude, but we may still think of the above-mentioned space as a container subsequently to be filled with intriguing findings from WFD scholarship.Research priorities are distributed unevenly with regards to countries, WFD requirements, and time periods.This leads us to suggest that some sectors of this 3-dimensional space are densely populated, whereas others remain largely unchartered terrain.We are not necessarily arguing that the research community should first and foremost seek to fill those 336 cells.After all, scholars may legitimately elect to use WFD implementation as a case study to further an agenda unrelated to the implementation of the directive.However, scholars wishing to contribute to scholarly debates on WFD implementation, and on EU policy implementation more broadly, may use this image as a device to reflect on areas of research that are well documented and on those that are understudied or widely ignored.

STUDYING EU POLICY IMPLEMENTATION
The previous section outlined a purely descriptive research program: What requirements could or should be taken into account in WFD implementation research?However, to explain the implementation patterns described in the matrix above, one would have to consider a set of independent variables.For this, one would have to make use of one of the explanatory frameworks developed in the EU policy implementation and Europeanization literature.
This literature did, in its early days, borrow much from its transatlantic counterpart in the United States (Pressman and Wildavsky 1984, DeLeon 1999, Hill and Hupe 2002).In doing so, the European community reproduced quite a few shortcomings characteristic of the U.S. literature, in particular its tendency to generate endless lists of potential causal factors.To recall, O'Toole (1986) counted more than 300 variables discussed in the literature.Since then, much progress has been made.We discuss three leading approaches in Europeanization research: the goodness-of-fit approach, the actor-based approaches, and the worlds-ofcompliance approach.
Goodness-of-fit approach: Relying on historical institutionalism, this approach argues that existing paths are resistant to change.EU policy implementation will be smooth if European requirements can be accommodated within current domestic paths, yet will be delayed or incorrectly implemented if they require substantive changes.To assess the degree of suitability and to predict compliance patterns, scholars compare European requirements and domestic policies.Often, authors distinguish policy fit, emphasising the compatibility of domestic and European policies, from institutional fit, highlighting the congruity between domestic institutions and institutional requirements of EU directives (Börzel 1999).
Increasingly, however, empirical work disconfirms the argument (Haverland 2000, Falkner et al. 2005).This is because the hypothesis is "rather static in nature" (Mastenbroek 2005(Mastenbroek :1110)).Plausibly, domestic actors are not necessarily interested in preserving the status quo.Instead, they may want to change existing policies and institutions and thus utilize the EU for domestic purposes.In response to these criticisms, proponents of the goodness-of-fit perspective added a number of auxiliary variables to the initial argument (Risse et al. 2001, Thomson 2007, Hartlapp 2009).However, these amendments lead to a theoretically elegant, though empirically inconclusive, hypothesis that makes the notion of goodness of fit overly complex and, because of its ad hoc character, does not allow for ex ante hypothesising.
Actor-based approaches: Other scholars, in contrast, abandoned the structuralist core of the goodness-of-fit argument entirely and suggested examining actors, interests, and beliefs directly.From a rational-choice perspective, this includes exchange, bargaining, and principal-agent models (Haverland andRomeijn 2007, Kaeding 2008).Sociological-institutionalist accounts instead theorize compliance with EU directives as a result of socialization, persuasion, and learning processes (Checkel 2001).
Worlds-of-compliance approach: Falkner et al. (2005) and Falkner and Treib (2008) argue that the substance of a particular EU policy is of little relevance for EU policy implementation.Instead, the authors focus on national compliance cultures, i.e., general We then examined the remaining set of papers and excluded those that did not meet our key search criterion: to report empirically on WFD implementation.This way we discarded articles describing the prehistory (Kaika and Page 2003) and content and ambition (Adshead 2004) of the directive; scholarship offering legal interpretations (Grimeaud 2001) or normative critiques, for instance, as to whether the WFD is compatible with concepts such as Integrated Water Resources Management (Rahaman et al. 2004); and works anticipating rather than studying implementation patterns (Hedelin and Lindh 2008).Among the empirical studies left, we did not consider articles reflecting physical and natural science research (de Toffol et al. 2005), discussing models based on WFD inputs (Crabtree et al. 2009), or studying phenomena not directly linked to the actual implementation of the directive in a member state.These would include, for instance, researcher-led experiments with public participation (Newig et al. 2008) or economic analysis (Martin-Ortega 2012), which may have informed WFD implementation but were not part of a country's official implementation schedule.We also excluded articles if the empirical parts were shorter than three pages (Carter 2007).
As a consequence, the findings reported in this article are based on 89 journal articles.See Table 1.
To analyze the research reported in each source article, we relied on a codebook based on more than 35 items covering author affiliations and countries, research priorities and questions, implementation requirements (for instance, river basin management, public participation, or economic analysis), and levels of analysis (national, RBD, or catchment), as well as the ambition of a study (descriptive, causal, or evaluative) and related theoretical and methodological choices.These items were chosen so as to draw a complete picture of the thematic, geographic, and temporal scales of the research undertaken so far.Because of the small sample size, we do not go beyond basic statistics when we present our findings.Appendix 1 provides a complete list of our codebook items.
We then used categorical, dichotomous, and ordinal variables to code our findings; manually written comments served to justify decisions and to provide additional material for the subsequent analysis and interpretation.The two authors of this article coded, independently from each other, all 89 source papers and resolved disagreements through deliberation.

PUBLICATION PATTERNS: WHO PUBLISHES WHEN AND WHERE?
We examined 89 journal articles studying, from various disciplinary, theoretical and methodological angles, WFD implementation in Europe.The number as such is impressive.Academic scholarship on the directive is booming, probably being the most-widely studied EU directive, and definitely the mostwidely studied piece of EU legislation in the field of environment.However, those 89 studies have not been published evenly across the years.The academic community began to pay attention to the empirical study of WFD implementation more systematically in 2007, followed by a remarkable increase in publications after 2010 (Fig. 1).In fact, more than 66% of all articles were published in the last five years.Furthermore, we are aware of more manuscripts being under way; in other words, the trend is likely to continue.Not surprisingly, those figures reflect WFD implementation patterns in many EU member states.In compliance with EU legislation, many countries had transposed the directive into national law by 2003 before defining RBDs, setting up the necessary institutions and characterizing water bodies.Early research largely reflects those priorities, mainly focusing on the establishment of RBDs (Moss 2004), the characterization process (Kirk et al. 2007), or pilot projects carried out in the context of the Common Implementation Strategy (Carter andHowe 2006, Watson andHowe 2006).However, as we will show further below, it is mainly the institutional novelties in the directive that intrigued the scholarly community, in particular the involvement of nonstate actors in water planning and river basin management.Obviously, those processes could only be described, theorized, and evaluated when they were already under way, i.e., when River Basin Management Plans were adopted in 2009-2010.Consequently, studies analyzing the actual planning process mushroomed in 2011, and figures have remained at high levels since.
What do we know about the authors of those studies?We looked into three dimensions: the country of origin of the first author, their professional background, and if the author is an academic, the departmental affiliation.First, we recorded the country of origin for the lead author of each article.Those authors work in institutions in 13 different member states (Fig. 2).Countries in northwestern Europe dominated; in fact, more than 61% of all studies were first-authored by scholars based in the United Kingdom, Germany, and the Netherlands.Scholars working in the Czech Republic, Denmark, France, Greece, Ireland, Luxembourg, Poland, Spain, Sweden, and outside Europe published the remaining 39%.We were unable to identify publications authored by scholars in the remaining 15 member states.In fairness, we only talk about lead authors here.However, the overall pattern did not look significantly different when we took into account all authors of a submission; the figures for authors based at institutions in southern and eastern Europe remained significantly low.

Fig. 2. Number of publications by country of origin (first author)
. http://www.ecologyandsociety.org/vol21/iss2/art19/These figures are not counterintuitive.Ultimately, they may simply reflect general patterns of academic productivity.There is no doubt that scholars in southern and eastern Europe carry out high-quality research across the board.Still, academics from northwestern Europe tend to publish in international peerreviewed journals more often than their colleagues in other parts of Europe.We may speculate as to why this is so, but whatever the causes are they say little about WFD research as such.However, it is plausible to argue that the directive is generally more widely studied in the United Kingdom, Germany, and the Netherlands.This may be because the WFD, as a water quality directive, addresses issues of more fundamental importance in northwestern Europe, setting in motion policy makers, civil society, national funding bodies, and the scholarly community.
Although troubled by water quality problems too, many regions in southern Europe put a higher premium on the question of water quantity and supply, a preference that may bind scarce research resources.Nevertheless, the observed pattern may have a significant influence on case selection, resulting in a noticeable imbalance as to the countries studied by the WFD community.
Second, a majority of our first authors, some 77 out of 89 articles in the sample, are academics from universities or national research institutes.Ten authors are practitioners, for instance, from government departments, environmental regulators, or charities; two authors have a double affiliation.We were not surprised to see a majority of academics here.Still, we believe that the 11% nonacademics in our sample is above average, highlighting the importance of WFD-related research for environmental practitioners in Europe.Third, the WFD community in Europe largely consists of scholars working at interdisciplinary environmental social science departments or, in a few cases, at natural science or engineering schools, in total 59 out of 79 in our sample.Only 17 scholars work at social science departments with a clear disciplinary focus: 6 in politics, 5 in law, and 6 in economics.
To be clear, this says nothing about their disciplinary background.However, it suggests that many authors operate in an interdisciplinary environment, often with a sympathy for applied research.As we argue below, this is likely to shape theoretical and methodological choices when studying WFD implementation in Europe.
Our analysis of target journals also highlights the interdisciplinary ambition of many scholars.Figure 3 summarizes the Web of Science subject categories of those journals that have published articles in our sample.There are two caveats.First, some journals are listed in several subject categories, so the absolute number of entries is higher than the number of articles in our sample.Second, a few journals are not listed in the Web of Science, and we manually assigned plausible subject categories to those outlets, based on editorial mission statements and other information found on the journal website.
Accordingly, manuscripts about the implementation of the WFD usually find their home in interdisciplinary journals, particularly in those specializing in environmental social science and water resources.Authors rarely target journals in social science subdisciplines such as political science (van der Arend and Behagel 2011, Koontz andNewig 2014a), public administration (Behagel and Arts 2014), economics (Bithas et al. 2014, Dehnhardt 2013), or legal studies (Adshead 2006, Keessen et al. 2010).At first sight, this is somewhat counterintuitive.After all, many authors explore topics such as public participation or the use of cost-benefit analysis in water planning.Arguably, their findings may be of great interest for a wider audience in political science or economics, for instance, for scholars working on deliberative democracy or evidence-based policy making.However, the approaches taken to study such phenomena, specifically the interdisciplinary and often applied nature of WFD-related research, make such work less appropriate for narrow disciplinary debates and more suitable for an interdisciplinary audience and their journals.Still, we are somewhat concerned that scholarly communities may talk past each other; in other words, interdisciplinary outlets may have become another niche for a community of experts rather than an arena of exchange across disciplines.

MAPPING SCHOLARLY INTEREST: COUNTRIES, POLICY LEVELS, THEMES
Now that we have established who publishes when and where, let us take a closer look at the object of those studies: the what.To this end, we organized our review around five dimensions: countries, policy levels, transboundary water management, the temporal stage of WFD implementation, and if the directive has not been studied in its entirety, the aspects of the directive that attracted scholarly attention.In other words, we map scholarly interest across a range of spatial, temporal, and substantial dimensions.
We begin with dimension one, i.e., countries studied.Figure 4 displays the number of publications per country.The absolute number of entries is higher than the number of articles in our sample; this is because some papers report on more than one country.Generally, our sample is characterized by an unhealthy imbalance.Five countries were studied quite thoroughly in the past: the United Kingdom (24 studies), Germany (19 studies), the Netherlands (16 studies), and Spain and Sweden (11 studies each).
Another group of countries is covered by between three to six articles, including Belgium, Denmark, France, Greece, Ireland, and Portugal.However, a majority of EU member states have received little or no scholarly attention at all.These include many countries that have joined the EU in 2004 or thereafter, for instance, Hungary, Romania, and Slovakia, but also the Baltic states.What should be clear from this brief survey is that much is known about WFD implementation in northern and western Europe, but relatively little about WFD implementation in Mediterranean countries, including founding members and heavyweights such as France and Italy, and in eastern Europe.
We offer two explanations.On the one hand, the above pattern simply reflects patterns of authorship.We have in our sample, for various reasons, a higher number of scholars based at British, Dutch, and German institutions, and this may influence case selection.There are good reasons for that: they speak the language, they have the contacts, they know the context, and they respond to requirements of national funding bodies.On the other hand, there may be an enlargement effect: the 2004, 2007, and 2013 accessions came with important transitory provisions and may have delayed WFD implementation in those countries.
Is all this a problem?Yes and no at the same time.Case selection is not random.Academics choose cases to make a specific argument, to test an established theoretical proposition, or to develop a new one.This argument may result in a specific interpretation or critique that remains valid and plausible beyond the case studied.Scholars in our sample sometimes study the WFD, or specific provisions of the directive, to make a claim that is entirely decoupled from the directive.For instance, Drazkiewicz et al. (2015) used WFD cases to test the hypothesis that participatory arrangements enhance the ecological quality of environmental decisions and their implementation.There is little reason to study Austria, Bulgaria, or Malta if there was no participation in those countries that could have had an effect on environmental outcomes.However, as we show further below, when we talk about research design choices, an overwhelming majority of studies in our sample are descriptive, sometimes with evaluative undertones.Authors report what is going on and sometimes use ad hoc or theory-informed benchmarks to assess what they observe, often, so it seems, with a view to supporting the implementation of WFD provisions in their country.From this perspective, the above-mentioned imbalance is unfortunate because it provides an incomplete picture.This is no trivial observation: many evaluations suggest that WFD provisions were implemented in an incomplete or suboptimal way.That may be true or not, but it would certainly be helpful to put such assessments into context by studying countries whose implementation record is, if we trust EU reports and media coverage, less than ideal too.In other words, selective cases may highlight deficits that are negligible if compared against other cases and may direct our attention away from more serious problems.
We now turn to dimension two: the policy level studied.We distinguish three levels: the national level, relevant for the domestic transposition of the WFD and sometimes for the general organization of WFD water planning in a EU member state; the RBD level, i.e., the reference level envisaged in the directive for the preparation of management plans; and the sub-RBD level, in this article conveniently called the catchment level, although we do not associate any specific ecological definition to this term.Accordingly, we have 30 articles in our sample that strictly analyze the national level.These are followed by research looking into processes at catchment level.There is a relatively low number of 16 articles studying the WFD at the RBD level.A further set of http://www.ecologyandsociety.org/vol21/iss2/art19/18 articles study combinations of, and sometimes the interactions between, several levels.We were surprised about the minor importance of RBDs and the larger number of studies focusing on catchments, given the prominence of RBDs in the directive.We offer three explanations.First, some member states may have found the RBD level somewhat impractical and organized important management activities at lower levels instead.This may be the case when RBDs conflict with administrative boundaries in a federal system such as Germany, where important planning activities are carried out at catchment level (Koontz and Newig 2014b).Obviously, this may direct scholarly attention away from RBDs.Second, some RBDs are inconveniently large to be studied in-depth through qualitative methods.In other words, the study of catchments is a methodological choice although, as we will argue below, such choices have not always been made explicit.
Third, when the actual policy level does not matter, scholars go where the data are.For instance, when Borowski et al. (2008) decided to study the effects of participatory arrangements on social learning, they chose the level where such involvement processes take place.Case selection follows theory.Still, in light of the fact that many studies in this field have no causal ambition and remain descriptive, it is fair to argue that such descriptions then tend to show an incomplete picture.Incomplete in the sense that the reader does not always know how representative a catchment is vis-à-vis other catchments.Furthermore, extensive knowledge about one particular policy level does not necessarily enhance our knowledge of other levels and their interactions.For instance, we possess an in-depth understanding of German WFD water planning at the catchment level and of important initiatives at the federal level.However, despite a wealth of publications on Germany we are still in the dark as to the coordinative mechanisms between federal states, i.e., mechanisms in place to integrate river basin planning and catchment level activities in various states.Future research will definitely benefit from perspectives that integrate various policy levels to provide a more complete picture of WFD water planning.
Dimension three looks into the transboundary aspect of river basin management.Almost all countries in continental Europe share at least one river basin with their neighbors.So far, no powerful mechanisms are in place to encourage river basin management beyond national borders.International commissions exist for each transboundary basin, but important parts of the planning process are excluded from such coordination, not least because countries implemented WFD provisions at a very different pace.At the time of writing, the scholarly community has not taken many steps to look into the transboundary aspect of WFD implementation.We have 33 studies studying RBDs, a vast majority of those RBDs being transboundary, but only one study (Mylopoulos and Kolokytha 2008) addresses this issue by studying both sides of a Greek-Bulgarian basin.All other studies remain on "their" side of the river; Meyer and Thiel (2012), for instance, study the German part of the Odra basin and ignore the Polish part.Likewise, the largest river basin in Europe, the Danube, has been managed for more than 150 years in the spirit of transboundary water management, through various international commissions established by international agreements.However, this has rarely attracted scholarly attention, and never in a WFD context.We believe this is a fundamental research gap.We acknowledge that there are limits to such a research program, exactly because river basin management beyond national borders currently meets various political, administrative, and perhaps, ideological obstacles.Still, scholarship on WFD implementation would greatly benefit from studies establishing the presence or absence of transboundary mechanisms.
We now move from spatial and scalar aspects to the temporal dimension of implementing the directive.This is dimension four: What stages of the WFD planning process have been studied so far?Member states were supposed to implement the directive in various steps.This involves transposing the WFD into national law, identifying RBDs, nominating a competent authority, designating water bodies, assessing pressures on aquatic environments and evaluating the risk of missing environmental targets (characterization process), and the drafting of River Basin Management Plans and Programmes of Measures.The early stages of the implementation process were accompanied by pilot measures.They were carried out in the context of the common strategy of implementation with a view to supporting the implementation of "tricky measures," such as the involvement of nonstate actors or the use of economic analysis in water planning.
Our sample includes three articles that analyze pilot projects: two of them on participation and one on cost-benefit analysis.All three report on the United Kingdom.Nine papers address, among other issues, the process of transposing the WFD into national law.A vast majority of papers in our sample, 86 out of 89, cover the preparation phase from 2003 to 2009 (and in some countries, 2010).This is not very counterintuitive per se.After all, many exciting innovations, from a social science perspective, could be observed only during or after the preparation phase.These include the establishment of RBDs, the involvement of stakeholders in the planning process, and the application of various economic decision-making tools.However, there is a caveat here: Many studies provide a snapshot, i.e., report about a specific moment in time or a particular step in implementing the directive.A few studies present a bigger picture or compare WFD implementation efforts over time.For example, we identified only three articles explicitly studying the first implementation cycle from 2009 to 2015 or systematically comparing the experience gained in different sectors and countries in the preparation phase and the first cycle.
Finally, in dimension five, we analyzed the various themes and requirements, usually institutional innovations, that can be found in the WFD and that may have attracted scholarly attention.We distinguished seven themes: the ecological goals of the directive and how achievement of those goals has been encouraged or enforced in EU member states, including the designation of water bodies; the use of economic principles as reflected in tools such as cost-benefit analysis in water planning; the establishment of RBDs and problems that arise as a consequence of rescaling; the involvement of nonstate actors in water planning; the integration of other policy areas in WFD water management, for instance, climate, forestry, flooding, and biodiversity; the planning process, if not specifically focused on participation, economic tools, policy integration, or river basin management; and a residual category: WFD in general.This last category is particularly useful for studies looking into the overall transposition of the directive into national law.However, the category also covers quantitative http://www.ecologyandsociety.org/vol21/iss2/art19/research aggregating several WFD themes across a large number of EU member states such that important lessons can be drawn regarding the general implementation of the directive in Europe, although at the same time, because of the step of aggregating information, little can be said in detail about, for example, participation, economic analysis, or policy integration.Figure 5 below reports our findings.Quite a few articles discuss more than one theme; the absolute number of entries is therefore higher than the number of articles in our sample.What do the data tell us?We observed a great imbalance as to the institutional novelties promoted by the WFD.The involvement of nonstate actors in water planning has inspired a rich literature.
In fact, 52% of all articles and (the figures differ here because articles often report on more than one theme) 41% of all themes are related to public participation in WFD water management.
To be clear, it is beyond the scope of this article to discuss the achievements made in this field.However, what can be said is that previous work centers on three themes.First, we have a number of publications taking a Europeanization perspective; i.e., they ask why specific domestic arrangements have come into place and what role the EU plays therein (Liefferink et al. 2011).Second, authors explore the effects of participation, typically on social learning processes (Borowski al. 2008), ecological outcomes and implementation (Drazkiewicz et al. 2015), or legitimacy (Behagel and Turnhout 2011).Third, we found a series of articles comparing instances of participation against legal or normative benchmarks, i.e., evaluative exercises (Watson and Howe 2006, Slavíková and Jílková 2011, Benson et al. 2014).With the exception of works focusing on ecological outcomes, an aspect still treated with neglect, the above questions represent strands of research that speak to a wider and already well-established literature.
Certainly, this enthusiasm for participation within the WFD research community calls for an explanation.We offer three.First, research interviews with national water managers and European Commission officials, carried out in the context of other projects, suggest that of all institutional novelties proposed by the WFD, public participation perhaps represents the greatest challenge to national administrative cultures and established water management traditions.Exploring public and stakeholder engagement therefore constitutes an excellent opportunity to study administrative reform, institutional change, and learning in contested settings.Second, these challenges resulted, in our mind, in an exceptionally high number of funded projects at national and EU levels, consultancy opportunities, and other incentives for researchers to collaborate with policy makers and stakeholders on participation.Third, participation resonates with a wide variety of literature, much more than other institutional novelties in the directive do.This includes communities working on EU policy implementation, on participatory and deliberative democracy, on regulatory reform, on policy effectiveness of participatory governance, on social learning, and many others.
We found less work, 20% of all articles, on water management at ecological scales.Once again, scholars study questions related to the Europeanization of domestic arrangements, i.e., of policy implementation.When authors identify misfits between EU requirements and domestic practice, and in this context this means between ecological and administrative scales, a link is usually made to the literature on spatial fit and institutional interplay (Moss 2004(Moss , 2012)).This literature is even more prominent in works investigating economic rationales for specific scalar arrangements (Thiel andEgerton 2011, Thiel 2014).
However, regulatory tools such as cost-benefit analysis remain understudied (but see Dehnhardt 2013Dehnhardt , 2014)).In other words, scholarship investigating the identification of environmental benefits and costs, approaches toward cost recovery of water services and incentive pricing, and the link between participation and economic analysis, in particular how they complement each other, is still in great demand.We also believe that the politics of exemptions is neglected so far, i.e., political strategies that often result in less stringent water quality objectives.Finally, policy integration is rarely considered in prior work, in particular with regard to climate policy-our sample includes two articles that exclusively analyze this relationship in Denmark and Scotland (Blackstock 2009, Larsen 2011)-and to EU policies based on similar management philosophies.These include, for instance, the Birds and Habitats Directives, the Marine Strategy Framework Directive, and the Floods Directive; we hypothesise they could be a source of both mutual reinforcement and conflict (Beunen et al. 2009).In fairness, the literature on institutional interplay (Young 2002; in a WFD context see, e.g., Moss 2004, Newig and Fritsch 2009, Hüesker and Moss 2015) has addressed such questions.However, we still see the promise of linking spatial approaches to the integration of different policy fields in WFD research more than in the past.

STUDYING WFD IMPLEMENTATION: THEORY AND METHODS
In this section we discuss how previous scholarship has approached the study of WFD implementation.To this end, we focus on three aspects.The first is the ambition of the research project; we hereby distinguish descriptive, causal, and evaluative work.The second is questions of research design and methodology.The third is choices made in terms of concepts and theory.
In terms of ambition, we identified three ideal types: (1) papers with a causal ambition that try, supported by theory and conceptual frameworks, to establish a causal link between two or (2) evaluative papers that compare patterns of WFD implementation against legal requirements or normative frameworks; and (3) descriptive papers that portray and detail a phenomenon without embedding it in an explanatory or normative framework.We complemented those ideal types with two additional categories: causal papers without theory, i.e., based on ad hoc explanations; and evaluative papers without normative framework, i.e., providing ad hoc evaluations.
We observed a large number of descriptive work in our sample.About 55% of all articles, 49 out of 89, describe the implementation of the directive, or of a specific WFD theme, in Europe.However, those authors do not use their empirical material to build, explore, or test theories or to appraise their subject of analysis using a normative framework.Furthermore, 13 articles in our sample are descriptive in nature, but come with ad hoc causal explanations.Likewise, 3 papers present ad hoc evaluations.Only 18 articles display a distinct causal ambition, relying on theory and hypotheses, and 6 papers offer a normative critique based on a previously defined framework against which authors compare their observations.In other words, only 24% of all articles go beyond descriptive or ad hoc ambitions.
According to our data, journal articles display a descriptive research interest more often if they appear in physical science, engineering, or interdisciplinary journals.Figure 6 below summarizes the type of ambition across the Web of Science journal subject categories in which the journals in our sample are listed.
Our sample of 89 articles includes quite a few articles published in physical science journals or outlets with a technical orientation, i.e., in journals belonging to Web of Science subject categories such as chemical engineering, civil engineering, environmental sciences, limnology, or metereology.Figure 6 suggests that those journals have a particularly strong preference for descriptive submissions.The picture is mixed with regard to interdisciplinary journals.We have many descriptive pieces in periodicals specializing in water resources, whereas the environmental Studies category is more balanced.Finally, journals in classic social science subdisciplines such as economics, political science, or public administration as well as in geography and urban studies tend to publish more research with causal or evaluative ambitions.
Our analyses also suggest that practitioners are more likely to publish descriptive work, and they prefer to do so in physical science, engineering, or interdisciplinary journals.
All studies with an evaluative ambition have one thing in common: They focus on public participation, including research on Ireland (Irvine and O'Brien 2009), the Czech Republic (Slavíková and Jílková 2011), Spain (Parés 2011), and the United Kingdom (Blackstock et al. 2012, Fritsch and Benson 2013, Benson et al. 2014).None of those countries had well-established participatory mechanisms in place prior to the adoption of the directive.It is therefore not surprising to see authors evaluating the progress made after the WFD was transposed into national law.
We observed more heterogeneity when we looked at causal projects in our sample.Articles cover areas as diverse as public participation, river basin management, and economic analysis, http://www.ecologyandsociety.org/vol21/iss2/art19/but also the directive as a whole.The majority of articles use WFD implementation as the dependent variable; i.e., they try to shed light on the political, economic, or societal causes of specific implementation patterns.To this end, authors either refer to the Europeanization literature (Liefferink et al. 2011, Moss 2004) or use public policy approaches, including institutional theories (Thiel and Egerton 2011, Nielsen et al. 2013, Thiel 2014), policy change models (Bourblanc et al. 2013), and the advocacy coalition framework (Dehnhardt 2014).Another set of work uses WFD implementation as the independent variable.Such studies explore, for instance, the effects of participatory exercises whereby social learning (Borowski et al. 2008, Lundmark andJonsson 2014) or policy implementation and environmental outcomes (Koontz and Newig 2014a;Drazkiewicz et al. 2015) constitute the dependent variables.
The Europeanization literature has developed a number of fruitful approaches to explain EU policy implementation.We distinguished earlier the goodness-of-fit, actor-based, and worlds-of-compliance approaches, plus theories combining the goodness-of-fit approach with additional variables.The WFD community has, to date, made only limited use of this literature.
In fairness, this may be because authors never meant to explain implementation patterns in the first place; instead, they may have selected the directive as an independent variable.However, only two papers in our sample analyze the implementation of the WFD as dependent variable and utilize the Europeanization literature systematically (Liefferink et al. 2011, Moss 2004); both papers employ a variant of the goodness-of-fit approach with additional variables.Although Albrecht (2013) mentions the term "Europeanization" in the title of her paper, no further reference to this literature is made in the remainder of the article.There is certainly potential to employ a wider array of approaches in a WFD context, ideally in a comparative setting with competing theories.
The overall dominance of descriptive projects in this literature is, in our estimation, problematic.To be clear, we recognize that explanation and evaluation are not necessarily key priorities for many authors studying the WFD.We also appreciate the fact that many interdisciplinary scholars seek impact outside rather than inside the academy.Finally, we agree that "mere description" (Gerring 2012) has its merits.To illustrate, descriptive research may constitute a springboard for subsequent evaluative or causal projects, including meta-analyses and comparative studies.Unfortunately, however, empirical documentation of such uses in a WFD context is minimal.So far we are left with a pile of articles that describe in more or less detail important features of WFD implementation in Europe.However, we as readers are somewhat left in the dark as to what the purpose, mission, or function of those articles is.After all, their observations cannot easily be translated across cases or disciplines; these articles often contain rich materials shy of a research question.To enhance the echo of such research beyond the community of WFD scholars, we think authors are well advised to carry the empirical torch slightly farther than to the nearest descriptive pit stop.
This is where theory kicks in.Essentially, WFD implementation research is theory-free territory.Almost 50 articles out of 89 do not mention theory at all.Others mention concepts and theories in between the empirical material, but those references do not really enhance our understanding of the overall argument; name dropping seems to outplay systematic utilization.There are a small number of theory-guided studies in our sample where conceptual considerations inform hypothesis building or normative frameworks.However, by and large, theory falls by the wayside.This is a direct corollary of the descriptive or ad hoc direction taken by many studies.If somebody prioritizes description or is content with ad hoc conclusions, there is little necessity for abstract reasoning.We find this lack of theory stunning.Theories are extremely useful vehicles to translate ideas across cases and disciplines, and we are convinced that the WFD community would benefit from a more systematic recourse to concepts, hypotheses, systematic classifications, and theory.
In terms of methodology, the WFD implementation literature is still in its infancy.We initially planned to map methodological choices made in this literature against classic approaches in social science.However, we failed to do so: More than 30%, 27 out of 89 articles in our sample, provide virtually no information on research design, methods, and data.We can sometimes infer from the list of references that policy documents have informed the analysis; we may make the educated guess that some conclusions must be based on interview data or direct observation.However, the sheer absence of any methodological statement in almost one third of all publications casts a shadow over the entire subfield.
Transparency and openness with regard to data sources and analysis are prerequisites for critical debate and enable the confirmation and refutation of claims.It is a professional standard that should not be given up lightly, and we do not see the merits of downplaying methodological precision.According to our data, authors operating in physical science or engineering schools are more likely to take a relaxed approach to methodology when they report on WFD implementation; likewise, such articles tend to be published in natural science or engineering journals.
Another set of 18 articles report on data and research design.However, this information does not form a coherent methodological section, but is presented as part of the introduction, in the empirical sections, or simply in a footnote.The average word count is 138; in 9 papers it is 75 words or fewer, which is, upon sober reflection, definitely insufficient to inform adequately about the methodological choices made.Finally, 44 out of 89 articles, fewer than half of the sample, provide a separate section dedicated to data and methods, with an average length of 579 words.Those studies usually rely on qualitative methods, particularly interview data, policy documents, participatory observation, and media analyses.Because of the overall lack of theory, there is little dialogue between theory and methods.In other words, methodological choices are rarely motivated by theory.Consequently, justifications for specific research designs and data analysis methods are very practical in nature.
With regard to research design, our sample looks very uniform.First, our knowledge about WFD implementation in Europe primarily relies on single case studies or small-n comparisons within one country.Our sample includes 74 single country studies, 2 comparisons between EU countries and jurisdictions overseas, as well as 11 comparative studies within Europe (between 2 and 11 countries each).There is a striking lack of large-n quantitative research: We only identified 2 studies (Kanakoudis andTsitsifli 2010, Kanakoudis et al. 2015).Second, those 11 small-n studies http://www.ecologyandsociety.org/vol21/iss2/art19/are not always comparative in methodological terms.This is because they frequently fail to sort cases into comparable sets, and cases are rarely chosen because of their properties.In other words, there are few attempts to use only most-similar, mostdifferent, or related strategies so as to maximize the benefits of a comparative research design.In this sense, many multicountry studies resemble case collections rather than theoretically and methodologically justified case comparisons.Not surprisingly then, 6 of those 11 studies are descriptive; 4 others have a causal ambition, and the remaining study offers an ad hoc explanation.
We argued before that the WFD community has submitted their research to a diverse range of journals, from economics and political science outlets to interdisciplinary journals and physical science periodicals.We now explore whether publication choices are linked to research ambition and methodological transparency.
The underlying intuition is that articles characterized by less ambitious research programs or weaker methodology parts are likely to be published in lower-ranked journals.
We first used the Web of Science to establish the relative position of a journal in its respective subject category ranking, based on the two-year impact factor at the time of publication of each article in our sample.To this end, we classified those journals as belonging to one of the following seven categories: top 5%, top 10%, top 25%, top 50%, top 75%, or top 100% of its respective Web of Science journal subject category; journals not listed at all form the last category.This way we are able to compare journals across Web of Science subject categories, although those categories differ as to their average impact factor and the number of journals listed.In other words, we argue that a journal on position 6 in a subject category with 138 journals enjoys, within its disciplinary community, a greater reputation than a journal on position 4 in a subject category of 59 journals.If a journal was listed in several categories, we used the best measure of this journal.The approach also enabled us to control for publication years; i.e., we considered the possibility that journals move up and down in the rankings over time.
We are aware that this approach has limits.On the one hand, we used the publication year of an article although the time of submission would be a more precise measure; this information, however, was not available to us.On the other hand, the position of a journal in a subject category ranking reflects its impact factor, a metric that has attracted widespread criticisms (Giles and Garand 2007, Plümper 2007, Garand et al. 2009, Eyre-Walker and Stoletzki 2013).The impact factor is statistically vulnerable because of the small sample size in many subject categories.Citation patterns often reflect journal availability rather than journal quality; authors cannot use a specific article if their institution provides no access.The impact factor is influenced by self-citations.Finally, the impact factor counts citations, but does not consider whether authors cite papers in a supporting or in a critical fashion.This implies that the impact factor says less about quality and more about reputation.However, we still believe that the relative position of a journal in the rankings provides the most useful metric in the context of this article.
Our findings are straightforward: Although there is no linear trend, our analysis suggests that theory-based causal and evaluative projects are more likely to be published in higherranked journals.Descriptive papers and those relying on ad hoc analyses, in contrast, tend to be published in outlets at the bottom of their subject category ranking or in outlets not listed in the Web of Science at all (Fig. 7). Figure 8 displays the relationship between methodological transparency and position in the rankings.Our findings suggest that, although there is one outlier category at 75%, higher-ranked journals tend to prefer submissions with a separate and, as we have shown earlier, longer methods section.Lower-ranked journals or outlets not listed at all are more likely to publish papers where information on data, cases and research design forms a shorter part of the introduction or empirical section, or papers with no methodological information at all.

CONCLUSION
We set out to systematically review studies dedicated to the implementation of the WFD in Europe.Based on a meta-analysis of 89 journal articles, we explored more than 35 dimensions in each study to report on authors and journals, countries, policy levels, and WFD themes, as well as on theory, methods, and research design.
Returning to our thought experiment described earlier, Table 3 below displays research priorities with regards to countries and WFD requirements, thereby ignoring the temporal dimension.The chart stresses that quite a few areas of research are well documented, whereas others appear as blind spots.We discuss them in turn below.We found, first, that there is a cluster of very well-researched countries, including the United Kingdom, the Netherlands, and Germany; however, member states that joined the EU in 2004 and 2007 as well as Mediterranean countries such as Italy and Greece are under-represented.These are gaps to be filled.However, we envisage two more promising research programs: On the one hand, we encourage scholars to compare groups of countries.EU policy implementation research talks about 'worlds of compliance' (Falkner et al. 2005), and the WFD would lend itself well to an empirical test of this claim.Furthermore, we believe a more systematic comparison of northern and southern EU member states will help understand the role of water quality and water quantity problems in EU environmental policy implementation.On the other hand, we suggest taking a closer look at candidate states, countries addressed by the European Neighbourhood Policy, and associated countries such as Switzerland and Norway (but see Baaner 2011).
Second, there is a certain imbalance as to the institutional novelties promoted by the WFD.Although the involvement of nonstate actors in water management has inspired a rich literature, there is less in-depth research on river basin planning and management at ecological scales.Most importantly, economic principles, as reflected in tools such as cost-benefit analysis, have not been studied in depth.This includes cost-effectiveness analysis, incentive tarification, adequate levels of cost recovery, and designation of heavily modified water bodies, all of them challenging in terms of knowledge, uncertainty, legitimacy, and social acceptability.The politics of exemptions, which often results in less stringent water quality objectives, also remains understudied.We identified three additional areas for future research.On the one hand, the directive and early guidance documents made no reference to measures supporting adaptation to climate change.However, the topic has become more prominent at the EU level (European Commission 2009b), and we suggest more systematic research exploring the link between WFD management activities and climate change adaptation.On the other hand, EU policy documents increasingly make reference to ecosystem services as a key concept to support WFD implementation (European Commission 2012b).Future research could take the cue and analyze whether, and if so how, this plays out on the ground.Furthermore, we encourage more empirical research on the interaction between the WFD, the Floods Directive, and the Marine Strategy Framework Directive, and with legislation unrelated to water, for instance the Birds and Habitats Directives.Intuitively, we would expect the potential for mutual reinforcement, for instance, when it comes to promoting a culture of participation in environmental management, but also for conflict and contestation in other areas (but see Newig et al. 2014).
Third, there is a lot of research on the preparation phase of WFD implementation, more specifically on the process of drafting the first set of river basin management plans.However, we know little about continuity and change from the preparation phase to the first cycle, and there is little comparative work over time.Further research priorities may include the achievement of the 2015 water quality goals and the role of governance innovations such as participation therein, and comparisons of the preparation phase, the first management cycle, and the on-going second management cycle.The latter may relate to a number of literatures, including those on policy learning and the role of guidelines, training, and capacity building in policy making (May 1992).
Fourth, there is a conspicuous lack of theory in WFD scholarship.
Authors tend to describe implementation patterns and, at times, to apply normative frameworks, but only a minority of studies refer to theory when explaining compliance with the WFD and embed observations in their social, economic, or political contexts.In this paper we outline a number of approaches and http://www.ecologyandsociety.org/vol21/iss2/art19/refer to others.All of them may be useful to help explain patterns of WFD implementation.We are convinced that theory deserves a more prominent place in future research on the WFD and its implementation in Europe.
Fifth, methods and research design are patently neglected and a serious cause of concern.Authors accord little attention to methodological questions, and papers mostly have a descriptive orientation.Overall, 21 out of 89 articles are descriptive and provide no information on data and methods.Moreover, our current knowledge about the implementation of the WFD in Europe relies mainly on single case studies or small-n comparative studies within one country.Cross-country comparisons are a minority, and there is a striking lack of large-n quantitative research.We believe future research would benefit from a departure from single-country studies.
Responses to this article can be read online at: http://www.ecologyandsociety.org/issues/responses.php/8411

Fig. 3 .
Fig. 3. Number of publications by Web of Science subject categories.

Fig. 4 .
Fig. 4. Number of studies per case study country.

Fig. 6 .
Fig. 6.Research ambition across Web of Science subject categories.

Table 2 .
Table 2 below summarizes our findings.Number of studies per policy level.

Table 3 .
Number of studies per country and WFD theme.