Ecology and Society
Home | Archives | About | Login | Submissions | Subscribe | Contact | Search
 E&S Home > Vol. 16, No. 1 > Art. 12

Copyright © 2011 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf version of this article

The following is the established format for referencing this article:
Morris, J. B., V. Tassone, R. De Groot, M. Camilleri, and S. Moncada. 2011. A Framework for Participatory Impact Assessment: involving stakeholders in European policy making, a case study of land use change in Malta. Ecology and Society 16(1): 12. [online] URL:

Research, part of Special Feature on Landscape Scenarios and Multifunctionality – Making Land Use Assessment Operational

A Framework for Participatory Impact Assessment: Involving Stakeholders in European Policy Making, a Case Study of Land Use Change in Malta

Jake Breton Morris 1, Valentina Tassone 2, Rudolf de Groot 2, Marguerite Camilleri 3 and Stefano Moncada 4

1Forest Research, 2Wageningen University, 3Malta Environment & Planning Authority, 4University of Malta


A Framework for Participatory Impact Assessment is presented for use within European land use policy impact assessment. The context and rationale for the development of the Framework are outlined, both in the context of European policy making and within a project called "Sustainability Impact Assessment: Tools for Environmental, Social and Economic Effects of Multifunctional Land Use in European Regions". A detailed description of the sequence of methods that make up the Framework is provided, followed by illustrations and details of the practical application and results from a case study in Malta, where the Framework was used to carry out an impact assessment of biodiversity policies. After reporting on the reflections of the research team and valuable feedback provided by Maltese stakeholders, the Framework’s ability to enhance the quality, credibility and legitimacy of European policy impact assessment is discussed.

Key words: European policy making; land use change; Malta; participatory impact assessment; stakeholder participation


The European Commission (EC) is committed to an inclusive and participatory approach to developing and implementing policies (CEC 2002). Impact Assessment (IA) was formally introduced into European policy making in 2002 and is essentially a form of decision support that analyzes the potential impacts of policies before their adoption (see Tscherning et al. 2008 for a useful overview of IA). Facilitating participation in policy making is central to the developing Impact Assessment (IA) agenda within the Commission, and is highlighted in the IA Guidelines themselves. The Guidelines state that the consultation of interested parties can usefully contribute to the quality, credibility and legitimacy of the IA process and its results:

Consulting those who will be affected by a new policy or initiative and those who will implement it is a Treaty obligation. It is an essential tool for producing high quality and credible policy proposals. Consultation helps to ensure that policies are effective and efficient, and it increases the legitimacy of EU action from the point of view of stakeholders and citizens (CEC 2009:19).

Despite the EC’s clearly stated commitment to participation, and despite the status of consultation as a Treaty obligation, there is little in the way of actual guidance for IA practitioners, beyond general descriptions of the consultation methods used in extended impact assessment reports.

In response to the lack of guidance and documented examples of participatory approaches to IA, a methodological framework for involving stakeholders in sustainability impact assessment (SIA) is presented and critically analyzed in terms of European land use policies developed within the SENSOR project (Sustainability Impact Assessment: Tools for Environmental, Social and Economic Effects of Multifunctional Land Use in European Regions), an Integrated Project funded under the EC’s sixth Framework Programme. A “Framework for Participatory Impact Assessment” (FoPIA) is presented, which is a structured set of sequenced research methods that, collectively, facilitate the involvement of national, regional and local stakeholders in assessments of European land use policy impacts at the case study level. The FoPIA was developed and tested through a structured program of research in the SENSOR project’s case study regions. To ensure sensitivity to criteria of sustainability appropriate not only at the European level, but also at regional and local levels, SENSOR partners engaged in a range of research activities in so-called Sensitive Area Case Studies (SACS), as follows: The Maltese Archipelago (Malta), Eisenwurzen (Austria), The High Tatras (Slovakia), Silesia (Poland), Western Estonia Coastal Zone and Saaremaa Island (Estonia), Valais (Switzerland), Saxony Lusatia (Germany). The deployment of the FoPIA to assess the sustainability impacts of biodiversity policy in Malta is used both to illustrate the framework of methods, and to critically assess the contribution in terms of the quality, credibility and legitimacy of the outputs of this worked example of a participatory approach to SIA.

By way of orientation, it should be recognized that the FoPIA complements a model-based Sustainability Impact Assessment Tool (SIAT), also developed within the project (Sieber et al. 2008). It is envisaged that the SIAT will be used to conduct “quick-scan” assessments of policy impacts across Europe to highlight areas or issues of significant impact that warrant further, more detailed investigation. The FoPIA could then be deployed in these selected areas to cross-check SIAT outputs by involving stakeholders in a parallel, deliberative assessment of the same policy, but focusing on a specific region or regions.


The FoPIA is designed to enable assessments of policy impacts that are sensitive to national, regional and local sustainability priorities by harnessing the knowledge and expertise of national, regional and local stakeholders who play a central role in the analytical process. The analysis of specific sustainability problems gives rise to realistic national and regional policy and land use change scenarios. Within SENSOR, scenarios refer to possible changes in European policy in response to perceived sustainability problems (Kuhlman 2008). Scenarios can involve combinations of policy instruments, such as legislation, subsidies or taxes that are identified in the description of a policy case. Scenarios are the subject of a SIA and the impacts of each scenario are analyzed through comparisons with a counterfactual, or baseline scenario, reflecting the situation in which the policy is not implemented. These scenarios are then the subject of the SIA, involving the selection and analysis of sustainability criteria and indicators that reflect key issues related to sustainable land use in the case study region. The analysis of impacts and the setting of sustainability limits are informed by stakeholders’ knowledge of the current, regional status of economic, social and environmental systems.

The FoPIA is structured around the five elements within the European Environment Agency’s Driver-Pressure-State-Impact-Response (DPSIR) framework (EEA 1999). Within SENSOR and so for the FoPIA, driver refers to the sustainability problems and issues that motivate a policy response, pressure to the changes in land use brought about by a change in policy, state and impact to the resulting changes in social, environmental and economic systems, as represented by changes in sustainability indicator values that are, in turn, compared with sustainability thresholds or limits. Response refers to the decisions made by the policy makers informed by the technical outputs of the assessment and so would, conventionally, lie outside the remit of an (S)IA tool. Indeed, the model-based SIAT presents changes in indicator values driven by changes in policy and land use, and indicates where impacts go beyond sustainability thresholds, but offers no guidance to the end user to inform the interpretation of these outputs, or to help shape the policy decisions that flow from them.

In contrast, however, and in keeping with developments in participatory methods designed precisely to involve stakeholders in decision making (see, for example, Costanza and Ruth 1998, Mayer and Geurts 1998, Van Asselt and Rijkens-Klomp 2002, Kasemir et al. 2003), the FoPIA uses the analysis of sustainability criteria precisely in order to engage with and inform the decision making that proceeds from the SIA. Building the analysis of criteria into the core functionality of the FoPIA stems from a recognition of the need to explore the values and preferences at play in processes of problem definition and policy decision making.

Two principal stakeholder research phases constitute the FoPIA: Defining policy scenarios (Phase 1), and conducting a SIA of each scenario (Phase 2). Phase 1 involves the use of semi-structured interviews with individual stakeholders, and in Phase 2 these stakeholders are brought together to participate in a SIA workshop.

Phase 1 starts with an examination of the national interpretations and implementations of the European policy in question as structured by perceptions of key sustainability issues and problems. This involves the use of semi-structured interviews with policy makers operating at the national level and responding to EU policy directives and targets. These are typically representatives of competent government departments and members of the working groups and advisory panels set up to assist with the policy design process. A second round of interviews with regional stakeholders focuses on the land use changes that are driven by the policy change. These stakeholders are representatives of the regional offices of government departments, those involved in spatial planning and decision making, local government employees, representatives of relevant land use sectors, landowner interest groups and associations, and landowners. Interviewees are selected on the basis of snowball sampling and are recruited by email or telephone. Each interview follows a topic guide, with questions relating to national and regional sustainability issues, policy design and implementation, and land use change. The interviews are recorded, transcribed and analyzed using a content analysis approach involving the identification of key themes. This analysis provides the basis for drafting provisional policy scenarios that are then taken forward to the SIA workshop (Phase 2).

Phase 2 involves the convening of a SIA workshop, bringing together all the stakeholders from Phase 1. During the workshop the stakeholders carry out an analysis of sustainability criteria and an assessment of the changes in the corresponding social, environmental and economic indicators that would result from implementing the proposed policy scenarios. New indicator values are then compared with sustainability limits set by the stakeholders. Finally, criteria are reassessed to elicit stakeholder preferences for the scenarios. Workshops are held at a convenient location within the case study region, usually in a hotel that provides adequate meeting facilities. Workshops typically last a whole day making it impractical to record and transcribe the discussions. However, detailed notes are kept by an appointed member of the research team and are later analyzed using a content analysis approach.

The FoPIA’s logical flow, from problem definition through policy scenario development, the analysis of criteria, impact assessment using indicators, sustainability impact assessment comparing impacts with limits, to elicitation of preferences, and the central role of stakeholders in driving the analysis in each of the sequenced steps is illustrated in Figure 1. A detailed description of the analytical process is provided.

Phase 1: Define policy scenarios

European member states are free to negotiate their own respective commitments and obligations to each of the indicative targets or standards set by the EC during the centralized policy drafting process. Furthermore, each member state is free to implement policies of its own design to deliver on its obligations. Changes in land use at national and regional levels and the consequent social, economic and environmental impacts of a European policy, therefore, will depend on this decentralized process of policy design. Within SENSOR this problem is overcome, as is customarily done in IA, by drafting policy scenarios and making the assumption that these will be adopted and implemented uniformly across all member states. In order to ensure sensitivity to national, regional and local sustainability priorities, and thereby enhancing the accuracy of the ensuing SIA process, the FoPIA sets out to identify where divergences from this centrally designed policy occur and to produce nationally and regionally relevant policy scenarios.

Semi-structured interviews are conducted with national and regional-level policy makers and experts over a one week period of intensive fieldwork. During this time national policy documentation is also reviewed to analyze policy objectives and proposed instruments. This is followed by a second week of fieldwork involving interviews with stakeholders from the case study area to analyze the likely land use changes resulting from the scenarios.

Phase 2: Define and analyze criteria

At the start of the SIA workshop the research team summarizes the findings from the Phase 1 interviews and presents provisional policy scenarios. These are then discussed by the stakeholders and, upon agreement, can be adjusted to reflect any recent developments in policy formulation. The next task is to define and assess the relative importance of a number of key sustainability criteria related to regional land use. This process involves a moderated discussion, followed by a simple scoring exercise, followed by further discussion to agree criteria scores.

To initiate discussion on sustainability criteria, nine “Land Use Functions”[1] (LUF) are presented by the workshop moderator (see Table 1). The LUFs facilitate the identification of those key social, economic, and environmental functions of the land that may be damaged or enhanced under a given policy scenario. The stakeholder group is then invited to discuss and refine them so that each most adequately reflects a corresponding sustainability issue. Changes to the definition of each LUF are recorded, yielding an agreed upon list of nine “Land Use Function Criteria” (LUFC).

The group then carries out a simple scoring exercise in which individual stakeholders give a score to each LUFC in accordance with their views about the importance of each criterion for the sustainability of the region. The range of scores is from 1 to 10, where 10 denotes very high importance and 1 denotes low importance. Scores are not exclusive, and can be attributed to more than one LUFC. Participants work independently and attach their scores to each of the posters using a post-it note. Average scores are then calculated by the moderator and written on each poster.

There follows a moderated discussion in which, upon agreement, the participants are able to adjust the average scores. Here the moderator uses variations in individual scores to prompt discussion about possible reasons and explanations. In this way, the group is encouraged to move towards a shared understanding of the relative importance of each of the LUFC in the context of the case study region. Each stakeholder can defend and/or reconsider his or her initial preferences, also taking into account the additional information and views provided by other participants, thereby enabling a process of social learning (Henkens et al. 2007). There follows an interactive and informative discussion on views, values, and goals for sustainability and about the importance given to different environmental, social, and economic factors. Average scores are only adjusted if agreement is reached among all participants. Otherwise, differences of opinion and justifications are recorded, but the LUFC keeps its original score. If changes are agreed upon, a new average score is written on the relevant poster. Upon conclusion of the discussion, a preliminary ranking order is presented back to the stakeholder group.

Phase 2: Select indicators

Following on from the analysis of regionally-specific sustainability criteria, the next task is to define and agree on a framework of sustainability indicators, which are then used to perform an impact assessment on each of the policy scenarios. These “Land Use Function Criteria Indicators” (LUFCI) are derived from the criteria (LUFC) generated in the previous session, with the aim of linking the analysis of impacts to the sustainability issues identified by the stakeholder group. Through this linked analysis of criteria and indicators, stakeholders not only forecast the impacts of each scenario, but also provide a basis for interpreting the results by displaying the relative importance, in sustainability terms, of each impact.

To initiate discussion, the moderator proposes a list of nine LUFCI, drawn from the indicator framework developed within the SENSOR project. The LUFCI are presented on posters and expressed as an impact assessment question, thereby clarifying their role in the subsequent exercise. To use an example from Malta, for the LUF “Provision of Work”, with corresponding LUFC “Employment generation”, the moderator proposed the LUFCI “Employment rate” expressed as the following IA question: “How would the employment rate change under Scenario x?”. There follows a moderated discussion during which the participants discuss and refine the provisional list of LUFCI and produce a set of IA questions representing the most adequate measures of change related to the agreed criteria. Suggested changes to indicators and questions are recorded on the posters by the moderator.

Phase 2: Impact assessment

The agreed list of LUFCI is then used to perform an impact assessment on each of the policy scenarios. Working individually, the participants are asked to score each LUFCI for each scenario. Scores are in the range -3 to +3, where -3 signifies a strong negative impact, and +3 signifies a strong positive impact. Stakeholders are asked to base their predictions of impacts on a timescale of 25 years. On completion of the individual scoring, average impact scores for each scenario on each LUFCI are calculated and recorded. As with the criteria scoring, the group is then asked to discuss the average scores and can make adjustments, on agreement. There follows a structured discussion aimed at exploring the main reasons behind differences of opinion expressed by individuals. Again, the discussion facilitates a process of social learning in which participants draw upon one another’s knowledge and experience, and question collectively their individual approaches to each impact assessment question. Upon agreement, adjusted average impact assessment scores are recorded.

Phase 2: Analyze sustainability limits

The next task is to assess the acceptability of the impacts of each of the policy scenarios. To that end, minimum standards, or sustainability “limits” are defined for each LUFCI through a process of individual scoring, followed by group discussion informing changes to average limit scores.

To introduce the concept of sustainability limits, the moderator presents available data relating to the current status of each LUF within the case study region, together with an indicative sustainability limit, using the same scale as the IA scoring exercise from -3 to +3. The concept of limits is explained as referring to the minimum standards required for the sustained functioning of the LUF. In other words, a limit score of -1 signifies that a slight loss in functionality is sustainable, while a limit score of +3 signifies that only a significant gain in functionality would be sustainable.

The participants are then invited to consider each LUFCI individually, and to decide whether their current status is sustainable or unsustainable. Individuals are then asked to give a score (-3 to +3) which denotes the minimum sustainability standard for that LUFCI. Scores are recorded on each LUFCI poster and averages are calculated. As with previous sessions, the group is then asked to discuss and adjust the average scores.

Phase 2: Stakeholder preferences

The final task of the workshop is to re-analyze the sustainability criteria (LUFC) in light of the impact assessment in order to elicit stakeholder preferences for the policy scenarios. The need to revisit the criteria is highlighted by the trade-offs implied by the positive and negative impacts of some or all of the scenarios. These trade-offs highlight the difficult political decisions to be made in light of the SIA, decisions that can be helped by stakeholders communicating the relative importance of the criteria. To that end the LUFC are rescored, but this time individual participants can only use each score once. In other words, the participants are required to sort the criteria into a ranking order of importance for sustainability.

To introduce the session, the moderator gives a brief summary of the trade-offs implied by the impact assessment scores generated earlier. Participants are then asked to rescore each LUFC from 1 to 9, whereby a score of 9 means most important. Each score can only be given once. As before, averages for each LUFC are calculated, discussed and adjusted. Upon conclusion of the discussion, a final ranking order for the LUFC is presented.

Phase 2: FoPIA results and stakeholder feedback

The final session of the workshop involves a discussion of the process and results of the SIA. This provides the opportunity for participants to reflect, not only on the outputs of the analyses they have performed, but to provide feedback on the various methods, materials and research inputs to the workshop.

The session is initiated by a summary presentation of the workshop results. The precise nature of this presentation will depend on the outputs, but the moderator will typically focus on key elements, such as the impacts on sustainability of each scenario with an emphasis on the trade-offs between gains and losses in functionality, how each scenario performs in relation to important criteria, and comparisons between impacts and sustainability limits. The group is then encouraged to discuss these results and individuals are given the opportunity to highlight aspects of the analysis with which they agree or disagree, and areas in which they need clarification either from the moderator or from other participants.

Finally, the group is given the opportunity to reflect on the analytical process. Here they are encouraged to comment on the various elements within the framework. The moderator may prompt discussion using direct questions related to the scoring exercises, the group discussions, and analytical devices such as the LUF. The group is finally asked to reflect on the FoPIA as a mechanism for involving national, regional and local stakeholders in the process of European policy making, whether they think the process is worthwhile, what they see as the main analytical and political gains, and whether they enjoyed the process.


A major workstream within the SENSOR project involved the implementation and testing of the FoPIA in a number of Sensitive Area Case Studies (SACS). The FoPIA was used to carry out a SIA on two key policy areas, namely biodiversity in Western Estonia and Malta, and bioenergy in Silesia (Poland), Lusatia (Germany) and the High Tatras (Slovakia). The results of the assessment of biodiversity policy in Malta are presented. It should be noted that verbatim quotes from interviews and workshop discussions are not provided, in accordance with the requests made by the research respondents.

Phase 1: Define policy scenarios for Malta

The eliciting of sustainability issues and indicators for Malta was informed by semi-structured interviews with 32 individuals from a range of public institutions, political parties, trade unions, universities, businesses, and business associations. Interview scripts were used to elicit stakeholders’ opinions about the key sustainability issues facing Malta. The most pressing sustainability issues facing Malta were land development, the loss of natural resources, economic competitiveness, and the lack of strong governance.

Follow-up interviews were conducted with biodiversity experts within the Malta Environment and Planning Authority and key stakeholders from the Ministry of Rural Affairs and Environment to discuss potential developments in Maltese biodiversity policy and resulting changes in land use. Reflecting the very restricted nature of land supply in Malta, many interviewees felt that policies to create new protected areas would be impracticable. Rather, existing designations should be brought under tighter management. This should be combined with tighter planning controls over development and more effective incentives for farmers to adopt less intensive agricultural practices. In addition, integrated sectoral approaches to promote adaptation and mitigate the damaging impacts of climate change were seen as critical for the conservation of Maltese habitats.

Upon completion of Phase 1, a workshop was convened, involving stakeholders representing the following organizations: Ministry for Rural Affairs and Environment, Malta Environment and Planning Authority, Farmers’ Association, farmers, the Rambler’s Association, and the Rural Development Department. After careful discussion during the first session of the workshop, the workshop participants agreed on the biodiversity policy scenarios described in Table 2. Of note is the fact that stakeholders conceive biodiversity policy in Malta as requiring cross-sectoral compliance, affecting the nature conservation, agricultural and construction sectors. Stakeholders felt that recognition of this was essential for the accuracy of the subsequent SIA process.

Phase 2: Define and analyze criteria for Malta

The next step was to agree on sustainability criteria with the stakeholders. Having discussed the LUF and a proposed set of LUFC presented by the moderators, workshop participants agreed on the LUFC listed in Table 3 (2nd column). In general, the definition of LUFC remain closely related to the “parent” LUFs, and the group felt the LUFs to be a useful representation of the key land use-related sustainability issues in Malta. Some participants did feel, however, that the sink function of land use, that is, for the dumping of waste, in Malta was missing, but it was agreed that this could be integrated within the environmental quality criterion. All the participants agreed that water status should be used to specify the LUF provision of “abiotic resources”, reflecting the critical status of fresh water quantity and quality in Malta.

The LUFC were then scored, and average scores were calculated, discussed and adjusted, yielding the results displayed in Table 3 (3rd column). The moderators noted the general uniformity in perceived importance across all LUFC, with slightly higher importance attached to environmental than to economic and social criteria. The group reflected that Malta currently faces serious environmental problems, and that policy performance should be judged primarily against these important criteria. However, one participant noted that the makeup of the group, with disproportionate representation of environmental organizations, may have produced a bias towards environmental criteria. Others contested this, stating that they had made a conscious effort to base their scoring on a careful, detached consideration of the relative importance of each criterion.

Phase 2: Select indicators for Malta

The next step was to associate numeric indicators with the sustainability criteria (LUFC), to be used in a systematic indicators-based impact assessment. Having discussed a provisional set of indicators presented by the moderators, the participants agreed on the list of LUFCI shown in Table 4, expressed as impact assessment questions, with corresponding LUFC. Of note is the way in which the definition of indicators enabled stakeholders to specify the best measurement of change related to a critical sustainability issue in Malta. For example, access to the countryside was seen as having a critical bearing on the issue of public health, whereas groundwater status was felt to encompass important measurements of both quality and quantity of water within Malta’s limestone aquifers.

Phase 2: Impact assessment for Malta

The participants then used the LUFCI to perform an impact assessment on each of the biodiversity policy scenarios. There followed a moderated discussion of individual scores, yielding the agreed averages presented in Table 5. As expected, stakeholders predicted largely negative impacts under “Low protection” and strong positive impacts under “High protection”. Under both scenarios the amount of land made available for development emerged as the key factor having negative and positive impacts.

Discussions of scores revealed some interesting insights into differences between individuals’ approaches to the task. For example, one participant gave a zero score (no change) for the indicator “How would the number of designated and managed heritage sites change in Scenario x?” under all three scenarios, explaining that the number of heritage sites would be unlikely to change irrespective of the strength of conservation policy. Another participant, who gave the indicator a score of -1 under the low protection scenario, explained that some sites may be degraded or lost due to extreme weather events if nothing was done to protect them. After discussion, it was agreed that a range of potentially influential factors could usefully be considered when assessing impacts and that this would yield more accurate predictions than a narrow focus on the impacts of policy change in isolation from all other factors.

Phase 2: Analyze sustainability limits for Malta

The research team introduced this session by presenting available data related to the current status of each LUF in Malta and, for illustration purposes, an indicative sustainability limit for each LUFCI. There followed a process of individual scoring and the discussion of average scores, yielding the agreed limit scores presented in Table 6. Stakeholders agreed that “land available for housing and employment” and “transport infrastructure” were the only LUFCI for which a loss in functionality could be deemed sustainable. Across all the other indicators, and particularly for the environmental indicators, significant or very significant improvements in functionality were deemed necessary for sustainability.

Again, the discussion of individual scores was productive and interesting. For example, one participant gave a zero limit score to the LUFCI “How would the land available for housing and employment change in Scenario x?”, explaining that although he had wanted to give a negative limit score to express his view that some developed land should be reclaimed, he saw this as pointless given current legal obstacles to such an initiative. Another participant made reference to the situation in Spain in which hotels had been bought and demolished as part of a government-run coastal rehabilitation program. Upon discussion, it was agreed that such an example could be followed by the Maltese government and an adjusted limit score of -2.3 was agreed upon, that is, respondents arrived at the view that Malta could lose some developed land in order to regain some more “open” land.

Phase 2: Stakeholder preferences for Malta

The LUFC ranking exercise was used to elicit stakeholder preferences by revealing the performance of each scenario in relation to priority criteria for sustainable land use in Malta. A ranking order for the LUFC was obtained after a second round of scoring and is displayed in Table 7. The results show the high importance attached to environmental criteria, revealing the group’s preference for Scenario 3 (high protection) which was predicted to perform positively against environmental criteria. These results could be interpreted by an end user as justification for the trade-off of losses of land available for housing and workplaces implied by the same scenario.

Discussion of the results of the first round of scoring revealed a division between participants approaching the exercise from eco-centric and anthropocentric perspectives. For example, some argued that physical and mental well-being should be regarded as the most important criterion which they saw as a precondition for improved environmental responsibility leading to better stewardship of the natural environment. In contrast, others argued that well-being itself was dependent on good environmental quality. Both sub-groups agreed that the scoring exercise was constrained by the fact that, in respondents’ minds, the criteria are not mutually exclusive or tradable, since people want both environmental quality and prosperity, and they are also inter-dependent. As such, it was difficult to avoid scoring strategically by prioritizing those criteria upon which other criteria were seen to rely, as in the example of human well-being and environmental quality. Strategic scoring had resulted in the three lowest ranking scores being given to the economic criteria, not because participants felt they were unimportant, but because they felt that attention given to environmental and social criteria would provide the necessary conditions for economic prosperity.

It was decided that a second round of voting was necessary, with the result that higher ranking positions were given to two of the economic criteria. These discussions and the markedly different outcome in the second round of scoring illustrate how discussion leading to shared understanding of the problem and the question being addressed can significantly affect the outcome. This demonstrates the importance of deliberation in public participation exercises, and the need to balance quantitative with qualitative forms of analysis (Jacobs 1997).

Phase 2: FoPIA results and stakeholder feedback for Malta

The final session started with the moderators providing a summary presentation of the workshop results, as shown in Figure 2, which displays the impact score for each LUFCI under each scenario. Impact scores can also be compared with the sustainability limits set for each LUFCI, denoted by the trend line. Also displayed are the results of the second criteria ranking exercise, denoted by the bubble size and position on the x axis (the most important criterion is the largest), reflecting participants’ prioritization of the criteria.

The results show that High protection achieves positive impacts in relation to highly prioritized criteria and performs best against the minimum sustainability standards set for each indicator. However, even High protection was predicted to have impacts that would fall short of minimum sustainability standards for the maintenance of ecosystem processes (ENV3) and cultural heritage (SOC1) functions. Medium protection is predicted to impact positively in relation to important criteria, but falls short of the minimum sustainability standards set for seven out of the nine indicators.


The Impact Assessment Guidelines state that the consultation of stakeholders as part of the IA process can significantly enhance the quality, credibility and legitimacy of the resulting policy proposals. Is this assertion borne out by the experience of implementing and testing the FoPIA?

In general, the experience of implementing the FoPIA in five case study sites has been very positive and the researchers involved are pleased to recommend the FoPIA to the EC for inclusion in the growing assemblage of impact assessment tools and procedures. These sentiments are strengthened by the largely positive feedback received from many of the stakeholders involved. However, FoPIA implementation provided the opportunity for reflection on the part of the research team and for stakeholders to provide constructive criticism of relevance to the issues of quality, credibility and legitimacy. Many of these reflections and criticisms are borne out of the inherent complexity of the analytical task of conducting sustainability impact assessments of land use change. As such, our intention is not always to offer solutions or to recommend changes in approach, but simply to report the issues as they arose and to raise awareness of them. Where appropriate, helpful pointers and suggestions are offered.

(1) Does the FoPIA enhance the quality and credibility of SIA?

The IA Guidelines emphasize the need for high levels of analytical accuracy to enhance the viability of the policy recommendations made on the basis of an IA:

Good quality data – facts as well as figures – are an essential part of any IA. You need them to define the problem and the baseline scenario, and to identify the impacts of alternative options for dealing with the problem. Particular attention needs to be paid to quality and credibility of data (CEC 2009:18)

When analyzing the complex human–nature systems involved in policy-driven land use change, there is a need for highly accurate and practicable forms of decision support. Some studies have shown, for example, how policies are often applied and, crucially, interpreted differently by practitioners operating in different contexts, sometimes with unforeseen and surprising consequences (Waterton et al. 2006). While the FoPIA does not make any false claims about its ability to represent exhaustively the inherent complexity of human–nature systems, it is, nonetheless, founded on a post-positivist ontology and epistemology and borne out of an acceptance of the need to accommodate the inherent diversity and complexity within the systems under analysis. The user of the FoPIA, therefore, is necessarily interested in exploring the detail of local and regional sustainability contexts through a “communicative-rational” approach to SIA, involving the direct participation of stakeholders in the discursive analysis of policy impacts (Majone 1989, Dryzek 1990, Fischer 1990, 1993, deLeon 1997, Fischer 1998, Morris et al. 2008). In particular, the FoPIA addresses the issues of complexity by facilitating integrated assessments. Here the term integration refers to a broad conceptual framework encompassing several meanings, as follows:

(a) Integration of economic sectors: The FoPIA is designed to assess policies that affect six land use intensive economic sectors, namely, agriculture, forestry, tourism, energy, transport infrastructure, and nature conservation. This sectoral integration is orchestrated both through stakeholders’ discursive analysis of competing claims for land made by the sectors in response to the policy under examination, and through their predictions of impacts that result from these changes in land use across a range of sustainability indicators.

(b) Integration across the three pillars of sustainable development: The FoPIA facilitates impact assessments that reflect changes in economic, environmental, and social systems, adopting what has been called the Triple Bottom Line approach (Eggenberger and Partidário 2000, Sheate et al. 2003, Twigger-Ross 2003). FoPIA design recognizes that economy, society, and environment are not independent from one another, however, but must be treated as inter-related functions of a complex system.

...dividing the holistic concept of sustainability into three pillars ...runs the risk of the sum of the parts being less than the whole. This is particularly true if the interrelations between the three pillars are not adequately understood and described... (Gibson 2001:12).

Accordingly, the FoPIA encourages stakeholders to consider the relationships between environmental, economic, and social issues, in order to assess policy impacts accurately and to inform decisions about unavoidable trade-offs, compromises and possible win-win situations.

(c) Integration of the multi-functional nature of land use: The FoPIA provides a mechanism for SIA that captures this notion of multifunctionality, through the selection of indicators that represent a range of social, economic, and environmental Land Use Functions relevant to a particular region (Perez-Soba et al. 2008, Tabbush et al. 2008).

The FoPIA allows detailed knowledge of context to be applied to assessments of policy impacts and, crucially, the relationships between impacts, thereby integrating sustainability pillars, multiple land use functions, and sectors. For example, to the surprise of the research team, Maltese stakeholders predicted that the value of agricultural outputs would increase under the high biodiversity protection scenario and that they would fall under the low protection scenario, which is a prediction that, at face value, seemed counterintuitive. Participants explained that, under the high protection scenario, Maltese farmers would be more likely to diversify their production base, and to explore and exploit specialist niche markets within the agro-industry sector, thereby increasing the financial value of their products. Specialist products from agriculture, such as woolen clothing, might be produced for the growing tourist market, a related impact of stronger biodiversity protection measures. This provides an interesting illustration of how knowledge of local culture, micro-economic factors, and the likely societal responses to changing conditions can be brought to bear in integrated impact assessment.

The FoPIA involves quantitative forms of analysis, with a number of scoring exercises used for the analysis of criteria, impacts and limits. A number of participants raised concerns about the use of quantification. Some found it difficult to make the conceptual link between reality and a set of essentially abstract numbers. Others felt uncomfortable with the idea of quantifying opinions and values. Some were concerned about the simple application of the numerical outputs of the analysis within decision making contexts without regard for the inherent complexity and uncertainty surrounding the judgments that informed them. All of these reservations raise important questions about the quality and credibility of FoPIA outputs. During discussions of these issues, the research team was at pains to allay fears about the use of quantification, drawing attention to the social learning approach, whereby individual scoring exercises were followed by group discussions that enabled participants to explore differences in opinion and understanding and to adjust average scores based on the outcomes of their discursive analysis. Furthermore, it was stressed that quantitative outputs need not be reported in isolation, but could be accompanied by detailed accounts of participants’ discussions, particularly when there was a high level of disagreement between participants, or when the group expressed concern about the accuracy of their predictions. It is worth highlighting, however, that despite these explanations some stakeholders remained critical of the quantitative forms of analysis.

Another issue affecting the quality and credibility of FoPIA outputs is the selection of stakeholders to take part in the workshops (Phases 3 and 4). During the discussion of FoPIA results in Malta, one participant observed that had the stakeholder group been composed of people representing other interest groups and sectors including, for example, property developers and bird hunters, the process might have produced quite different results. Indeed, these issues were discussed among SENSOR project partners involved with the organization of the workshop. It was clear during these discussions that the ideal of equal representation of all potential interest groups would be almost impossible to achieve. Firstly, in practical terms it was proving difficult to find a date when all the invited stakeholders could attend. Secondly, given the extreme political and social sensitivities surrounding the issue of biodiversity conservation in Malta and the strongly conflicting interests of land development, conservation, agriculture, and hunting, it was felt that equal representation could be achieved in one group, but only at the expense of reasoned debate and discussion. As such, the makeup of the workshop participants did, perhaps, introduce a bias in favor of environmental criteria. This is reported here to raise awareness of the potential complexities and analytical consequences of stakeholder analysis and selection. While simple solutions do not immediately present themselves, the research team did reflect on the importance of reporting stakeholder selection candidly so that end users can interpret FoPIA outputs with a full understanding of any potential bias.

(2) Does the FoPIA enhance the legitimacy of SIA?

The IA Guidelines state that consultation is key to bringing legitimacy to the policy recommendations that are informed by an IA process:

Consultation helps to ensure that policies are effective and efficient, and it increases the legitimacy of EU action from the point of view of stakeholders and citizens (CEC 2009:19).

The issue of legitimacy has driven the development of a vast array of participatory methods to involve stakeholders in decision making (see, for example, Costanza and Ruth 1998, Mayer and Geurts 1998, Van Asselt and Rijkens-Klomp 2002, Kasemir et al. 2003). These methods have emerged in concert with a general questioning of the relationship between science and policy making and between science and society. Other commentators have called for a new contract between science and society, involving a shift in emphasis away from the production of reliable knowledge to the production of a socially robust knowledge that is sensitive to real-life contexts in which a particular problem field is experienced (Stirling 1997, Durning 1999, Gibbons 1999). For these commentators, ideas of legitimacy have as much to do with the involvement of stakeholders as holders of opinions about appropriate courses of action in the light of their experiences of particular issues and problems as they do about stakeholders as knowledge bearers.

Owens et al. (2004) note that concepts of sustainability and by association, therefore, the practice of SIA, necessarily follow some predetermined value system or normative presuppositions. Of particular relevance here is the fact that policy-related SIA is oriented towards decision making processes that are informed by the principle of intergenerational equity and guided by the need to balance or, sometimes unavoidably, to trade-off between the three pillars of sustainability. The inherently normative nature of SIA begs the question: whose criteria should be taken into account? Depending on their position within the heirarchy of European policy making, or on their own beliefs about the appropriate spatial and temporal distribution of positive and negative policy impacts, different stakeholders will draw quite different conclusions about what constitutes an appropriate course of action in the light of a SIA. This subjectivity is particularly apparent when one considers that policy making for sustainable development necessarily starts from the perception of a particular sustainability issue or problem and that perceptions are likely to differ between the political centre and the political periphery. Given that decisions will be made in Brussels based on the forecasting of impacts in regions across Europe, there is a strong case for involving regional stakeholders in the assessment of criteria that can guide, inform and, therefore, legitimate difficult and delicate policy choices. Stirling captures this point succinctly, arguing that the facilitation of public deliberation on issues of policy and decision making can provide “the essential empirical inputs concerning the selection, definition and prioritization of the appraisal criteria” (1997:127).

Many stakeholders provided a positive endorsement for the application of sustainability criteria analysis within the SIA process, which was seen as a potentially powerful means of raising awareness of key regional sustainability issues within centralized processes of European policy making. Furthermore, stakeholders felt that the analysis of limits provided a valuable line of influence over the interpretation of assessment results and the political decisions flowing from them. In the case of Malta, for example, the analysis provided a means by which stakeholders could communicate that Low and Medium protection scenarios would be largely unacceptable in sustainability terms.

Another important outcome from the Maltese case relates to the comparison of sustainability issues in the EU Impact Assessment Guidelines (CEC 2009) and those discussed with Maltese stakeholders during Phase 1 (Moncada et al. 2010). The Guidelines do not include important impact issues such as peripherality and insularity, which are crucial for the vast majority of European islands. Their inclusion would allow a greater representation of the issues involved in European land use changes, thereby enhancing the legitimacy of this important impact assessment reference document.


The rationale, design, and implementation of a Framework for Participatory Impact Assessment (FoPIA) have been discussed. A strong case for the development of such a participatory approach is offered, focusing on the issues of quality, credibility, and legitimacy, which are driving the development of participatory approaches to Impact Assessment (IA). We have paid considerable attention to the careful description of the sequenced methods within the FoPIA, and have deliberately illustrated ways in which this approach can not only enhance the accuracy of the IA process, but also provide a mechanism by which policy decision making at the political centre can be made responsive to sustainability issues and problems as they are experienced at the political periphery.

The presentation of the FoPIA shows how a deliberative research approach involving stakeholder discussions and instances of social learning can coexist with quantitative forms of analysis, offering the potential for integration across methodological and epistemological boundaries. Facilitating a combination of stakeholder-based discussions and quantified impact assessments of policy scenarios, the development of the FoPIA thus far has set a useful precedent for moving towards methodological, epistemological, and disciplinary integration by balancing the conflicting demands placed on it by two broad schools of analysis, one requiring the purposeful complication of the analysis through discussion, the other imposing a regime of simplification through forms of quantification. It is envisaged that the FoPIA can significantly enhance the deliberative scope of European policy making.


Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link.


The authors acknowledge the important contributions to this work made by a large number of individuals and organizations. In particular, they thank the many people who gave up valuable time to take part in interviews and workshops as part of the FoPIA process. This publication is, in no small measure, a result of their enthusiasm and generosity.


CEC (Commission Of The European Community). 2002. Towards a reinforced culture of consultation and dialogue - general principles and minimum standards for consultation of interested parties by the Commission. Communication from the Commission. Commission of the European Community, Brussels, Belgium.

CEC (Commission Of The European Community). 2009. Impact assessment guidelines. Secretariat of the Commission (SEC), Brussels, Belgium.

Costanza, R., and M. Ruth. 1998. Using dynamic modelling to scope environmental problems and build consensus. Environmental Management 22(2):183-195.

deLeon, P. 1997. Democracy and the policy sciences. State University of New York Press, Albany, New York, USA.

Dryzek, J. S. 1990. Discursive democracy. Cambridge University Press, Cambridge, Massachussetts, UK.

Durning, D. 1999. The transition from traditional to postpositivist policy analysis: a role for Q methodology. Journal of Policy Analysis and Management 18:389-410.

EEA (European Environment Agency). 1999. Environmental indicators: typology and overview. Technical report No 25. European Environment Agency, Copenhagen, Sweden.

Eggenberger, M., and M. Partidario. 2000. Development of a framework to assist the integration of environmental social and economic issues in spatial planning. Impact Assessment Project Appraisal 18(1):201– 7

Fischer, F. 1990. Technocracy and the politics of expertise. Sage, Newbury Park, California, USA.

Fischer, F. 1993. Citizen participation and the democratization of policy expertise. Policy Sciences 26:165-187.

Fischer, F. 1998. Beyond empiricism: Policy inquiry from a postpositivist perspective. Policy Studies Journal 26:129–146.

Gibbons, M. 1999. Science’s new social contract with society. Nature 402(2):C81-C84.

Gibson, R. 2001. Specification of sustainability-based environmental assessment decision criteria and implications for determining ‘‘significance’’ in environmental assessment. Online [URL]:,EA.doc

Henkens, R., V. Tassone, S. Grafakos, R. De Groot, and J. Luttik. 2007. A participatory decision support tool to assess costs and benefits of tourism development scenario. Application of the adaptive model to Greater Giyani, South Africa. Alterra-report 1583. Wageningen University, The Netherlands.

Jacobs, M. 1997. Environmental valuation, deliberative democracy and public decision-making institutions. in J. Foster, editor. Valuing nature? Economics, ethics and the environment. Routledge, London, UK.

Kasemir, B., J. Jäger, C. Jaeger, and M. Gardner. 2003. Public participation in sustainability science: a handbook. Cambridge University Press, Cambridge, UK.

Kuhlman, T. 2008. Scenarios: driving forces and policies. Pages 35-54 in K. Helming, M. Perez-Soba, and P. Tabbush, editors. Sustainability impact assessment of land use changes. Springer, Berlin, Germany.

Ling, C., J. Handley, and J. Rodwell. 2007. Restructuring the post-industrial landscape: a multifunctional approach. Landscape Research 32:285-307.

Majone, G. 1989. Evidence, argument and persuasion in the policy process. Yale University Press, New Haven, Connecticut, USA.

Mayer, I., and J. Geurts, 1998. De instrumentele mogelijkheden van de argumentatieve beleidsanalyse: participatieve methoden. Pages 187-204 in A. Hoppe, and A. Peterse, editors. Bouwstenen voor argumentatieve beleidsanalyse. Elsevier, The Hague, The Netherlands.

Moncada, S., M. Camilleri, S. Formosa, and R. Galea. 2010. From incremental to comprehensive: towards island-friendly European Union policymaking. Island Studies Journal 5(1):61-88.

Morris, J., M. Camilleri, and S. Moncada. 2008. Key sustainability issues in European sensitive areas – a participatory approach. Pages 451-470 in K. Helming, M. Perez-Soba, and P. Tabbush, editors. Sustainability impact assessment of land use changes. Springer, Berlin, Germany.

Owens, S., T. Rayner, and O. Bina. 2004. New agendas for appraisal: reflections on theory, practice and research. Environment and Planning A 36(11):1943-1959.

Perez-Soba, M., S. Petit, L. Jones, N. Bertrand, V. Briquel, L. Omodei-Zorini, C. Contini, K. Helming, J. Farrington, T. Mossello, D. Wascher, F. Kienast, and R. De Groot. 2008. Land use functions – a new conceptual approach to assess the impact of land use changes on land use sustainability through multifunctionality. Pages 375-404 in K. Helming, M. Perez-Soba, and P. Tabbush, editors. Sustainability impact assessment of land use changes. Springer, Berlin, Germany.

Sheate, W., S. Dagg, J. Richardson, R. Aschemann, J. Palerm, and U. Steen. 2003. Integrating the environment into strategic decision-making: conceptualizing policy SEA. European Environment 13(1):1–18.

Sieber, S., K. Muller, P. Verveij, H. Haraldsson, K. Fricke, C. Pacini, K. Tscherning, K. Helming, and T. Jansson. 2008. Transfer into decision support: the Sustainability Impact Assessment Tool (SIAT). Pages 107-128 in K. Helming, M. Perez-Soba, and P. Tabbush, editors. Sustainability impact assessment of land use changes. Springer, Berlin, Germany.

Stirling, A. 1997. Multi-criteria mapping: mitigating problems of environmental valuation? Pages 186-210 in J. Foster, editor. Valuing nature? Routledge, London, UK.

Tabbush, P., P. Frederiksen, and D. Edwards. 2008. Impact assessment in the European Commission in relation to multifunctional land use. Pages 35-54 in K. Helming, M. Perez-Soba, and P. Tabbush, editors. Sustainability impact assessment of land use changes. Springer, Berlin, Germany.

Tscherning, K., H. König, B. Scöβer, K. Helming, and S. Sieber. 2008. Ex-ante impact assessments (IA) in the European Commission – an overview. Pages 17-33 in K. Helming, M. Perez-Soba, and P. Tabbush, editors. Sustainability impact assessment of land use changes. Springer, Berlin, Germany.

Twigger-Ross, C. 2003. Emerging approaches to integrated assessment. Paper presented at the 23rd Annual Meeting of the International Association for Impact Assessment. Marrakech, Morocco. June 2003.

Van Asselt, M., and N. Rijkens-Klomp. 2002. A look in the mirror: reflection on participation in integrated assessment from a methodological perspective. Global Environmental Change 12:167-184.

Waterton, C., J. Morris, and L. Norton. 2006. Understanding Loweswater: interdisciplinary research in practice. Journal of Agricultural Economics 57(2):277–293.

[1] Conventionally, land use has been understood to describe the allocation of land to different land cover types. It is clear, however, that land provides for a multitude of human needs in complex ways, ranging from the intellectual, cultural and aesthetic, to the purely resource-based and economic. This realization has given rise to the concept of multifunctional landscapes (Ling et al. 2007). SENSOR sets out an approach to SIA which captures this notion of multifunctionality, through the selection of indicators that represent a framework of social, economic, and environmental Land Use Functions (LUF). The LUF are the private and public goods and services provided by the different land uses, and together they summarize the most relevant social, economic, and environmental dimensions of sustainable land use in a region (Perez-Soba et al. 2008).

Address of Correspondent:
Jake Breton Morris
Forest Research
Alice Holt Lodge
Farnham, Surrey
United Kingdom
GU10 4L


Home | Archives | About | Login | Submissions | Subscribe | Contact | Search