Factors influencing successful collaboration for freshwater management in Aotearoa , New Zealand

Public participation in freshwater management has been widely advocated as an effective way to resolve the tensions between contested values and objectives while maintaining ecological integrity. However, questions remain regarding which processes and factors contribute to successful processes and outcomes for freshwater. Using a comparative case-study methodology, we unravel the “noise of participation” to assess the factors that influence the success of participatory decision making in collaborative processes currently underway in Hawke’s Bay and Northland, Aotearoa, New Zealand. In Hawke’s Bay, participants have been periodically surveyed to solicit their perceptions of how the process is working and the likelihood of achieving desirable outcomes. In Northland, five identical processes are currently underway, one per catchment, providing the basis for an intraregional assessment of collaboration. Our results suggest that participants’ perceptions change within the process, and that those changes may involve complex, dynamic, and reciprocal interactions within the collaborative group. Results also show the strong influence of external conditions. The choice of stakeholder participants is also critical to ensuring the viability of collaboration. Key factors include participants’ previous interactions and relationships, which may help to prime them for collaboration. These factors are dynamic and evolve through different cycles. Although both collaborative processes are still underway, these insights may help focus greater attention to process design and stakeholder selection from the outset to ensure successful outcomes. Ultimately, a successful collaborative process is one that is able to incorporate feedback and adapt to changing the dynamic and often complex external environment.


INTRODUCTION
Collaborative forms of planning and decision making are widely used in efforts to resolve the often intractable conflicts over scarce resources (Conley and Moote 2003, Sabatier et al. 2005, Cullen et al. 2010, Chaffin et al. 2012, Benson et al. 2013, Hurlbert and Gupta 2015).By bringing together those individuals or groups most affected by planning outcomes, collaborative planning and decision making seek to achieve consensus outcomes that will deliver the greatest benefits to the widest number of stakeholders, while also achieving desirable outcomes for natural resource management.It is a participatory approach that relies less on experts and more on stakeholders sharing power with authorities and having the mandate to reach consensus outcomes (Conley and Moote 2003, Sabatier et al. 2005, Innes and Booher 2010).Advocates suggest this approach is an effective means of resolving the "wicked problems" of managing resources in the face of multiple contested values and objectives while continuing to respect planetary boundaries (Milly et al. 2008, Hartmann 2012, Patterson 2016).
In 2009, the New Zealand government signalled a need to significantly change the way in which the country's freshwater resources were managed (Weber et al. 2011).The multistakeholder Land and Water Forum carried out an extensive review and public consultation, culminating in a series of recommendations.Many of these were enacted in national policy in 2011, including the National Policy Statement for Freshwater Management.Among the policy changes, regional councils (regional councils are the regulatory authority in New Zealand responsible for managing freshwater resources) were encouraged to use collaborative processes for developing and operationalizing freshwater-related regional plan changes (New Zealand Government 2011).
Collaborative processes are now underway throughout the country (Allen et al. 2011, Fenemor et al. 2011, Duncan 2014).In Canterbury, for example, the Canterbury Water Management Strategy (CWMS) is using collaborative processes to develop and implement water management plans in all 10 of the region's watermanagement zones using a relatively informal collaborative governance model based on Ostrom's (1990) approach to selfgoverning communities (CWMS 2016).Other jurisdictions have employed or are exploring the potential of collaborative planning to address aspects of land and marine management (Chapin et al. 2012).
The use of collaborative processes in several New Zealand regions provides a unique opportunity to evaluate the approach.When successful, collaboration can lead to higher-quality and enduring agreements (Innes and Booher 2010, Lubell and Lippert 2011, Rinkus et al. 2015).It can raise social capital and improve relationships between stakeholders (Mandarano 2009, Floress et al. 2011) and can enhance social learning and result in more creative solutions (Fischer et al. 2014, Ayre andNettle 2015).When collaborations fail, however, hard-earned trust between stakeholders and public authorities can be eroded, adversarial positions may be resumed, and stakeholders' confidence in the use of alternative planning forums may be undermined.Determining which factors contribute to achieving successful outcomes is therefore essential (Midgley et al. 2013, Hurlbert andGupta 2015), yet such determination requires continued evaluation of the strengths and weaknesses of a collaborative approach and identification of best practices to mitigate the challenges associated with face-to-face negotiations (Antunes et al. 2009, Cullen et al. 2010, Chaffin et al. 2012).https://www.ecologyandsociety.org/vol22/iss2/art14/Studies evaluating collaborative processes have employed a wide range of criteria and approaches.In the early evaluative literature, the focus was on the rules of the game (Mandarano and Paulsen 2011), i.e., how participants and stakeholders in collaborative planning processes ought to behave (Rowe andFrewer 2000, Leach et al. 2002).These normative and procedural guidelines for good process design included ensuring all affected stakeholders were involved, were negotiating in good faith, and conducted transparent and open discussions (Susskind and Cruikshank 1987, Gray 1989, Gray and Wood 1991, Mattessich and Monsey 1992, Wilson et al. 1996, Moote et al. 1997, Innes and Booher 1999).
Subsequent scholarship has broadened the definition of successful collaboration to include the social benefits and outcomes that accrue through such processes, such as trust, relationships, and networks (Susskind et al. 2003, McKinney and Field 2008, Mandarano 2009, Allen et al. 2011, Fischer et al. 2014).Social learning is also an important criterion used to evaluate success.Collaborative processes, for example, enhance transformative (double loop) learning, which can help with finding enduring solutions to complex environmental issues (Ansell and Gash 2008, Leach et al. 2013, Curtin 2014, Fischer et al. 2014).
Resilience scholars have also proposed characteristics of successful collaboration (Berkes 2009, Armitage et al. 2011, Curtin 2014).The characteristics of resilient systems, i.e., selforganizing, buffered, and adaptable, can be applied to collaborations (Folke 2006).From this perspective, a high-quality collaborative process should be self-organizing and evolving, be effective at information gathering, and foster networking and relationship building among its participants (Berkes and Turner 2006).
Building on this complex-systems approach, Hassenforder et al. (2015Hassenforder et al. ( , 2016) ) developed a framework to compare participatory processes.Their framework consists of three dimensions: context, process and outcomes, and outputs and impacts (Hassenforder et al. 2015).They apply the framework to five processes in Asia and Africa.The strength of their framework is that it synthesizes previously separate bodies of literature from social ecological systems, governance and policy, monitoring and evaluation, and collaborative and participatory processes into a single evaluative framework.
Many of the factors described have featured in attempts to define and measure the success of collaboration.Evaluation frameworks synthesizing normatively desirable characteristics of quality consensus building are in broad agreement with respect to the necessary criteria (Innes and Booher 1999, Rowe and Frewer 2000, Conley and Moote 2003, Frame et al. 2004, Mandarano 2008, Innes and Booher 2010, Teitelbaum 2014, Hassenforder et al. 2015).Although there is a well-developed empirical literature evaluating the success (or failure) of collaborative processes, much of this has been ex-post (Frame et al. 2004, Cullen et al. 2010, Morton et al. 2012, Blackstock et al. 2012, Carr et al. 2012, Bohnet 2015, Rinkus et al. 2015).Evaluations after the fact provide insight into participants' views on collaboration but fail to capture the dynamic workings of how those perceptions might change over time.Participants' views on their experience of face-to-face negotiations may change following the experience, or they may shift during the negotiations themselves in response to complex interactions between external and internal factors (Thomson and Perry 2006).
Drawing on the work described, as well as our own experience working with stakeholders, we developed and empirically applied an evaluation framework to assess two collaborative processes for freshwater management in New Zealand.Using cross-case analysis and a consistent evaluation framework, the objective was to determine the factors most likely to influence the overall success of collaboration."Success" is used here to refer to a multifaceted evaluation framework capturing the procedural rules and guidelines of good collaboration, but also the extent to which collaboration is likely to lead to better outcomes for freshwater.We look not only at the individuals involved, i.e., the people around the negotiation table, but also at the process itself (the methodology, procedural rules, and steps followed in a collaborative group) to assess their influence on achieving consensus recommendations for freshwater management.
There are limitations to our approach, which should be considered when interpreting the results.First, we are evaluating processes that are still underway, limiting the extent to which we are able to comment on the success, or failure, of the collaborations to reach consensus.Second, we are soliciting stakeholders' perceptions without controlling for any pre-existing biases, such as previous experience with collaboration or knowledge of planning processes, which is a limitation of other, similar studies (Coglianese 2003, Conley and Moote 2003, Sabatier et al. 2005, Patterson 2016).Some participants, for example, had been involved in previous unsuccessful collaborations, which may have influenced their views; others may have expressed overly positive views regarding the process because of their desire for a successful outcome.Nor have we done a controlled comparison of the collaborative processes with the traditional planning process used in New Zealand, involving submissions and hearings.However, despite the low response rate for some of the surveys and the small number of cases used, the results demonstrate the complex, messy, and often clumsy process of collaboration, and provide rich insight into the dynamics of movement toward negotiated outcomes.

METHODS
The basis of the research was a cross-case comparison of collaborative processes in two regions of New Zealand's North Island: Hawke's Bay, in which a single group of stakeholders is tasked with making consensus recommendations across four hydrologically connected catchments, and Northland, in which five separate collaborative processes are running in parallel (see Figs. 1 and 2; refer also to Table 1).

Overview of case studies
TANK (Tūtaekurī, Ahuriri, Ngaruroro, and Karamū) process, Hawke's Bay Hawke's Bay is located on the east coast of the North Island (Fig. 1).It is economically dependent on primary production, including dairying, high-value horticulture and viticulture, dry stock (sheep and beef) farming, and cropping.There is a growing wine industry due to the warm climate, ideal soils, and reliable aquifer water supplies.A strong tourism sector capitalizes on many of these strengths.Freshwater management issues in the region include potential overuse of groundwater and river-gravel resources; reduced surface water (rivers, lakes, wetlands) quantity and quality; and the risk of groundwater contamination, contaminant discharges, and spillages (Hawke's Bay Regional Council 2015).In 2012, the Hawke's Bay Regional Council (HBRC) convened a collaborative stakeholder group, the TANK (Tūtaekurī, Ahuriri, Ngaruroro, and Karamū) group, to advise on future land and water management and associated policy approaches for four interconnected catchments and their associated estuarine and coastal receiving environments.The group consists of approximately 30 stakeholder representatives tasked with providing the Hawke's Bay Regional Council with consensus recommendations on freshwater objectives and policies to be included in the regional resource management plan.https://www.ecologyandsociety.org/vol22/iss2/art14/

Collaborative catchment groups, Northland
The Northland region comprises the area north of Auckland, in the North Island (Fig. 2).It has a modest regional economy and approximately 3.6% of the country's population.With a subtropical climate, the region's agricultural sector has a competitive advantage.Higher rainfall and pasture growth have seen the pastoral sector (dairy, beef, sheep, and deer) double productivity in the last 15 years.Beef farming is an important sector, producing around 20% of New Zealand's beef output.Farming and forestry occupy over half the land, and high-value horticulture and aquaculture are growth industries.The region also has a strong tourism economy.
Under the National Policy Statement for Freshwater Management, Northland Regional Council must set freshwater objectives and quantity and quality limits for all water bodies in Northland.Beginning in 2014, the council convened five collaborative catchment groups in Doubtless Bay, Māngere, Poutō, Waitangi, and Whangārei Harbor.A sixth catchment was added in November 2015, but is not part of this study.The catchments range in size and have a diverse mix of land uses, from low-lying, intensively farmed agricultural land to important recreational sites and high-value forestry.Water-quality issues affect the potential for dairy-farm conversions from other land uses, and rising stocking rates on beef units and sediment are also concerns given the large areas of erosion-prone soils in the upper reaches of some catchments and the potential for sediment and nutrient runoff.
These collaborative groups are tasked with developing catchment plans, which include freshwater values and environmental states to be achieved.They will also develop regulatory (e.g., limits, policies, and rules) and nonregulatory management options to achieve the agreed environmental objectives and provide consensus recommendations for the water and soil plan for Northland.
The groups, convened in 2013-2014, range in size from 15 to 25 members and are a mix of nominees from industry and sector groups and direct appointments from the council.The context for each of the processes was different and one of three council staff was nominated to manage and facilitate each of the processes.The procedural rules for all the groups are similar to the TANK process, in which individuals are generally mandated sector or group representatives.If members do not have a mandate from their sector or interest group, they participate as individuals and convey ideas and perspectives from their wider networks (Northland Regional Council 2015).

Evaluation method
The primary evaluation tool was an electronic survey administered to participants in the collaborative process online or via tablet computer.The TANK process evaluations also made use of feedback forms provided after meetings, stakeholder interviews, detailed observation, and reviews of meeting minutes and other process-related documentation.
The survey questionnaire was adapted from the evaluation literature and the basic framework of Frame et al. (2004), which includes 14 process design criteria and 11 outcome criteria associated with quality collaborative processes.The framework was tailored to the New Zealand context with the addition of several criteria (Table 2).In New Zealand, the government has obligations to "tangata whenua," New Zealand's indigenous Māori population, under the Treaty of Waitangi.The treaty was signed by representatives of the British Crown and Māori chiefs from around New Zealand.It established a British governor of New Zealand while recognizing Māori ownership of their lands, forests, and other properties.Criteria were added, therefore, to account for this distinctive relationship between Māori and the Crown (Memon and Kirk 2012).Morton et al. (2012) made a similar modification to gauge the extent to which a two-staged collaborative process effectively engaged with aboriginal participants in Western Canada.
New criteria on social learning and process outcomes were also incorporated into the evaluation framework (Baird et al. 2014a).Table 2 lists the criteria used to assess the design and implementation of the collaborative processes, social learning outcomes, and perceptions of outcomes for freshwater.These criteria provided the basis for the design of the survey instruments and how success is being evaluated in this analysis.
The TANK process evaluation was a longitudinal study involving four surveys of the same process.Two survey instruments were used.The survey for the first two evaluations (hereafter, TANK survey 1 and TANK survey 2) consisted of three separate sections.The first section contained 35 criteria statements with which participants were asked to indicate their level of agreement using a five-point Likert scale (strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree).The second section contained 20 criteria statements, which participants ranked in order of importance.The final section was a series of open-ended questions to assess stakeholder perceptions of the strengths and weaknesses of the process.
The survey for the subsequent evaluations (hereafter, TANK survey 3 and TANK survey 4) followed a similar format but used an 11-point Likert scale to allow a greater scoring continuum.
Participants were asked to score a series of process-and outcomerelated statements and to answer open-ended questions to capture any narrative they wanted to include, but they no longer had to rank outcome and process statements.This change to the survey was made to reduce the response burden for participants, to minimize some of the redundancy in the questions, and to allow better comparison with other processes in New Zealand, such as those in Northland.Copies of the complete survey instruments are available upon request.
The timing of the survey evaluations was an important factor in the research design.The TANK evaluations were conducted at the beginning of the process, soon after the group agreed on its terms of reference (November 2012), and then following a field trip (April 2013) aimed at building social capital and familiarizing participants with issues in the catchment.The next two evaluations were undertaken after the collaborative group had released interim reports on its progress and the status of its agreements (in December 2014 and July 2015).The Northland process evaluations used the same survey instrument as TANK surveys 3 and 4. The survey has been administered once, midway through their processes (June 2015).
The aim of the evaluations was to determine the extent to which process criteria were being met and respondents' confidence that Table 2. Process design, implementation, and outcome criteria for collaboration (Susskind and Cruikshank 1987, Crowfoot and Wondolleck 1990, Cormick et al. 1996, Wilson et al. 1996, Moote et al. 1997, Gunton et al. 1998, Innes and Booher 1999, Wondolleck and Yaffee 2000, Beierle and Cayford 2002, Susskind et al. 2003, as cited in Frame et al. 2004, Morton et al. 2012)

Dimension
Criterion Indicator Philosophy Buy-in I would prefer that freshwater resources are managed through collaborative decision making than legislative processes.

Process design Shared purpose
The process is driven by a shared purpose.

Clear terms of reference
Participants agree on the terms of reference.

Deadlines
The deadlines are clear.

Incentives
Collaboration is more effective than legislation for managing freshwater resources.

Māori engagement
Tangata whenua (Māori) viewpoints are represented in the collaborative process.

Representation
All other viewpoints are represented in the collaborative process.

Process design
Participants have contributed to the design of the process.

Adaptability
The process is able to adapt to new information and changing circumstances.

Process implementation Equity
The process provides balanced opportunity for all viewpoints to be heard.

Mutual respect
There is a high level of mutual respect in all parts of the process.

Trust
There is a high level of trust in all parts of the process.

Agency
My participation is making (or will make) a difference to the outcome of this process.

Accountability
As a participant in this process, I am accountable to an external organization/sector/group. Communication I am regularly informed about progress in the collaborative process.

High quality information
Decision making in this process incorporates the best information available.

Procedural rules
The procedural rules set out in the terms of reference are followed.

Deadlines
The process is managed to meet established deadlines.

Facilitation
Facilitation in this process is unbiased.Outcomes for freshwater management Information New and useful information for decision making is being generated as a result of this process.

Public interest
The outcomes of this process are meeting broad community objectives.

Understanding and support for collaboration
The process is resulting in increased public support for collaboration.

Other outcomes (capital)
Knowledge My understanding of environmental health has improved as a result of this process.

Knowledge
My understanding of economic issues has improved as a result of this process.

Knowledge
My understanding of social values has improved as a result of this process.

Knowledge
My understanding of cultural values has improved as a result of this process.

Empathy
My understanding of different perspectives on freshwater management has improved as a result of this process.

Collaboration
My skills in collaborative decision making have improved as a result of this process.
successful outcomes would be achieved.Survey results were analyzed using a combination of qualitative analysis, descriptive statistics, and statistical inference to test significant factors influencing the success of collaboration.

Survey administration
The sample population for the surveys comprised the active participants in each of the TANK and Northland processes.For the TANK process surveys, email was the primary method used to contact participants and manage the survey.For those without Internet access a printed copy was also available.Seventy-four surveys were completed by participants in the TANK process, with an average response rate of 54%.Response rates for online surveys are typically low, with rates between 25% and 30% generally considered good (Cook et al. 2000), so the response rate in TANK can be considered high and sufficient for the type of statistical analysis that we undertook (Carley-Baxter et al. 2009, Fulton 2016).
In Northland, email response rates were too low, so we switched to administering the survey via computer tablets immediately before a collaborative group meeting.This change resulted in a 100% response rate of those who attended the respective meetings.Overall, there was an average response rate across all groups of 51% because not all group members attended all meetings (Table 3).
The highest proportion of respondents in the TANK surveys, averaged over the four surveys, was from the primary sectors, including agriculture, horticulture, and viticulture at 43%, followed by council and government at 20% (Table 4).In Northland, agricultural stakeholders (27%) and those selfidentifying with environmental interests (26%) were those most represented in the five processes.The lowest proportions of respondents in Northland were from commercial/industrial, local government, and tangata whenua (i.e., Māori), at less than 10% each.

Survey findings
The percentages of people who strongly agreed or agreed with each statement on design and process management criteria in the TANK surveys are shown in Table 5 (refer also to Appendix 1, Figs. 2-4).For TANK surveys 1 and 2, the average percentages for each process criterion were calculated using the percent of positive responses across all survey statements related to a particular criterion.For example, to ascertain the groups' perception of representation, we used the following statements: 1.All appropriate interests are represented in the stakeholder group.
2. The regional council is adequately represented.

3.
There is equal representation of different stakeholders in the group.
In TANK surveys 3 and 4, a single statement "To what extent are all other viewpoints represented in the process?" was used to assess representation.To compare across all the surveys, the 11-point Likert scale was rescaled to the 5-point Likert scale, where 1 is strongly disagree and 5 is strongly agree.
Longitudinal results from the TANK surveys showed high scores on satisfaction with the process for the majority of process design and implementation criteria.For example, the process began with and continues to have a strong mandate from its stakeholders, who support the approach being taken.In TANK survey 1, 90% and 93% of respondents strongly agreed and agreed, respectively, with the following statements: I am involved in this project because it is the best way to achieve water management goals in the catchment area for myself and/or my sector/group.

and
Collaborative decision making is a step in the right direction to managing freshwater in the Hawke's Bay.
This perception continued in TANK survey 2, where 87% and 90% of respondents strongly agreed and agreed, respectively, with the same statements.The statement: I am fully committed to the collaborative decisionmaking process, also received a high level of support from respondents (87%), and there was widespread agreement (93%) that freshwater management in the TANK catchments was a significant concern, requiring a timely resolution.From the outset there was a clear preference for collaboration and a willingness by participants to invest in the process.
By the time TANK survey 3 survey was administered, 77% of respondents indicated that collaborative processes were still the preferred choice for managing freshwater.However, this figure dropped to 62% in the fourth survey.These results are captured in other process design and implementation criteria, which show that the process started strongly, and although it faced immediate hurdles, including mistrust between participants from previous failed attempts at collaboration, the group made significant gains early on.The process is still ongoing, but the most recent results show declines for several criteria.
Respondents were generally supportive of the way the TANK process was being managed and coordinated, with the terms of reference being followed (77% on average across the four surveys).They understood the extent to which they were accountable to their sector or stakeholder group (76%) and perceived that facilitation was unbiased (71%).Support for all three criteria improved since the first evaluation, suggesting that the process is, for the most part, being run well, that the rules are being followed, and that it is being conducted fairly.
The extent to which the process was meeting deadlines did not rate highly.When the council convened the process it was anticipated to run for 12-18 months.However, the process length will ultimately be more than double the original projections, so it is not surprising that agreement with the statement: The process is managing to meet deadlines, dropped from 85% to 23%.This result reflects the time required for collaboration, especially when participants have varying skills, capacities, and understanding of the issues at stake (Leach et al. 2002, Keast et al. 2004, Measham 2013).Internationally, the average time to reach consensus recommendations through a collaborative process is four years (Morton et al. 2012).
Other related process criteria with low levels of agreement include having realistic time limits and the degree of flexibility within the process to accommodate additional time.Unrealistically short timelines were also the most frequently cited weakness in the openended responses.As one respondent said: [There is] not enough time to understand hydrology and fill in science gaps.
Scores on other criteria also differed between surveys.In TANK survey 1, 66% agreed or strongly agreed that a consensus-based outcome was likely.By TANK survey 4, only 31% stated that consensus was likely.Participants' sense of agency was also lower in TANK survey 4 than it was in the first survey.In TANK survey 4, fewer than half of the respondents felt their participation was making a difference to the outcome, and the percentage that perceived the group had a shared purpose was 64% at the beginning but only 54% by the time the fourth survey was administered.
In Northland, five collaborative groups are running identical processes in parallel (i.e., each group is using a similar set of methods to build consensus over a series of monthly meetings).All five groups have been surveyed once, using the same survey instrument as the TANK surveys 3 and 4 evaluations in Hawke's Bay.The Northland surveys allow a comparison with the TANK process and also between each another, enabling us to explore inter-and intra-regional variations.Selected results showing support for design and implementation criteria in the five catchments are shown in Table 6.(Refer also to additional survey results Appendix 1, Figs. 5-7.) Survey results show that individuals' perceptions of each of the Northland processes vary considerably.In the collaboration literature, for example, having clear rules of procedure has been identified as a critical factor for success (Frame et al. 2004, Ansell andGash 2008).In two catchments, however, less than half the participants believe procedures are being followed, whereas in another catchment 80% of participants believe they are.There are also differences in the way in which participants want freshwater decisions to be made.In the Pōuto process, only 55% prefer collaboration, compared to 86% for the Māngere process.Perceptions of learning also varied, with Waitangi having the largest number of participants perceiving that the process led them (90%) or other participants (80%) to reconsider which issues they thought were more important.Interestingly, Sinner et al. (2016a, b) found that, in terms of public perceptions of freshwater management, Pōuto scored highest among the five Northland catchments.
Statistical analysis was used to determine the factors that influence the relative success of different collaborative processes.This analysis used one-way ANOVA, which tested for the equality of mean scores across all groups, while pairwise comparisons based on the Studentised Range distribution with the Fisher-Hayter method (Kirk 2013) were used to identify groups that systematically differed from all others.
The TANK group provides a baseline for the comparison.The group was convened by the council and is proceeding through a clearly defined process.Although there have been some changes to the membership of the TANK group, these occurred at various times and the ability of a process to accommodate them can itself be seen as an internal process factor.
The five Northland processes are nearly identical.All of the collaborative groups have been convened by the regional council, have similar terms of reference, and are proceeding through a series of steps at a similar pace.The groups vary only in size and in the composition of participants, while holding process relatively constant.
To assess how design characteristics influence the perceived success of collaboration in the TANK process, variables relating to representation, accountability, and mutual respect were analyzed independently using the one-way ANOVA test (Table 7, column 1) and pairwise comparisons (Table 7, columns 2-5).These variables have been identified in the literature as influencing the success of collaboration.Because the participants in the TANK process have been involved throughout the process, we have a longitudinal record of changes in process-related variables.Hence, these tests allow us to analyze how the influence of design elements and the process evolve over time.
Comparing the means of the four surveys shows that participants' perceptions of the design and implementation of the TANK process change over time for six variables; namely, confidence in reaching consensus, following the terms of reference, allowing all viewpoints to be heard, mutual respect, adapting to changing circumstances, and managing to meet established deadlines (p < 0.10).
Our results suggest that participants' perceptions change within the process, and that those changes may involve complex, dynamic, and reciprocal interactions within the collaborative group (Vandenbussche et al. 2015).The findings also highlight the influence of external conditions and the importance of stakeholder selection.
The pairwise comparison of the means shows that support for process design and implementation criteria increased between TANK surveys 1 and 2, peaked at the time the second survey was administered, and subsequently declined.Two criteria, "confidence in reaching consensus" and "managed to meet deadlines," were at their lowest level when TANK survey 4 was administered.This result demonstrates the dynamic nature of collaborative processes and suggests that attempts to measure success at a single point in time may be problematic.Longitudinal evaluations are probably required to investigate the dynamics of collaboration.Ex-post assessments of collaboration are helpful, but we need to know more about the complex interplay between multiple factors during the course of face-to-face negotiations and how these affect the success factors.
The survey results from the Northland processes facilitate comparing means for process design, implementation, and outcomes across five collaborative processes.Because the processes use the same methodology and because the surveys were administered at the same point in time in each process, we are able to control for the influence of design criteria.Statistically, the processes differ in seven criteria: (1) Is your participation making a difference to the outcome?; (2) Are participants' opinions closer than when the process started?; (3) Has the process led you to reconsider what you think is important?; and (4) Has the process led others to reconsider what they think is important?(p < 0.10; Table 8).Several criteria related to learning outcomes were also significant, including participants' (5) understanding of environmental, (6) social and cultural priorities, and (7) economic interests (refer to Appendix 1, Figs. 7-14).
The results from Northland also show considerable variation among the catchments.Using the Fisher-Hayter estimator for pairwise comparisons, the Pōuto lakes process statistically has the lowest support for nearly one-third of the 36 criteria, whereas the Waitangi process has statistically higher support than any other in 11 of 36 categories.The Māngere group is also proceeding well, with sense of community and the increasing levels of public support for the process ranking highly.This group also had a strong level of preprocess cohesion, having worked together in an effort to reduce phosphorus levels in the Māngere River.Across all five catchments, learning was evident, particularly with respect to the environment, economic impacts, and social and cultural values.Given the dynamic nature of the results for the TANK process, we expect that Northland perceptions will also be dynamic.

Discussion of variables influencing the success of collaborative process
The results of the surveys and analyses suggest that success in collaborative processes is influenced by good process design and procedural rules, and by the individuals involved in the negotiations.The effects of process can be mitigated through good design, paying particular attention to meeting milestones, and the influence of other internal and external factors that might have a bearing on collaboration.Considered attention to participant selection and structured approaches to recruitment can also be considered (Cradock-Henry 2013), though this can have implications for perceptions of legitimacy (Sinner et al. 2015).
External events also play a role in determining success.
In the TANK process, delays in the provision of science have also impinged on the success of collaboration.After TANK survey 2, it became evident that the science information would not be delivered within the promised time frame(s).The council had overextended itself, focusing significant resources on planning a large dam project in a neighboring catchment.As a result, the science for the TANK group was delayed, leading to some frustration with the amount of time the process was taking.
Participants noted in the survey responses: . Timeframe and frequency of meetings seems to be a moving target.
. We are constantly waiting for information.
. Considerable science information has to be completed to enable sound decisions to be made -that info is not there yet.
In addition, the council's project leader resigned and it was nearly a year before he was replaced, during which time the process lost momentum.In this case, careful consideration of the science requirements should have been undertaken earlier in the process.This is not to suggest that a collaborative process should not begin until the science is ready, but rather that unmet expectations of science delivery are likely to result in frustration by the members of the process, which may spill over into other areas.We believe that basic information about the state and trend of environmental conditions is sufficient for collaboration to begin.
Within a single process there were dynamic changes in participants' perceptions.Although findings from other longitudinal studies suggest that process performance tends to improve over time (Leach et al. 2002), results from the four TANK surveys show declines in stakeholder satisfaction.In the TANK process, these changes have been most evident since the second survey, in which declines in satisfaction were recorded across nearly all of the process design and implementation criteria.In addition to the internal factors, such as delays in science delivery, external factors may also have played a role in these fluctuating evaluations.Decisions by the regional council to support a large dam and to grant consent to a water-bottling plant might have have had a negative effect on the group.From the timing of these decisions, falling between TANK surveys 2 and 3, and between TANK surveys 3 and 4, it appears that these caused some participants to perceive a loss of trust and an erosion in feelings of agency.As one respondent said: I am committed to a genuine collaborative process, but am aware that our group only provides recommendations to HBRC who appear to have already decided what the outcome will be.
Finally, the longitudinal assessment of the TANK process suggests that time matters.Measham (2013) found evidence for social learning, i.e., learning as a result of working together to understand and develop solutions to environmental challenges, in a group of landholders after one year of working together to address problems with dryland salinity.This learning was limited, however, to a better understanding of the problem.Evidence of learning to identify potential solutions took an additional two years (Measham 2013).In other words, success takes time (Leach et al. 2002).At a minimum, three to four years might be required for collaborative processes (Morton et al. 2012).Sufficient time is essential to build the necessary social capital, trust, and goodwill and to enable upskilling and learning (Wondolleck and Yaffee 2000, Keast et al. 2004, Innes and Booher 2010).
Forcing a recommendation or abrogating the consensus requirement may have long-term, negative consequences for collaborative processes (Ostrom 2005, Burton et al. 2006, Hassenforder et al. 2015).Negative initial experiences with collaboration may jeopardize future efforts at collaboration and have negative implications for other efforts at community building (Midgley et al. 2013).
The people around the table also matter, which has implications for the recruitment process, and highlights the importance of stakeholder composition when scoping and designing a collaborative process.It also suggests that the competencies of potential participants should be considered alongside their views on collaboration and their attitudes toward the convening agency.
Although various options for group composition are described in the literature (e.g., election, volunteerism, self-selection, purposeful sampling), our findings suggest that more methodological approaches to selecting participants in a collaborative process should be considered.One solution might be to use a social-ecological inventory to identify relevant actors (Baird et al. 2014b).Considering both the social and ecological aspects of a system and interacting with stakeholders in a systematic way can help to establish a basis for stakeholders' participation in governance activities and to lay a foundation for further learning and participation (Edmunds and Wollenberg 2001, Baird et al. 2014b, Curtin 2014, Baird et al. 2016).The value of working together before convening a collaborative process is highlighted in the results from the Waitangi catchment survey.Survey results showed that 80% of respondents favored the collaborative model, and that, for most measures, Waitangi was the most successful collaboration of the five groups.One explanation for this success might be the presence of long-running subcatchment community groups and a recently completed threeyear project working with local farmers on land management practices (Hampson 2013).The results of these small-scale community initiatives have seen farmers in the catchment implement management practices to reduce runoff from productive land, with noticeable improvements in on-farm conditions (Hampson 2013).In addition to the tangible benefits arising from improved water quality, experience, social capital, and relationships have also been developed.These small successes may have primed the Waitangi collaborative group for success (Baird et al. 2014b).By having pre-existing relationships and foundational social networks between stakeholders, it is likely that social capital and trust have been developed more quickly, potentially providing the basis for a successful collaboration (Leach and Sabatier 2005, Floress et al. 2011, Leach et al. 2014).
Participants' attitudes to collaboration may also have an impact on the process.Nearly half the respondents (46%) in Pōuto preferred something other than a collaborative group to manage freshwater in the catchment, including a legislated process.This feeling may have led to internal opposition from the outset, creating additional barriers and friction within the collaborative group.Participants with negative views of collaboration, strongly held views on its use for decision making, or previous negative experiences with collaboration may slow consensus building.Such internal conflict needs to be overcome before decisions are made, highlighting the need either for careful participant selection or for activities to build trust and social capital, and foster relationships before more formal collaborative activity begins (Baird et al. 2014b).Membership considerations must also include the extent to which participants will be accountable to stakeholder groups, and the extent to which the process will have legitimacy within the wider community (Sinner et al. 2015).
Finally, the differences between processes in Northland may also be explained by differences in contextual factors between the five groups (e.g., perceived urgency, drivers of change, and biophysical conditions).Although each groups is following an identical process, we have yet to determine whether or not other factors might explain those differences.Further interviews and analyses are planned to explore this in more detail.

CONCLUSION
Collaborative processes have been widely applied in a range of contexts to deal with value-laden and often contested issues of resource management.However, there are still very few examples of longitudinal evaluations of collaboration and limited insights into the factors that influence success.
Using a comparative case-study methodology, we looked at different variables that influence success in collaborative processes.The results of the analyses suggested that procedural rules and other process-related variables are important and change over time.In addition, external factors and conditions unrelated to the collaboration may affect how participants interact in a process.In the case of the TANK process, these external factors related to processes to approve water storage in a nearby catchment and water bottling at the end of the catchment.Although these decisions did not have a direct bearing on the conditions in the TANK catchments, they did signal to participants that they may ultimately have less agency than they believed.The extent to which these events may have had a bearing on participants' perceptions of the collaborative process has yet to be fully explored; however, evidence from other jurisdictions has shown that processes operating at higher scales (e.g., nationally or regionally) can have a detrimental effect on local processes (Patterson 2016).To ensure a greater likelihood of successful collaboration, it may be necessary to delay major decisions that have a direct bearing on collaboration until consensus has been reached.
The results from the TANK process also showed that allowing sufficient time for collaboration is essential.Collaboration is a lengthy process, and to ensure participants do not become disenchanted, the time expectation should be clearly communicated from the outset.Unrealistic expectations regarding the time needed to undertake a collaborative process may also have implications for resourcing, staffing requirements, science delivery for the convening agency, and dedication to the process for the stakeholders.
The people around the collaborative table also have a significant influence on the success of a process.Survey results from five groups following an identical approach to collaboration and surveyed at a similar point in the collaboration process showed marked differences in the ways in which the process was perceived.
For councils or other agencies considering collaboration as a means of resolving disputes, greater attention might need to be paid to recruitment to ensure the stakeholders involved are willing to set aside personal differences and work together.Agencies exercising greater control over membership, however, may have adverse effects on accountability to stakeholder groups.If councils select group members, processes may be more successful in terms of achieving consensus and meeting internal evaluation criteria, but less successful in reducing conflict away from the negotiating table (Sinner et al. 2015).Interest groups and the wider community might question whether the outcome reflects a https://www.ecologyandsociety.org/vol22/iss2/art14/consensus of all interested groups and parties, or just a consensus of people selected by the council.The results of our analysis did show that in one catchment, previously established social capital and trust from other community initiatives helped to prime the process for success.Resourcing small-scale collaborations and catchment groups can foster legitimacy and increase the likelihood of a truly representative process when the time comes, and they should be considered.
Although collaborative processes continue to be promoted as a means of resolving intractable problems of resource management, they are not a silver bullet.Collaboration does provide the opportunity for building social capital and trust, and, if successful, can have outcomes that extend far beyond the scope of the original collaboration.However, identifying the right participants, ensuring (as far as possible) that there is support for collaboration from the outset, and having a well-designed process will likely result in improved outcomes.
There are many criteria for evaluating the success of collaborative processes, including the degree of inclusiveness, adequate resourcing and facilitation, and responsiveness to the existing context.However, no collaborative process can be designed for all eventualities at the outset, and collaborative processes are often large-scale, long-term projects that evolve through different cycles of goal setting and key political relationships.Therefore, the ultimate success factor is building in both the capacity to generate feedback on the collaborative process and the flexibility to redesign the process based on feedback from stakeholders and in light of a dynamic and often complex external environment.
Responses to this article can be read online at: http://www.ecologyandsociety.org/issues/responses.php/9126 Appendix 1

Figure A1. 1
Figure A1.1 Results from the TANK process over time, "Confidence in reaching consensus"

Table 1 .
Land use and management issues for case study catchments.

Table 3 .
Summary of survey response rates.

Table 4 .
Distribution of respondents by affiliation.Data for the TANK group is an average of the four surveys completed to date. ‡

Table 5 .
Process design and implementation criteria and stakeholders' agreement.The percentage figure in some cases represents the average from several questions and/or statements used to evaluate a single criterion.Each criterion was prefaced with "To what extent..." Note: TANK = Tūtaekurī, Ahuriri, Ngaruroro, and Karamū.

Table 8 .
Northland equality of means for a single point in time.One-way ANOVA † Pairwise comparison based on studentized range distribution with the Fisher-Hayter method.p < 0.10.Are participants opinions closer together now than when the process started?