Operationalizing resilience: co-creating a framework to monitor hard, natural, and nature-based shoreline features in New York State

There is growing interest in the application of nature-based solutions to adapt to climate change and promote resilience, yet barriers exist to their implementation. These include a perceived lack of evidence of their functioning in comparison to conventional solutions and an inability for existing design, policy, and assessment processes to capture the multiple benefits of these solutions. Positing this as a challenge of operationalizing and measuring resilience, we argue that the concept of resilience needs to be given concrete meaning in applied management contexts. Starting with shoreline vulnerability as a policy problem and natural and naturebased shoreline features as a promising solution, we present a case study of a co-creative process to produce an interdisciplinary and locally relevant approach to understanding and capturing the benefits of natural and nature-based solutions. We develop the notion of resilience service to enable a concreteness to resilience that simultaneously takes into account ecological, technical, and social dimensions. Through the co-creative process, our researcher-practitioner network developed a monitoring framework for shoreline features in New York State to facilitate the comparison of natural and nature-based features with conventional shoreline approaches. We describe the process and assess the advantages and drawbacks of integrating scientific input and local knowledge. We present the monitoring framework, showing how the co-creative character of the process is consequential in the formulation of the final framework through the selection of parameters, indicators, and protocols. We argue that interdisciplinarity, co-creation, pragmatism, multi-scalar applicability, and policy relevance are critical principles to understand the functioning and facilitate the implementation of naturebased solutions, while recognizing that this work necessitates compromise and as such will lead to continued deliberation. We posit this is a strength of the process for it acknowledges the creation of resilience as a social process in which values are central and subject to change.


INTRODUCTION
Resilience is increasingly a key goal in many policy domains, including in shoreline management. The call for resilient shorelines from a multitude of actors stems from a diversity of concerns, including the ability to deal with challenges imposed by climate change (e.g., storm surges and flooding), the delivery of social benefits (e.g., recreation, human health, and well-being), and a desire to limit habitat destruction and loss of biodiversity associated with traditional shoreline stabilization measures. Recently, the notion of nature-based solutions (NbS) has gained traction in both policy and academic circles as a way to pursue different understandings of resilience simultaneously (Raymond et al. 2017, LaFortezza et al. 2018, Hobbie and Grimm 2020 and has interested shoreline managers as a result.
Although varied in their applications, NbS emphasize the use of nature (living organisms, substrates, and landscape features) in tackling socio-environmental challenges in ways that are considered cost-effective; additive of environmental, social, and economic benefits; and help build resilience (Temmerman et al. 2013, Reguero et al. 2018). NbS are supported or inspired by natural systems and constructed to mimic characteristics or incorporate elements of natural features. Even with growing interest in the application of NbS by communities, designers, and government agencies, their implementation remains limited because of a number of barriers (Sarabi et al. 2019). These include inadequate financing , institutional fragmentation and overlapping jurisdictions of decision-making bodies (Wamsler 2015, existing land use policies (Santiago Fink 2016, Davies and Lafortezza 2019), inadequate regulations Naumann 2017, Denjean et al. 2017), and uncertainty around the performance of NbS (Eggermont et al. 2015, Arkema et al. 2017, Nesshöver et al. 2017, Raymond et al. 2017.
Despite some documented empirical observations in academic and non-academic literature of the multifaceted shoreline management benefits of NbS, hard structural features are often perceived as the best and most reliable option to implement (Scyphers et al. 2015, Moosavi 2017) through a process referred to as shoreline armoring. Hard structural features such as seawalls, bulkheads, and revetments however can have negative consequences on shoreline systems through exacerbated shoreline erosion, loss of biodiversity, and habitat degradation (Van Slobbe et al. 2013, Firth et al. 2014, Moosavi 2017; and detriments to community health, livelihoods, and well-being. For example, certain shoreline armoring can provide a false sense of safety and enable underestimation of risk (Kimura 2016), can be more costly in maintenance and repair for homeowners than natural shorelines (Smith et al. 2017), and can hinder activities of value to communities such as fishing (Kimura 2016). In contrast, NbS -in the context of shoreline management referred to as natural https://www.ecologyandsociety.org/vol26/iss3/art10/ and nature-based features (NNBF) and including features like wetlands, dunes, and floodplains-potentially provide similar risk reduction benefits to hard structures while having fewer negative consequences (Narayan et al. 2016, Arkema et al. 2017.

Study context and problem statement
A decisive barrier to the implementation of NbS in shoreline management is the lack of confidence in their performance (Scypers et al. 2015, Sarabi et al. 2019. This is the result of recorded impacts generally coming from single case studies, onetime evaluations, or limited applicability over different types of features and impacts (Raymond et al. 2017, Dumitru et al. 2020 and the lack of knowledge systemization (Fernandes andGuiomar 2018, Albert et al. 2019). Although various frameworks have previously been created to measure the benefits of NbS (e.g., Van Slobbe et al. 2013, Reguero et al. 2014, Arkema et al. 2017, Raymond et al. 2017, Sterling et al. 2017, they come with limitations that include the absence of social indicators (Brink et al. 2016, Dumitru et al. 2020; the high costs to their execution; the lack of broad field testing; and limited applicability over different shoreline types, which is important for comparative evaluation to inform decision making. Also, the operationalization of resilience is a largely unresolved issue (Baravikova et al. 2021) hindering the use of the concept on the ground (e.g., Davidson et al. 2016), despite growing efforts to quantitatively and qualitatively assess the performance of specific systems through the lens of resilience (e.g., Quinlan et al. 2016, Allen et al. 2018. How can resilience be operationalized in the context of shoreline management? What processes and practices can inform the creation of specific and replicable assessments of the resilience of different shoreline features? Starting with the desire to trace and evaluate the performance of NbS in general and NNBF in particular relative to hardened shoreline measures, in this paper we describe a process through which an interdisciplinary and locally relevant approach to understanding the benefits of NNBF in shoreline management can be accomplished that takes into account the above mentioned limitations of existing frameworks. Through a multi-year process of co-creation, a monitoring framework was developed, which we present here and can be used to better understand the functioning of various shoreline features. Although co-creation is advocated as crucial in the uptake of NbS (Nesshöver et al. 2017, Frantzeskaki 2019, DeLosRíos-White et al. 2020, limited attention has been given both to how co-creative processes shape the understanding of concepts like resilience (a concept whose meaning is often assumed rather than negotiated) and to the challenges inherent to co-creation. By creating and documenting a co-creative process to operationalize resilience we aim to better understand how to apply resilience and how cocreative processes matter in natural resources management, while tracking the performance of shoreline features.
The development of this process and of the resulting monitoring framework took place in New York State (NYS). In the U.S. context, federal and state agencies regulate construction and other activities in water bodies, while land use typically falls under the jurisdiction of county and local governments (municipalities). This results in a complex overlay of regulation and management for shorelines that typically involves all levels of government, making effective shoreline management difficult . Agency coordination at multiple levels of governance is advocated for effective shoreline management (Lowry 1985), with the relationship between planning and permitting agencies at state and county level being most crucial in many states (Kittinger and Ayers 2010). In a comparative study between coastal zone management programs in Hawai'i and North Carolina, Kittinger and Ayers (2010) show that more local approaches result in disparate and fragmented policies that hinder strategic planning for environmental protection and coastal development, resulting in more coastal development and shoreline armoring, and shifting the fiscal burden of risk onto the public rather than littoral property owners. They argue that comprehensive coastal policy developed at the state level is necessary to address persistent issues like erosion and emerging threats like sea-level rise in strategic, cohesive, and holistic ways. Here, we start from the premise that in the U.S. the intermediary level of the state can play an important role in bridging federal and local jurisdictions to enable a more systems-based approach, making interventions less ad hoc and potentially more coordinated. However, community input and acceptance are integral to shoreline management (O'Riordan and Ward 1997, Gopnik et al. 2012, Palomo et al. 2016, which is why we propose a process of co-creation through which local contexts and preferences can shape and be integrated in the monitoring framework. In NYS, the water's edge and shorelines are regulated, and to a high degree managed, at the state level: the Department of Environmental Conservation (NYSDEC) regulates the state's tidal wetlands, while the Department of State (NYSDOS) is in charge of the coastal zone management program and applies regulations related to coastal zone hazard areas in the state. Most guidance and guidelines for coastal and shoreline management are issued from the state level and localities within the state are accustomed to interacting with state agencies concerning issues of shoreline management. In multiple studies and plans, NYS agencies have highlighted shoreline management as a critical issue (NYSDOS 2015(NYSDOS , 2017. Although implementation of NNBF is currently limited, recent programs (e.g., NYSDOS 2017), plans (e.g., NYS DHSES 2019), and law (e.g., New York State Assembly 2014) explicitly support the implementation of NNBF, following similar acknowledgements of the value of NNBF shorelines at the federal level (FEMA 2013, Bridges et al. 2015, NOAA 2015. The 2014 NY Community Risk and Resiliency Act (New York State Assembly 2014), for example, requires the NYSDEC and NYSDOS to "develop additional guidance on the use of resiliency measures that utilize natural resources and natural processes to reduce risk" (New York State Assembly 2014). This law heightens the interest and imperative for NYS to better understand the benefits and trade-offs for utilizing these types of measures for shoreline management. In 2017, NYSDOS partnered with New York State Energy Research & Development Authority (NYSERDA) to co-sponsor a project to (1) identify key indicators to monitor the performance of shorelines with specific attention to their contribution to resilience; and (2) develop standardized protocols to generate comparable data across diverse shorelines of NYS. This effort included the ocean-facing coast and tidally influenced bays and estuaries of Long Island, New York City, and the Hudson River Valley; and the coastal zone of the Great Lakes (Fig. 1).

Theoretical and conceptual foundations: resilience and cocreation
The concept of resilience can take on many meanings depending on context and application. Broadly speaking, approaches to resilience can emphasize stability (engineering approachbouncing back) and/or transformative change (evolutionary approach -bouncing forward). Which of these approaches to emphasize is a question of import because it involves trade-offs across temporal and spatial scales (Chelleri et al. 2015, Baravikova et al. 2021, ultimately impacting "for whom, to what, where, when, and why" resilience is (Cutter 2016, Meerow andNewell 2019). Thus, the way resilience is defined and operationalized is crucial not only for its uptake in planning and policy-making processes but in shaping who does and does not benefit from them (Chelleri et al. 2015, Meerow andNewell 2019). Here, we therefore understand resilience as a process characterized by negotiation, contestation, and trade-offs (Harris et al. 2018), and its operationalization to be contingent on understanding decision making and political contexts Mitchell 2017, Davoudi 2018).
In the context of shoreline management there is no scientific or policy consensus about whether to emphasize stability or transformative change as the key characteristic or goal for resilient shorelines. Discussion about whether to prioritize hard, natural, and nature-based shoreline features often hinges on implicit adherence to one rationale or the other, in which hard structures come to represent the status-quo-enforcing engineering approach and NNBF an adaptive evolutionary approach. However, understanding these as opposites contributes little to understanding how varying shoreline features function compared to others and whether or not these logics and their material representations are mutually exclusive. This distinction also fails to address the question of who benefits from them or resolve temporal and spatial issues caused by hard structures, which need to be rebuilt after a few decades and can make things worse for unprotected nearby areas. One possible way to overcome this conundrum is if we embrace an understanding of resilience consistent with Bridges et al. (2015) that focuses on maintaining or recovering desired functions and services of a social-ecological system in the face of disturbances such as storms, and increasing pressures caused by urbanization and sea level rise. This understanding allows for stability (through the notion of maintaining and recovery) as well as change (though the notion of desirability) and provides a mechanism to consider the question "who for?" But what are the desired services and functions of shorelines? What services and functions need to be maintained or recovered for a shoreline to be considered resilient? The answers to these questions are time and place specific, and the process of answering them requires interdisciplinary and place-specific engagement as exemplified in our NYS case study. Nevertheless, three categories of benefits related to resilience of shoreline features can be identified and used in such a process, representing both stability and adaptability logics without assuming one to be more important than the other or to match onto specific shoreline features.
The first category is ecological function, referring to the potential to enhance biodiversity and increase biological function through creating new and connecting existing habitat isolated because of development and shoreline hardening (Warren et al. 2002, Zedler and Kercher 2005, Bracken et al. 2007), and the maintaining or restoring of environmental processes that improve water quality and aquatic ecosystems (Zedler andKercher 2005, O‛Donnell 2017). A second category is structural integrity and hazard mitigation, referring to the ability of a shoreline feature to resist breaking or deforming under various pressures (e.g., wave energy, drought, sea level rise) and reducing or eliminating risks to life and property from hazard events (e.g., storms or droughts). Whereas hard structural features tend to suffer costly damage under severe storm events, certain NNBF can naturally recover (Gittman et al. 2014) and adapt to sea level rise (Morris et al. 2002(Morris et al. , 2018. A third category consists of social and economic outcomes, which include direct and indirect effects of shoreline features on people and their livelihoods. NNBF can support livelihoods, recreation, and human-health and well-being by providing productive and green spaces (Scyphers et al. 2011, Hobbie andGrimm 2020). The consideration of socioeconomic benefits, critical to providing a complete picture of shoreline resilience (Wamsler 2015, Keeler et al. 2019, should be understood in relation to demographic data and local context, especially given historic unequal exposure to risks, unequal access to urban green space, and exclusion from resources and decision-https://www.ecologyandsociety.org/vol26/iss3/art10/ making processes on the basis of race/ethnicity and socioeconomic status (Hobbie and Grimm 2020).
The above categories build on the idea of ecosystem services, as the tangible and intangible benefits people obtain from ecosystems and the species that make them up (Moberg and Folke 1999). Shoreline management is largely driven by social values (Loomis and Paterson 2014). The notion of ecosystem services connects social and biophysical sciences and interests in environmental management in general and shoreline management in particular (Barbier et al. 2011, Biedenweg 2017, Ryfield et al. 2019) by linking particular sets of ecological characteristics to social value (Dore and Webb 2003). We broaden the focus by not exclusively focusing on ecosystems, but also on social and technological systems that together make up the specific forms shorelines take. We use the term "resilience service" (Fig. 2) to center resilience as a goal in planning, and use it to examine the functioning of shoreline features; though resilience service is a useful conceptual move to comparing interventions in other domains as well. In particular, resilience service allows for comparative assessment of benefits and burdens derived both from natural, man-made, and hybrid systems, including the production and maintenance of ecosystem services. For example, hard shoreline features are usually evaluated for adverse environmental impact upon construction, but not evaluated for the lack of provisioning of ecosystem services at the feature scale during its lifetime, because an engineered structure does not count as an ecosystem that can be assessed for ecosystem services. As a result, engineered structures are not evaluated along ecological principles, even though NNBF are assessed along principles of engineering given the centrality of shoreline stability to shoreline management. The resilience services concept allows for the inclusion of benefits rooted in ecological, technical/engineering, and social/economic domains simultaneously, expanding the universe of desired shoreline functions. As such, resilience service is a way to potentially highlight a lack of ecosystem services, and to gain insight into choices and trade-offs between various shoreline features in terms of the types of benefits and burdens they produce and for whom. It enables explicit discussions rather than assumptions about desired functions and services of shoreline features or other interventions in the landscape, and thus what resilience means in situated contexts. This means that concerns about equity or environmental justice, which are not adequately captured under the umbrella of ecosystem services (Langemeyer and Connolly 2020), can be considered in the planning and policy process around shoreline interventions.
In order to give explicit meaning to these overarching categories of resilience services, we argue that operationalization needs to happen through an interdisciplinary and co-creative process, and we show how the co-creative nature of our process was foundational to operationalizing resilience and creating a monitoring framework for shoreline resilience. Knowledge produced across boundaries (co-creation) is more salient, credible, and legitimate (Cash et al. 2006), and this approach has been identified as crucial to the uptake of NbS in general (Nesshöver et al. 2017, Frantzeskaki 2019, DeLosRíos-White et al. 2020) and coastal projects in particular (Seijger et al. 2015). This form of stakeholder engagement is increasingly seen as necessary because the resulting integrated knowledge provides a more inclusive understanding of complex social-ecological systems. It can bring together engineering, ecological, and social perspectives, while taking diverse stakeholder perspectives and tacit knowledge types into account, thus breaking away from siloed thinking Mitchell 2017, Frantzeskaki 2019). It makes knowledge more relevant and accessible to stakeholders and is more likely to lead to locally implementable solutions (Baravikova et al. 2021). This is particularly true when monitoring needs to happen in both locally relevant and scientifically defensible ways , Biedenweg et al. 2017, Raymond et al. 2017, Sterling et al. 2017. In theory, a co-creative process further allows for discussion and negotiation on the meaning and characteristics of resilience, including on how certain conceptualizations and operationalizations benefit, burden, or neglect certain groups, thus foregrounding rather than hiding contestation and trade-offs between different approaches.

Case study approach
This paper presents a case study of a co-creative process of operationalizing shoreline resilience by a researcher-practitioner network. The authors are situated to comment on the case reflexively, because the authors were members of the network. This participatory approach starts from the premise that a topdown or solely technocratic approach is inappropriate and insufficient in dealing with the multiple facets of shoreline management, in part because it can ignore the role that communities play in managing the shoreline environment. Instead, collaboration with numerous stakeholders in a cocreative process seeks to rework the relationships between government agencies, researchers, residents, and other community stakeholders in order to base decision making on more than technical expertise alone (Thomas-Slayter 1995, Ludwig 2001, Campbell et al. 2016).
The case study method is a detailed investigation of an event, certain actors, and relationships. Case study research is especially useful to explore interactions between variables and follow how outcomes are generated. Over the course of almost three years, in-depth data were generated and used to explore how the cocreative process shapes the operationalization of resilience in the context of shoreline management while chronicling challenges of this type of process. These data were generated mostly through participant observation, with note-taking happening throughout the process at all meetings. Sometimes these notes were the observations of individuals; at other times, especially at larger meetings, they were a group process in which some questions of interest were predetermined and various note takers would record conversations and observations simultaneously, which were subsequently discussed post-meeting. Photographs were used to keep track of information about the process in general (who was in which group, how many people were present, etc.) and about the operationalization choices in particular (e.g., how many votes were cast in a certain category?). Working papers were developed for the project sponsors, chronicling the process and interim results. In developing this case narrative, we triangulated across these data sources, drawing upon published research and grey literature, as well as the field notes collected. Our analysis centered around the dual question of how the operationalization of resilience changed by the involvement of varying stakeholders and the benefits and challenges of a co-creative process. Our approach was abductive, meaning that while we looked for these overarching themes in the analysis of our data we remained open to how they became expressed; in other words, we did not have predefined categories of benefits and challenges. Thus, in striving for reflexivity, we remained attuned to problems and challenges encountered, rather than merely focusing on the resulting operationalization. Finally, to strengthen the credibility and validity of the analysis, we conducted a "member check" by having the draft reviewed by each other and by practitioner collaborators involved in the project, subsequently revising the narrative.

Developing the framework
In addition to being applicable across the state, shoreline types, and types of benefits, the framework needs to be field tested, lowcost, and low-technology to encourage wide uptake. This is in line with recommendations from Kirbyshire et al. (2017), who note that "many of the existing frameworks for measuring resilience are too data-demanding, too academic, or too time-sensitive to meet practitioners' needs" (p. 37) to actually lead to monitoring and evaluation. Given this diversity of needs, the co-creative process we developed brought together people from different disciplinary backgrounds (e.g., biophysical, social), sectors (e.g., design and planning, regulatory, research, resource management), and regions of NYS. Including participants from all four regions was crucial because each region has different levels of acceptance and experience implementing NNBF; differences in the types and scale of shoreline protection desired varying with factors such as physical conditions, population density, and tidal vs. non-tidal influences; and different socioeconomic, political, cultural, and regulatory customs. This diverse array of actors participated in selecting parameters to allow evaluation, indicators to operationalize the parameters into measurable units, and monitoring protocols to enable data collection. The co-creative effort occurred in five phases (Fig. 3).

Phase 1
In the first phase, the need for a monitoring framework, an interdisciplinary and multi-sector core project team, and the framework goals and basic structure were established. Interest at municipal and state levels for NNBF previously led to a research agenda for green coastal infrastructure (Zhao et al. 2014). A broad interest in collaboration on NNBF research and the creation of a NNBF monitoring framework became evident in a series of interviews conducted by the Consensus Building Institute (personal communication). A framework could improve understanding of the functioning of NNBF and facilitate comparison over space and time, addressing critical uncertainties that are inhibiting wider permitting and implementation of NNBF. Grant proposals written by a consortium of organizations including design and consultancy companies, non-profit organizations, government agencies, and academic institutions eventually led to a partnership with sponsors NYSDOS and NYSERDA. Individuals from these organizations formed the core team and their expertise included planning, design, biophysical (e.g., ecology, marine science) and social sciences (e.g., political science, urban planning, law), natural resources management, and policy. This collaboration officially started with a workshop in early 2017, where decision makers, researchers, practitioners, and other experts on NNBF discussed the current state and future of designing, implementing, and monitoring NNBF in NYS. This workshop provided input to the core team on useful parameters and indicators and framework structure, including the decision to focus on three "resilience services." Thus, it was in the initial phase that the problem of measuring the performance of NNBF and comparing them to hard shoreline features merged with the question of operationalizing resilience for shoreline management.

Phase 2
During the second phase, the core team undertook a document analysis of existing monitoring frameworks and convened technical working groups (TWG) around the three resilience service areas to develop a draft monitoring framework combining biophysical and social indicators. The goals of this phase were to identify and develop preliminary parameters, indicators, and protocols for each resilience service, and articulate related technical considerations. Thirteen existing U.S.-based shoreline monitoring frameworks with attention to NNBF were reviewed for their conceptual and organizational strengths and weaknesses, paying attention to their applicability to different shoreline types, geographic conditions, and disciplinary emphasis. Comparing the differences and similarities between these frameworks resulted in a list of considerations about how to organize the new monitoring framework. The TWG were asked to contemplate this list in their development of a draft framework. TWG consisted of 10-15 members, each representing a mixture of expertise areas and experience from the private sector, public agencies, and academia. The ecological function group, for example, had representatives from ecology, hydrology, and biology based at academic institutions and public organizations; the structural integrity and hazard mitigation brought together experts in ecology, coastal engineering, hydrology, and landscape architecture from public and academic institutions and private companies; and the socioeconomic outcomes group had representatives from political science, economics, landscape architecture, ecology, natural resources management, and planning. Each TWG conducted one or two in-person and several video-conferencing meetings during which parameters and indicators were identified and prioritized, and monitoring protocols were outlined based on verbal and written feedback from all working group members. This was achieved through a pre-meeting survey, structured brainstorming exercises during meetings, voting exercises to rank parameters and indicators, and discussions facilitated by members of the core group to further deliberate on the results of brainstorming and voting exercises. The core team, of which two or three representatives were present at these meetings, then consolidated the recommendations of the TWG and their goals, assumptions, and process, resulting in a draft monitoring framework.

Phase 3
In the third phase, the core team and representatives from the TWG worked with representatives from (non)governmental organizations, academics, property owners, private sector professionals, and local, state, and federal agencies to revise and refine the draft framework with two goals. The first was to ensure local applicability/relevancy and user value by accounting for both philosophical and pragmatic considerations of stakeholders across the state. The second was to develop a state-wide network of monitoring partners. In a state with ~2600 miles of shorelines, site-specific or local partnerships are required to implement at the scale necessary to generate enough data to effectively inform policy and management. Convening a cadre of partners from across the state and including their insights and data needs into the framework makes it more robust and more likely to be used.
Three types of engagement to revise and refine the framework were organized. The first engagement group was the project advisory committee (PAC), consisting of 10 experts on shoreline management from state and federal governmental agencies, academic institutions, and environmental organizations. This group advised on framework conceptualization and organization as well as engagement with other groups and individuals. The meetings with the PAC were organized virtually, with the first meeting focusing on the project framing, approach, and planning; and the second meeting on indicators and protocols. Committee members reviewed the proposed list of parameters and indicators in an online spreadsheet, and assessed the framework based on their own expertise, on their understanding of the feedback from stakeholders, and on the justifications as provided by TWG. They also provided feedback on the developed protocols using feasibility of the effort as a guiding principle.
The second form of engagements were interactive in-person workshops entitled "regional working groups" (RWG) held during the summer of 2018 in each of the four coastal regions of New York State. The RWG were critical to the process of cocreation as well as the ultimate viability and implementation of the monitoring framework. Involving end-users who own, manage, and use the shorelines was viewed as imperative to creating a relevant monitoring framework that includes indicators and protocols that landowners and managers can and will use and that will produce information relevant to landowners', shoreline managers', regulators', and policy-makers' future shoreline management or policy decisions, particularly as they relate to NNBF. This on-the-ground perspective is particularly Fig. 4. Recruitment criteria for regional working groups used to identify and engage regional working group participants within each New York State coastal region. TWG = technical working groups.
important for understanding critical information desired from stakeholders regarding the resilience of NNBF. Prior studies have indicated that misperceptions regarding the environmental impacts and cost-effectiveness of different shoreline conditions is common among landowners in particular, and may promote armoring over use of NNBF (Scyphers et al. 2015(Scyphers et al. , 2019. Recruitment criteria were developed to help identify 20-30 individuals from each region who represented the diverse perspectives and expertise across resilience service areas, and the organizations, agencies, and individuals who would ultimately implement the monitoring framework or use its results (Fig. 4).
Using these criteria, members of the core team, i.e., regional leads, identified and invited potential participants in their respective regions. Each regional lead had two or more years of extension and outreach experience building relationships and partnerships with community leaders, local stakeholders, government, and universities. The regional leads engaged participants individually and identified times, locations, and formats for RWG workshops accessible to the greatest number of target participants. Potential groups included civic/non-governmental organizations (including education, recreation, conservation, home-owners associations, and groups who work, live, or play in New York State's coastal areas); representatives from local, state, and federal governments; tribal groups; permitting agencies; shoreline practitioners and consultants; emergency managers; and scientists and researchers. Some potential RWG participants were invited but could not attend, though most invitees were interested in updates on the progress and outcomes of the project. Prior to the RWG workshops, participants were encouraged to attend a webinar that explained the project background, goals, and assumptions. Participants also received a copy of the draft framework to review, while representatives of the TWG were present to lead discussion and to ensure that the feedback from the RWG would be communicated to other TWG members. RWG participants were tasked to provide feedback on the draft monitoring framework with emphasis on evaluating regional applicability; identifying likely challenges to implementation and offering potential strategies for overcoming them; becoming a partner or helping to identify partners to pilot test monitoring; and joining the statewide network of practitioners, researchers, agencies, and non-profit organizations to support the project goal. The workshops themselves consisted of multiple rounds of feedback and input on the goals and assumptions of the framework; the proposed parameters, indicators, and protocols; and the creation of a monitoring network. Multiple ways of soliciting feedback from participants were utilized to ensure that all voices were considered in prioritizing indicators and developing protocols: plenary discussion with time for questions, small break-out groups with facilitators, voting exercises in which participants were free to distribute 20 dots per round to the parameters and indicators they wanted to prioritize (Table 1), and written feedback forms.
The third engagement type was a "permit reviewer meeting," which was organized in a virtual format to accommodate the geographical spread of participants from the United States Army Corps of Engineers, New York State Department of State, and New York State Department of Environmental Conservation, as well as municipal agencies involved in permitting. During two 1.5- Table 1. Prioritization of resilience service indicators across four stakeholder regions. This table details the outcomes of a dot voting exercise conducted at each of the regional work groups. Each participant was given 20 dots that could be used to indicate support for (+) or against (-) each performance parameter across all three resilience areas: ecological function, structural integrity and hazard mitigation, and socioeconomic outcomes. The top 10 parameters with the most votes for or against it within each regional work group are marked with an X. hour meetings, permit reviewers were asked for feedback and input on the content, relevance, and utility of the framework from a permitting perspective through voting exercises using an online, real-time voting tool and discussion. In the first meeting, indicators from the draft frameworks for each of the resilience services created by TWG were prioritized and then ranked for feasibility, with participants allocating 100 points between the various indicators proposed in both rounds, after which the results were deliberated and regional differences were considered. The TWG incorporated feedback from this round and the RWG to narrow the list of indicators. In the second meeting with permit reviewers, they were asked to rank on a scale of 0 to 5 (strongly disagree to strongly agree) whether the selected indicators were useful and reasonable to ask for in a simple monitoring plan and whether they would be willing to recommend collection of the data in a permit.

Phase 4
Phase four consisted of the modification of parameters, indicators, and protocols based on the feedback and input from the PAC, RWG, and permit reviewers. The core team with TWG leads collected and organized the feedback and input and communicated this to the full TWG. The TWG then incorporated the feedback to create a final framework. This effort started with a workshop bringing together TWG members to discuss the feedback and turn this into modification tasks, which were subsequently executed by TWG members. Although the input from phase 3 was crucial in this phase, TWG members did not simply execute requests for modifications, but instead deliberated how these modifications impacted the scientific soundness and actual measurement, before revising the framework and monitoring protocols for field testing.

Phase 5
The fifth and final phase of framework development consisted of testing the monitoring framework in the summer of 2019. The goal of this piloting phase was to test whether both desktop and field-based protocols were user friendly, complete, and clear in their application. In addition, the data gathered would establish a baseline for each of the pilot sites. The testing was done on four sites in each of the four regions (16 in total) that included one hardened, one natural, and two nature-based shoreline features for each region. The sites were selected by the core team on the basis of suggestions made through engagement with the PAC, RWG, and permit review meetings. Field monitoring was done by select members from the core team, student research assistants and local volunteers. The field teams collected pictures and detailed notes about how the protocols functioned in the field. Lessons learned were shared with the core team and TWG and used to refine the protocols. Figure 5 depicts the sectoral background of the individuals participating in the co-creative effort in one or more of the phases outlined above.

Fig. 5.
Background of co-creators operationalizing resilience for shorelines in New York State. A count of representatives (n = 155) is provided to depict the diversity of affiliations of cocreators. Numbers on top of bars represent the number of cocreators representing an affiliation type. The Federal-State category listed includes organizations/programs that are jointly funded or administered by a federal and a state agency.

Co-creation construes the meaning and measurement of resilience iteratively
The co-creative and interdisciplinary process for developing the monitoring framework shaped the selection and substance of monitoring parameters, indicators, and protocols in profound ways. The list of potential resilience indicators for NYS shorelines is long but over the months was refined and prioritized. Here, we center our analysis on each group of participants outlined in the prior section and how they contributed in distinct and different ways to operationalizing resilience and influencing the structure and content of the framework.
Core team: providing structure for the co-creative process The first group of influencers was the core team that led the process and organization of the framework. Throughout the development of the framework, the core team played a key role in documenting, developing, and shaping its structure and organizing participation in its creation. This process took place over multiple years and the team ensured that institutional knowledge and documentation of the many decisions was retained, including macro-level decisions that affected the structure of the framework. For example, from the document analysis, a goal-oriented framework (as opposed to a typologyoriented framework) was chosen to allow for comparisons across all shoreline types. After deliberation, the decision was made to formulate goals broadly as resilience services, rather than more specific goals for particular indicators, because specific goals for shoreline projects are contextual and vary from project to project. Although there was agreement among TWG and RWG on what was important to monitor (e.g., shoreline elevation, real estate value), there was no consensus on the particular value associated with each (e.g., whether an increase, decrease, or no change was the goal in the context of resilience). Rather than deciding a priori on values by including detailed goals in the framework, or circumventing the issue by leaving particular indicators out, the inclusion of the indicators in the framework encourages further deliberation in the contextual application of the framework. The core team also made numerous micro-level decisions about organizing and synthesizing the list of performance parameters, indicators, and protocols that were presented at workshops and meetings; the format and the level of detail included in protocols; and many other aspects of the framework. These small decisions were the result of diligent documentation, synthesis, and discussions of input from different groups and from published literature. They allowed for productive conversations at workshops and meetings and ensured that feedback from all groups was incorporated into the framework. The core team thus shaped the operationalization of resilience by positing resilience as being about ecological function, structural integrity/hazard mitigation, and socioeconomic outcomes at once, acknowledging possible trade-offs between these themes, while arguing that the details of what resilience means should be derived from context through deliberation.
Project advisory committee, regional working groups, and permit reviewers: providing local and policy-relevant expertise A second group of influencers integrated local and policy priorities into the framework by evaluating and suggesting modifications to parameters, indicators, and monitoring protocols in line with these stakeholders' preferences and professional commitments. This group consisted of the RWG, permit reviewers, and the PAC. Throughout the process, there was an expansive list of possible parameters and indicators that could be monitored and protocols that could be used. Participants agreed that the list of parameters needed to be limited to make use of the framework feasible. Thus, one of the most important functions of these influencers was to help narrow and prioritize the list of parameters, indicators, and protocols to include the most relevant and feasible for monitoring shoreline features in New York State. In general, RWG and permit reviewers agreed on the most useful indicators. Vegetation cover was the top ecological function indicator for both groups although the RWG rated benthic species richness highly as well. Change in shoreline position, visible signs of scour/erosion, and change in water elevation were consistently rated as the top structural integrity/ hazard mitigation indicators. The top three socioeconomic outcome indicators were the number of households exposed to flooding, public/critical facilities protected by the shoreline feature, and the monetary value of recreation and tourism surrounding the shoreline feature. Although the same indicators were consistently considered crucial, RWG urged the core group and TWG to consider the inclusion of previously omitted indicators such as shoreline access. The PAC reviewed the general structure of the framework during early drafts as well as the final list of indicators and generally agreed on both the structure and content of the final framework.
This group of influencers consistently expressed concerns over the capacity and funding to implement the monitoring. Thus simplicity of execution by a broad range of user groups became a key factor in indicator selection and protocol development. For example, carbon value was excluded because monitoring carbon storage and sequestration requires access to laboratory equipment that may not be available to many.
Ecology and Society 26(3): 10 https://www.ecologyandsociety.org/vol26/iss3/art10/ The RWG also provided important feedback based on characteristics of each region. For example, the framework needed to be relevant to unique riverine and lacustrine shoreline systems of The Great Lakes region. In the Hudson River Valley and New York City, many shoreline monitoring projects are already underway and thus existing indicators and protocols could be leveraged where possible. In Long Island and New York City, addressing a diverse range of site-specific goals in the framework was recommended to increase local buy-in. This is especially relevant to privately owned sites and those with complicated governance structures, where permissions to monitor shoreline features are required from multiple governmental agencies and/or private persons or organizations.
Permit reviewers approached the framework from a regulatory perspective. They rated ecological function and structural integrity/hazard mitigation indicators as more feasible and relevant to monitor than socioeconomic ones and quantitative indicators as more feasible to monitor than qualitative ones. Their perspective was influenced by an existing practice by permit holders of collecting quantitative, biophysical data on shoreline projects to comply with federal, state, and/or local environmental protection regulations. Although permit reviewers agreed that socioeconomic outcomes are important to monitor, few regulatory mechanisms exist that require permit holders to monitor the socioeconomic impacts of their projects. In addition, permit reviewers noted that this type of monitoring typically occurs only for larger scale projects and is often done in collaboration with researchers rather than by permit holders alone because of the need to monitor a system of sites to adequately assess socioeconomic impacts. There was consistent agreement across all groups regarding the importance of including socioeconomic outcomes in our framework.
In sum, this second group advanced an understanding of the concept of resilience in a practical way: while the universe of what resilience could mean is expansive, prioritization is necessary if actual monitoring of the performance of shoreline features is to happen. This prioritization was shaped by a pragmatic approach geared toward a simplicity of execution, while emphasizing the ongoing nature of the monitoring itself.

Technical working groups: providing scientific, engineering, and other technical expertise
The third group of influencers, the TWG, crucially integrated scientific preferences into the framework, ensuring that the framework was scientifically rigorous, relevant to management, and that measurements were relevant to the indicators. The integration of scientific knowledge in monitoring efforts is both critical and challenging, with calls to scientists to engage in transdisciplinary and two-way interactions to jointly develop research and policy increasing (Young et al. 2014, Rose 2015. The TWG, representing both scientific practice and industry practice, iteratively worked on integrating scientific knowledge and technical practice to the framework in conversation with other stakeholders. The criteria TWG used to identify, rank, and evaluate parameters and indicators and create monitoring protocols were largely in line with those outlined by Kershner et al. (2011) and Biedenweg et al. (2017). For example, TWG made sure the parameters and indicators were theoretically sound, responsive and sensitive to changes, concrete, operationally simple, with a broad spatial coverage, consistently measurable, and focused on detecting variation.
The TWG, each led by a member of the core team, drew on existing rapid assessment protocols to avoid "reinventing the wheel" and to take advantage of the experience of earlier efforts to develop effective protocols. New performance parameters, indicators, and protocols were continually added to the framework, reflecting the diverse interests and expertise of all participants. By the middle of Phase 3, the framework had grown to 20 performance parameters, over 80 indicators, and over 60 potential protocols. The TWG narrowed this down to 11 performance parameters, 19 indicators, and 17 protocols based on their expertise and feedback from RWG, permit reviewers, and PAC (Fig. 6). Similar or redundant performance parameters and indicators were merged, and many were excluded because of their low feasibility of implementation, low applicability across a variety of shoreline features and/or geographic areas, and/or too indirectly related to the presence of a shoreline feature. For example, because ice impacts were significant in the Great Lakes and Hudson regions and not others, it was determined that distinct protocols for detecting ice scour were not needed, and protocols for documenting erosion and scour more broadly were sufficient. The presence of toxins and contaminants was excluded because it was more likely to be influenced by broader factors than relatively small-scale shoreline features, and any change would be difficult to link directly to a shoreline feature. Some parameters were added as well. For example, two socioeconomic outcome parameters, recreational and cultural use and environmental justice, were added to the framework to address concerns of access and inequality that had been expressed by participants during RWG. Because feasibility played an important role in parameter selection, these parameters were added after rapid protocols were identified (for recreational and cultural use) or created (for environmental justice) by the socioeconomic outcomes TWG that could be implemented to assess these parameters at the feature scale. In addition, the majority of the protocols selected for the final framework were ones that the TWG's had implemented in previous projects elsewhere, so there was confidence that parameters could be easily monitored. Suites of indicators that could be monitored using a single protocol were also prioritized over indicators that required multiple, separate protocols. For example, multiple indicators selected for structural integrity/ hazard mitigation could be monitored using one protocol.
In this influencer group, the operationalization of resilience meant a further prioritization from its possible meanings to a smaller set of measurable units. On the one hand, resilience thus was narrowed by the focus on using existing protocols and the desire to measure multiple items with one action as leading. On the other hand, resilience was expanded when TWG found ways to concretize more abstract concerns in measurement protocols.
Field monitoring staff: providing expertise on implementation A fourth and final group consisted of people piloting the monitoring protocols to evaluate the feasibility of applying the framework in practice. After pilot monitoring was conducted in all four regions, the performance parameters and indicators remained unchanged, because most protocols could be implemented across all site types and regions given modest resources. However, this group identified important considerations Ecology and Society 26(3): 10 https://www.ecologyandsociety.org/vol26/iss3/art10/ Fig. 6. Comparison of draft parameters and field-tested parameters.
on how certain indicators should be interpreted. Execution of some protocols and quality of some data, such as plant richness, varied depending on field expertise or staff time available. Others, such as the business activity index and the number of households and public facilities exposed to flooding/erosion, were difficult to implement because variations in population density made it difficult to define spatial boundaries for data collection. As a result, further refinements to these protocols were needed in order to collect data in a manner that would be comparable across sites. In addition, many of the socioeconomic protocols in the framework were adapted from protocols that were originally designed to assess indicators at larger scales (e.g., to assess the use and value of public open space in general, to assess overall community resilience) and needed explaining. Thus, the protocols evolved based on experience with their application. This highlights the need for a core group or organization to refine the indicators and protocols, analyze data, and interpret results from the pilot monitoring, and other tasks to ensure the framework can effectively evaluate shoreline features and is widely implemented across New York State. Thus, the testing done by this group did not change the content of the monitoring framework, but showed that the process to execute measurement needed adaptations to work along the desired principles of ease of execution and valid comparisons. The field-tested framework is shown in its entirety in Figures 7, 8, and 9.
Although some parameter, indicator, and protocol measurement preferences varied between people from different regions and people from different professional backgrounds (e.g., science, industry, policy, communities), the final framework was able to combine the priorities of all groups through the iterative nature of the process through which parameters, indicators, and protocols were identified, modified, and prioritized in ways that balanced credibility, legitimacy, and relevancy.

Five characteristics and challenges to operationalizing shoreline resilience
We reflect here on how the process of developing the monitoring framework affected the content and form of the framework, focusing on five challenging characteristics: interdisciplinary work; co-creation; pragmatic orientation; multi-scalar applicability; and policy implications. We argue that the iterative nature of cocreation, in which top-down theoretical input and bottom-up applied contributions were in conversation, generates a radically different output than any of the individuals or groups by themselves could have developed. However, the co-creation process comes with a distinct set of challenges that are inherent and need to be reflexively engaged with in order to gauge the legitimacy, salience, robustness, and effectiveness of the outcomes of the co-creative process. Based on lessons learned from the process and its challenges, we also offer recommendations for next steps.
The first challenge we highlight is on interdisciplinary work. The integration of science in general and social science more Ecology and Society 26(3): 10 https://www.ecologyandsociety.org/vol26/iss3/art10/ Fig. 7. Field-tested performance parameters and indicators for ecological function.
specifically into natural resources policy and management decisions has been argued to be crucial, including in indicator development (Biedenweg et al. 2014. Whereas biophysical knowledge tends to be well represented in governmental agencies and departments tasked with shoreline management, social science is less so although its insights can facilitate planning in ways that match local context and are socially acceptable (Nursey-Bray 2011, Bennett et al. 2017). To do interdisciplinary work effectively requires a problem-based approach and codevelopment of research objectives, methods, and outputs (Campbell 2005, Viseu 2015, Bennett et al. 2017). Yet the integration of natural and social science is challenging because of disciplinary silos and an unwillingness or inability to engage with different epistemologies and methods (Eigenbrode et al. 2007, Evely et al. 2008. Collaborations can fail if meaningful interactions are not facilitated and managed (Walley 2002, Wyborn 2015.
To bridge the gap between sectors and disciplines, the interdisciplinary core team spent significant time early in the process carefully defining terms and objectives to create a common working language for collaboration, discussion, and reflection between participants. Later in the process, care was taken to translate input into consistent and clear language understandable across sectors and disciplines. The process of translation extended to recalibrating technical matters, including determining appropriate assumptions for the monitoring effort, a necessary yet not always anticipated step. For example, defining assumptions about knowledge and experience of those monitoring early on would have minimized revising indicators and protocols further along the process.
The commitment to interdisciplinarity also meant an insistence on the inclusion of socioeconomic indicators alongside biophysical ones, which brought unique challenges. With limited tested examples from other monitoring efforts, operationalizing priorities like "well-being" into measurable indicators required more time and deliberation than some of the biophysical indicators. Determining the appropriate scalar level of analysis was another hurdle. For example, the risk of property damage to households and public facilities was regarded as both important and difficult to monitor because of questions of correlation and causation of socioeconomic outcomes and specific shoreline features. Differing ontological, epistemological, and ethical commitments (or starting points) across disciplines here led to contrasting interpretations of what and how to monitor the resilience of shorelines (Wijsman and Feagan 2019); yet these foundational differences were hard to address given the separation of TWG by disciplinary areas (ecological, technical, social) and the foundational assumption of epistemological commensurability, which especially in the highly diverse social sciences is not a given. Overall, the effort required constant interpretation, translation, and communication across disciplinary and sectoral boundaries, necessitating interdisciplinary teams with staff expertise in coordination and time and resources to spend toward these activities. Interdisciplinary collaboration thus requires a measure of flexibility in terms of allocating resources because not all challenges can be anticipated, yet the organization of work plans, funding, and personnel does not always allow for this. It also requires a reflexive engagement with ontological and epistemological questions, while acknowledging that these issues cannot always be resolved in ways satisfactory to all involved. Because foundational questions of ontology and epistemology ultimately shape the outcomes of the work and thus the interventions proposed, we posit that it is important to dedicate time and resources to such deliberations and report them clearly in interdisciplinary co-creative processes. Second, the evolution of the framework and details of the process of its creation were influenced by the commitment to co-creation and regional collaboration in critical ways. Working with people with different professional expectations and commitments means negotiating between scientific rigor, public accessibility, and individual and communal relevance. Throughout the project's lifespan pros and cons of various approaches were continually discussed, re-visited, and weighed. As such, the final framework reflects the outcome of a lengthy negotiation process focused on balancing comprehensiveness and pragmatism (see below). Over the course of deliberation a breadth of participants made sure that varied perspectives were carefully considered, providing a sense of confidence in the eventual prioritization of indicators and criteria for protocols. The process centered around deliberation and discussion to establish broad support and applicability; whether this is achieved can only be established in the future by following the afterlife of the produced framework. Such follow-up can give important insight into the functioning of co-creative processes, given that deliberation does not automatically lead to consensus, support, and future uptake. Although the core team explicitly tried to create open and varied ways for participants to express their positions, managing the dynamics in physical and virtual rooms to work toward respectful discussion, whether this successfully translated into the quality of participation is uncertain. For example, it is difficult to gauge whether people are willing and able to dissent in front of others, whether a participant's approval extends to outside of the deliberation spaces, and if participants advocate for the framework. The space of participation is not neutral (Sprain 2016) and can sometimes be hierarchical and exclusive rather than accommodating and inclusive (Cooke and Kothari 2001).
Although some of the input given was not taken up by subsequent groups after deliberation, no particular group or individual had sole decision-making power. Nevertheless the core group made some executive decisions when consensus or additional deliberation was unattainable, for example because of project time lines or incommensurable positions. We found that for a cocreative effort like this, in which many people invest a little bit of time, it is critical to have a smaller group of people to synthesize and take editorial responsibility in order to keep moving the project ahead. In this regard, we observed that some members of TWG and the core team contributed substantially more to certain parts of the project than other members; the likely result of schedule clashes, personal and professional interests and stakes, commitments outside of this project, and structural incentives and obstacles (e.g., seniority, gender, organizational culture). It is also due to budgets not matching high ambitions, leading to individuals and organizations investing their time and energy pro Fig. 9. Field-tested performance parameters and indicators for socioeconomic outcomes.
bono because of a belief in the importance of the project, which may exclude certain participation. This points to a paradox of co-creative work: the process is meant to encourage ownership and active participation, yet the presence of others can lead to a less active stance, particularly when an individual's or group's responsibilities are interpreted in differing and contradictory ways.
In addition, because the co-creative process requires heavy investment of time and resources that is often unpaid and/or not fully acknowledged in time lines, this poses a challenge for smaller organizations or for individuals from disenfranchised communities (e.g., racial and ethnic minorities) to participate, which is a problem in environmental work more broadly speaking (Finney 2014, Taylor et al. 2019. Acknowledging this is critical to a more comprehensive understanding of co-creative processes beyond the ideal of participation, toward how it works in practice and when it encounters outside limitations. This is especially important in order to acknowledge and give meaning to those strikingly absent. In this particular instance, professionalized non-and for profit organizations and governmental staff worked closely with academic researchers, leading to an overrepresentation of highly educated and white people compared to the region. One of the noticeable absences not only in the room but thus in the framework and meaning of resilience is that of Indigenous communities, even though various nations exist in each of the four NYS regions on or near bodies of water. Although leadership from Indigenous communities was invited to participate in RWG, the outreach was not specifically tailored to recognize that Indigenous peoples have expressed skepticism and research fatigue (Blair 2016) and the approach was not explicitly designed to make it more relevant, reciprocal, respectful, and responsible to Indigenous communities (Peltier 2018). Although the subsequent exclusion of Indigenous worldviews and priorities in shoreline resilience was not intentional, it is hardly surprising and indeed problematic. Omissions like these create a need for fundamental changes in how environmental management projects are designed and executed. More generally, a key issue is what is the right quality and quantity of participation in the co-creation process (Hugel and Davies 2020).
A pragmatic orientation is a third characteristic and challenge to the process described here, expressed at different levels. On a foundational level, the choice to think about and operationalize https://www.ecologyandsociety.org/vol26/iss3/art10/ resilience through the notion of services was prompted by the familiarity and prominence of the ecosystem services concept in all sectors dealing with shoreline management questions. As a result, participants quickly embraced the idea of resilience services, avoiding fundamental discussions on the allowances and exclusions of the metaphor of a service. Although this brought uniformity to an interdisciplinary framework and allowed for efficiency in the operationalization process, critics will argue that it also means that the conceptualization of nature-society relations and thus the human-centeredness of the framework went unchallenged.
On a more applied level, participants throughout the process emphasized the importance of easy implementation of the monitoring framework because costly, time consuming, and complex monitoring would hamper the framework's wide-scale uptake to generate and share monitoring data. Pragmatic concerns, including simplicity and cost-effectiveness, thus led to the development of relatively simple, "low-tech" monitoring protocols and the prioritization of indicators that could be assessed with such protocols. Without these pragmatic concerns it is likely that indicators requiring more extensive or hypothetical data inputs and more complex data collection protocols would have been included in the framework, in line with some of the existing monitoring frameworks examined during the document analysis. Many of these were highly technical and, although extensive evaluative procedures were spelled out, these rarely included protocols for data collection. The reports that did offer less intensive methods did so only in pairing with a more ideal form of intensive monitoring, a distinction that can complicate data comparability across geography and time. Instead, uniform, simple, and cost-effective protocols were developed. Although few indicators were excluded because of these reasons, protocols were less exhaustive as they otherwise would have been. For example, instead of instructions on how to plot an environmental justice map using geographic information system software and data from the census and American community survey, a web-based map was developed for the entire state so that the monitoring task is one of a mere look up. Ease of use was thus prioritized over use of the most recent data, because the alternative of asking monitors to plot their own maps was seen as time consuming and requiring skills and technology not accessible to everyone. As a result however, the current protocol gives a static depiction of the environmental justice situation, and in the future requires updating to prevent it from becoming dated. Based on the field testing through pilot monitoring we conclude that the protocols can generate data that is sufficiently robust to enable comparative analysis, while they are simple enough to be used by a wide spectrum of stakeholders. Pilot monitoring did reveal some considerations for improvement of the protocol to ease aggregating and post-processing of data collected. Overall, practicality importantly shaped the development and prioritization of indicators and protocols, to facilitate wide adoption of the framework.
Fourth, multi-scalar applicability shaped the process of operationalization in profound ways. The key indicators and metrics developed are broadly applicable across the diverse regions of the state. This makes a more systems-based approach to monitoring beyond individual sites possible, through aggregation and comparison of data gathered across locales. Such a regional perspective takes seriously that certain biophysical and social processes are not bound by jurisdictional borders of municipalities, and that the performance of shoreline features is impacted by and impacts processes outside of the specific shoreline sites. Nevertheless, the ambition to address shoreline resilience from a systems perspective requires site-specific measurement. When developing and prioritizing the indicators and protocols geared toward straightforward data collection, some indicators addressing systems performance were eliminated. For example, water quality was excluded from the framework. Participants argued that factors broader than relatively smallscale shoreline features where the monitoring would take place are of consequence, and any change to water quality would be difficult to link directly to the design or management of a particular shoreline feature. This conundrum of correlation or causality between the shoreline being monitored and the indicator itself similarly became an issue in determining socioeconomic indicators, with some advocating for exclusion of indicators because of a perceived lack of explanatory power and others valuing its descriptive possibilities regardless. The desire to understand both a locality and to understand the shoreline system more broadly were occasionally at odds. In a pragmatic move, it was decided to include some and exclude other indicators, and work toward collection and data analysis over time to further assess these relationships. Still, the need to aggregate data over multiple shorelines to better understand larger scale processes, such as flood protection, was seen as a future ambition. The puzzle to resolve is how to go beyond site-specific monitoring alone toward widespread and possibly coordinated monitoring to get at the systems level.
Fifth and finally, policy relevance was a crucial characteristic and challenge to the process. Although this effort acknowledges the value of monitoring with an eye toward biophysical systems, it also acknowledges that jurisdictional borders matter for the management and functioning of shoreline features, for example through policy priorities and permit requirements, by taking NYS as a focus. Although cities and smaller municipalities are undoubtedly a crucial geographical scale to plan and implement NbS, the intermediary scale of the state might prevent misalignment between (local) governance and policy and (landscape) environmental systems (Sayles 2018) by providing policy guidance on an intermediary level between federal and local requirements. The comparative character of the monitoring framework in particular can inform policy decisions, for example, by expanding the types of resilience services shoreline managers are encouraged to consider. Because communities in NYS can be impacted by shoreline use decisions made with support of the framework, it is important to identify and consider differential impacts on socio-demographic groups (Dumitru et al. 2020). In that regard, the fact that many potential indicators were discarded and only a few remain in the final framework leads to a somewhat narrowly focused data collection that impacts eventual decision making. Though the process aimed to incorporate local priorities and benefited from input and deliberation throughout, and was made as transparent as possible, participation was still in some ways selective and whether the process and products are seen as legitimate remains to be answered. That being said, the indicators themselves are not prescriptive toward goals because they do not assign weight or directional (good versus bad) values, thus leaving https://www.ecologyandsociety.org/vol26/iss3/art10/ shoreline managers and decision makers space to set their own priorities. The ambition to create an open-access database that stores collected data would allow experts and decision makers to analyze the data in ways that are consistent with community goals. For example, combining quantitative socioeconomic data with qualitative perceptions of shoreline benefits could inform holistic and proactive decisions about shoreline interventions that consider environmental justice and community well-being.

CONCLUSION
The meaning of resilience is the result of a social process of deliberation. Combining the management goal to operationalize resilience to evaluate the performance of NNBF in comparison to hard structural features with the query of how a process of cocreation matters for resilience operationalization, we showed that the iterative nature of co-creation generates the meaning of resilience in context. With our NYS case study, we laid out an iterative process to develop a monitoring framework for shoreline performance and showed that this process generates a radically different framework than any of the individuals or groups involved by themselves could have developed. Using the notion of resilience service, this framework brings together ecological considerations with technical and socioeconomic ones, recognizing that shorelines function in multiple ways and that the prioritization of one function over another reflects preferences and thus requires explicit deliberation.
The commitment to interdisciplinary work, a process of cocreation, pragmatic and tested protocols, multi-scalar applicability, and policy relevance shaped both the process and the outcome of the framework. We argue that engaging these principles is necessary to make monitoring efforts relevant on both local and regional levels, and to enable data collection and comparative analysis on shoreline performance necessary for policy decisions. Time will tell if this proves true, that people use it and we are able to generate the longitudinal data necessary to provide the desired data to inform design, management, and policy for shorelines state-wide. In this regard fostering the network of individuals and organizations involved in monitoring shorelines, both technical experts and practical monitors, will be critical in the following phases of data collection, management, analysis, and interpretation. We understand these future phases as possibly further informing the framework through user feedback, and thus conceptualize the framework as a living framework. By following the afterlife of the framework we will be able to assess whether the deliberative nature of our process resulted in a balanced and useful compromise between rigor and practicality, social and biophysical aspects, and local and systems relevance, and thus whether compromises reached in participatory settings can contribute in meaningful ways to ambitious resilience and nature-based solutions projects. We posit the continued deliberation as a strength of this process, acknowledging how the value and meaning of resilience is created through social processes and may change over space and time.
Responses to this article can be read online at: https://www.ecologyandsociety.org/issues/responses. php/12182