Ecology and SocietyEcology and Society
 E&S Home > Vol. 24, No. 4 > Art. 11
The following is the established format for referencing this article:
Stanton, J. C., J. Marek, L. S. Hall, B. E. Kus, A. Alvarado, B. K. Orr, E. Morrissette, L. Riege, and W. E. Thogmartin. 2019. Recovery planning in a dynamic system: integrating uncertainty into a decision support tool for an endangered songbird. Ecology and Society 24(4):11.
https://doi.org/10.5751/ES-11169-240411
Research

Recovery planning in a dynamic system: integrating uncertainty into a decision support tool for an endangered songbird

1U.S. Geological Survey, Upper Midwest Environmental Sciences Center, La Crosse, Wisconsin, 2U.S. Fish and Wildlife Service, Ventura, California, 3Western Foundation of Vertebrate Zoology, Camarillo, California, 4U.S. Geological Survey, Western Ecological Research Center, San Diego, California, 5California State University Channel Islands, Camarillo, California, 6Stillwater Sciences, Berkeley, California, 7Nature Conservancy, Ventura, California

ABSTRACT

Along the Santa Clara River in California, populations of the federally and state-listed Least Bell's Vireo (Vireo bellii pusillus) are recovering from near extirpation. Habitat protection and restoration, as well as controlling rates of brood parasitism, are thought to be the primary drivers of this recovery. Continuing successful management of this population faces multiple challenges due to the highly dynamic and unpredictable nature of the system, lack of clearly defined and measurable recovery criteria, parametric and stochastic uncertainty, and data limitations. Many of these management challenges are not unique to Least Bell's Vireo and require careful balancing of limited resources into the future. We developed a decision support tool as a user interface for exploring the underlying uncertainty in a population viability analysis under an array of different management scenarios. The tool was designed to assist with the planning and coordination between conservation partners in the region in three distinct aspects of the decision-making process: defining the problem and setting clear goals and objectives, exploring the consequences of potential alternative actions, and identifying criteria for ongoing evaluation and monitoring. The general framework for the design of this decision support tool is broadly applicable to many management and decision-making scenarios that share these common challenges.
Key words: Arundo donax; Brown-headed Cowbird; conservation; endangered species; Least Bell's Vireo; population viability analysis; recovery planning; uncertainty; Vireo bellii pusillus,

INTRODUCTION

Decision making for the conservation and recovery of threatened and endangered species is often a complicated and high-stakes endeavor (Bottrill et al. 2008, Gregory and Long 2009, Joseph et al. 2009, Wilson et al. 2011) in which poor decisions can result in long-term damage to fragile populations or irreplaceable loss of biodiversity. Despite these risks, decision making for species conservation is often done in resource- and data-limited environments (Bottrill et al. 2008, Joseph et al. 2009). For these reasons, approaching species management with a thoughtful framework and tools to aid in decision making is advantageous.

One common approach to evaluating the possible outcomes of alternative management actions on a population of conservation concern is to use population viability analysis (PVA; Gilpin and Soule 1986, Boyce 1992, Morris and Doak 2002). Population viability analysis is used to make predictions about the viability of species or populations under current conditions or under possible future conditions, such as those resulting from proposed management actions. There are many different types of modeling approaches for conducting a PVA but typically the approaches involve an integration of the intrinsic traits of the species with extrinsic drivers that may alter or impact the species viability (Stanton and Akçakaya 2013).

An important aspect of PVA is the treatment of uncertainty. Uncertainty can derive from environmental stochasticity, variability in individual demographic rates, lack of data, measurement error, observer bias, or model uncertainty (Regan et al. 2002, Gregory and Long 2009, McGowan et al. 2011). Further, even when accurate and unbiased measurements of past conditions are available, they may not be predictive of future states under changing environmental drivers and conditions (Peterson et al. 2003). Ignoring or improperly accounting for uncertainty can lead to wasted resources on unnecessary conservation actions based on type I errors (falsely detecting a population decline when there is none), prescribing ineffective conservation measures because the true cause of decline was misunderstood, or focusing too much attention on threats or drivers that are of little importance to the long-term viability of a population (Milner-Gulland and Shea 2017).

However, delaying conservation action in favor of reducing uncertainty by gathering more data can have negative consequences as well. The decision to delay action to allocate time and resources to improve knowledge and reduce uncertainty has an opportunity cost if those resources could have been used directly for conservation actions (Grantham et al. 2009, Canessa et al. 2015).

We developed a PVA-based decision support tool to assist in coordinating management actions for Least Bell’s Vireo (Vireo bellii pusillus), which breed in the riparian zone of the Santa Clara River in southern California. Uncertainty has been a significant challenge for making management decisions for this population. The Multiple Objective Vireo Explorer (MOViE; https://www.usgs.gov/apps/MOViE) tool was designed to allow the user to consider multiple types of uncertainty, such as lack of clearly defined and measurable recovery criteria, parametric and stochastic uncertainty, and data limitations (Carroll et al. 2006, Earl et al. 2018) in the decision-making process. The MOViE tool is a dynamic interface to a PVA that gives the user the opportunity to explore model output sensitivity to uncertain model parameters while accomplishing three common objectives in the decision-making process (Gregory and Keeney 2002, Gregory and Long 2009, Runge 2011, Johnson et al. 2015): defining measurable population goals, exploring the potential consequences of alternative management actions, and identifying criteria for monitoring of action outcomes (see Box 1 for details).

We aim to provide a useful template for others needing to make management decisions in the face of uncertainty. The MOViE tool and the underlying PVA model were built for the management of a specific population. However, our approach provides a general framework for addressing challenges that are common to the management of many threatened and endangered species.

Box 1:

Decision making with the Multiple Objective Vireo Explorer (MOViE) tool

Defining measurable population goals

Without clearly defined and measurable goals, it is difficult to make management decisions (Johnson et al. 2015). For species listed as threatened or endangered under the Endangered Species Act (ESA), recovery plans are required to contain measurable criteria that, when met, could initiate a review process to determine whether a species should be delisted or downlisted. In practice, however, even when recovery plans do contain quantifiable recovery criteria, they only rarely provide guidance on the amount or configuration of required habitat or probabilistic assessments of persistence over time (Neel et al. 2012). In addition, species recovery plans often lack measurable objectives clearly stipulating explicit spatial and temporal bounds for assessment (Himes Boor 2014). This deficiency can mean that management objectives at the scale of subpopulations can be difficult to interpret from the recovery plan.

The Least Bell’s Vireo draft recovery plan stipulates that the Santa Clara River should support several hundred pairs in a self-sustaining population that could also serve as a source to bolster populations to the north in the Santa Ynez and Salinas Valleys of California, and for recolonization of extirpated populations in the Central Valley (U.S. Fish and Wildlife Service 1998). However, the plan does not specify a timeline for achievement of recovery goals nor minimum population thresholds that could signal the need for additional conservation efforts.

The MOViE tool, and the underlying population model and analysis, address this objective by exploring multiple possible model results analogous to measurable future population-level consequences such as mean population size, minimum population size, and population growth rate. The population model results are summarized in terms of the probability of meeting specified goals. The MOViE tool allows the user to explore these different population-level model results by selecting separate tabs along the top of the user interface. The MOViE tool also allows the user to explore model results at different threshold values that automatically update the probability values.

Exploring alternative management actions

The task for managing vireos on the Santa Clara River is deciding on the optimal balance between the restoration and maintenance of habitat, and effort invested to control the impact of Brown-headed Cowbird (Molothrus ater) parasitism on vireo fecundity. Both habitat restoration and cowbird control have proven to be effective actions for the conservation of birds in general (Sutherland et al. 2018), as well as for the recovery of the Least Bell’s Vireo (Kus 1998, 1999, Kus and Whitfield 2005). However, both activities are labor-intensive and financially costly.

The challenge is to find the right balance of activities to restore and maintain the population of Least Bell’s Vireo with limited resources. However, uncertainty about many aspects of the habitat dynamics and life history of the vireos at this site has an impact on the ability to make precise predictions about effects of management on the population. Our approach was to simulate the population across broad ranges of parameter uncertainty. Model results are summarized by the probability that a given combination of management actions was likely to meet a stated management goal despite broad uncertainty.

The MOViE tool allows the user to explore parameter uncertainty for each combination of management actions by providing range sliders on most model parameters. The range sliders filter model results to user-specified ranges and automatically update the model summaries.

Evaluation and monitoring

Deviations from the appropriate course of management action are inefficiencies that can cost time and resources. Setting up a monitoring regime specifically designed to address sources of uncertainty that are most likely to impede progress toward goals can help guide future management decisions (Keith et al. 2011). A monitoring plan may be constructed to proceed alongside management actions while learning about the system, thus paving the way toward development of an adaptive management program (Williams et al. 2009, Runge 2011, Conroy et al. 2011). Alternately, monitoring regimes might be established to act as a trigger to alert managers to potential problems and needed actions.

The Least Bell’s Vireo could be considered a conservation-reliant species (sensu Goble et al. 2012) because current habitat management and cowbird control are expected to be necessary into the future. To understand the efficacy of those actions, ongoing monitoring would be beneficial.

The MOViE tool summarizes model results such that once the population goals and thresholds are determined, habitat restoration and fecundity (as achieved through cowbird control) can be used to provide the relevant insight necessary for shaping future monitoring actions around key uncertainties and information gaps, thus informing adaptation of management strategies through time.

METHODS

Study system: Least Bell’s Vireo on the Santa Clara River

Least Bell's Vireo, one of four subspecies of the more abundant and widespread Bell’s Vireo (Vireo bellii), was once common in riparian corridors through much of California and northern Baja California. The subspecies declined rapidly through the latter half of the twentieth century because of loss and degradation of habitat and the expanding range of the Brown-headed Cowbird (Molothrus ater), a brood parasite. The Least Bell’s Vireo was listed as endangered by the State of California in 1980 and federally listed in the United States under the Endangered Species Act (ESA) in 1986. At the time of its listing under ESA, the subspecies was reduced to only a few hundred pairs in the southern portion of its former range and was nearly extirpated from riparian areas along the Santa Clara River, 1 of 14 subpopulation units identified for recovery (U.S. Fish and Wildlife Service 1998).

The riparian vegetation that the vireo depends on for nesting is highly dynamic. Least Bell’s Vireo relies on early successional stands of native vegetation such as willow (Salix spp.) or mule-fat (Baccharis salicifolia). The amount and quality of habitat at any given place or time depends on multiple factors such as the physical configuration of the river, timing and severity of flooding events and the degree of vegetative scouring in the streambed, vegetative succession, and native plant competition with invasive species (Downs et al. 2013, Parker et al. 2014). Significant habitat degradation has occurred through formation of dense stands of the nonnative perennial grass giant reed (Arundo donax) and further losses are possible with the recent arrival of the polyphagous shothole borer beetle (Euwallacea spp.), which transmits a pathogenic fungus that kills native host trees (Coffman et al. 2010, Umeda et al. 2016).

Since the late 1980s, when only a few pairs were documented in the Santa Clara River watershed in Ventura County (U.S. Fish and Wildlife Service 2006), Least Bell’s Vireo has greatly expanded its distribution through the watershed, to the point that there are at least 250 breeding pairs documented for this location (L. Hall, personal observation). However, the total number of pairs along the entire watershed, as well as a full inventory of the potential amount of suitable habitat, is still unknown because not all potentially suitable areas have been surveyed.

A conceptual model describing the influence of extrinsic drivers on Least Bell’s Vireo population dynamics was developed through regular conversations with a working group of experts with specialized knowledge of the subspecies and/or region (the authors except JCS and WET). The process of developing conceptual models can be a useful exercise in the decision-making process because they can help illustrate how life history parameters relate to environmental conditions (Smith et al. 2018). The conceptual model sought to include the key processes driving the dynamics of both the habitat quality and quantity in terms of vegetation cover as well as factors with direct impacts on vireo population growth such as brood parasitism by Brown-headed Cowbirds (Fig. 1). We presented the conceptual model to a broader group of managers in the region for their input and feedback at a one-day workshop in November 2016.

Population viability analysis (PVA) development and approach to uncertainty

While attempting to parameterize the conceptual model into a simulation model for use in a PVA, it was determined that many of the factors and drivers in the conceptual model were either not tractable or not supportable with sufficient data. For example, we have data on the historical frequency and severity of scouring events on the river but lack predictive models of how that might change in the future with climate variability. Also absent is an ability to quantify the interaction of native vegetation with giant reed establishment, particularly following significant flooding events, and how that translates to the number of breeding pairs of vireos the habitat can support. In addition, we also lack the ability to directly parameterize how cowbird control efforts affect rates of nest parasitism and, in turn, fecundity rates.

Lacking the data to fully parameterize many of the relationships outlined in the conceptual model (Fig. 1), we constructed a simplified population growth model (Fig. 2) that became the basis for the PVA model. This approach captured the essential dynamics of the system by collapsing some of the fine-grained processes into coarse-grained parameters with correspondingly wider uncertainty bounds (Fig. 2). This reduced model approach has been used previously for assessing the current and long-term viability of populations (McGowan et al. 2017, Smith et al. 2018). Because of the wide uncertainty parameters incorporated for each management scenario, the model does not yield precise predictions about future vireo populations or the predicted amount or configuration of habitat at any specific location or property along the river; rather, the population model structure is designed to estimate the probability that specific management goals will be achieved under a given set of potential management actions (McGowan et al. 2014) and given parameter uncertainty. In this way, decision makers can evaluate one course of action relative to other courses of action (or inaction) without requiring the model to produce precise predictions of future conditions (Cook and Robinson 2017).

To account for both parameter uncertainty and irreducible uncertainty in the form of environmental stochasticity, we used a framework similar to McGowan et al. (2011, 2017) by constructing the model with three distinct levels. The top level is the replication loop that takes a single draw from each parameter distribution to create a set of parameters to simulate through time. The replication loop accounts for the parameter uncertainty by sampling across the range of uncertainty for each model parameter. The next level is the management loop that repeats each set of parameters from the replication loop over every combination of six potential habitat management scenarios and six levels of cowbird removal effort. This arrangement is similar to the matched runs of Cook and Robinson (2017) or the paired comparisons of Aiello-Lammens and Akçakaya (2017). Finally, in the time-step loop, the scenario is simulated through time with environmental stochasticity (year-to-year variability) using the parameter set and management scenario conditions from the upper-level loops.

The order of the replication loop and the management loop could theoretically be reversed. We chose to draw the parameter sample sets first and replicate over the management scenarios in an inner loop to be assured that the differences in model outcome between management scenarios were due solely to differences in the management parameters and not because of differences in random draws from the parameter distributions.

Habitat model

We simulated the available habitat along the Santa Clara River in terms of the theoretical carrying capacity tracked as the potential maximum number of breeding pairs the system could support in each model year.

We divided the river into a series of management units, each composed of one or more spatially adjacent properties, which either are currently managed or are being considered for future management to restore or maintain Least Bell’s Vireo habitat. We conducted a spatial assessment of each property, as well as reviewed current data on the number of pairs observed nesting on several of the properties (Appendix 1, Table A1.1). We used this information to estimate the current availability of suitable vegetation as well as the potential maximum carrying capacity each property might achieve given active management for vireo habitat. We divided the river into six management units as well as an unmanaged unit composed of all the remaining area on the river not contained within a management unit, but which nonetheless may contain vegetation suitable for nesting vireos. The total carrying capacity of the model at each timestep was determined by summing the carrying capacity across the six management units and the remaining unmanaged area. The carrying capacity within each management unit at each timestep was determined by the parameter set coming from the replication loop as well as the parameters of the habitat management scenario being simulated.

Frequency and severity of scouring events were parameterized (Table 1) using an analysis of vegetation cover following major flooding events on the Santa Clara River dating back to the 1930s (Stillwater Sciences 2007). This analysis provided insight on changes to riparian vegetation in general along the Santa Clara River, but may not proportionally translate to changes in the total carrying capacity of Least Bell’s Vireo; however, this vegetation-cover analysis represented our best current understanding of vegetation dynamics due to stochastic river flow events. We randomly assigned each simulation year to follow one of three river flow categories (Table 1) such that the probability of being in each category was proportional to the probability of that flow type event. Once the flow category was determined for a simulation year, the vegetation impact was sampled from a uniform distribution defined by the upper and lower bounds for that river flow category (Table 1). The Stillwater Sciences (2007) report provided insights on the impact of moderate to severe flooding events but did not allow for the parameterization of the typical variation in riparian vegetation under more common (low/no scour) years. The upper and lower bounds for the low/no scour scenario were set through consultation with the expert working group based on patterns of annual loss resulting from normal scouring in a typical year as well as invasion by giant reed and habitat gains through natural succession following scouring events.

The flooding and scouring scenarios were parameterized to simulate the extrinsic forces determining the amount of suitable habitat available on the river in the absence of active management for Least Bell’s Vireos. We parameterized habitat loss from moderate to severe scouring to take precedence over habitat-management activities. This meant that when the simulation year river-flow category model indicated a moderate to severe scour, the trend in all managed and unmanaged units followed the trend for that river-flow category impact regardless of the management unit and habitat management scenarios. Additional details on the habitat model are available in Appendix 1.

Management actions

To simulate Brown-headed Cowbird management, we generated scenarios exploring the impact of parasitism on Least Bell’s Vireo fecundity (i.e., seasonal productivity per adult). Although other factors such as predation influence fecundity, cowbird parasitism is the primary determinant of seasonal productivity of Least Bell’s Vireo (Kus and Whitfield 2005) and is amenable and responsive to management. To simulate cowbird control, we added a scalar adjustment to the fecundity value in the life stage population matrix and recalculated the population growth rate as the dominant eigenvector of the matrix. We simulated six levels of Brown-headed Cowbird impact at each of the following multipliers on fecundity: 0.1, 0.2, 0.4, 0.6, 0.8, and 1.0, where a level of 1.0 indicated full control of Brown-headed Cowbirds resulting in no reduction in vireo fecundity, and a level of 0.1 indicated a 90% reduction in fecundity associated with little or no cowbird control (i.e., a very high parasitism rate; Fig. 3). During model construction, data were not sufficient to parameterize explicit links between a given amount of Brown-headed Cowbird control effort (e.g., number of traps operated) and a subsequent reduction in parasitism rates and/or vireo fecundity. Therefore, our approach did not result in a model that can predict how a specific cowbird management program (e.g., specific placement or density of Brown-headed Cowbird traps, when and how long to activate the traps) would influence population outcomes. However, it does allow managers to establish observable monitoring goals in terms of minimum vireo fecundity levels and use that information to refine the particulars of the cowbird control effort to achieve those goals.

To simulate habitat restoration, we parameterized the actions as directly influencing the carrying capacity. We first drew a random restoration rate from a uniform distribution ranging from 1.5 to 15 pairs/year. The restoration rate signified the maximum rate at which additional carrying capacity could be added to the area within a management unit depending on whether the scenario being simulated included that unit. The carrying capacity at each managed unit increased at the restoration rate unless the management program ended, the maximum carrying capacity for that unit was reached, or there was a scouring event (moderate or heavy scour) that reduced the carrying capacity. If a management program was not active in a management unit, we simulated the carrying capacity at that unit as an unmanaged unit (i.e., followed the trajectory prescribed by the stochastic river flow described in Table 1).

The initial carrying capacities at each individual management unit were determined based on the total carrying capacity at the initiation of the simulation (determined by the parameter draws of the replication loop) and the proportion of that total apportioned to each unit, as in Table 2. The maximum carrying capacities of each unit were similarly apportioned.

We simulated five scenarios to signify different strategies for habitat restoration and management along the river based on different treatment of the management units. The parameters for each management unit under each scenario are outlined in Table 2. The first three scenarios simulated management on units that comprised approximately ¼, ½, or ¾ of the total riparian habitat area available, respectively. In each of these management scenarios, the carrying capacities would either increase at the restoration rate each year or be reduced by a scouring event until the maximum carrying capacities for the units were reached. Once the maximum carrying capacity was reached on a managed unit, it would be maintained at that level unless it was reduced by a scouring event.

We also simulated a scenario called “walk-away,” which was similar to the ¾-area managed scenario except that as soon as the total carrying capacity of the system reached 70% of the maximum carrying capacity, active restoration ceased, and all management units reverted back to an unmanaged state. This scenario represented the real possibility that restoration could not be fully concluded or sustained because of a lack of continued financing for restoration. In the “flood” scenario, it was assumed that the natural scouring that occurs during a flood could be taken advantage of to restore vegetation suitable for vireos. Under this scenario, all management units were treated as unmanaged until a stochastic heavy flood occurred. Once a significant scouring event occurred, the scoured area available for restoration was determined by the size of the event. From that point, restoration would proceed at the restoration rate until the carrying capacity proportional to the amount scoured was reached. Under this scenario, restoration could proceed on any of the numbered management units. For comparison, we also considered a scenario with no habitat management in which all management units were treated as unmanaged for the duration of the simulation.

Population model

Once the trajectory of total carrying capacity was determined based on the scouring events and habitat management scenario, we simulated the population size through time based on a simple scalar population model, tracking the total number of pairs of Least Bell’s Vireos. Scalar population models have been shown to be sufficient for providing unbiased estimates of population trends (Rueda-Cediel et al. 2015, 2018). During the replication loop, we drew a set of fecundity and survival values to generate a two-stage matrix (see Appendix 1 for details). This matrix was modified by the cowbird control scenario during the management loop, and the modified matrix was used to estimate the population growth rate for the scalar models by calculating the dominant eigenvalue.

Sensitivity analysis

We analyzed the sensitivity of model results to parameter uncertainty through boosted regression tree analysis. This approach allows for assessing the impact of individual model parameters on model results while also allowing for interactions between model parameters (Elith et al. 2008) without having to specify interaction terms a priori as one would in a linear regression model. For the sensitivity analysis, we were primarily interested in the impact of uncertainty of the parameters in the replication loop on model results that would be relevant for management decisions. For that reason, we held management actions constant by filtering model results to those in which fecundity was between 1.0 and 1.5 and habitat management was under the ½ area managed scenario. We conducted separate boosted regression tree analyses for separate model response variables: mean number of pairs over the duration of the simulation, minimum number of pairs at any point during the simulation, number of pairs in simulation year 50, and the geometric mean of year-to-year change in number of pairs. The boosted regression tree analyses were conducted in R (R Core Team 2018) using the gbm and dismo packages (Hijmans et al. 2017, Ridgeway 2017).

MOViE tool

We designed the MOViE tool to provide a user interface for exploring the population-model output and to facilitate decision making (see Box 1). The MOViE tool displays a table of management action scenarios and the probabilities of achieving specific threshold-level management goals under each scenario. Parameter uncertainty ranges are made explicit and adjustable whereas the model output instantaneously updates. This creates an interactive sensitivity analysis without the need to run and analyze additional model simulations. The decision support tool also allows users the flexibility to modify management goal thresholds.

The MOViE tool presents a series of tables showing the proportion of model replications in which a specified management goal (e.g., mean population size > = 200 pairs) was achieved. The tables are arranged to explore the probability of each goal being met under different management scenarios.

The tool was implemented using the Shiny package in R (Chang et al. 2018). Shiny provides a translator to create interactive web applications based on R code. The R scripts for implementing the MOViE tool are available at https://doi.org/10.5066/P9VNZI1W.

RESULTS

Management actions

The simulated impact of Brown-headed Cowbird control had a large effect on model outcome. Considering fecundity as a driver of the population’s ability to fill the available habitat measured as the ratio of the mean number of pairs to mean carrying capacity, the response showed marked increase around a fecundity value of ~1 fledglings per adult (Fig. 4). Models with fecundity values less than 1.0 had a mean ratio value of 0.14 (95% CI: 0.01-0.89) whereas models with fecundity values greater than or equal to 1.0 had a mean ratio of 0.71 (95% CI: 0.06-0.99).

Other model results showed similar responses to cowbird control. For example, holding the effect of habitat restoration constant at the ½-area managed restoration scenario, the mean number of pairs over the duration of the study varied depending on the postcowbird control fecundity value (Fig. 5), showing large increases when fecundity values were ≥ 1.0 but fewer gains as fecundity values exceeded 1.5. When fecundity was < 1.0 fledglings per adult, the expected mean number of pairs was 42 (95% CI: 9-686). When fecundity was ≥ 1.0, the expected mean number of pairs was more than an order of magnitude greater at 596 (95% CI: 39-1142). Similar patterns were observed for other summarized model results predicting minimum population size, future population size, and population growth rate (Appendix 2).

The total additions to carrying capacity varied by habitat restoration scenario, with the most habitat added under the ¾ area managed scenario followed by the walk-away scenario (Fig. 6). The marginal effect of habitat restoration on model results can be measured by holding cowbird control constant over a narrow band of fecundity values. Over a range of fecundity values from 1.0-1.5, the mean number of vireo pairs over the duration of the study was consistent with the amount of total habitat restored under each habitat scenario (Fig. 5). Within this range of fecundity values, the expected mean number of pairs was 232 (95% CI: 32-744) for the none scenario, 356 (95% CI: 32-885) for the 1/4 area managed scenario, 450 (95% CI: 31-1097) for the 1/2 area managed scenario, 486 (95% CI: 32-1272) for the 3/4 area managed scenario, 484 (95% CI: 32-1253) for the walk-away scenario, and 331 (95% CI: 32-923) for the flood scenario. Similar patterns were observed for other summarized model results (Appendix 2).

Sensitivity analysis

Filtering the total number of model replicates (720,000) to only those with fecundity between 1.0 and 1.5, and habitat management under the ½-area managed scenario, resulted in a total of 28,476 replicates to include in the boosted regression tree analysis.

Both adult and juvenile survival explained large portions of the variation across all model results (Fig. 7). Both mean and minimum population sizes were sensitive to the initial number of pairs whereas the final population size and the mean growth rate were only mildly influenced by this parameter. The habitat restoration rate explained a small amount of variance in the mean population size and the final population size. Final population size was slightly sensitive to the two parameters related to flooding (number of floods, mean flood impact). The remaining parameter uncertainties had minimal influence on model results of interest (Fig. 7).

MOViE tool

The final design of the MOViE tool (https://www.usgs.gov/apps/MOViE) included several features that were modified from earlier prototype versions based on iterative feedback and modifications between the expert working group and the modeling team. For example, the experts working on this subspecies typically measure population recruitment in terms of fledglings per pair rather than per adult. To make the tool maximally user friendly to our intended audience, we converted the fecundity parameter from the model for display in the tool.

The final design of the decision support tool includes the following tabs: (1) an informational page that gives some background information on this project as well as instructions on how to use the tool, (2) a quality check tab that summarizes the total number of model iterations for each scenario, and (3) four model summary result tabs: mean population size, minimum population size, population size in the final simulation year, and the mean population growth rate.

DISCUSSION

Decision making in the face of uncertainty is often paralyzed by an unclear understanding of the probability of occurrence of some outcome of interest or the magnitude of its consequence. Conversely, when decisions are made in the face of this uncertainty, the risk of adverse outcomes is often poorly understood. Previous efforts in developing decision-support tools in uncertain environments for endangered species focused largely on imperilment determination. Numerous statistical models, tools, and programmatic assessment criteria exist for determining the magnitude and timing of risk faced by imperiled species (e.g., Carter et al. 2000, IUCN 2012, Crimmins et al. 2015, Semmens et al. 2016, Stanton et al. 2016). A wide array of tools identify key demographic and habitat parameters associated with species extinction risk (e.g., Erickson et al. 2014). Another important area of tool development has focused on optimal resource allocation under defined sets of constraints (Converse et al. 2011, Gerber et al. 2018). A novel development of the MOViE tool is that it provides conjoint decision insight in the face of uncertainty among and across each of these areas.

The MOViE tool was designed to be used by federal, state, and nongovernmental agencies and organizations to inform Least Bell’s Vireo management and recovery on the Santa Clara River. The U.S. Fish and Wildlife Service, which commissioned this tool, requested it as part of a strategic habitat conservation approach (NEAT 2006) to reviewing the status of the Least Bell’s Vireo. The process of building the conceptual model (Fig. 1), the PVA model, and the tool has generated discussion on what might be appropriate quantitative management and recovery goals for this population, something that had not previously been clearly articulated. The process has also lent insight into how the balance of cowbird control and habitat restoration might interact in the probability of achieving those goals.

An important design feature of the MOViE tool is presenting probabilistic results over a range of potential management scenarios. Because the tool summarizes the model in terms of probabilities of successfully meeting quantitative goals, it facilitates dialogue regarding the level of acceptable risk to the population. Whenever decisions are made in which outcomes cannot be guaranteed, the level of acceptable risk is subject to the risk attitudes of the decision makers. Models can aid in illustrating the differences in risk between alternative actions and can enable frank discussions about risk, but models cannot by themselves resolve what amount is considered acceptable (Cook and Robinson 2017). There will always be other considerations beyond species-specific risk that will weigh into decisions, such as cost and competing interests for land or resources (Rose 2015). For the vireo, finding an efficient way to allocate available resources will mean finding a balance between habitat restoration and efforts to control Brown-headed Cowbird parasitism.

The simulated impact of Brown-headed Cowbird parasitism had a large effect on the population. These results were not surprising given that Brown-headed Cowbird parasitism has long been known to detrimentally have an impact on native passerine hosts in the United States (Elliott 1978, Smith 1981, Brittingham and Temple 1983), including Least Bell’s Vireo (Kus 1999). Cowbird control has been used since at least the 1970s to lessen these impacts (DeCapita 2000, Kus and Whitfield 2005). However, the model results also indicate that it is unnecessary to strive for total elimination of Brown-headed Cowbird parasitism of Least Bell’s Vireo nests to achieve and maintain management goals. In the simulated system, fecundity values > 1.0 fledglings per adult substantially increased population size relative to fecundity below 1.0, but above this threshold, further increases in fecundity added little to improve model outcomes. This flexibility could have great significance for managers of vireos, as well as other endangered songbirds, because it indicates that an adaptive approach to cowbird control that maintains fecundity above a target level can achieve recovery goals. Our model results support previous conclusions that the total removal of Brown-headed Cowbirds from vireo nesting areas is neither a practical nor necessary goal for the long-term recovery of Least Bell’s Vireo (Kus and Whitfield 2005, Peer et al. 2005).

In our simulations, cowbird control influenced not only fecundity but also the response of the vireo population to habitat restoration. Although overall population sizes were larger as restoration increased from ¼ to ¾ of the river being managed, no level of restoration yielded population growth sufficient for recovery when vireo fecundity was below 1.0 fledglings per adult. Visualizing the effects of cowbird control on fecundity and restoration simultaneously allows managers to identify the appropriate combinations of cowbird control and restoration to achieve desired population outcomes.

Finally, the tool also was designed to assist in identifying the most important factors impacting the population and to aid in establishing a plan for future monitoring and data collection. Having adjustable bounds on model parameters allows users to explore how those parameters have an impact on the model results and conclusions. This identification of influential parameters can help guide future research by indicating where having more precise parameter estimates would likely improve the robustness of future population models or analyses. For example, we found that both juvenile and adult survival rates explained large portions of the variation across all model results (Fig. 7). Population models are frequently sensitive to survival and productivity estimates (Anders and Marshall 2005), yet these parameters are often the least available, or at least the most uncertain, in studies of wildlife species. Obtaining accurate, population-specific estimates of survival rates can be prohibitively labor-intensive and require several years of data collection on large numbers of marked individuals over large areas. This is particularly true for juveniles of species like Least Bell’s Vireo, which typically disperse away from the natal area, confounding attempts to distinguish mortality from dispersal (Anders and Marshall 2005). However, refinement of these estimates, and a better understanding of the factors influencing them, is critical to improving projections of population outcomes and the management needed to achieve them.

The sensitivity of our model results to adult and juvenile survival rates as well as the importance of fecundity highlight the value of ongoing monitoring and the importance of revisiting and revising population models and subsequent management decisions when additional data are available (Beissinger and Westphal 1998). Additional variables were influential, for example, we found that projected mean and minimum population sizes were sensitive to the initial number of pairs, again serving to emphasize the importance of continued assessments of vireo pairs in the study system. Final population size also was influenced by two flood variables, including flood impact, emphasizing the need to monitor and further study vegetative responses to environmental perturbations. And lastly, the habitat restoration rate explained a small amount of the variance in mean and final population sizes, indicating that current restoration work on the Santa Clara River by partners such as Nature Conservancy and the University of California at Santa Barbara can contribute to the long-term persistence of this special status species.

Our incomplete understanding of dynamic, complex natural systems complicates species conservation. Decision making in this uncertain environment is more thoughtful and accountable when it recognizes and accommodates these uncertainties. Decision support tools that can clarify the magnitude and consequences of these uncertainties can lead to sounder conclusions and, as a result, improve the chances for species recovery.

RESPONSES TO THIS ARTICLE

Responses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link.

ACKNOWLEDGMENTS

We thank Sanger Hedrick for his devotion to habitat conservation on the Santa Clara River, and Sam Rojas for assisting with data and access to the upper reaches of our study area. This manuscript benefited from the thoughtful comments and suggestions of Conor McGowan and two anonymous reviewers. The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the U.S. Fish and Wildlife Service. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government. The Western Foundation of Vertebrate Zoology provided logistic and research support for the team and to LSH.

LITERATURE CITED

Aiello-Lammens, M. E., and H. R. Akçakaya. 2017. Using global sensitivity analysis of demographic models for ecological impact assessment. Conservation Biology 31(1):116-125. https://doi.org/10.1111/cobi.12726

Anders, A. D., and M. R. Marshall. 2005. Increasing the accuracy of productivity and survival estimates in assessing landbird population status. Conservation Biology 19(1):66-74. https://doi.org/10.1111/j.1523-1739.2005.00543.x

Beissinger, S. R., and M. I. Westphal. 1998. On the use of demographic models of population viability in endangered species management. Journal of Wildlife Management 62(3):821-841. https://doi.org/10.2307/3802534

Bottrill, M. C., L. N. Joseph, J. Carwardine, M. Bode, C. Cook, E. T. Game, H. Grantham, S. Kark, S. Linke, E. McDonald-Madden, R. L. Pressey, S. Walker, K. A. Wilson, and H. P. Possingham. 2008. Is conservation triage just smart decision making? Trends in Ecology and Evolution 23(12):649-654. https://doi.org/10.1016/j.tree.2008.07.007

Boyce, M. S. 1992. Population viability analysis. Annual Review of Ecology and Systematics 23(1):481-497. https://doi.org/10.1146/annurev.es.23.110192.002405

Brittingham, M. C., and S. A. Temple. 1983. Have Cowbirds caused forest songbirds to decline? BioScience 33(1):31-35. https://doi.org/10.2307/1309241

Canessa, S., G. Guillera-Arroita, J. J. Lahoz-Monfort, D. M. Southwell, D. P. Armstrong, I. Chadès, R. C. Lacy, and S. J. Converse. 2015. When do we need more data? A primer on calculating the value of information for applied ecologists. Methods in Ecology and Evolution 6(10):1219-1228. https://doi.org/10.1111/2041-210X.12423

Carroll, C., M. K. Phillips, C. A. Lopez-Gonzalez, and N. H. Schumaker. 2006. Defining recovery goals and strategies for endangered species: the wolf as a case study. BioScience 56(1):25-37. https://doi.org/10.1641/0006-3568(2006)056[0025:DRGASF]2.0.CO;2

Carter, M. F., W. C. Hunter, D. N. Pashley, and K. V. Rosenberg. 2000. Setting conservation priorities for landbirds in the United States: the Partners in Flight approach. Auk 117(2):541-548. https://doi.org/10.1093/auk/117.2.541

Chang, W., J. Cheng, J. J. Allaire, Y. Xie, and J. McPherson. 2018. Shiny: web application framework for R. R Foundation for Statistical Computing, Vienna, Austria. https://CRAN.R-project.org/package=shiny

Coffman, G. C., R. F. Ambrose, and P. W. Rundel. 2010. Wildfire promotes dominance of invasive giant reed (Arundo donax) in riparian ecosystems. Biological Invasions 12(8):2723-2734. https://doi.org/10.1007/s10530-009-9677-z

Conroy, M. J., M. C. Runge, J. D. Nichols, K. W. Stodola, and R. J. Cooper. 2011. Conservation in the face of climate change: the roles of alternative models, monitoring, and adaptation in confronting and reducing uncertainty. Biological Conservation 144(4):1204-1213. https://doi.org/10.1016/j.biocon.2010.10.019

Converse, S. J., K. J. Shelley, S. Morey, J. Chan, A. LaTier, C. Scafidi, D. T. Crouse, and M. C. Runge. 2011. A decision-analytic approach to the optimal allocation of resources for endangered species consultation. Biological Conservation 144(1):319-329. https://doi.org/10.1016/j.biocon.2010.09.009

Cook, A. S. C. P., and R. A. Robinson. 2017. Towards a framework for quantifying the population-level consequences of anthropogenic pressures on the environment: the case of seabirds and windfarms. Journal of Environmental Management 190:113-121. https://doi.org/10.1016/j.jenvman.2016.12.025

Crimmins, S. M., P. Boma, and W. E. Thogmartin. 2015. Projected risk of population declines for native fish species in the upper Mississippi River. River Research and Applications 31(2):135-142. https://doi.org/10.1002/rra.2741

DeCapita, M. E. 2000. Brown-headed Cowbird control on Kirtland’s Warbler nesting areas in Michigan, 1972-1995. Pages 333-341 in J. N. M. Smith, T. L. Cook, S. I. Rothstein, S. K. Robinson, and S. G. Sealy, editors. Ecology and management of cowbirds and their hosts: studies in the conservation of North American passerine birds. University of Texas Press, Austin, Texas, USA.

Downs, P. W., S. R. Dusterhoff, and W. A. Sears. 2013. Reach-scale channel sensitivity to multiple human activities and natural events: Lower Santa Clara River, California, USA. Geomorphology 189:121-134. https://doi.org/10.1016/j.geomorph.2013.01.023

Earl, J. E., S. Nicol, R. Wiederholt, J. E. Diffendorfer, D. Semmens, D. T. T. Flockhart, B. J. Mattsson, G. McCracken, D. R. Norris, W. E. Thogmartin, and L. López-Hoffman. 2018. Quantitative tools for implementing the new definition of significant portion of the range in the U.S. Endangered Species Act. Conservation Biology 32(1):35-49. https://doi.org/10.1111/cobi.12963

Elith, J., J. R. Leathwick, and T. Hastie. 2008. A working guide to boosted regression trees. Journal of Animal Ecology 77(4):802-813. https://doi.org/10.1111/j.1365-2656.2008.01390.x

Elliott, P. F. 1978. Cowbird parasitism in the Kansas tallgrass prairie. Auk 95(1):161-167. https://doi.org/10.2307/4085507

Erickson, R. A., W. E. Thogmartin, and J. A. Szymanski. 2014. BatTool: an R package with GUI for assessing the effect of White-nose syndrome and other take events on Myotis spp. of bats. Source Code for Biology and Medicine 9(1):9. https://doi.org/10.1186/1751-0473-9-9

Gerber, L. R., M. C. Runge, R. F. Maloney, G. D. Iacona, C. A. Drew, S. Avery-Gomm, J. Brazill-Boast, D. Crouse, R. S. Epanchin-Niell, S. B. Hall, L. A. Maguire, T. Male, D. Morgan, J. Newman, H. P. Possingham, L. Rumpff, K. C. B. Weiss, R. S. Wilson, and M. A. Zablan. 2018. Endangered species recovery: a resource allocation problem. Science 362(6412):284-286. https://doi.org/10.1126/science.aat8434

Gilpin, M. E., and M. E. Soule. 1986. Minimum viable populations: processes of species extinction. Pages 19-34 in M. E. Soule, editor. Conservation biology: the science of scarcity and diversity. Sinauer Associates, Sunderland, Massachusetts, USA.

Goble, D. D., J. A. Wiens, J. M. Scott, T. D. Male, and J. A. Hall. 2012. Conservation-reliant species. BioScience 62(10):869-873. https://doi.org/10.1525/bio.2012.62.10.6

Grantham, H. S., K. A. Wilson, A. Moilanen, T. Rebelo, and H. P. Possingham. 2009. Delaying conservation actions for improved knowledge: how long should we wait? Ecology Letters 12(4):293-301. https://doi.org/10.1111/j.1461-0248.2009.01287.x

Gregory, R. S., and R. L. Keeney. 2002. Making smarter environmental management decisions. Journal of the American Water Resources Association 38(6):1601-1612. https://doi.org/10.1111/j.1752-1688.2002.tb04367.x

Gregory, R., and G. Long. 2009. Using structured decision making to help implement a precautionary approach to endangered species management. Risk Analysis 29(4):518-532. https://doi.org/10.1111/j.1539-6924.2008.01182.x

Hijmans, R. J., S. Phillips, J. Leathwick, and J. Elith. 2017. Dismo: species distribution modeling. R Foundation for Statistical Computing, Vienna, Austria. https://CRAN.R-project.org/package=dismo

Himes Boor, G. K. 2014. A framework for developing objective and measurable recovery criteria for threatened and endangered species. Conservation Biology 28(1):33-43. https://doi.org/10.1111/cobi.12155

International Union for the Conservation of Nature (IUCN). 2012. IUCN Red List categories and criteria. Version 3.1, second edition. IUCN Species Survival Commission, Gland, Switzerland. [online] URL: https://iucn-csg.org/red-list-categories/

Johnson, F. A., M. J. Eaton, J. H. Williams, G. H. Jensen, and J. Madsen. 2015. Training conservation practitioners to be better decision makers. Sustainability 7(7):8354-8373. https://doi.org/10.3390/su7078354

Joseph, L. N., R. F. Maloney, and H. P. Possingham. 2009. Optimal allocation of resources among threatened species: a project prioritization protocol. Conservation Biology 23(2):328-338. https://doi.org/10.1111/j.1523-1739.2008.01124.x

Keith, D. A., T. G. Martin, E. McDonald-Madden, and C. Walters. 2011. Uncertainty and adaptive management for biodiversity conservation. Biological Conservation 144(4):1175-1178. https://doi.org/10.1016/j.biocon.2010.11.022

Kus, B. E. 1998. Use of restored riparian habitat by the endangered Least Bell’s Vireo (Vireo bellii pusillus). Restoration Ecology 6(1):75-82. https://doi.org/10.1046/j.1526-100x.1998.06110.x

Kus, B. E. 1999. Impacts of Brown-headed Cowbird parasitism on productivity of the endangered Least Bell’s Vireo. Studies in Avian Biology 18:160-166. [online[ URL: https://sora.unm.edu/node/139540

Kus, B. E., and M. J. Whitfield. 2005. Parasitism, productivity, and population growth: response of Least Bell’s Vireos (Vireo bellii pusillus) and Southwestern Willow Flycatchers (Empidonax traillii extimus) to Cowbird (Molothrus spp.) control. Ornithological Monographs(57):16-27. https://doi.org/10.2307/40166811

McGowan, C. P., N. Allan, J. Servoss, S. Hedwall, and B. Wooldridge. 2017. Incorporating population viability models into species status assessment and listing decisions under the U.S. Endangered Species Act. Global Ecology and Conservation 12:119-130. https://doi.org/10.1016/j.gecco.2017.09.004

McGowan, C. P., D. H. Catlin, T. L. Shaffer, C. L. Gratto-Trevor, and C. Aron. 2014. Establishing endangered species recovery criteria using predictive simulation modeling. Biological Conservation 177:220-229. https://doi.org/10.1016/j.biocon.2014.06.018

McGowan, C. P., M. C. Runge, and M. A. Larson. 2011. Incorporating parametric uncertainty into population viability analysis models. Biological Conservation 144(5):1400-1408. https://doi.org/10.1016/j.biocon.2011.01.005

Milner-Gulland, E. J., and K. Shea. 2017. Embracing uncertainty in applied ecology. Journal of Applied Ecology 54(6):2063-2068. https://doi.org/10.1111/1365-2664.12887

Morris, W. F., and D. F. Doak. 2002. Quantitative conservation biology: theory and practice of population viability analysis. Sinauer Associates, Sunderland, Massachusetts, USA.

National Ecological Assessment Team (NEAT) 2006. Strategic habitat conservation: a report from the National Ecological Assessment Team. U.S. Fish and Wildlife Service, Washington, D.C., USA and U.S. Geological Survey, Reston, Virginia, USA. [online] URL: https://www.fws.gov/nc-es/habreg/NEAT_FinalRpt.pdf

Neel, M. C., A. K. Leidner, A. Haines, D. D. Goble, and J. M. Scott. 2012. By the numbers: how is recovery defined by the U.S. Endangered Species Act? BioScience 62(7):646-657. https://doi.org/10.1525/bio.2012.62.7.7

Parker, S. S., E. J. Remson, and L. N. Verdone. 2014. Restoring conservation nodes to enhance biodiversity and ecosystem function along the Santa Clara River. Ecological Restoration 32(1):6-8. https://doi.org/10.3368/er.32.1.6

Peer, B. D., S. I. Rothstein, M. J. Kuehn, and R. C. Fleischer. 2005. Host defenses against Cowbird (Molothrus spp.) parasitism: implications for Cowbird management. Ornithological Monographs(57):84-97. https://doi.org/10.2307/40166816

Peterson, G. D., G. S. Cumming, and S. R. Carpenter. 2003. Scenario planning: a tool for conservation in an uncertain world. Conservation Biology 17(2):358-366. https://doi.org/10.1046/j.1523-1739.2003.01491.x

R Core Team. 2018. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. [online] URL: https://www.r-project.org/

Regan, H. M., M. Colyvan, and M. A. Burgman. 2002. A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecological Applications 12(2):618-628. https://doi.org/10.1890/1051-0761(2002)012[0618:ATATOU]2.0.CO;2

Ridgeway, G. 2017. gbm: generalized boosted regression models. R Foundation for Statistical Computing, Vienna, Austria. https://CRAN.R-project.org/package=gbm

Rose, D. C. 2015. The case for policy-relevant conservation science: policy-relevant conservation science. Conservation Biology 29(3):748-754. https://doi.org/10.1111/cobi.12444

Rueda-Cediel, P., K. E. Anderson, T. J. Regan, J. Franklin, and H. M. Regan. 2015. Combined influences of model choice, data quality, and data quantity when estimating population trends. PLOS ONE 10(7):e0132255. https://doi.org/10.1371/journal.pone.0132255

Rueda-Cediel, P., K. E. Anderson, T. J. Regan, and H. M. Regan. 2018. Effects of uncertainty and variability on population declines and IUCN Red List classifications: assessments of IUCN Red Lists. Conservation Biology 32(4):916-925. https://doi.org/10.1111/cobi.13081

Runge, M. C. 2011. An introduction to adaptive management for threatened and endangered species. Journal of Fish and Wildlife Management 2(2):220-233. https://doi.org/10.3996/082011-JFWM-045

Semmens, B. X., D. J. Semmens, W. E. Thogmartin, R. Wiederholt, L. López-Hoffman, J. E. Diffendorfer, J. M. Pleasants, K. S. Oberhauser, and O. R. Taylor. 2016. Quasi-extinction risk and population targets for the eastern, migratory population of monarch butterflies (Danaus plexippus). Scientific Reports 6(1):23265. https://doi.org/10.1038/srep23265

Smith, D. R., N. L. Allan, C. P. McGowan, J. A. Szymanski, S. R. Oetker, and H. M. Bell. 2018. Development of a species status assessment process for decisions under the U.S. Endangered Species Act. Journal of Fish and Wildlife Management 9(1):302-320. https://doi.org/10.3996/052017-jfwm-041

Smith, J. N. M. 1981. Cowbird parasitism, host fitness, and age of the host female in an island Song Sparrow population. Condor 83(2):152-161. https://doi.org/10.2307/1367420

Stanton, J., and H. R. Akçakaya. 2013. Conservation planning to ensure viability of populations and metapopulations. Pages 271-291 in F. L. Craighead and C. I. Convis, editors. Conservation planning: shaping the future. ESRI, Redlands, California, USA. [online] URL: https://life.bio.sunysb.edu/ee/akcakayalab/PVA.pdf

Stanton, J. C., B. X. Semmens, P. C. McKann, T. Will, and W. E. Thogmartin. 2016. Flexible risk metrics for identifying and monitoring conservation-priority species. Ecological Indicators 61:683-692. https://doi.org/10.1016/j.ecolind.2015.10.020

Stillwater Sciences. 2007. Analysis of riparian vegetation dynamics for the lower Santa Clara River and major tributaries, Ventura County, California. Santa Clara River parkway floodplain restoration feasibility study. Stillwater Sciences, Berkeley, California, USA. [online] URL: http://parkway.scrwatershed.org/wkb/scrbiblio/floodplainfeasibility/attachment_download/Veg%20Dynamics_Report_FINAL.pdf

Sutherland, W. J., L. V. Dicks, N. Ockendon, S. O. Petrovan, and R. K. Smith, editors. 2018. What works in conservation 2018. Open Book, Cambridge, UK. [online] URL: https://www.openbookpublishers.com/product/696

Umeda, C., A. Eskalen, and T. D. Paine. 2016. Polyphagous shot hole borer and fusarium dieback in California. Pages 757-767 in T. D. Paine and F. Lieutier, editors. Insects and diseases of Mediterranean forest systems. Springer, New York, New York, USA. https://doi.org/10.1007/978-3-319-24744-1_26

U.S. Fish and Wildlife Service. 1998. Draft recovery plan for the Least Bell’s Vireo. U.S. Fish and Wildlife Service, Portland, Oregon, USA.

U.S. Fish and Wildlife Service. 2006. Least Bell’s Vireo (Vireo bellii pusillus) 5-year review summary and evaluation. U.S. Fish and Wildlife Service, Carlsbad, California, USA. [online] URL: https://www.fws.gov/carlsbad/SpeciesStatusList/5YR/20060926_5YR_LBV.pdf

Williams, B. K., R. C. Szaro, and C. D. Shapiro. 2009. Adaptive management: the U.S. Department of the Interior technical guide. U.S. Department of the Interior, Washington, D.C., USA. [online] URL: https://www.doi.gov/sites/doi.gov/files/migrated/ppa/upload/TechGuide.pdf

Wilson, H. B., B. E. Kendall, and H. P. Possingham. 2011. Variability in population abundance and the classification of extinction risk: classifying extinction risk. Conservation Biology 25(4):747-757. https://doi.org/10.1111/j.1523-1739.2011.01671.x

Address of Correspondent:
Jessica C. Stanton
2630 Fanta Reed Road
La Crosse, Wisconsin
54603 United States
jcstanton@usgs.gov
Jump to top
Table1  | Table2  | Figure1  | Figure2  | Figure3  | Figure4  | Figure5  | Figure6  | Figure7  | Appendix1  | Appendix2