The Challenge of Forest Diagnostics

Ecologists and practitioners have conventionally used forest plots or transects for monitoring changes in attributes of forest condition over time. However, given the difficulty in collecting such data, conservation practitioners frequently rely on the judgment of foresters and forest users for evaluating changes. These methods are rarely compared. We use a dataset of 53 forests in five countries to compare assessments of forest change from forest plots, and forester and user evaluations of changes in forest density. We find that user assessments of changes in tree density are strongly and significantly related to assessments of change derived from statistical analyses of randomly distributed forest plots. User assessments of change in density at the shrub/sapling level also relate to assessments derived from statistical evaluations of vegetation plots, but this relationship is not as strong and only weakly significant. Evaluations of change by professional foresters are much more difficult to acquire, and less reliable, as foresters are often not familiar with changes in specific local areas. Forester evaluations can instead better provide valid singletime comparisons of a forest with other areas in a similar ecological zone. Thus, in forests where local forest users are present, their evaluations can be used to provide reliable assessments of changes in tree density in the areas they access. However, assessments of spatially heterogeneous patterns of human disturbance and regeneration at the shrub/sapling level are likely to require supplemental vegetation analysis.


INTRODUCTION
Forests are of immense value to humankind, acting as carbon sinks, protecting biodiversity, providing essential ecosystem services, and enhancing the livelihoods of millions of people.However, the same forests that are of such immeasurable value to us are fast shrinking, degrading, and transforming.To better manage forest change, it is critical to have methods that can provide reliable, inexpensive, and rapid assessments of the manner in which forest condition is changing over time.Unfortunately, despite extensive acknowledgment of the criticality of such information for better management and conservation, there remains a substantial lack of rapid assessment methods that can be used to collect reliable data across a number of forested locations for the purpose of monitoring (Walpole et al. 2009, DeFries et al. 2010).Consequently, although widespread awareness of the issue exists, the available data on forest change are patchy, and not always consistent or reliable.
Remote sensing has proved to be very useful for global studies of forest-cover change, but forest degradation and regrowth processes often occur at levels undetectable by Earth observation alone (Nagendra andRocchini 2008, DeFries et al. 2010).Further, remotely sensed data require substantial ground truthing for monitoring applications (Ostrom andNagendra 2006, Tang et al. 2010).Thus, this approach is best suited for assessing changes in forest cover at broad scales from landscapes to larger bioregions, and less suited for the assessment of fine-scale changes in forest condition at specific locations.
Many conventional approaches to forest monitoring use protocols such as quadrants and transects to assess changes in forest biodiversity, density, biomass, and regeneration (Stohlgren 2007).These tools have been widely used for decades in forests around the world and are generally accepted as reliable approaches for evaluating performance, not only of protected areas but also of other forms of governance in diverse social-ecological settings.However, monitoring forest change through field plots alone is expensive, time-consuming, and difficult to conduct across large spatial scales.
Consequently, several prominent studies evaluating the effectiveness of forest management approaches have relied on qualitative judgments by park officials and professional foresters (Bruner et al. 2001, Ervin 2003).The continued use of these informal and rapid assessments can be seen from a recent article examining the management of over 1000 protected areas in Australia, which found that conservation practitioners overwhelmingly tend not to use evidence-based knowledge in making judgments.Instead, they rely almost entirely on knowledge acquired from their own prior experience (Cook et al. 2010).Such experiential assessments have not been verified through comparison with evaluations derived from conventional ecological forest monitoring protocols, and their widespread use has been categorized by these authors as "worrying" (Cook et al. 2010).
A number of recent studies have also suggested that forest users and volunteers should be able to provide rapid inputs for forest and biodiversity monitoring (Chhatre and Agrawal 2008, Holck 2008, Jones et al. 2008, Schmeller et al. 2009, Danielsen et al. 2010).Many studies have suggested incorporating local ecological knowledge into wildlife monitoring, because this is an area where accurate scientific knowledge on population trends is very limited, and where local knowledge can potentially serve as a valuable supplement (Jones et al. 2008, Danielsen et al. 2009).A recent debate in Ecology and Society addresses this issue from two different viewpoints.Gilchrist et al. (2005) suggest that Inuit communities are able to provide reliable information on population trends of marine bird species, but some species appear to be more accurately predicted than others.Thus, that paper, along with Gilchrist and Mallory (2007), concludes that local ecological knowledge can be a very useful supplement to scientific data but requires (1) a standardized approach to collection and (2) some rigorous testing at a local scale before it is widely incorporated into management planning.Brook and McLachlan (2005) strongly disagree with the second of these suggestions, stating that user evaluations constitute legitimate perspectives in and of themselves and should not be evaluated against the questionable standards of "objective" science.Both groups in the debate, however, agree on one aspect: that the difference in the scale and location of assessments conducted by locals and scientists can greatly impact the nature and specifics of assessments of ecological change.
Both groups also indicate, along with the more recent studies mentioned above, that user evaluations can be timely, cost effective, and provide reliable data.However, they run counter to beliefs widely held by many conservation practitioners and scientists who think that user and volunteer evaluations are biased, noisy, and unreliable.Unfortunately, as with assessments by practitioners and trained professionals, these arguments cannot be resolved by data, as the wider problem remains that forest assessments by local users have not been systematically compared with those derived from conventional, scientific protocols.Such comparisons are necessary if these monitoring mechanisms are to be used more widely as supplements to expensive, time-consuming, and labor-intensive biological monitoring (Jones et al. 2008, Danielsen et al. 2009, Schmeller et al. 2009, Danielsen et al. 2010).
In this paper, we address this issue by using data collected by the International Forestry Resources and Institutions (IFRI) Research Program, a longterm multicountry effort to collect data on forests and users using standardized interdisciplinary data collection protocols across multiple continents (Ostrom and Nagendra 2006, Tucker et al. 2008, Persha et al. 2011).From the start of this research program, we have used three methods for evaluating forest change: forest plots, assessments by professional foresters, and assessments by forest users.This dataset thus provides a unique opportunity to compare assessments of forest condition derived from conventional ecological assessments using field plots randomly distributed across a forest with assessments of change made by local forest users, which may be biased toward more intensively visited or utilized locations.Because so many research studies as well as field assessments used by managers and practitioners rely on the opinions of professional foresters (e.g.Bruner et al. 2001, Cook et al. 2010), we believe this is also an important type of assessment.Although they are often believed to be more authentic than evaluations http://www.ecologyandsociety.org/vol16/iss2/art20/provided by users, the reliability of this assertion has rarely been investigated using actual data.We also address this aspect in our investigations.

METHODS
For this study, we used methods developed by the IFRI Research Program that was initiated in 1992, when our research team at Indiana University was asked by the Food and Agricultural Organization of the United Nations to develop a multicountry effort to evaluate how diverse institutions affect forest conditions and forest sustainability.We designed a set of data collection instruments that would enable us to obtain multiple measures of forest conditions (Wollenberg et al. 2007).These instruments were applied in diverse forests located in a range of ecosystems and countries by a network of collaborating centers located in multiple countries around the world (Gibson et al. 2000).At present, collaborating research centers exist in Bolivia, Colombia, Ethiopia, Guatemala, India, Kenya, Mexico, Nepal, Tanzania, Thailand, Uganda, and the United States.The IFRI program is currently coordinated by Arun Agrawal at the University of Michigan (more details about IFRI are available at www.sitemaker.umich.edu/ifri/).
Our comparative assessment of forest change focused on a subset of 53 IFRI forests for which data were available from at least two visits at different times (mean = 5.6 years, standard deviation = 1.6 years).We selected five countries, i.e., India, Nepal, Kenya, Uganda, and the USA, where at least four forest locations had original visits and one or more revisits with complete data for the variables relevant to our study.When forests had data from more than two visits, we selected the two most recent visits.The distribution by country of the 53 forests selected for analysis was: 7 in India, 18 in Nepal, 5 in Kenya, 19 in Uganda, and 4 in the USA.Forests were sampled from a diversity of management regimes including governmentowned, strictly protected areas, forests managed and used by communities, and private forests managed by small groups such as families or church groups.
From the start of the IFRI Research Program, we used three different methods for monitoring forests, based on forest plots, assessments of relative condition by professional foresters, and evaluations of change by forest users.First, we used randomly located forest plots to obtain rigorous information on the number of trees, saplings, and shrubs, tree size, number of species, etc.At each time point, forest condition was assessed using randomly distributed 10-m circular plots, within which plant species' identity, height, and girth were recorded for all trees greater than 10-cm diameter at breast height (DBH).Within this, a 3-m circular plot was used to collect information on saplings and shrubs below 10-cm DBH.Between 20 and 60 random plots were located in each forest in our study.Although complete data on the sizes of forests were available for only protected areas accessed by communities and some community forests, more plots were laid in forests that appeared to be larger, denser, or richer in plant species.Although IFRI teams did not use a stratified sampling approach, the random sampling approach should have ensured assessment of a diversity of different patches and strata within each forest.Because the object of our study was to conduct comparisons across two points in time, the impacts of within-forest heterogeneity were minimized, especially because they should not have changed much during the relatively short average period of five years between samplings.Thus, the differences can largely be attributed to changes in human impact and management.
Forest plot data for our study were evaluated for changes in tree and sapling/shrub density.A nonparametric, one-tailed Mann-Whitney U test (p < 0.10) was used to categorize forests into those with a significant decrease or increase in tree or sapling/shrub density, and those where no significant change was observed over time.
It is problematic to compare forest condition across different ecological zones using measures of forest condition collected from plots sampled at one point in time (Tucker et al. 2008).Forests assessed at a single time point but located in different ecological zones, such as tropical moist forests and pine forests, will differ greatly in vegetation density and species richness.These differences cannot be attributed solely to differences in management, because the carrying capacity for species abundance and diversity for different ecological zones must be taken into account.In practice, this is very difficult to do.In contrast, changes in vegetation density and species diversity over time in the same forest can be more comparable even for forests located in different ecological zones.Thus, changes in forest condition can be attributed to a greater extent to changes in the intensity of forest use and the types of forest management and monitoring (Ostrom and Nagendra 2006).http://www.ecologyandsociety.org/vol16/iss2/art20/Thus, the IFRI research team adopted two additional approaches for assessing forest conditions and change.We asked the opinions of local forest user groups about changes in tree and shrub densities over time.Forest user groups are defined as people who harvest from, use, and/or maintain a forest and who share the same duties and rights to products from the forest.They can range from a formal community forest user group in some forests to, for example, groups of woodcutters and honey gatherers in another forest.After the initial days of a forest visit, once an IFRI team had an opportunity to become familiar with the rules of access and different user groups, some or all members of each group were interviewed and asked about their opinions of forest change.Within the group meeting, discussions about change normally ensued, until the group came up with an assessment of change with which the majority of members concurred.Care was taken to include a majority of users from different backgrounds; for instance, if the women in a group were not represented during these group discussions, separate discussions were held with them.In some cases, although very infrequently in our study, different sets of users, for instance, woodcutters and nontimber forest produce collectors, might have had different opinions on changes in the forest.In such cases, we combined the data from multiple groups to provide the dominant opinion about change based on the overall majority opinion.
Specifically, forest users in our study were asked whether the densities of trees and the bushy shrub/ sapling layer had changed in the previous five years.Answers were recorded on a three-point scale, which indicated whether, in the opinion of the majority of forest users interviewed, the forest had stayed the same, improved, or deteriorated.We then compared ratings by users based on changes in tree and shrub/sapling density, with evaluations of change based on an analysis of changes in tree and sapling/shrub density from the forest plots, using a Spearman rank correlation.
IFRI was also interested in assessments of forest condition provided by professional foresters, who accompanied each IFRI interdisciplinary research team to supervise collection of forest plot data.The forester who accompanies each team was not necessarily the same forester who accompanied the team on a previous visit.Thus, this person was not always familiar with the previous condition of the forest and could not be asked to directly judge changes over time.However, foresters or botanists accompanying IFRI teams had professional experience with the local area and were consequently familiar with the quality of the local forests in the region.They were also aware of the variations in forest biodiversity that are expected to occur in different ecological zones in the region, and would know, for instance, that the vegetation density and range of species found even in a degraded tropical rain forest can often be expected to be greater than those found in a good quality pine forest.
Therefore, we asked the forester to compare the forest being studied to other forests in the same ecological zone and assess if it was comparable, worse, or better in terms of vegetation density and species richness.This evaluation was done after the team finished collecting ecological data from forest plot sampling, so an informed assessment could be made.During forest sampling, professional foresters were asked, "In your best judgment, given the topography and ecological zone in which this forest is located, how would you judge the following attributes of this forest?"Answers with respect to vegetation density were recorded using a five-point scale at each time point, ranging from "very sparse" to "very abundant." Although such assessments do not provide direct evaluations of change, they provide important snapshots of the relative effectiveness, or uselessness, of forest management strategies.Repeated evaluations, which could be by the same person or by different foresters over time, provide an opportunity to also use them to evaluate change.We looked for changes in this evaluation at both time points to characterize forests as having deteriorated, improved, or experienced no perceptible change in vegetation density over time.Although this measure is not directly comparable to a users' evaluation, it can be compared to the forest plot analysis, where information collected from forest plots at different points in time is similarly compared to provide an evaluation of forest change.This challenge is also faced in other forest contexts, where users are familiar with their local forests and can readily provide evaluations of change, but where it is difficult to find professional foresters with detailed knowledge of specific locations, and who can reliably assess changes in forest condition for a diversity of forests over time.Thus, we compared foresters' and users' evaluations of change in vegetation density, with plot-based http://www.ecologyandsociety.org/vol16/iss2/art20/evaluations of change in tree density over time, using a Spearman rank correlation.We did not conduct direct comparisons of foresters' and users' evaluations, because they were derived using different approaches, necessitated by differences in the familiarity of users and foresters with specific forest locations.

RESULTS
Tree plot data analysis indicated that a large number of forests (41%) showed no significant change in tree density over time, while roughly equal proportions of forests showed an increase (32%) and a decrease (27%) in tree density (Fig. 1).User assessments provided a more positive picture of changes in tree density, indicating that 51% of forests exhibited an increase, and just 36% exhibited a decrease in tree density between time periods.Forester assessments provided a more neutral picture of change, indicating that 46% of forests did not show a significant change in tree density, while an almost equal proportion of the dataset recorded an increase (25%) and a decrease (26%) in vegetation density.
Users indicated a similarly neutral picture of changes at the shrub/sapling level, with just 12% of all forests recording no observed change in density, while equal proportions recorded an increase and decrease in shrub/sapling density.Plot data however provided a very different story, indicating that a scarce 11% of all forests showed a significant increase in density at this strata, compared with 42% of the forests that recorded a significant decrease (Fig. 1).
Pairwise comparisons of the data enabled us to further assess the relationship between assessments provided by different methods.There was a strong positive relationship between plot-based assessments and user assessments of changes in tree density.A Spearman rank order correlation between the two variables was strong, with a Spearman r of 0.48, and highly significant, with p < 0.01.The assessments were identical for just over half (51%) of the forests (Fig. 2).Twenty-three percent of all forests recorded an increase in both assessments, 8% did not record a significant change, and 21% recorded a decrease in both assessments.We categorized disagreements into two types: strong disagreements, where one assessment indicated an increase while the other indicated a decrease, and weak disagreements, where one assessment indicated an increase and the other indicated no perceived or significant change.When looking at the discrepancies between assessments, strong disagreements accounted for only 10% of the overall forests, while weak disagreements accounted for the remainder (39%).
Plot-based assessments of change in tree density were not well related to forester assessments of change in vegetation density, however, with agreement taking place in only 34% of the forests (Fig. 3).A Spearman rank correlation between the two variables was weak, with a Spearman r of 0.14, and not statistically significant, with p = 0.32.While these two assessments disagreed in two out of every three forests (66%), as indicated by the off-diagonal columns in Fig. 3, weak disagreements accounted for over half of the forests surveyed (56%), with strong disagreement in 10% of the cases.
Plot-based assessments of change in sapling/shrub density showed a weaker relationship with user assessments at this level (Fig. 4), with a Spearman r of 0.26, which was weakly significant at p = 0.06.These two assessments provided similar responses for 32% of the forests, and conflicted strongly in 15% of the forests, while weak disagreements accounted for over half of the forests surveyed (53%).

DISCUSSION
Although plot-based indicators provide widely accepted, time-tested, reliable, and useful assessments of forest change, such data are expensive, time-consuming, and difficult to collect for large spatial scales.Thus, in practice, many forest managers appear to rely on assessments from foresters (Bruner et al. 2001, Danielsen et al. 2009, Cook et al. 2010), while many recent studies also suggest that users can provide important evaluations of ecological change based on local ecological knowledge (Gilchrist et al. 2005, Jones et al. 2008, Danielsen et al. 2009).Therefore, it is important to understand the extent to which these assessments provide similar or contrasting perspectives of forest change.Across 53 forests located in five countries, our results indicate that assessments of change in tree density made by forest users are significantly and strongly related to assessments made based on statistical analysis of forest plot data.Assessments of change in density at the shrub/sapling level http://www.ecologyandsociety.org/vol16/iss2/art20/appear more difficult for users to gauge accurately, and user assessments paint a fairly neutral picture of overall forest change.In contrast, statistical analysis of forest plot data reveals a more alarming picture of decrease in density at the shrub/sapling level.Nevertheless, these assessments do appear to be somewhat related, albeit at a lower level of significance and strength.It must be noted here that user evaluations are likely to be biased spatially toward areas more frequently visited or used (as also suggested by Gilchrist et al. 2005), whereas distant locations may be better represented, and spatial coverage may be less biased, by the use of random plots.Although users were asked about changes over five years, in some cases the time between visits was greater, and this may also introduce some discrepancies.However, the overall correspondence between these evaluations is interesting and provides us with significant corroborations of their overall reliability.
It appears to be far more difficult to use professional foresters to provide reliable assessments of forest change, possibly because of the difficulty in obtaining direct information on change from foresters familiar with different forested locations.Because forest users are expected to be familiar with the forest on which they depend, they could be directly asked for their opinion of how the forest had changed in density during the previous five years.Professional foresters on the IFRI research teams could not be asked similar questions, because it would be difficult to find a professional forester with intimate knowledge of each local forest, and awareness of its change over time.Instead, we asked foresters to compare the condition of a forest patch with similar patches in the same ecological zone, and looked for changes in these assessments over multiple visits.Most foresters on IFRI research teams were familiar with the general condition of forests in the larger region within which the study forests were embedded, and were able to provide reliable assessments for single points of time (Varughese and Ostrom 2001).However, these assessments may be less useful to perceive changes in forest condition.Although we are not suggesting that the use of qualitative perceptions of foresters should be avoided, policy makers and managers need to take into account the background and familiarity of specific individuals with specific forest areas, and accordingly frame appropriate questions.
Thus, we found a strong correspondence between plot and user assessments of change in tree density, and a weaker correspondence between plot and user http://www.ecologyandsociety.org/vol16/iss2/art20/assessments of change in sapling/shrub density.Human disturbance including fire, grazing, and charcoaling has a large impact on the sapling/shrub layer, which is especially responsive to such disturbances (Ostrom and Nagendra 2006).These disturbances on forests are not uniformly distributed, tending to be greater in areas with greater accessibility and lower in areas where there is greater monitoring and enforcement (Schweik 2000).Similarly, forest plantations and protected community-managed or sacred patches are also located in specific parts of the forest.The forest plots are randomly distributed and hence provide information from areas with variable levels of human impact.Our field observations, however, suggest that forest users tend to focus on change in areas that they visit most frequently, which they know are increasing or decreasing in density.This could explain the greater variation between plotbased and user-derived assessments of change at that level, with users documenting information from specific parts of the forest that are protected and planted, or degrading, while randomly distributed forest plots provide a more comprehensive, statistically representative picture.Gilchrist et al. (2005) suggest similarly that users will be more familiar with changes in bird populations in areas they visit more frequently, whereas isolated locations will be less monitored.Brook and McLachlan (2005) corroborate this, stating that differences in the scales and locations of observation can impact differences in the assessments contributed by local ecological knowledge and scientific studies.

CONCLUSIONS
There is widespread agreement about the importance of maintaining forests for protecting biodiversity, sequestering carbon, and providing ecosystem services to the people of the world.However, considerable disagreement exists about how best to govern and manage forests, whether through large-scale, government-owned reserves, relatively small-scale community-owned andmanaged forests, or a variety of other approaches.This debate is largely exacerbated by the lack of available, reliable data that measure changes in forest condition across multiple forests and facilitate data-based assessment of how changes in management impact forest condition and conservation.Detailed information on forest change is also essential for forest managers and local users who seek to protect their forests and manage them for sustainable use without impacting forest condition.However, monitoring forest change remains a critical challenge for the future (Cook et al. 2010, DeFries et al. 2010).Although quantitative evaluation of change using plots and transects constitutes the most widely accepted approach for monitoring forest condition and change (Stohlgren 2007), it is expensive, logistically challenging, and time-consuming, and therefore incapable of scaling up for applicability across multiple forests.
Consequently, many forest practitioners and managers have relied on their own qualitative observations of forest change to make management decisions (Bruner et al. 2001, Cook et al. 2010), while a growing number of recent studies seek to also incorporate evaluations by forest users (Chhatre and Agrawal 2008, Holck 2008, Danielsen et al. 2009).These evaluations, however, have been largely unverified against conventional ecological monitoring approaches.
Based on a study of 53 forests located in India, Nepal, Uganda, Kenya, and the USA, we find that evaluations of changes in tree density derived from forest plots are largely congruent with assessments made by forest users.There is a greater divergence between plot-based assessments of changes in sapling/shrub density and assessments made by forest users.This difference may indicate the spatially variable nature of changes in density at the shrub/sapling level because of the patchy nature of forest protection and human disturbance.Such disturbance can be better perceived by randomly distributed forest plots because users are likely to report change in the areas of the forest most visited by them, often either protected and actively managed or most disturbed because of human use and extraction.Even here, there is weak disagreement between these two assessments, and perhaps user assessments can be adopted to identify areas at greatest risk, where plot-based assessments could then be made.
Naturally, we cannot become fixed on any one method of measuring forest conditions over time.Just as forests are managed in a variety of ways across the world, we will probably need to use a variety of approaches to assess changes in forest condition, depending on practical considerations of time, cost, and logistics, as well as what information is most important to the evaluator.Forest plots cannot be deployed in all forests because of constraints of time and money, challenges of logistics, and availability of human resources.Foresters' evaluations are useful for the quick assessment of forest condition in comparison to nearby forests in the same ecological habitat that have different management regimes, but it is difficult to use them to assess changes in forest density.
Evaluations by forest users of changes in tree density appear to provide reliable indicators of changes, which may be particularly useful in a world where more forests will be evaluated for their ability to store carbon, and quick assessments of changes in tree density need to be made at frequent intervals.Evaluations made by forest users of changes in http://www.ecologyandsociety.org/vol16/iss2/art20/shrub/sapling density are less reliable but still provide a slight indicator of forest change, related, of course, to the level of knowledge they have about their local forests.If the users are self-organizing, their evaluation of how the forest is responding to their efforts and to external disturbances will be very well informed, perhaps as much so as information collected from forest plots.Indeed, local communities' participation in monitoring can lead to greater levels of collective action and participation in other management activities, as has been observed in fisheries in the Philippines (Uychiaco et al. 2005), protected forests in the Philippines (Danielsen et al. 2005(Danielsen et al. , 2010)), and forests in Tanzania (Danielsen et al. 2010).If a forest has no local users, or users are restricted from accessing a forest, as is the case in many protected areas, plots and other quantitative ecological approaches will be essential to accurately measure changes in resource quality and use.

Acknowledgments:
The IFRI Research Program has received funding from the FAO of the United Nations, the Ford and MacArthur Foundations, and the U.S. National Science Foundation.We thank Ashwini Chhatre, Arun Agrawal, and Burney Fischer for feedback on earlier versions of this manuscript; the Department of Science and Technology, Government of India, for financial support through a Ramanujan fellowship; Lionel Sujay Vailshery, Madhumitha Jaganmohan, and Suparsh Nagendran for assistance with data analysis; our IFRI colleagues around the world for their collaboration; and the numerous forest users and foresters who assisted us in our enquiries.

Fig. 1 .
Fig. 1.Assessments of changes in forest vegetation density based on indicators derived from forest plots and assessments by forest users and professional foresters.

Fig. 2 .
Fig. 2. Plot assessment of change in tree density vs. user assessment of change in tree density.

Fig. 3 .
Fig. 3. Plot-based assessment of change in tree density vs. forester assessment of change in vegetation density.

Fig. 4 .
Fig. 4. Plot assessment of change in sapling/shrub density vs. user assessment of change in sapling/shrub density.