Improving attitudes and knowledge in a citizen science project about urban bat ecology

. In order to deal with the current, dramatic decline in biodiversity, the public at large needs to be aware of and participate in biodiversity research activities. One way to do this is citizen science projects, in which researchers collaborate with volunteering citizens in scientific research. However, it remains unclear whether engaging in such projects has an impact on the learning outcomes of volunteers. Previous research has so far presented mixed results on the improvement of citizens’ attitudes and knowledge, mostly because such research has focused only on single aspects of citizen science projects in case studies. To address these limitations, we investigated the impact of an urban bat ecology project on citizens’ attitudes and knowledge about bats, and on their engagement with citizen science. We also examined whether the degree of citizen participation (i.e., collecting data vs. collecting and analyzing data) had an influence on the outcomes. We conducted four field studies and used a survey-based, experimental, pre-/post-measurement design. To vary the degree of participation, we assessed the post measurement in one group directly after data collection, whereas, in a second group, we assessed it after data collection and analysis, at the end of the project. Across all studies, the results demonstrated that citizens’ content knowledge of urban bat ecology increased, and their attitudes toward bats and toward their engagement in citizen science improved during their participation. Citizens’ degrees of participation did not influence these outcomes. Thus, our research illustrates that citizen science can increase awareness of urban bat conservation, independently of citizens’ degree of participation. We discuss the implications of our findings for the citizen science community.


INTRODUCTION
Projects for public participation in scientific research have the potential to address scientific and societal issues (Shirk et al. 2012), such as the conservation of biodiversity. One way citizens can participate in such research is to volunteer for citizen science projects, in which professional scientists collaborate with volunteers in scientific research (Heigl et al. 2019). For examples of citizen science see https://www.birds.cornell.edu/citizenscience (Bonney et al.2009), https://www.ispotnature.org (Silvertown et al. 2015), and https://www.zooniverse.org (Cox et al. 2015). Previous research has highlighted the potential benefits of citizen science projects for citizens' individual learning outcomes, among other outcome categories (Shirk et al. 2012, Phillips et al. 2018). In particular, it has been suggested that citizens might gain knowledge and skills, or change their attitudes or behavior (Bela et al. 2016). It has also been assumed that such projects increase citizens' feelings of psychological ownership for the citizen science project (Pierce et al. 2001(Pierce et al. , 2003 and feelings of pride in their participation (Rotman et al. 2014, Jordan et al. 2015, Haywood et al. 2016, Lewis 2016).
However, the potential of citizen science to increase such learning outcomes is not well understood, because robust scientific evidence is lacking (Toomey and Domroese 2013, Jordan et al. 2015, Phillips et al. 2018. Even though most citizen science researchers agree that such projects should increase citizens' content knowledge of the project topic and improve their attitudes toward the topic and toward citizen science and science in general (see also Bruckermann et al. 2021a), research results are mixed. Although some research has demonstrated increases in content knowledge (Brossard et al. 2005, Trumbull et al. 2005, Jordan et al. 2011, other research has found little or no improvement in scientific understanding or attitudes (Trumbull et al. 2000, Crall et al. 2013), or has not systematically investigated outcomes such as ownership and pride (for an exception see Greving et al. 2020).
There may be two reasons for these mixed findings. First, they may be caused by a lack of clearly conceptualized measures of learning outcomes (Becker-Klein et al. 2016, Phillips et al. 2018, Peter et al. 2019. For example, instruments consisting of several questions were used that were based on participants' self-reports, but these instruments had low internal consistencies (Brossard et al. 2005, Crall et al. 2013. Other research used only indicator variables, e.g., interest for motivation (Rotman et al. 2014), and relied on subjective assessments of knowledge and attitude changes (Toomey and Domroese 2013). Second, the mixed findings may have been caused by a lack of rigorous study designs, e.g., experimental studies (Phillips et al. 2018, Dickinson and Crain 2019, Aristeidou and Herodotou 2020, Kloetzer et al. 2021). Indeed, many previous studies only described citizen science projects without using any statistical tests (Fernandez-Gimenez et al. 2008, Toomey andDomroese 2013). If using statistics, these studies were mostly pre-/post-test studies (Druschke and Seltzer 2012, Sickler et al. 2014, Peter et al. 2019. Such methods may have prevented previous researchers from drawing more general conclusions concerning the extent to which citizens' participation can improve their individual learning outcomes (Masters et al. 2016).
According to models of public participation in scientific research (PPSR; Shirk et al. 2012), the degree of citizens' participation is the extent to which they are involved in different steps of the scientific research process. These models of PPSR represent different project models that vary in the possible degree of citizens' participation. In contributory project models, citizens only collect and contribute data to scientific research, whereas in collaborative projects, they additionally engage in data analysis to interpret the research findings (Shirk et al. 2012). There is, however, a lack of systematic investigation of whether the degree of participation influences the outcomes of citizen science projects. One experimental study focused on the effects of projects on individual learning outcomes, and used a rigorous experimental design (Dickinson and Crain 2019), comparable to before-after control-impact designs (Christie et al. 2019). Although this study found no difference between a participant and a control group, it also did not consider the different degrees of participation.
In the research presented here, we used an experimental design and rigorous measures in order to analyze data from four field studies of a citizen science project about urban bat ecology. We investigated whether participating in the project increased citizens' content knowledge of urban bat ecology, and improved their attitudes toward bats and engagement in citizen science, and their feelings of psychological ownership and pride. To answer these research questions, we used an experimental pre-/postmeasurement design, and varied the point in time of the post measurement between two groups to which participants were randomly assigned (Fig. 1). In the data collection only group, participants engaged in data collection and, directly afterwards, completed the post measure. In the data collection and analysis group, we assessed the post measure after both data collection and data analysis were completed. Although we expected, overall, that participation in the project would have a positive effect on citizens' attitudes and knowledge, we also assumed that a higher degree of participation (i.e., participating in both data collection and analysis) should be even more beneficial for improving citizens' learning outcomes (Lawrence 2006, Bonney et al. 2009). Therefore, we stated the following hypotheses: . Attitudes toward bats improve during participation (hypothesis 1a); this improvement is stronger for the data collection and analysis group than for the data collection only group (hypothesis 1b).
. Content knowledge for bat ecology increases during participation (hypothesis 2a); this increase is stronger for the data collection and analysis group than for the data collection only group (hypothesis 2b).
. Attitudes toward engagement in citizen science improve during participation (hypothesis 3a); this improvement is stronger for the data collection and analysis group than for the data collection only group (hypothesis 3b).
We exploratively tested the effects of participation and degree of participation on psychological ownership and pride.

METHODS
In order to test our hypotheses, we conducted four field studies using identical procedures (Table 1). These studies were part of a citizen science project about urban bat ecology called "Bat Researchers" that took place in a German metropolitan city. The biological aim of the project was to investigate the presence of bats in the urban ecosystem. The citizens' task was to walk along a pre-defined route on two evenings during a two-week period and record the echolocation calls of flying bats with a bat detector capable of detecting and recording ultrasonic frequencies. After the data collection only group had completed their walks, they returned the bat detectors to the project scientists. Then, the data collection and analysis group did their evening walks and after completion handed over the bat detectors to the project scientists. Based on the ultrasonic recordings on the bat detectors, the scientists identified the bat species and provided the data to both groups for further analysis and discussion of the results.
We used an online platform for all the other activities that participants could perform in the project besides data collection with the bat detector. In particular, the platform provided tutorials for the identification of bat species and information about urban bat ecology to support participants in data collection and analysis. On this platform, participants uploaded their collected data and https://www.ecologyandsociety.org/vol27/iss2/art24/ Note: We tested for differences between the field studies concerning the dependent variables and the explorative variables. But the field studies mostly did not differ from each other on each of these variables: attitude toward bats: downloaded the species identifications provided by the scientists.
They had the opportunity to analyze their own data as well as the complete dataset of all routes on which participants collected data. They could, for example, examine the correlations between bat activity and environmental features, such as proximity to water or tree cover. To analyze the data, participants followed a structured analysis process comparable to the usual scientific analysis process, i.e., formulate the research question, formulate hypotheses, specify the independent and dependent variables as well as their relationship, run tests for differences or for associations, and inspect, visualize, and interpret the findings. Citizens could discuss their findings and questions concerning the project and the topic with other citizens and with the project scientists in a forum.
Via this platform, participants also filled out questionnaires. After filling out the pre-measure questionnaire (T1), participants in the data collection only group (N = 64) filled out the post-measure questionnaire (T2) after data collection was completed. Participants in the data collection and analysis group (N = 75) filled out the post-measure questionnaire (T2) after both data collection and data analysis were completed (Fig. 1). All of the measures (

Dropout analysis and participants
We recruited participants via public outreach campaigns targeted at the general public. These participants themselves chose to participate in the project, were very likely quite interested in bats, and were willing to invest their leisure time in participating in the project. Each recruited participant could only participate once in one of the field studies, and each participant recorded the bats' echolocation calls with the bat detector. Across all four field studies, 224 participants filled out the pre-measure questionnaire, and 139 participants also completed the post-measure questionnaire. This was a dropout rate of 37.9% of those filling out the questionnaire. However, these participants did not drop out of the project. Participants who dropped out did not differ from those participants who completed both questionnaires in their gender, χ²(2) = 0.78, p = 0.678. Participants who completed both questionnaires were older, t(222) = -2.88, p = 0.004, and had a higher level of education, t(222) = -0.29, p = 0.023, than those participants who only filled out the pre-measure questionnaire. Thus, we included 139 participants in our analyses; see Table 1 for demographics.

Measures
The details of each measure are presented in Table 2. We assessed participants' attitudes toward bats with 12 rating-scale items based on general attitude approaches (Bohner andDickel 2011, Albarracin andShavitt 2018). To measure citizens' content knowledge of urban bat ecology, we pre-identified the most relevant topics from the perspective of citizens and scientists Note: RS = rating scale on a 5-point scale ranging from 1 (does not apply at all) to 5 (completely applies), SC = single-choice questions, MC = multiple-choice questions. (Bruckermann et al. 2022) by means of a Delphi approach (e.g., Blanco-López et al. 2015). Using these topics as a basis, we then constructed 29 single-and multiple-choice questions. Finally, we divided participants' correct answers by the total number of questions and assessed their content knowledge as the percentage of correct answers.
We assessed participants' attitudes toward engagement in citizen science with five underlying dimensions (Summers and Abd-El-Khalick 2018) following the theory of planned behavior (Ajzen 1991, Fishbein and Ajzen 2010). With three rating-scale items each, we measured attitudes toward citizen science, intentions to engage in citizen science projects, behavioral beliefs, control beliefs, and normative beliefs. Similarly, we measured psychological ownership (Pierce et al. 2001, 2003, Peck and Shu 2009) as well as pride (Lewis andSullivan 2005, Lewis 2016) with three rating-scale items each.

Statistical analysis
To test our hypotheses, we conducted mixed analyses of variance (ANOVAs) with degree of participation (data collection only group vs. data collection and analysis group) as between-group factor and participation (between the measurement points T1 vs. T2) as within-group factor in all analyses. We used SPSS Version 22.0 for this purpose (IBM Corporation 2013). We set the level of significance < 0.05 and used two-tailed tests throughout all analyses.

RESULTS
All test statistics are presented in Table 3. Compared with T1, all participants had a more positive attitude toward bats and more content knowledge of urban bat ecology at T2 ( Table 2, Fig. 2), which supported hypotheses 1a and 2a. There were no further effects, which did not support hypotheses 1b and 2b. For the five underlying dimensions of attitudes toward engagement in citizen science, there were similar results. Compared with T1, participants in both groups had a more positive attitude, higher  Fig. 3), which supported hypothesis 3a. None of the other effects was significant, with the exception of the interaction effect between participation and degree of participation for normative beliefs (Table 3). However, the degree of participation groups did not differ at each of the measurement points. Thus, overall, there was no support for hypothesis 3b. Finally, when we included the five underlying dimensions of attitudes as an additional within-group factor into the mixed ANOVA, this analysis also found that all participants had a more positive attitude toward engagement in citizen science in general at T2 than at T1 ( Table 2). With respect to psychological ownership, we did not find any significant differences. The data collection only group experienced more pride than the data collection and analysis group, but the other effects were not significant (Fig. 2).

DISCUSSION
The research presented here investigated the impact of a citizen science project about urban bat ecology on citizens' content knowledge about bats, and attitudes toward bats and toward engagement in citizen science. Our findings demonstrated that knowledge increased and attitudes improved during citizens' participation in the research process. In particular, the increase in citizens' content knowledge about urban bat ecology was more pronounced than the improvement in their attitudes, which is in line with previous research (Peter et al. 2019). Most previous studies agree that citizen science projects enhance citizens' content knowledge (Druschke and Seltzer 2012, Bela et al. 2016, Haywood et al. 2016. Findings on citizens' attitudes have been less conclusive and revealed small to negative changes in attitudes (Brossard et al. 2005, Druschke andSeltzer 2012). Our study adds to the picture by showing significant and medium-sized changes for both attitudes toward bats and toward engagement in citizen science. Moreover, our findings extend previous studies by not only distinguishing between attitudes toward bats and sciencerelated attitudes (e.g., Peter et al. 2019), but also differentiating among various attitudinal domains, which we captured using multi-items measures. Based on the theory of planned behavior (Ajzen 1991, Fishbein and Ajzen 2010), our findings showed different changes in the attitudinal domains. They revealed stronger changes in citizens' beliefs about their ability to participate in citizen science, along with no changes in their beliefs about its usefulness for their personal lives. Thus, our research has demonstrated that "Bat Researchers" project had the potential to improve citizens' learning outcomes.
Furthermore, this research set out to investigate whether the degree of citizens' participation in the research process has an impact on their learning outcomes. Inquiry-based learning opportunities combine citizens' participation in the different steps of the research process and in scaffolding structures that support Ecology and Society 27(2): 24 https://www.ecologyandsociety.org/vol27/iss2/art24/ their understanding (Aristeidou et al. 2020). If citizens participate in the data collection and are supported with a tutorial to record bats' echolocation calls, they could increase their knowledge of distinguishing among bat species. For instance, Prather et al. 2013 demonstrated the influence of identifying galaxies on citizens' knowledge about galaxy morphology. If citizens participate in data analyses and are provided with the data and a tool to test their assumptions on the influence of environmental features on bat species, they may develop their knowledge in a different way and increase their understanding of urban bat ecology. Our findings demonstrated that these degrees of citizens' participation did not seem to have an influence on the outcomes. This means that citizens' additional engagement in data analysis did not affect any improvements in attitudes toward bats or toward engagement in citizen science, or any increase in knowledge acquisition about bats. Our findings extend previous research on the relationship between degree of participation and learning outcomes by using the exact same citizen science project and research context (i.e., bat ecology) in both conditions and by directly comparing learning outcomes of citizens who could participate on a contributory level (i.e., providing data) with those learning outcomes of citizens who could participate on a collaborative level (i.e., analyzing data and discussing findings with citizens and scientists).
Our findings may also contradict previous assumptions that were derived from the so-called "Arnstein's ladder" (Arnstein 1969; see Haklay 2013 for an overview). These assumptions postulated that the higher the degree of participation, the better for citizen science outcomes. On the one hand, participating in the offered activities of the project was in our study enough to increase learning outcomes, independently of the degree of participation. This finding may be good news for the citizen science community, because learning from participation in projects does not seem to be limited to higher degrees of participation but may depend on the offered activities. On the other hand, it could also be that citizens did not engage in the data analysis sufficiently enough (T. Bruckermann, H. Greving, M. Stillfried, A. Schumann, M. Brandt, and U. Harms, unpublished manuscript) to have any additional effect on the outcome measures. In line with previous research, learning outcomes may indeed be more closely related to the prerequisites in projects, such as citizens' goals and abilities for participation, e.g., motivation (Phillips et al. 2019) and scientific reasoning skills (Stylinski et al. 2020), than to the degree of participation (Shirk et al. 2012). Behavioral data from future research on how citizens actually participate in different scientific activities may help explain why engaging in the data analysis had no additional effects. Future research may also need to specify either the prerequisites of the participants, e.g., scientific reasoning skills (Bruckermann et al. 2021b), or the prerequisites of the project, e.g., training on data analyses (Gray et al. 2017), under which citizens' opportunities to analyze data have beneficial effects for outcome measures in similar projects.
Finally, the findings of the explorative measures were informative. We found that the data collection only group experienced more pride in their participation. Here, the assumption that a higher degree of participation would benefit outcomes also did not hold (Shirk et al. 2012). In contrast, asking citizens directly after their evening walks and data collection could have activated their feelings of pride more readily. These feelings might have already faded away for those citizens in the data analysis group who answered the post measurement at the end of the project. Apart from this, citizens could have seen their contribution as just collecting data, not analyzing it (Phillips et al. 2019). This suggestion is also in line with recent research that analyzed activity patterns of citizens who used an online platform during a citizen science project (T. Bruckermann, H. Greving, M. Stillfried, A. Schumann, M. Brandt, and U. Harms, unpublished manuscript). These data showed that citizens were mainly active during data collection and more passive during data analysis. But more research needs to be conducted about the conditions under which the engagement in data analysis has beneficial effects on citizen science outcomes.
The strengths of the studies were their standardized and rigorous approaches. We conducted externally valid studies and used samples of participants that were representative of typical citizen science volunteers. The sample size was also large enough to generate sufficient statistical power. We used established and objective measures that, overall, had sufficient internal consistencies. Moreover, by employing both a data collection only group and a data collection and analysis group, the long-debated construct of degree of participation (Shirk et al. 2012) was successfully implemented and experimentally tested, a relevant step forward for the citizen science community.
There were also some limitations. First, the results revealed that citizens' attitudes toward bats and toward engagement in citizen science improved. On the one hand, these are merely attitudes and it is unclear whether citizens would also act in accordance with their attitudes. Thus, there may be a gap between attitude and behavior in the areas of bat conservation and engagement in citizen science. On the other hand, there is a solid body of research and several frameworks that clearly indicate that attitudes are highly relevant predictors of behavioral intentions and actual behavior, e.g., theory of planned behavior (Ajzen 1991, Fishbein and Ajzen 2010; for other models see Sheeran et al. 1999, Webb and Sheeran 2006, Albarracin and Shavitt 2018. This means that, although we did not assess actual behavior, the changes that we found in attitudes have the potential to initiate behavioral changes in citizens. Second, we developed our questionnaire on content knowledge based on questions that citizens frequently ask about bats living in the city. The changes in knowledge might have been different if we had asked citizens for their formal scientific knowledge of bat ecology instead of their specific local knowledge (Stocklmayer and Bryant 2012).
Third, our project about urban bat ecology was open to and directed at the public, and citizens could apply for it if they were interested in participating. Thus, we did not analyze a sample that was representative of the general population, but rather a selfselected sample of citizens who showed a general interest in bats, meaning the findings of our studies may be limited to people who are already enthusiastic about bats. We also had a high dropout rate of citizens who did not fill out the post measurement, although they continued participating in the project itself. This dropout may have been caused by the fact that participation in the whole project and the questionnaires was completely voluntary; we did not give citizens any incentives for their participation. Researchers could possibly pay monetary incentives to their participants for completing questionnaires in https://www.ecologyandsociety.org/vol27/iss2/art24/ future studies. This could help address both concerns, i.e., create a more diverse and representative sample, and decrease dropout across the points of measurement.
Finally, the sample size of the participating citizens across the field studies created enough statistical power to conduct mixed ANOVAs. But with an even larger sample size, we could have calculated larger path models with latent variables to test our hypotheses (Bruckermann et al. 2021a). Because of the expected sample size, we also implemented the variation in degree of participation with two groups. With a larger sample size, we could have also measured citizens' actual level of participation in the different scientific tasks and could have used those measurements as predictors in the models. On the other hand, such an approach could have produced subsamples with totally different sample sizes, because citizens might have rather engaged in data collection than data analysis (T. Bruckermann, H. Greving, M. Stillfried, A. Schumann, M. Brandt, and U. Harms, unpublished manuscript).

CONCLUSION
In summary, our research investigated the impact of a citizen science project about urban bat ecology on citizens' knowledge acquisition about urban bat ecology, and their attitudes toward bats and toward engagement in citizen science. Our findings present evidence that attitudes and knowledge improved during citizens' participation, largely independently of their degree of participation (i.e., whether they only engaged in data collection, or in data collection and analysis). Thus, if citizen science practitioners wish to conduct a project in order to increase citizens' attitudes and knowledge, it may be enough to engage them in data collection along with the other offered activities (e. g., tutorials), because additional data analysis did not alter the effect. However, we acknowledge that, if citizens understand and learn in the future that they can also be valuable data analysts, additional engagement in data analysis may have the potential to further improve attitudes and increase knowledge.