Learning , Signaling , and Social Preferences in Public-Good Games

This study compares the empirical performance of a variety of learning models and theories of social preferences in the context of experimental games involving the provision of public goods. Parameters are estimated via maximum likelihood estimation. We also performed estimations to identify different types of agents and distributions of parameters. The estimated models suggest that the players of such games take into account the learning of others and are belief learners. Despite these interesting findings, we conclude that a powerful method of model selection of agent-based models on dynamic social dilemma experiments is still lacking.


INTRODUCTION
Social dilemmas are situations in which behavior that is rational for and in the self-interest of individuals results in socially suboptimal outcomes.Most environmental problems, such as clean air, the management of common-pool resources, recycling, etc., involve social dilemmas.
Experimental research has contributed a great deal to the understanding of the factors that affect the level of cooperation in repeated social-dilemma games, such as games that provide public goods and common-pool resources (CPR).Many scholars now agree that some players do not seem to be interested in maximizing their own incomes, that players are heterogeneous on several dimensions, and that rates of cooperation are affected by the pay-off functions, matching protocols, and other institutional features of experimental treatments (Ledyard 1995, Ahn et al. 2003, Ones and Putterman 2004).
To date, however, no widely accepted models of individual decision making exist that provide the micro-foundations for such empirical regularities.What are the motivations and learning rules used by the players?Do players differ from one another in significant ways?If so, what are the key dimensions of such heterogeneity?To what extent, and in what manner, do players in repeated social-dilemma games learn from past experiences during the game?Do some players behave strategically to increase their future pay offs?In sum, we need micro-level models of individual behavior that are open to heterogeneity across players to advance our knowledge of the dynamics in repeated socialdilemma games.
Significant progress has been made by behavioral economists, game theorists, and experimentalists who have developed rigorous models of behavior in game settings and tested these with controlled experiments with human subjects (see Camerer 2003 for a review).These models are often called models of learning in the sense that they explain the emergence of equilibrium over time.Socialdilemma research can greatly benefit from taking these efforts seriously when providing micro-level explanations of macro-level regularities in N-person social-dilemma games.
On the other hand, the study of learning models can expand its horizons greatly by taking N-person social-dilemma games seriously.Most of the learning models have been applied to rather simple games, which is understandable given that the formulation and testing of those learning models are often very complicated tasks.The increasing level of sophistication in the formulation of the models and their tests now allows us to expand the horizon of learning models to more complicated game settings.N-person social-dilemma games can test http://www.ecologyandsociety.org/vol11/iss2/art21/alternative learning models in more demanding contexts and provide an opportunity to develop more relevant models of behavior.This paper attempts to expand the horizon of behavioral models to repeated N-person socialdilemma games.Specifically, this study compares the empirical performance of several alternative learning models that are constructed based on the social preferences model of Charness and Rabin (2002) and the behavioral learning model of Camerer and Ho (1999).The models are tested with experimental data drawn from the public-good experiments by Isaac and Walker (1988) and Isaac et al. (1994).
Motivated by experimental observations that are not consistent with equilibrium predictions, researchers have developed models of learning in which players learn to play the equilibria as a game repeats.Earlier efforts to model a learning process in repeated games include reinforcement learning or routine learning (Bush andMosteller 1955, Cross 1983), fictitious play and its variants (Robinson 1951, Fudenberg and Kreps 1993, Young 1993, Fudenberg and Levine 1995, Kaniovski and Young 1995), and replicator dynamics (Fudenberg and Maskin 1990, Binmore and Samuelson 1992, 1997, Ellison and Fudenberg 1993, Schlag 1998).To test the explanatory power of these models more rigorously, many game theorists and experimentalists began to use specific experimental data (Crawford 1995, Crawford and Broseta 1998, Cheung and Friedman 1997, Broseta 2000).However, these studies tend to test a single model, usually by estimating the parameters of a specific model using a set of experimental data.
More recently, researchers began to compare the explanatory power of multiple models using data from multiple experimental games, which represented a step forward from the previous approaches.These studies include Boylan and El-Gamal (1993), Mookherjee andSopher (1994, 1997), Ho and Weigelt (1996), Chen and Tang (1998), Cheung and Friedman (1998), Erev and Roth (1998), Camerer and Ho (1999), Camerer andAnderson (2000), Feltovich (2000), Battalio et al. (2001), Sarin and Vahid (2001), Tang (2001), Nyarko and Schotter (2002), Stahl and Haruvy (2002), and Haruvy and Stahl (2004).Another noticeable aspect of the current research is the careful examination of the testing methods themselves and the use of multiple criteria of goodness-of-fit (Feltovich 2000, Bracht and Ichimura 2001, Salmon 2001).Thus, the research has progressed from parameter fitting of a single model to rigorous testing of alternative models in multiple game settings and to careful examination of testing methods.
We extend this comparative approach in several ways.First, the decision-making setting that we study involves the provision of public goods in which the predicted equilibrium of zero contribution has repeatedly been shown to misrepresent actual behavior.Thus, in framing our research, we find that "learning" is not necessarily the general theme.
There are dynamics at individual and group levels.However, it is still an open question, especially in repeated social dilemma games, whether those dynamics result from learning or other mechanisms such as forward-looking rational and quasi-rational choices.In general, we entertain the hypothesis that heterogeneity across players on multiple continuous dimensions is the key aspect of the microfoundations that generate the observed dynamics in repeated public-good games.
Second, several other factors posed challenges to estimating model parameters and developing goodness-of-fit measures.They included (1) the large number of players, which ranged from four to 40 in our data; (2) the number of the stage game strategies for each player, which varied from 11 to 101 in our data; and (3) the variation in the number of rounds, which ranged from 10 to 60 in our data.Previous studies have used various estimation methods such as regression (Cheung and Friedman 1998), maximum-likelihood gradient search (Camerer and Ho 1999), and grid search (Erev andRoth 1998, Sarin andVahid 2001).A number of recent studies show that structural estimation of the true parameters using regression methods is problematic for modestly complicated models (Bracht and Ichimura 2001, Salmon 2001, Wilcox 2006).Salmon shows that maximum-likelihood estimation of learning models is not capable of discriminating among contending learning models.Econometric approaches that assume a "representative player" lead to serious biases in the estimated parameters when there is structural heterogeneity across the players (Wilcox 2006).With such problems in mind, we can perform only some modest comparative analyses.In this study, maximum-likelihood estimation of representative agents is used as a starting point, but we also compare alternative models in terms of their performance of macro-level metrics.http://www.ecologyandsociety.org/vol11/iss2/art21/Third, the experimental results of public-good games are multilevel.This poses the question of which aspects of the experimental data need to be explained.Using only average behavior as the target of calibration may severely distort empirical tests in the public-good games.This is because the same average can result from widely different combinations of strategies at the player level.In addition, players change their contributions over time, some quite frequently and dramatically, others not so often and in small steps.We develop multiple indicators that characterize behavior at individual and group levels and changes in behavior over time.These include average contribution level, variance across individual contribution in a given round, and variability of change in contribution between rounds at the individual level.
Fourth, the analyses performed in this paper provide a number of examples of how to develop and test agent-based models using experimental data.Although behavioral game theorists estimate their formal models in a similar fashion, we focus on heterogeneity within the player population and on determining how to estimate and formalize this heterogeneity.Agent-based modelers are also interested in macro-level results of agent-agent interactions.Therefore we also compare the macrolevel patterns between our empirical data and the simulated data based on the tested models.
The remaining sections of this paper are organized as follows.In the second section, we discuss the experimental environment of linear public-good games.We use experimental data from Isaac and Walker (1988) and Isaac et al. (1994) and discuss the main stylized facts from that data set.In the third section, we present the formal model in detail.This model combines basic models of other studies such as the experience-weighted attraction model of Camerer and Ho (1999) and the hybrid utility formulation of Charness and Rabin (2002), and we formalized the signaling process suggested by Isaac et al. (1994).In the fourth section, we report parameter estimates using maximum-likelihood estimation.We applied maximum likelihood to different levels of scale, including the representative agent, different types of agents, and the level of the individual.We summarize our findings and suggest directions for further research in the final section.

LINEAR PUBLIC-GOOD PROVISION GAMES
This section introduces the notations related to Nperson linear public-good games and reviews the most prominent features of behavior at both individual and group levels in such experiments.We will use experimental data from Isaac and Walker (1988) and Isaac et al. (1994) throughout this paper.

Public-good provision experiments
The standard linear public-good provision experiment (Marwell and Ames 1979, 1980, 1981, Isaac et al. 1984, 1985, 1994, Isaac and Walker 1988, to name only some of the pioneering researchers) can be characterized by the number of players (N), the marginal per capita return (r), the number of repetitions (T), and the initial endowment for each player (ω).An experimental linear publicgood provision game involves a free-rider problem if r < 1 and N * r > 1.
Suppose that, in a given round, player i contributes x i of ω for the provision of the public good.His monetary pay off (π i ) is: (1) in which α is the conversion rate by which monetary earnings are calculated from experimental endowment units such as "tokens" and experimental pay offs.The equilibrium prediction, assuming that players maximize their own monetary pay offs, is that the public good will not be provided at all.This prediction still holds when the situation is repeated for a known finite number of rounds.However, experimental studies regularly find that, in such experiments, public goods are provided at substantial, though usually suboptimal, levels.In addition, many aspects of the experimental results seem to vary systematically depending on the aforementioned experimental parameters, such as the size of group and the marginal per capita return.http://www.ecologyandsociety.org/vol11/iss2/art21/Stylized facts from the public-good games: What needs to be explained?
We present three observations or stylized facts from linear public-good provision that any attempt to offer coherent theoretical explanations should address.The stylized facts are illustrated with data on the six experimental treatments, defined by the marginal per capita return (MPCR hereafter), and the group size, shown in Figs. 1 and 2.
Observation 1.The time course of the average contribution at the group level is a function of group size and the MPCR.
The average level of contribution for public-good provision and its change over time differs across experimental settings.Some extreme experimental conditions with low MPCR show a rapid convergence to an almost complete free-riding, whereas other treatments with relatively high MPCR show a pattern of stabilization of the contribution level at approximately 50% of the total endowment.Still other experimental conditions exhibit trends in between these two extremes, typically showing an overall decrease in contribution level.Experiments with longer durations of 40 or 60 rounds (Fig. 2) also show declining trends toward zero.Controlling for MPCR, it appears that, the larger the group size, the higher the contribution level.This can be seen most clearly in Fig. 1 when one compares three treatment conditions.For an MPCR of 0.3, groups of size 4 (filled diamond) show the lowest contribution, groups of size 10 (filled triangle) show a noticeable increase in contribution level compared to that of groups of size 4, and groups of size 40 show contribution levels of around 50% without a clear declining trend.However, this apparently benign effect of group size is not present for the MPCR value of 0.75.Both groups of size 4 and 10 show very similar trends of contribution when the MPCR is 0.75.
Observation 2. For a given level of average contribution in a round, there is a substantial variance in the level of contribution at the individual level.
Variance in contribution levels across players in a given round is another important factor characterizing public-good experimental results.In some rounds, all players contribute a similar proportion of their endowment; obviously, this is more likely when the average contribution is near zero.In other rounds, there is a diversity of contribution levels ranging from 100% to 0. An interesting observation comes from a session in Isaac et al. (1985), with MPCR = 0.3 and group size 40.The players in the session were all experienced.As Fig. 3 shows, there is a tendency for contribution levels to bifurcate toward the extremes of 0 and 100% over time.In the experimental session, about 20% of players contribute all of their endowments to the public-good account.This type of complete contributors increases to 40% by the final round of the experiment.At the same time, the proportion of complete free-riders also increases from 10% in the first round to more than 30% in the 10th.Thus, by the final round, the complete contributors and the complete free-riders together comprise more than 70% of the group.This micro-mechanism generates the stable group-level contribution shown in Series (40, 0.3), marked by hollow circles, in Fig. 1, with increasing variance shown in the corresponding series in Fig. 4.
Observation 3. Players change contribution levels between rounds.The extent and direction of such changes vary across players.Variability across players and between rounds for a player appears to be dependent on the experimental parameters and the number of rounds remaining.
Third, the variability of contribution across rounds differs from one player to another.Some players change their contribution levels rather dramatically between rounds; others maintain relatively stable levels of contribution across rounds.From the perspective of agent-based modeling, we are interested in seeing whether we can observe patterns and distributions at the population level.Figure 5 shows the relative change in contribution levels at the player level between rounds.We derived this figure by calculating for each observation the relative change in contribution between every two rounds.Thus, when a player invested 10 tokens in one round and six in the subsequent round, we registered 40% for this agent between these two rounds.This was done for all rounds and for all agents.We then calculated the relative frequency of the occurrence of different categories of change, e. g., -100% to -95%, -95% to -85%, -85% to -75%, etc.By doing this we derived a distribution.We plotted the relative frequencies on a logarithmic scale to emphasize the observed distribution.We saw a dominance of situations in which players did not change their investment, but also a relatively high frequency of situations in which the players changed their investments by 100%.The challenge for the model exercise is therefore not only to replicate the data at an aggregate level but also to generate results that incorporate between-player variability and variability over time at the player level.

MODEL
A general formal model is presented in this section that represents the decision making of agents in social dilemmas.The model will be tested on

Probabilistic choice
The general structure of probabilistic choice is the same across several models that we test.Let P i x denote the probability that agent i contributes x units of total endowment ω for the public-good provision.Then, (2) in which the parameter φ i is called the response sensitivity and A i x is the attraction of choice x to agent i.A high value of φ i leads to a sharper discrimination among strategies, and a high value of A i x implies that contribution level x has a large chance of getting chosen by agent i.

Learning behavior
The way players learn in repeated public-good games is modeled as the updating of attraction parameter A i x .The learning model is based on the experience-weighted attraction (EWA) model of Camerer and Ho (1999).This model assumes that http://www.ecologyandsociety.org/vol11/iss2/art21/Fig. 3. Increasing variance across players over time in a public-good experiment with marginal per capita return equal to 0.3 and group size equal to 40.There seem to be a bifurcation within the population of those who invested everything and those who invested nothing in the public good.each strategy has a numerical attraction that affects the probability that it will be chosen.Agent i's attraction to strategy x, i.e., contribution of x units, in round t is denoted as A x i (t).The initial attraction of each strategy is updated based on experience.The variable H(t) in the experience-weighted attraction (EWA) model captures the extent to which past experience affects an agent's choice.The variables H(t) and A x i (t) begin with initial values of H(0) and A x i (t).The value of H( 0) is an agent-specific parameter to be calibrated.Updating is given by two rules.First, (3) The parameter λ i represents forgetting or discounting of the past experience, and κ i determines the growth rate of attractions.Together they determine the fractional impact of previous experience.The second rule updates the level of attraction as follows.The model weighs hypothetical pay offs that unchosen strategies would have earned by parameter δ i and weighs pay offs actually received by an additional 1 -δ i .Define an indicator function I(x, y) to be 0 if x ≠ y and 1 if x = y.The EWA attraction updating equation is the The parameter λ i is a discount factor that depreciates previous attraction.When δ i is equal to 0, EWA mimics reinforcement learning as used by Erev and Roth .When δ i is equal to 1, the model mimics belief learning as used by Sarin and Vahid (1999).
x Ecology and Society 11(2): 21 http://www.ecologyandsociety.org/vol11/iss2/art21/Following Wilcox (2006), we assume that the initial value of H is (5) which means that agents do not have much previous experience.The term u i represents the utility of player i, which is a function of his own pay off as well as pay offs to others.The details of this social utility function are explained below.

Social preferences
The fact that many players in public-good games do contribute to the provision of a public good at a substantial level, even in the final rounds, indicates that their preferences are not entirely dictated by the monetary pay offs they receive in the experiments.Thus, allowing for social preferences is crucial in explaining the dynamics of these games.In addition, the extent to which agents deviate from purely selfish motivation differs from one agent to the next.There are multiple ways of representing these heterogeneity preferences (Fehr and Schmidt 1999, Bolton and Ockenfels 2000, Charness and Rabin 2002, Cox and Friedman 2002, for example).
The utility functions are modified to reflect the specifics of the repeated N-person public-goods provision experiments.That is, instead of the exact distribution of the pay offs to others, an agent is assumed to consider the average of the pay offs of others: π -i .We use the average because, in the experiments that generated the data being used, the players did not have information about the exact distribution of pay offs to other group members; they could only infer the average pay off to others.Charness and Rabin (2002) developed a general model for social preferences that embeds other models.The utility function is defined as (6) where χ ≤ ρ ≤ 1.A lower value of χ compared to ρ implies that a player gives a larger weight to his own pay off when his pay off is smaller than the average pay off of others than when it is larger.When χ ≤ ρ ≤ 0 the player is highly competitive.The players like to have their pay offs higher than those of the other players.An alternative model is that players prefer the pay offs among the players to be equal.This so-called inequity aversion holds when χ < 0 < ρ < 1 (see Fehr and Schmidt 1999).The third model is the so-called social welfare consideration, which holds when 0< χ ≤ ρ ≤ 1.The parameter ρ captures the extent to which a player weighs the average pay offs of the other N-1 agents compared to his own pay off when his own pay off is higher than the average payoff of the others.If ρ = χ = 0, we have the condition that a player cares only about his own welfare.

Signaling
Another component in the utility function has to do with the forward-looking signaling behavior of the players in repeated games.Isaac et al. (1994) propose the hypothesis that these players are involved in a forward-looking intertemporal decision problem.Players may signal their willingness to contribute for a public good in the future by contributing at a high level in the current round.A player may benefit from this signaling if others respond positively in the following rounds.If this is the case, the potential benefit of signaling depends on the number of rounds that are left before the game ends.Therefore, one would expect less signaling toward the end of a game.This is consistent with their findings in experiments with more than 10 rounds (Figs. 1 and 2).That is, the decline of contribution level depends not so much on the number of rounds played as it does on the number of rounds remaining.
We assume that the attraction of strategy x i as formulated in Eq. 2 is adapted to include the signaling component in the following way ( 7) Ecology and Society 11(2): 21 http://www.ecologyandsociety.org/vol11/iss2/art21/positively affects others contribution in the future.In addition, the larger the marginal per capita return (MPCR) is, the more positive a player's assessment of the effect of his own contribution on the future contributions of others.The two individualized parameters, θ i and η i , also affect the signaling strength of i, generating another dimension of heterogeneity across agents.Specifically, θ i represents player i's belief about how large the positive effect of his contribution will be on the future contributions of others.The parameter η i models player i's end behavior, given that (T -t)/T is smaller than 1, a larger (smaller) η i .

MODEL ESTIMATION
For the eight treatments shown in Table 1, which contains 278 players, we have estimated the parameters listed in Table 2. Three types of estimations were conducted: (1) representative agent estimation, (2) multiple-type estimation, and (3) individual estimation.In the representative agent estimation, we assume that all the players are of the same type and estimate the parameters of the model.In the multiple-type estimation, we use the methodology of El-Gamal and Grether (1995) that divides the players into multiple segments to find the best fit.In the individual-level estimation, the parameters are estimated for each individual player.
Because of the stochastic nature of the model, we use conventional maximum likelihood (L) estimation to estimate the parameters.Fitting the model, however, is not an adequate approach for evaluating model performance (Pitt and Myung 2002).The main problem is that more complicated models have more degrees of freedom to fit the data.The trade-off is between the fit of the data and the complexity of the model.We use two criteria to evaluate the different estimated model versions.
The first criterion is the Akaike Information Criterion (AIC), which is defined as (8) where k is the number of parameters of the model.Thus, for each parameter added to the model, the maximum likelihood needs to increase more than one unit to justify this extra parameter.The Bayesian Information Criterion (BIC) also includes the number of observations N used in the equation: (9) This means that, the more observations are used, the more an extra parameter must contribute to improving the maximum likelihood to justify this extra parameter.For example, when N is 8, the improvement of the maximum likelihood must be slightly more than one unit, but, when N is 80, the improvement must be more than 2.2 units.Both AIC and BIC are ways to strike a balance between the fitness and complexity of models and favor the models with lower AIC/BIC values.

Representative agent estimation
Here, we estimated four variants of the general model.In each of the estimated models, agents are assumed to be homogeneous, i.e., they have the same set of parameters.The four models include different elements of the general model denoted "SP" (social preference according to the Charness-Rabin social welfare utility function), "L" (experience-weighted attraction learning model of Camerer and Ho), and "S" (signaling).They are listed below: 1. Model SP: Probabilistic choice with social preferences, without learning and signaling.

2.
Model SP+L: Probabilistic choice with social preferences with learning and without signaling.

3.
Model SP+L+S: Probabilistic choice with social preferences with learning and with signaling.
4. Model L: Probabilistic choice of income maximizers with learning and without signaling.
Tables 3 and 4 show the results of the maximum likelihood estimation.In both the 10-and 40-round data, the most comprehensive model (SP+L+S) gives the lowest AIC and BIC, and the simple SP The added component indicates that a player thinks that his contribution level in the current round, x, Table 1.Experimental conditions and parameters.The parameters are defined in Table 2. IW88 signifies that the data are from Isaac and Walker (1988); IWW94 signifies that the data are from Isaac et al. (1994).In all of these eight experimental conditions, the participating players were experienced in the sense that they had previously participated in another public-good game.Also, cash payments were made in all of the eight conditions.The eight conditions are divided into two subsets: six experimental conditions with 10 rounds and two experimental conditions with 40 and 60 rounds.model performs the worst.The differences in AIC and BIC values are relatively small between Model SP+L and Model SP+L+S and, thus, one might wonder whether the added complexity of SP+L+S is worth the trouble.However, recall that both the AIC and BIC values already account for the degree of complexity in evaluating performance of the models.Thus, we consider that the three features of social preference, learning, and signaling are all essential in explaining behavior in repeated publicgood games.

Condition
In the 10-round data, the estimated parameters are quite similar between Models SP+L and SP+L+S (Table 3).The positive values of ρ and χ suggest that the players on average had a social-welfare utility function in which utility is a weighted average of one's own pay off and the average of the pay offs of others.Admittedly, this is somewhat different from the more widely accepted wisdom that the players in the social-dilemma games typically exhibit a conditionally cooperative behavior as suggested by Fehr and Schmidt (1999) or Bolton and Ockenfels (2000).The utility function of Charness and Rabin (2003) that we used in this study embeds the inequality aversion as a special case.That is, if the estimation results were a positive ρ and a negative χ, that would be consistent with a preference for inequality aversion.It is possible that, if we used either Fehr and Schmidt's or Bolton and Ockenfel's utility functions, we could have found estimates that are consistent with an inequality aversion.However, because the main focus of our study is to test the significance of some broad features such as learning, signaling, and social preference, we did not conduct a separate estimation using an inequality aversion function.Instead, we limit the result as suggesting that some level of other-regarding preferences is present among the players, not necessarily that a social-welfare utility function is superior to an inequality-aversion utility function.
Also, notice that in Model SP without learning or signaling, the representative agent appears to be competitive, i.e., a difference maximizer, as  suggested by negative ρ and χ.However, because Model SP has a significantly poorer fit compared to Models SP+L and SP+L+S, and the estimates are quite similar between Models SP+L and SP+L+S, we consider the results of Model SP estimation to be invalid.
The discount of the past, parameter λ, is approximately 0.85 in both the 10-and 40/60-round data.The weights of forgone pay offs δ are 0.55 and 0.72, respectively, which suggests that the players are more belief learners than reinforcement learners.The rates of attraction growth are 0.06 and 0.03, which represent a rapid attraction to particular choices.
The estimated signaling parameters differ between the two data sets.The 10-round experiments lead to a short and strong effect of signaling, with θ equal to 2.05 and η equal to 10.However, the 40-and 60round experiments lead to a weaker effect of signaling, although it does have an effect over a relative longer period than the 10-round experiments.This might indicate that the relative effect of signaling differs when the duration of the game changes.

Estimation and model evaluation with multiple types of agents
Now that we have estimated the representative agent, we will perform maximum likelihood estimation with different types of agents.Using the methodology of El-Gamal and Grether (1995), we maximize the likelihood function and at the same time classify different types of agents.Because the full SP+L+S model came out the strongest in our representative agent estimation, we used it in the estimation of multiple types.Because the model specification is identical, the only difference among the estimated models is the number of types allowed.3.Estimated parameter values for different model versions using the data from experiments with 10 rounds.The model versions are defined as follows: SP = probabilistic choice with social preferences, without learning and signaling; SP+L = probabilistic choice with social preferences with learning and without signaling; SP+L+S = probabilistic choice with social preferences with learning and with signaling; and L = probabilistic choice of income maximizers with learning and without signaling.See Table 2 for a list of parameters and their definitions.InL stands for maximum likelihood, AIC for Akaike Information Criterion, and BIC for Bayesian Information Criterion.Once the number of types is exogenously given in an estimation, the maximum likelihood estimation endogenously distributes the 248 players into different categories until the likelihood is maximized.Starting from the two-types model, we increased the number of types until the model started to perform more poorly than a model with a smaller number of types.Here the focus is on whether allowing for multiple types improves the fit.Thus, the substantive details of the estimation results are suppressed.For comparison purposes, the AIC and BIC values of the representative agent model estimation and the individual estimation, i.e., 248types model, which will be discussed in the next subsection, are included in Tables 5 and 6.
We find that eight different types of agents best explain the data on the 248 players in the 10-round experiments when we take into account the increasing complexity of the model with a larger number of parameters.We also find that two types of agents provide the best explanation for the 30 players in the 40/60-round experiments.the fully heterogeneous model.The optimal number of types is rather large, probably because of the complexity of the general model.Again, however, given that the AIC and BIC scores take into account model complexity, including the number of types, we cautiously conclude that it is essential to incorporate multiple types of agents defined on multiple and continuous dimensions of heterogeneity to understand the results from repeated experiments involving the provision of public goods.

Individual level estimation
Finally, we estimated the parameters for each player.This leads to a distribution of parameter values.Figure 6 provides the cumulative distribution of the estimated parameter values, which gives an indication of distributions.For most parameters, these distributions are remarkably similar among the 10-and 40/60-round data sets.
Besides the distributions of the estimated parameters of the two data sets, we defined general distribution functions (Table 7) that mimic the observed distributions.This is the third line, i.e., the one with triangle legends, in each of the parameter figures in Fig. 6.We did not do a formal estimation of the general distributions in Table 7, but defined some simple forms that mimic the general features so that we could use this in simulation models as a general model that represents the statistics.Note that one of our aims is to derive agent-based models based on empirical data, and therefore a more general description is preferred.The generalized distributions might provide educated information for other agent-based models when heterogeneity of agents is assumed.In terms of style of learning, most of the players are identified as belief learners, including Cournot-type learners, who take into account not only their experienced pay offs but also the pay offs the agents could have gotten had they made other decisions.
Given the large number of decision options, i.e., 11 to 101 possible token investment options, the fact that most players are identified as belief learners is not a surprise because learning from only experienced observations, i.e., reinforcement learning, would take much longer.Also interesting is the fact that most of the players identified as reinforcement learners have short memory spans as indicated by large λ parameters.This seems to suggest that they are not, in fact, learning systematically from their past experiences.With regard to social preferences, the inequity-aversion preference is the most frequently identified utility function.Note that 216 out of 248 players are identified as having either inequity-aversion or social-welfare preferences, again suggesting that incorporating social preference is essential in understanding the results of repeated socialdilemma experiments.Fewer than 10% of the agents are identified as interested only in maximizing their own pay offs.
In Appendix 1 we provide a more in-depth analysis of the models generated by the three different estimation techniques.In particular, macro-level statistics generated by the models are compared with the same statistics obtained from the data.Some of the macro-level statistics, such as those from Fig. 5, are not produced with great accuracy by the simulation models.

DISCUSSION
In this paper we evaluated versions of a hybrid model of decision making and learning in repeated public-good experiments.Our analyses show that most players have other-regarding preferences, and that the types of other-regarding preferences differ among the players.The players learn in different ways from their experience, but the most dominant result from our analysis is a belief learning process in which players take into account the potential benefit they could have derived if they had made different choices.Some players signal their willingness to invest in public goods in the hope that others will increase their investments too.
In sum, even in the baseline public-good experiments without additional institutional features such as punishment (Ostrom et al. 1992, Fehr and Gächter 2000, Anderson and Putterman 2006) or endogenous group formation (Coricelli et al. 2003, Ahn et al. 2005, Cinyabuguma et al. 2005), it is essential that the dynamics at the individual and group levels be explained as interactions among multiple types of players defined on multiple dimensions of heterogeneity.In this sense, as Ones and Putterman (2004) suggest, repeated N-person dilemmas need to be studied from the viewpoint of an ecology of interacting types.Consistent with experimental studies that specifically address the problem of heterogeneous preference types in repeated public-good games (Fischbacher et al. 2001, Kurzban and Houser 2001, Fischbacher and Gächter 2006), we find that most of the subjects have other-regarding preferences of inequality aversion or conditionally cooperative preferences.
In addition, our simulation results suggest that most subjects, although they do have other-regarding preferences, are at the same time quite rational.They seem to form and update their beliefs about the behavior of others and then choose their actions based on their beliefs and preferences.This finding may explain why punishment opportunity might encourage contribution even before punishment is exercised (Fehr and Gächter 2000) and why certain forms of endogenous group formation, especially expulsion, induce very high levels of contribution from the very beginning of an experiment (Cinyabuguma et al. 2005).Our results also suggest that the rationality of some, if not a majority of, subjects extends to signaling their intentions in an attempt to induce higher levels of contribution from others.An interesting venue for future research would be to derive the implications of the types identified in our study to richer institutional settings and test whether the results of such experiments can also be systematically explained in terms of the interaction of the types.
Methodologically, this paper is an attempt to expand the horizon of empirically grounded agent-based modeling practices.Our analysis combines rigorous tools from behavioral economics and cognitive science (maximum likelihood estimation) with agent-based models (emergent properties and macro-level metrics).For the empirical testing of agent-based models in laboratory experiments involving group dynamics, we derive a good starting point from statistical tools like maximum likelihood.Nevertheless, it is not sufficient to generate all the emerging properties from agent interactions.A problem with maximum likelihood estimation is the focus on the calibration of observations at the individual level.However, emergent patterns at the group level, such as the patterns in Fig. 5, are not necessarily generated when the model is calibrated at the individual level.Hence, agent-based models require methods for multilevel calibration and evaluation.The balance between fitting the data and generalizability remains another problem.Although we can include some penalties within the maximum likelihood estimation, such as the number of parameters, it is not clear whether this penalizes model complexity for agent-based models.For example, computational time might also be a consideration to be included in the penalty.
Despite the problems of model estimation and evaluation, we were able to develop a general model that mimics the most important elements of the experimental data.We found that other-regarding preferences, learning, and signaling all had to be included to explain the observations.Adding all these components was still beneficial after including penalties for model complexity.Assuming that there is agent heterogeneity improves the maximum likelihood estimation; this also occurs when additional complexity is penalized.Therefore, a representative agent model for public-good experiments is not justified based on our findings.We were able to derive parameter distributions based on the individual-level calibration of the experimental data.These parameter distributions can be used to inform applied agent-based models in which social dilemmas are involved.
Based on the distributions of parameter values, we found that players seem to use different learning models, namely belief learning and reinforcement learning, and other-regarding preferences, e.g., inequity aversion and social welfare.The largest group, about 25%, is classified as inequity-aversion players with reinforcement learning and forgetting.Only 10% of the players are classified as selfish.
The results of our model analysis depended on the specific functional forms we used.Although we based our model on hybrid model versions of experimental economics studies, we also considered some of the additional functional forms used in the literature.Nevertheless, our results showed the potential of using laboratory experiments to develop empirically tested agentbased models.Most notably, to explain the dynamics of social dilemmas, we had to incorporate multiple types of agents or distributions of parameter values.We have shown that we can detect this agent heterogeneity by various methods in analyzing the data.These empirically tested agentbased models might guide the parameterization of applied agent-based models.

Fig. 1 .
Fig. 1.Average contribution over the round for the six different treatments.The number of participants and the marginal per capita return are given in parentheses.
experimental data.The model is built on three components: (1) the probabilistic choice model to define the decision, (2) the learning model that captures the change in behavior over time at the player level, and (3) the social utility function by which a player evaluates outcomes of the game.The social utility function is embedded in the learning model, which in turn is embedded in the probabilistic choice model that determines the relative probabilities of choosing different levels of contribution.

Fig. 2 .
Fig. 2. Average contribution per round for two different treatments.The number of participants, the marginal per capita return, and the number of rounds are given in parentheses.

Fig. 4 .
Fig. 4. Average standard deviation of the contribution level within a group for the six different treatments.The number of participants and the marginal per capita return are given in parentheses.

Fig. 6 .
Fig. 6.Cumulated distribution of the estimated parameters for the two data sets.The parameters are defined in Table2.

Table 2 .
Parameters of the model.

Table 5
and 6 show how the indicators of goodness of fit and the generalization indicators are affected by the number of agent types.Table5shows that, up to eight different types of agents, the performance of the model improves.From Table6it can be seen that two distinct types of agents improve the performance of the model, whereas it performs less well when we add more types of agents, i.e., an increase in BIC.In both 10-and 40/60-round data sets, the best multiple-types models (8-types in 10 rounds and 2-types in 40/60 rounds) perform much better than either the representative agent model or http://www.ecologyandsociety.org/vol11/iss2/art21/

Table 4 .
Estimated parameter values for different model versions using the data of experiments with 40 rounds and 60 rounds.The model versions are defined as follows: SP = probabilistic choice with social preferences, without learning and signaling; SP+L = probabilistic choice with social preferences with learning and without signaling; SP+L+S = probabilistic choice with social preferences with learning and with signaling; and L = probabilistic choice of income maximizers with learning and without signaling.See Table2for a list of parameters and their definitions.InL stands for maximum likelihood, AIC for Akaike Information Criterion, and BIC for Bayesian Information Criterion.

Table 5 .
The value of maximum likelihood (InL), the Akaike Information Criterion (AIC), and the Bayesian Information Criterion (BIC) for different numbers of types distinguished by maximum likelihood estimation using the method of El-Gamal and Grether (1995) for 10-round data.

Table 8 .
Classification of players based on the estimated parameter values.The parameters are defined in Table2.The 248 players in the 10-round experiments are classified into 16 possible cases.The total number for each type of learning model is given in the last row, and the total number for each type of utility function is given in the last column.The 30 players of the 40/60-round experiments are also classified, and these numbers are given in parentheses.