The main body of the first edition of this paper is published in Games and Economic Behavior, 28 (1999), 155-170 .  Copyright and all rights on the published paper therein are retained by Academic Press (Copyright 1999 by Academic Press).    Copyright and all rights on the  parts of this document which are not published in the paper therein are retained by the author (Copyright 1999 by Ariel Rubinstein). 

Abstract

This is a revised version of my paper with the same title published in Games and Economic Behavior, 28 (1999), 155-170. The paper summarizes my experience in teaching an undergraduate course in game theory in 1998 and in 1999. Students were required to submit two types of problem sets:pre-class problem sets, which served as experiments, and post-class problem sets, which require the students to study and apply the solution concepts taught in the course. The sharp distinction between the two types of problem sets emphasizes the limited relevance of game theory as a tool for making predictions and giving advice. The paper summarizes the results of 43 experiments which were conducted during the course. It is argued that the crude experimental methods produced results which are not substantially different from those obtained at much higher cost using stricter experimental methods. 
 
 

My sincere thanks to my two excellent research assistants, Yoram Hamo, during the 1998 course, and Michael Ornstein, during the 1999 course and in the writing of the paper and its revised version. 
 
 

1. Introduction

Teaching game theory to undergraduates has become standard in economics and other social science disciplines . This is "great news" for game theorists. Academic knowledge is created and circulates within a small circle of researchers for a very long time before the "happy" moment it enters undergraduate textbooks. In the case of game theory, that moment should not only be a cause for celebration. As game theorists, we have a responsibility for the way it is taught. We are the only people who can control and influence the content of the material taught. In particular, we have a duty to control the message that game theory transmits to the broader community. 

It is my impression that most students approach a course in game theory with the belief that game theory is about the way that game situations are played and that its goal is to predict strategic behavior. They hope the course will provide them with the tools to play game-like situations better. During the course, they are disappointed with the poor performance of game theory on the descriptive level and its lack of relevance to practical problems. 

The disappointment, of course, leads the student to ask "what is the relation between the 'game theoretic prediction' and the real world?" My impression is that undergraduate textbooks are vague on this question. This is probably because we, game theorists, are confused about what the theory is trying to accomplish. 

In the past, I held the "radical" view that undergraduate studies of game theory may  influence students negatively. Students may recognize the legitimacy of manipulative considerations.  They may come to believe that they need to use mixed strategies.  They may tend to become more suspicious and to put less trust in verbal statements . They may adopt game theoretic solutions dogmatically. However, a pilot experiment which I conducted together with a group of graduate students at Tel Aviv University (Gilad Aharanovitz, Kfir Eliaz, Yoram Hamo, Michael Ornstein, Rani Spiegler and Ehud Yampuler) made me less certain about this position. When we compared the responses of economics students to daily strategic situations before and after a course in game theory, we found little difference before and after the course, though there was a clear correlation between their responses and their second major. 

My method of teaching an undergraduate course in game theory is derived from my views on the relationship between theory and real life. I perceive game theory as the study of a set of considerations used (or to be used) by people in strategic situations. I do not, however see our models as being in any way constructions or depictions of how individuals actually play game-like situations and I have never understood how an equilibrium analysis can be used as the basis for a recommendation on how to play real "games". My goal as a teacher is to deliver a loud and clear message of separation between game theoretic models and predictions of strategic behavior in real life. 

Students in my class were asked to complete two types of assignments. The "post-class" problem sets were standard exercises that cane be found in any game theory text. Students were asked to fit games to verbal situations, apply standard game theoretic solution concepts and investigate them analytically. The answers to these problem sets were categorized as right/wrong. 

The new feature here was that students were also requested to respond regularly to "pre-class" problems sets, which were posted weekly in the course website. The responses were collected in a log file, which allowed me to enter the class with statistics regarding the results. It was stressed that there are no right/wrong answers to the pre-class problems. The results were compared with the standard game theoretic treatment, and it was pointed out that some of the results fit the game theory analysis well while many others do not. Game theory was described as a collection of considerations which could or could not be used. 

The pre-class problem sets served two additional purposes: First, they helped the students to concentrate on the examples discussed later in class. It is not a trivial task for students to absorb several games in one class and this method facilitated their understanding of the material taught. Second, they provided a cheap and convenient tool for experimentation. I am fully aware of the potential for criticism of this method: No monetary rewards were offered. However, comparisons between the results achieved in class with those received in more standard frameworks show, in my opinion (and I know this may be controversial), insignificant differences.

Whereas the first version of this paper included the results in the 1998 course only, I am including the results of both 1998 and 1999 in the new version. Most of the experiments in 1999 repeated those in 1998. Very few experiments are new. Several versions were changed, sometimes in order to test some framing effect. In 80% of the experiments which were repeated in both years, the results were remarkably similar. In only two experiments, "he will play first" and " randomization2") are the results qualitatively different. In the first case, the result last year was suspicious and I do not rule out a technical mistake in processing the information. I do not have an explanation for the second case. In any case, I find the comparison of the results in the two classes important as a self-disciplinary devise to detect such mistakes.

The following table summarizes almost all the experiments conducted (though not in the order presented). I omitted only those 1998 experiments which contained a clear problem in their wording. The reader may view the experiment by clicking the box in the left-hand column. The experiment page is linked to a result page. The second column is anchored to a discussion of the experiment in this text.
 
 
  Note:you need javascript to be enabled in order to see this paper properly.

Name and Link	Anchor to Discussion	Description	Similar Previous Experiments
		Non-cooperative games
average of all average of other		Guess what is "2/3 of the average".	Nagel (1995) and Thaler (1998).
secure play		Player 1 in a game with payoffs  (5,5) (-100,4) (0,1) (0,0)	Beard and Beil (1994)
elimination
Battle of the sexes		Battle of the sexes: Original version
he will play first		Battle of the sexes: The other player will play first.	Cooper, DeJong, Forsythe and Ross (1993)
yesterday's_effect		Battle of the sexes: You know what your opponent played yesterday.	---
cheap_talk1		Battle of the sexes: The other player made an announcement	Cooper, DeJong, Forsythe and Ross (1980)
cheap_talk2		Battle of the sexes: Interpretation of silence in the pre-game communication.	Cooper, DeJong, Forsythe and Ross (1980)
cheap_talk3		Battle of the sexes: Interpretation of silence in the pre-game communication.
burn_money		Battle of the sexes: Your opponent has an opportunity to burn money.	---
deleting_actions		Do you think that deleting an action can help ?	---
coordination1		Coordinate on one of the items: - Fiat 97, Fiat 96, Saab 95, Fiat 97.	Mehta, Starmer and Sudgen (1994)
coordination2		Coordinate on one of six different apartments
		Zero-sum games
zero-sum11		Player 1 in the zero-sum game 2 0 0 1	Fox (1972)
zero-sum12		Player 2 in the zero-sum game 2 0 0 1	Fox (1972)
zero-sum20		Try to be as close as possible to player 2.	---
minmax		What is greater, minmax or maxmin ?	---
4_boxes		Find a prize hidden in one of four boxes labeled: A B A A (98 seeker 99 hider):	Rubinstein, Tversky and Heller (1996)
hide in a table		Hide a treasure in a 5x5 table.	Ayton and Falk (1995)
		Dictator / Ultimatum
dictator1 dictator1a		Divide a sum between yourself and a second player (98 - x for me, 99 - x for him)	Forsythe, Horowitz, Savin, and Sefton (1994)
ultimatum1 ultimatum1a		Ultimatum game: You have to make an offer(98 - let the other keep X, 99 - i will keep x).	Guth, Schmittberger and Schwarze (1982), Camerer and Thaler (1995)
ultimatum2 ultimatum2a		Ultimatum game: Would you accept 10% (98 - give the other 10 shekels, 99 - keep 90 shekels)?	Roth and Prasnikar (1992), Forsythe, Horowitz, Savin, and Sefton (1994)
ultimatum3		Ultimatum game: What is the minimal offer you will accept?	Forsythe, Horowitz, Savin, and Sefton (1994)
		Extensive form games
shop_transform		What is your advice to a potential entrant facing an incumbent?	Schotter, Weigelt and Wilson (1994)
knowledge_value		How much would you pay for transforming the battle of the sexes into a game in which you will be informed of the second player's action?	---
trace_i		Does a player analyze a tree from its root or from its  end?	Camerer, Johnson, Rymon and Sen (1993)
bargaining-discounting		Predict the outcome of a two-neighbor bargaining game in which one neighbor is paying his bank a higher interest rate.	Ochs and Roth (1989)
bargaining-proposer		Bargaining: do you prefer to be the proposer or the responder?	---
bargaining-reputation		A seller of a used car refused to lower the price. What is your prediction about the car's condition?	---
		Finite horizon games
centipede		The centipede game (100 rounds)	McKelvey and Palfrey (1992),  Nagel and Tang (1998)
stop_or_pass		Stop or pass game ? (20 players)	---
repeated PD a+b		Repeated prisoner's dilemma (4 rounds)	Selten and Stoecker (1986)
repeated BOS a+b		Repeated battle of the sexes (4 rounds)	---
		Randomization
randomization1		Randomly choose an integer in the set {1,2,...,9}	Simon (1971) and Simon and Primavera (1972)
randomization2		Randomly choose 4 boxes from a row of 8.	---
guess departments		Guess the majors of five students.	Rubinstein and Tversky (unpublished), Lowenstein and Read (1995)
mall_gates mall_gates1		At which gate will you wait for your friend? You are ambushing a suspect, where will you wait for him?	---
		Failures (and "Paradoxical Behavior")
count the F's		Count the number of "F"s in a text	---
winner's curse		If you offer a price above v, will you be able to sell it for 1.5v? How much would you offer?	Samuelson and Bazerman (1985)
envelopes		Would you switch your envelope with another containing x/2 or 2x of what you have?	---
allais 1 allais 2		Choose between getting $4000 with probability 0.2 and $3000 with probability 0.25	Kahneman and Tversky (1979)
shafir_tversky11 shafir_tversky12 shafir_tversky13		Prisoner's dilemma - 2 have played C  Prisoner's dilemma - 2 have played D  Prisoner's dilemma - simultaneous	Shafir and Tversky (1992)
		Ethical Views
election		Are you prepared to manipulate election results?	---
selling_car		Would you increase a price offer you have already made?	---

2. The Experiments

We now move to a summary of the pre-class problem sets. 

Non-cooperative Games

The games in this category were meant to introduce the students to basic strategic considerations. The students' attention was directed to considerations which affect the outcome of a game but are excluded from the game theoretic analysis. 

The game guess the 2/3, where a subject must announce a number between 0 and 99 with the aim of guessing "the highest integer which is no higher than 2/3 of the average of all the responses", has become a standard tool for demonstrating game theoretic considerations ("I think that they think that...") and pointing out the tension between real-life behavior and analysis. The game has a unique Nash equilibrium outcome in which all players choose the number 1. The results fit well with those in the literature (see for example, Camerer (1997), Nagel (1995) and Thaler (1998)). The winning number was 19 in 1998 and 23 in 1999. (Nagel's winning number was higher at 24, and Thaler's large experiment with Financial Times readers received a result of 13). Our result is lower than Nagel's due to the fact that a significant number of students (versus almost no one in Nagel's experiment) chose the lowest number. In Thaler's experiment, I suspect that the relatively low number is an outcome of the participants' bias in favor of "more sophisticated" subjects. 

The full game theoretical analysis of this particular game is not trivial and therefore, in 1999 I offered part of the class a modified version of the question where the winning rule was changed to "guess 2/3 of the average of the other players". The results were not significantly different.

The next game was meant to test whether player 1 is certain that player 2 will take the action which is "clearly" optimal for player 2. 

	A	B
A	5, 5	-100, 4
B	0, 1	0, 0

The outcome (A,A) is the best outcome for both players, and player 2 has no reason to "punish" player 1 by playing B. Nevertheless, player 1 may be uncertain as to whether player 2 will employ the correct reasoning. Results: In both years less than a quarter of the students did not trust the other player and chose the safe action B. Beard and Beil (1994) tested a similar effect in a two-stage extensive game where player 1 could either take a safe action or allow player 2 the option of making a choice.  If player 2 makes the irrational choice, player 1 suffers a loss. Though the payoff numbers are different, the results here are in line with those in experiment 6 in Beard and Beil (1994). 

The question whether players follow a more complicated process of successive elimination of dominated strategies was tested in a 4X4 matrix game. Though no student chose the weakly dominated action "A", only 34% of the subjects chose "B", the only action which survives the successive elimination process.  The choice of C and D is probably a result of high payoffs attached to some entries in the rows of those actions. 

The battle of the sexes was used to explore several fundamental issues. 

	A	B
A	2, 1	0, 0
B	0, 0	1, 2

In 1999 students were asked  to play the battle of the sexes where the row player was called "He" and the column player was called "She". Students were asked to play the game in the role of the player who fits their gender. Results:  68% of the students chose their preferred action. Since we asked students to play the game in the role which fits their real gender, we can compare the choice of the students according to their gender. 75% of the males chose their preferred action whereas the females were divided equally between the two actions. In comparison, in Cooper, DeJong, Forsythe and Ross (1993), about 63% of the subjects in any of the two roles chose their preferred action. I am not aware of any study of the battle of the sexes where the report of the results classified the subjects by their gender.Back to the table

An essential assumption of simultaneous games is that each player makes a move independently of the other. Thus, information that one of the players plays at an earlier point of time is not a part of the model as long as the other player plays the game without being informed about the taken already action. Is this a significant piece of information? 
In the next game, a student plays the role of player 2 in a battle of the sexes game: 

He is told that player 1 moves first, without informing him of player 2's action. Cooper, DeJong, Forsythe and Ross (1993) found (with slightly different numbers) that 70% of the players in the role of player 2 chose A whereas in a standard simultaneous game, the proportion was only 35%. (Note that in their experiment, each player played the game a large number of times). Here, in 1999, I got the same results as those of Cooper, DeJong, Forsythe and Ross (1993) whereas in 1998 I received strangely a very different result (was it just a mistake of mine?). 

Information about  outcomes of similar situations played in the past is another example of an additional type of information which is not included in the description of a game. In the next game, a situation which fits the battle of the sexes was described verbally. Students were asked to imagine that in the last play of a similar game, they had conceded, in other words, chosen the "inferior" action. This piece of information made the subjects (especially in 1999) choose their preferred action in proportions higher than that they did in the standard battle of the sexes.

The next four problems tested signaling effects in the battle of the sexes. 
In cheap_talk1 , player 2 has just announced that he will play his favorite action. This statement was sufficient for almost all students (80% in 1998 and 94% in 1999) who were posted in the role of player 1 to believe the announcement and to play B.  In comparison, the effect was even stronger in Cooper, DeJong, Forsythe and Ross (1980) where 96% of the subjects played B. (However, note again that the results there are reported for the case where each player played the game a large number of times). 

In cheap_talk2, a conversation takes place in which player 1 announces that he will play his favorite action (T), player 2 insists that he will play his favorite action (R), and player 1 responds by remaining silent. Is silence interpreted as "agreeing" or as "disagreeing"?   In this case, students were asked to predict the outcome of the game. The results of the experiment ,conducted only in 1998 : A vast majority (85%) of the subjects predicted the outcome (T,L), implying that they interpreted silence as a confirmation of player 1's first announcement. The closest comparable game is found in Cooper, DeJong, Forsythe and Ross (1980), where players made their moves after a cheap talk stage in which they made simultaneous announcements. In those cases where one player announced T and the other remained silent, 80% of the outcomes were indeed (T,L). 

In cheap_talk3 (conducted only in 1999), I tried to investigate the way that subjects interpret "silence". Subjects were asked to predict the outcome of the battle of the sexes after player 2 had an opportunity to make an announcement and remain silent. My guess is that "silence" was interpreted as "weakness". I think that the subject investigated is of much interest; however, no definitive conclusions could be drawn from this single experiment. The least which is needed to draw any meaningful conclusions is to compare the results with an experiment where the subject is asked to predict the outcome of the standard battle of the sexes but where the players are named "1" and "2".

Ben-Porath and Dekel (1992) provide the setting for the next problem. Player 1 is notified that player 2 did not burn money although he could have done so. I doubt if any of the students had in mind the considerations which Ben-Porath and Dekel described; however, the results were in the "right" direction moderately: 46% of the students in 1998 and 33% in 1999 chose action B, only a slightly more than expected without this information. An interesting comparison with the results can be made to the problem studied in Cooper, DeJong, Forsythe and Ross (1980), where the mere existence of an outside option for player 2 (independent of the values of that option) resulted in a stronger tendency for player 1 to "yield" than in the standard BoS. 

The next problem in this group is quite different: I attempted to test the students' intuitions as to whether eliminating a player's action can be harmful for that player. In 1999 71% of the students expressed an intuition that the removal may help, whereas in 1998, (with a more confused statement of the problem), the group was split equally in their answers. I incorporate the question here primarily to emphasize the point that pedagogically, it is interesting to survey intuitions before discussing them formally in class. 

Coordination games are well suited to experiments. The common finding is that people succeed in coordinating on the salient option (see for example, Mehta, Starmer and Sudgen (1994)). The question now is what are the characteristics of the salient option. Here, subjects were asked to coordinate on one of four alternatives labeled Fiat 97, Fiat 96, Saab 95 and Fiat 97. I wanted to test a conjecture made by Michael Bacharach: When each alternative is described in terms of a number of characteristics, the salient option is the one which is distinctive in most of the characteristics. My own conjecture was that the salient option is the one which is most distinctive from the set of most common alternatives (the Fiats in this experiment). This question was presented only in 1998 and the findings are that Michael was right! 

In 1999 I tried  another problem which demonstrates that it is not so easy to coordinate in cases where each option is described by a vector of characteristics. In the experiment, each subject has to choose one of the six alternatives (Gad,4,c), (Gad,3,a), (Dan,4,a), (Gad,4,d), (Dan,3,e) and (Gad,4,b). The alternatives were presented in a vertical list. The order of the top and bottom options was exchanged for half of the class. According to the results, if two subjects were randomly chosen to play the game, the chances that they would succeed to coordinate was 33% in the first order and 20% in the second order, not very high, although higher than the 17% expected if the subjects were choosing their actions uniformly randomly. In the results, we can observe a strong tendency to avoid both the bottom option and the alternative (Gad,4,d) which is received as the most undistinctive alternative. 

Zero-Sum Games

The class of zero-sum games is attractive as a teaching device since students are familiar with such games from daily life. Given the sharply defined predictions (in payoff terms) of equilibrium in zero-sum games, comparing results with equilibrium is simple. All the problem sets were given to the students prior to the classroom discussion of the notion of mixed strategies. 

The first zero-sum game has a unique mixed-strategy equilibrium in which player 1 plays T with probability 1/3 and player 2 plays L with probability 1/3. 

	L	R
T	2, -2	0, 0
B	0, 0	1, -1

Quite surprisingly, the results were close to the game theoretic prediction. T was played by 37-38% of the subjects in both years! 

Immediately after responding to this problem, the students were asked to play the game in the role of the column player.    In both years , 86% of the students chose the action R, well above the "predicted" 67%. I suspect that the fact that player 2’s payoffs were presented in negative numbers was the main reason for this finding. The results inspired several interesting explanations. One of the more interesting stated that in the second game, many subjects sought to justify their previous choice. 

A similar experiment is found in Fox (1972), though it did not use negative numbers. Fox found that the row player's distribution is close to 50-50 whereas the column player is concentrated on R. 

In the next game (presented only in 1998), a subject chooses  a number in the interval [0,100] while aiming to be as close as possible to his opponent, who wants to avoid him.  The game has many equilibria.  Every choice is consistent with some equilibrium. A clear majority of the students chose the middle or the edge points. 

Prior to the presentation of the maxmin theorem, students were asked to express their views as to whether "minmax is greater than maxmin" or "maxmin is greater than minmax". The maxmin criterion was presented as a pattern of reasoning whereby a player thinks that his opponent will always successfully predict his action. The minmax criterion was presented as a pattern of reasoning where the subject is a "magician" who always correctly guesses the other player’s intentions, and the other player knows it. Though the minmax is never below the maxmin, in both years, the respondents split almost evenly in their voting. This split demonstrates how difficult and unintuitive this elementary inequality is. 

The next two problems were designed to demonstrate systematic deviations from the game theoretic predictions about zero-sum games resulting from framing effects.   4_boxes is a repetition of an experiment conducted by Rubinstein, Tversky and Heller (1996). The subjects were asked to hide a treasure in one of four boxes placed in a row and labeled A,B,A,A. The seeker is able to open only one box. In 1998 the subjects were assigned the role of the seeker. The distribution of answers (16%, 19%, 54%, 11%) is strongly biased towards the central A box, avoiding the edges. These results were even more pronounced than those of the original experiment (13%, 31%, 45%, 11%). In 1999 the subjects were assigned the role of the hider. Once again, the results (16%, 18%, 45%, 22%) were similar to the results (9%, 36%, 40%, 15%) obtained originally. Where a random seeker of the 98's class was playing the game against a random player of the 99's class the chances that he would find the prize was 33%, much above the game theoretic "prediction" of 25%. By the way, the presentation of the results in class created a feeling of real "discovery". 

The strong tendency to avoid the edges was also obtained in "hide a treasure in a 5X5 table", where the subject hides a treasure in one of the table's 25 boxes.   Here, the 64% of the boxes placed at the edges received only 47%-49% of the choices. This is certainly in line with the results of Ayton and Falk (1995), who asked subjects to hide three treasures in such a layout. They found avoidance of the edges and strong concentration on boxes B4 and D4 (in our case, the most frequent choices were D2 and D4). 

Dictator and Ultimatum Games

The dictator and ultimatum games were used to make the point that monetary and game payoffs are not identical. This is an extremely important point from an educational perspective. 

The dictator game illustrates two principal modes of behavior: "taking the entire sum of money" (52%) and "sharing it equally" (35%).  In other words, half of the class (in 1998) exhibit preferences which are not purely monetary in nature. The concentration of subjects in the above two modes of behavior is similar to the results of Forsythe, Horowitz, Savin, and Sefton (1994). They found that even when the players played for real money, only 35% chose to grab the whole sum. In 1999, the question was framed slightly differently: a subject had to choose the sum of money he gives to the other player (rather than the sum he takes for himself). I did not find any significant difference. Still, 37% chose to split the sum equally and a bit more than half the class "grabbed" the entire sum.

I think that no other game has been used in more experiments than the ultimatum game. Having to agree on the partition of 100 shekels, the offers in 1998 were split into three groups: About 35% of the offers equalled 1, 39% offered 50, and 26% offered a sum between 10 and 40. In comparison, previous experiments such as those of Guth, Schmittberger and Schwarze (1982) and Forsythe, Horowitz, Savin, and Sefton (1994), found that with or without the payment of real money, a higher percentage of subjects offered an equal split of the pie and almost no subjects offered to retain almost all of the money. In 1999, the game was framed differently: the proposer had to declare the amount of money he demands to himself. Once again a framing effect was not traced. About 1/3 of the subjects demand half the sum and 39% (almost) demanded the entire sum.

Reversing the roles, such that students in 1998 responded to a hypothetical offer of 10% (and students in 1999 responded to the a demand for 90%), about 4/5 accepted the offer. In comparison, Guth, Schmittberger and Schwarze (1982) found that 60% of the (small number of) subjects rejected offers of 10%, while Roth and Prasnikar (1992) found that offers below 35% were overwhelmly rejected. 

Finally, when students were asked to determine a minimal cutoff point for acceptance when facing an unknown offer, about one half of the the students in both years said that they would accept all offers (other than 0) and 35% of the students in 1998 and 23% in 1999 set the cutoff point at half the divided sum. Previous results ( see Harrison and McCabe (1992)) showed a much higher proportion of subjects setting the cutoff point at 50%. 

Although there is no difference in the basic modes of behavior appearing among the students and the subjects in laboratory experiments, it seems that the students in my class were more aggressive than the subjects in the previous experiments.  This is not a very surprising fact, considering the prevailing mood in Israel (see Roth, Prasnikar, Okuno-Fujiwara, and Zamir (1991)). 

Extensive Form Games

In the first problem in this category, subjects were presented with a situation, described verbally, similar to the one-shot chain store paradox game: A tailor is considering transforming his shop into a mini-market in a location where a grocery store already exists. He is afraid of a possible harmful response from the grocery store, one which will be harmful to the grocery store as well. About 53% of the students in 1998 and 58% in 1999 recommended that the tailor "enter" the food market (in line with the sub-game perfect equilibrium). In contrast, Schotter, Weigelt and Wilson (1994) found that a much higher percentage of subjects chose to enter. The difference, in my opinion, is due to the fact that the problem here was presented verbally whereas Schotter, Weigelt and Wilson (1994) presented the subjects with an explicit tree which assisted them in the backward inductive reasoning. In fact, when they gave the subjects the opportunity to play the corresponding normal-form game, only 57% of the subjects chose to enter. 

The next problem was intended to demonstrate that in a game situation, more rather than less information may be harmful. Students were asked how much they were willing to "pay" to exchange a play of the battle of the sexes game for a play in a similar game in which the subject would be informed "publicly" about the other player's move before the subject made his own choice.  The three equilibria payoffs for the BoS version are 20, 10 (pure equilibria payoffs) and 6.7 (a mixed equilibrium payoff), which is also its maxmin value. The value of the only sub-game perfect equilibrium of the extensive game which fits the alternative game is 10. Thus, even under the most pessimistic view, a player should not value the offer at more than 3.3. Yet, 56% of the students in 1998 (and 51% in 1999) were ready to pay more than 3.3 and only 26% of the students in 1998 (and 42% in 1999), found the offer valueless. 

In the next problem I followed the ideas of Camerer, Johnson, Rymon and Sen (1993), who performed one of the most beautiful experiments I have ever come across. Subjects had to choose the order by which they would expose information in a two-stage extensive game. The payoff numbers were chosen to be complicated (some were negative and had many digits after the decimal point) in order to create the impression that analyzing the game was not a trivial task and required memory.   Revealing "B" first makes the analysis easier. However, in both years, only 36% of the subjects analyzed the game from it's end, whereas 64% first investigated the content of consequence A. This is definitely in line with the conclusion of Camerer, Johnson, Rymon and Sen (1993) that people tend to analyze an extensive game forward rather than backward. 

The last three problems regard bargaining situations: 

Discounting: Students were asked to predict the outcome of a bargaining session between two bargainers possessing identical characteristics except that one is more impatient than the other. Results: about half of the students in each year, predicted an equal split. The other subjects in 1998 were equally divided as to whether the more impatient person would get more or less than half of the sum whereas in 1999 there was a tendency to predict that the more impatient bargainer will get more than the patient bargainer. Thus, the results do not confirm the intuition that people evaluate impatience as a negative factor in bargaining. This result is in line with the results of Ochs and Roth (1989), who demonstrated the negligible effect of different discount rates on bargaining outcomes. 

Being a proposer or being a responder: Game theoretic models suggest that being a proposer provides a strategic advantage over being a responder.  However, students seem to consider the role of the responder more attractive. The contrast between this finding and the standard game theoretic models begs for an explanation!

Reputation: A seller has refused several offers made by the subject. How does the subject interpret the rejections? Results: 62% of the students in 1998 (and 49% in 1999) did not find the refusal informative, but almost all the rest considered the refusal to be an indication that the value of the item is higher than what was initially thought. 

Finite Horizon Games

The family of finite horizon games has been used mainly as a tool in teaching the meaning of strategy in extensive games. 

The first game in this class was the 100-period centipede game. The results: Very few students (about 10%) chose to stop the game immediately, as Nash equilibrium "predicts", 50% of the students in 1998 and 60% in 1999 chose to "never stop" and 22% chose to stop the game at the last or the penultimate opportunity. The closest comparison to these results is Nagel and Tang (1998), who found (in a six-period centipede game) that almost no subjects were following Nash equilibrium strategies and that the vast majority of subjects were stopping two or three periods before the end of the game.  (See also McKelvey and Palfrey (1992)). 

Not all problems were intended to refute the game theoretic "predictions". In a game with a similar structure to that of the centipede game, the subject was the first in a sequence of players to decide whether "to stop" or "to pass the game to the next player in line", with payoffs that made "all players pass" the unique sub-game prefect equilibrium. Results: about 60% of the students indeed chose "pass". 

In two other experiments, students were asked to play a four-period repetition of the prisoner's dilemma game and of the battle of the sexes. Students were asked to specify their strategies as "plans of actions", that is, they were not asked to specify actions following histories which contradict their own plans. The objective was to emphasize the contrast between the formal concept of a strategy and the intuitive notion of a strategy as a plan of action (see Rubinstein (1991)). 

In the repeated PD, only 34% of the subjects in 1998 and 42% in 1999 chose C in the first period. The results contain a large number of strategies, many of which were difficult to interpret. This fact motivated me to ask the students in 1999 to describe their strategies in words, as well. About half of the students in 1998 and 25% in 1999 chose to play constant D , 5% of the students in each of the years chose to play constant C, and 5% in 1998 and 13% in 1999 chose the Tit-for-Tat strategy. The closest previous experiment is Selten and Stoecker (1986); however, the results are difficult to compare. 

In the repeated BoS, only 10% of the subjects in 1998 (and 28% in 1999) started the game by playing the less favorable action. Once again, the results contained a large variety of strategies. Two strategies were most frequent: 22% of the students started the game by playing the more favorable action in 1998 (and 25% in 1999) and continuing to play the best response against their opponent's last played action; 12% of the in 1998 (and 8% in 1999) students chose the strategy "play the favorable action unless, in the past, the opponent played his favorable action in a strict majority of the periods" . 

It is interesting to note that in the last two experiments, although asked to describe their strategies, many of the students chose to the describe their considerations.

Randomization

In class I presented various interpretations of mixed strategies (see Osborne and Rubinstein (1994, ch. 3)): 
· the "naive" interpretation - a player chooses a random device such as a roulette.
· the "purification" idea - a player’s behavior depends deterministically on unobserved factors.
· the "beliefs" interpretation- a player's mixed strategy is what other players think about a player’s behavior.
· the "large population" interpretation - a mixed strategy is a distribution of the modes of behavior displayed by a large population of players who are matched randomly in order to play the game. 

Adopting either of the first two interpretations led to a discussion of "random behavior". My aim was to demonstrate to the students that when people choose "random behavior", they create patterns of behavior which are not so random. 

In randomization 1,  students were asked to choose an integer between 1 and 9. In Simon (1971) and in Simon and Primavera (1972), the number 7 was clearly the most frequent choice (33% and 24% of the subjects, respectively, chose 7 from among the numbers 0,1,...,9). Here, the conjecture that people over-choose "7" was confirmed.  In fact, "7" and "5" were the most frequent choices at 17% each in both years. Another clear phenomenon is the avoidance of the "edges" ("1" and "9"), which were the least chosen alternatives.

In randomization2, students were given the following task: "Randomly choose 4 of the integers {1,2,...,8}."  In 1998, three patterns were observed in the results: 

1) Out of the 70 possible answers, only two were chosen by more than 3 students: 7 students chose "1234" and 6 chose "1357".
2) Although 21% of the possible sequences do not include either 1 or 2, only 6% of the students actually excluded either 1 or 2 from their chosen sequence.
3) The number 6 was chosen by 30% of the students, far less than all other numbers. Even when we exclude the students who chose the sequences "1234" or "1357", the proportion of answers which did not include 6 was well above 50%. One explanation is that subjects who chose three numbers from 1,2,3,4,5 felt they must also choose one of the last two numbers: thus, they skipped 6. In 1999, the results were entirely different. No student chose the sequence "1234" and only 2 students chose "1357". One common observation: the number 8 was chosen well below 50%. This is one of the few experiments I conducted in class in which I cannot explain the sharp differences between the results in the two years.

Excess randomization is also well documented in the literature, under the name "matching probabilities". In "guess the departments" (following an idea I was working on with the late Amos Tversky), students had to guess the second major of five randomly chosen students who study economics in a double-major program. Though the distribution of the second major was not given to the students, one could expect that they had some idea, and, in any case, maximization of the probabilities to win the prize should have led them to choose the major they believe is the most frequent among their five guesses. Theresults clearly demonstrate an excess use of randomization. Only 28% of the students in 1998 and 6% of the students in 1999 repeated the same guess five times. All the others diversified their answers and included at least 3 different choices in their list of guesses. The results correspond well with Loewenstein and Read (1995), who demonstrated a strong tendency towards diversification when subjects had to choose a sequence of three items even though one item was viewed by them as superior to the others. 

In the next experiment, students were asked to choose one out of four possibilities, which yield prizes with probabilities of 21%, 27%, 32% and 20%, respectively. The two versions presented in the two years were quite different. In 1998,  the students had to choose at which gate to choose a friend and in 1999, at which gate to ambush a suspect. In both years the students were explicitly allowed to randomize but in 1998, one of the explicit options was to choose one of the gates. The differences in the responses to the two versions is very clear. In 1998, only 28% of the students chose to randomize. In 1999, once the students had explicitly to allocate the probabilities among the four options, 67% chose to randomize (in particular, 30% chose the numbers which were given as the probabilities of each gate and 14% assigned equal probabilities to the four gates).

Failures

A course in game theory is not a course on rational behavior. However, I consider it important to demonstrate the limits of rational behavior to the students . 

The problem of count the number of F's was distributed this year on the Net (I do not know who initiated this beautiful problem). A subject was asked to count the number of F's in a 90-letter text. The question was given, first of all,...for  but it was also supposed to remind the students that people often make systematic mistakes. Counting the number of "F's" in an 81- letter text is supposed to be a trivial task, but only one third of the students did it right. The common answer (at least among my friends), "3", was given by a quarter of the students in 1998 and 30% in 1999. 

 only 

The next problem is the simplest problem yet devised which demonstrates the winner's curse phenomenon. A student has to bid for an object; he will receive  only it if he offers more than its real value, a number which is distributed uniformly in the interval [0,1000]. The bidder will then be able to sell the item for 150% of its real value. The problem was studied first (with and without real money) by Samuelson and Bazerman (1985). The results here were not significantly different (the differences seem to depend on the fact that I allowed the subjects to bid for more than 1000). Only about 10% of the students gave the "optimal" offer 0; the majority of the students in each of the two years offered 500 or more! 

Another puzzle which has been widely discussed in the last few years, is the exchange of envelopes.  Two positive numbers, one twice the size of the other, are placed in two sealed envelopes. The subject randomly receives one of the envelopes and another person tries to persuade him to exchange the envelopes with the claim:  "If the number x is in your envelope, the expected value of the other envelope is 3x/2".  This problem was offered to the students at a late stage in the course followed by a post-class problem where the students had to understand the Brams and Kilgour (1995) game theoretic treatment of the problem.  In each of the two years, only about 20% of the students expressed, willingness to exchange envelopes. 

Some other problems dealt with the basic assumptions of the VNM theory of decision making under uncertainty. 

A variant of the Allais paradox (originating in Kahneman and Tversky (1979)), was presented to the students. Students had to choose between two lotteries: one which yields $4,000 with probability 0.2 and a second which yields $3,000 with probability 0.25. The results: 72%-74% of the students chose the first lottery, very close to the results of Kahneman and Tversky (1979), where 65% of the subjects chose the first lottery.  In 1999, I confronted the students with the variant choice between the certain $3,000 and a lottery which yields $4,000 with probability 0.8. The results : 86% of the subjects chose the certain prize. As is well known, the two results sharply conflict with expected utility theory. 

A basic principle of rationality in decision making under uncertainty is the "sure thing" principle: If action D is better than action C under any of  two exclusive circumstances, then D is better than C when the decision maker does not have information what circumstance had occured. Shafir and Tversky (1992) showed that more people choose to cooperate in the prisoner's dilemma than in either of the two cases in which they are told that the other player had cooperated or defected. Here, the problems were given to the students in the order "player 2 has made up his mind to cooperate" , "player 2 has made up his mind to defect"  and a regular prisoner's dilemma. The results were in line with Shafir and Tversky (1992): In 1998, only 9% cooperated when the other player did so, only 4% cooperated when the other player defected, and 16% cooperated when they did not know what the other player chose (12%, 0%, 16% were the corresponding numbers in 1999 and 3%, 16% and 37% were the corresponding figures in Shafir and Tversky (1992), with similar though not identical payoffs). 

Ethical Values

Does game theory affect the ethical attitudes of students concerning behavior in strategic situations? The suspicion may arise that game theory intensifies "selfish motivations", strengthens manipulative attitudes, reduces the importance of ethical considerations, and so forth. In collaboration with a group of students, Gilad Aharanovitz, Kfir Eliaz, Yoram Hamo, Michael Ornstein, Rani Spiegler, and Ehud Yampuler, we gave a series of questions to the students at the first meeting of the course. We compared the results with those of a similar group of students who had just completed a similar course given by another teacher. The results did not show any clear difference between students' behavior before and after taking the course. We still feel that more experiments should be conducted on the subject. In the meantime, let it suffice to present the results of two of our problems presented ti the students. 

In order to examine the tendency of students to behave manipulatively in elections, they were presented with some hypothetical election situations in which their candidates were doomed to lose but where they could increase the chances of their second best choice. Indeed, 76-79% of the students were prepared not to vote for their favorite candidate in order to help their second-best choice to win. (For a related experiment see Eckel and Holt (1989).) 

In another problem, students were placed in the role of auto dealers who had offered a price and then received information that the potential buyer was ready to pay more than they had offered him. They then had to decide whether or not to raise their price.  Here, 55% of the students in 1998 and an even higher proportion, 71% in 1999, stated that they would not raise the price. 

3. A Short Comment on Experimental Methods

The main purpose of this paper was to summarize my experience in teaching an undergraduate course in game theory. However, in retrospect, I feel that the experience presents an opportunity to evaluate the experimental methods used in game theory. Researchers are split into two camps: Some create careful laboratory environments and pay the subjects monetary rewards for their performance in the experiment; others ask subjects to fill out questionnaires requiring them to speculate on hypothetical situations. 

I fall into the second category.  (Compare with the view in Camerer and Hogarth (1999)).  My impression is that the results are as significant as those obtained under more sterile conditions, in the sense that the same modes of behavior appear in both sets of results. If we were interested in obtaining precise statistics regarding the appearance of those modes of behavior in the general population, then both methods are deeply flawed since our subjects are never chosen from random representative samples. In cases where the previous results differ quantitatively (as they do in the dictator and ultimatum games, for example), the distribution of modes of behavior is clearly affected by culture, education and personal characteristics; hence, there is no reason to expect uniform results. I would therefore like to stress my doubts as to the necessity of laboratory conditions and the use of real money in experimental game theory. 
 
 

References

Ayton, P. and R. Falk (1995), "Subjective randomness in hide-and-seek games", in Books of Abstracts of the 15th bi-annual conference on Subjective Probability, Utility and Decision-Making, p. 37. 

Beard, T. and R. Beil, Jr. (1994), "Do people rely on the self-interested maximization of others? An experimental test", Management Science, 40, 252-262. 

Ben-Porath, E. and E. Dekel (1992), "Signaling future actions and the potential for self-sacrifice", Journal of Economic Theory, 57, 36-51. 

Brams, S. and M. Kilgour (1995), "The box problem: To switch or not to switch", Mathematics Magazine, 68, 27-34. 

Camerer, C., E. Johnson, T. Rymon and S. Sen (1993), "Cognition and framing in sequential bargaining for gains and losses", in Frontiers of Game Theory, K. Binmore, A. Kirman, and P. Tani (eds.), Boston: MIT Press, 27-48. 

Camerer, C. and R. Thaler (1995), "Anomalies: ultimatums, dictators and manners", Journal of Economic Perspectives, 9, 209-219. 

Camerer, C. (1997), "Progress in behavioral game theory", Journal of Economic Perspectives, 11, 167-188. 

Camerer, C. and R.M. Hogarth (1999), "The effects of financial incentives in experiments: A review and capital-labor-production framework, mimeo, CALTECH.

Cooper, R., D. V. DeJong, R. Forsythe and T. Ross (1980), "Alternative institutions for resolving coordination problems: Experimental evidence on forward induction and preplay communication" in Experiment in non-cooperative oligopoly, by J.W. Friedman and A.C. Hoggat (eds.), CT: JAI Press, 129-146. 

Cooper, R.,D.V. DeJong, R. Forsythe and T. Ross (1993), "Forward induction in the battle of the sexes games", American Economic Review, 83, 1303-1316. 

Eckel, C. and C. Holt (1989), "Strategic voting in agenda-controlled committee experiments", American Economic Review, 79, 763-773. 

Forsythe, R. , J. Horowitz, N. Savin and M. Sefton (1994), "Fairness in simple bargaining experiments", Games and Economic Behavior, 6, 347-369. 

Fox, J. (1972), "The learning of strategies in a simple two-person zero-sum game without a saddlepoint", Behavioral Science, 17, 300-308. 

Guth, W., R. Schmittberger and B. Schwarze (1982), "An experimental analysis of ultimatum bargaining", Journal of Economic Behavior and Organization, 3, 367-388. 

Harison, G. and K. McCabe (1992), "Expectations and fairness in a simple bargaining experiment", mimeo. 

Kahneman, D. and A. Tversky (1979), "Prospect theory: An analysis of decision under risk", Econometrica, 47, 263-292. 

Loewenstein, G. and D. Read (1995), "Diversification bias: Explaining the discrepancy in variety seeking between combined and separated choices", Journal of Experimental Psychology, 1, 34-49. 

McKelvey, R. and R. Palfrey (1992), "An experimental study of the centipede game", Econometrica, 60, 803-836. 

Mehta, J. , C. Starmer and R. Sugden (1994), "The nature of salience: An experimental investigation of pure coordination games", American Economic Review, 84, 658-673.

Nagel, R. (1995), "Unravelling in guessing games : An experimental study", American Economic Review, 85, 1313-1326. 

Nagel, R. and F. Tang (1998), "Experimental results on the centipede game in normal form: An investigation on learning", Journal of Mathematical Psychology, 42, 356-384. 

Ochs, J. and E. Roth (1989), "An experimental study of sequential bargaining", American Economic Review, 79, 355-384. 

Osborne, M. and A. Rubinstein (1994), A Course in Game Theory, MIT Press , Ch. 3, 37-43. 

Roth, E., J. Murnighan and F. Schomaker (1988): " The deadline effect in bargaining : Some experimental evidence" American Economic Review, 78, 806-823. 

Roth, E. ,V. Prasnikar, M. Okuno-Fujiwara and S. Zamir (1991) " Bargaining and market behavior in Jerusalem, Ljubljana, Pittsburgh and Tokyo: An experimental study", American Economic Review, 81, 1068-1095. 

Roth, E. and V. Prasnikar (1992), "Considerations of fairness and strategy: Experimental data from sequential games", Quarterly Journal of Economics, 107, 865-888. 

Rubinstein, A. (1991), "Comments on the interpretation of game theory", Econometrica, 59, 904-921. 

Rubinstein, A., A. Tversky and D. Heller (1996), "Naive strategies in zero-sum games", in Understanding Strategic Interaction - Essays in Honor of Reinhard Selten, W. Guth et al. (editors), Springer-Verlag, 394-402. 

Samuelson, W. and M. Bazerman (1985), "The winner's curse in bilateral negotaiotions", in V.L. Smith (editor), Research in Experimental Economics Vol. 3, Greenwich, CT: JAI Press, 105-137. 

Schotter, A., K. Weilgelt and C.Wilson (1994), "A labratory investigation of multiperson rationality and presentation effects", Games and Economic Behavior, 6, 445-468. 

Selten, R. and R. Stoecker (1986), "End behavior in sequences of finite prisoner's dilemma supergames", Journal of Economic Behavior and Organization, 7, 47-70. 

Shafir, E. and A. Tversky (1992), "Thinking through uncertainty: Nonconsequential reasoning and choice", Cognitive Psychology, 24, 449-474. 

Simon, W. (1971), "Number and color responses of some college students: Preliminary evidence for the 'Blue Seven Phenomenon' ", Perceptual and Motor Skills, 84, 658-673.

Simon, W. and L. Primavera (1972), "Investigation of the 'Blue Seven phenomenon' in elementary and junior high school children' ", Psychological Reports, 31, 128-130. 

Thaler, R. (1998), "Giving markets a human dimension", Mastering Finance in Financial Times, 6 , 4.