Social Intuitionism

Extracted text from SocialIntuitionism.pdf:

The Emotional Dog and its Rational Tail: A Social Intuitionist Approach to Moral Judgment Jonathan Haidt University of Virginia October 31, 2000

This is the manuscript that was published, with only minor copy-editing alterations, as: Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review. 108, 814-834 Copyright 2001, American Psychological Association To obtain a reprint of the final type-set article, please go through your librarys journal services, or contact the author directly

Abstract Research on moral judgment has been dominated by rationalist models, in which moral judgment is thought to be caused by moral reasoning. Four reasons are given for considering the hypothesis that moral reasoning does not cause moral judgment; rather, moral reasoning is usually a post-hoc construction, generated after a judgment has been reached. The social intuitionist model is presented as an alternative to rationalist models. The model is a social model in that it de-emphasizes the private reasoning done by individuals, emphasizing instead the importance of social and cultural influences. The model is an intuitionist model in that it states that moral judgment is generally the result of quick, automatic evaluations (intuitions). The model is more consistent than rationalist models with recent findings in social, cultural, evolutionary, and biological psychology, as well as anthropology and primatology.

Author notes I thank Sara Algoe, Jon Baron, Fredrik Bjorklund, Talbot Brewer, Daniel Fessler, Alan Fiske, Nico Frijda, Chip Heath, Jeff Greenberg, Dacher Keltner, Angeline Lillard, George Loewenstein, Charles Mathewes, Ed Royzman, Paul Rozin, John Sabini, Jonathan Schooler, Stacey Sinclair, Barbara Spellman, Stephen Stich, Stephen Stose, Daniel Wegner, Daniel Willingham, and Timothy Wilson for helpful comments on earlier drafts. Preparation of this paper was supported by the National Institute on Drug Abuse, grant #1-RO3-DA12606-01. Correspondence concerning this article should be addressed to Jonathan Haidt, Dept. of Psychology, 102 Gilmer Hall Box 400400, University of Virginia, Charlottesville VA 22904-4400. Electronic mail may be sent to haidt at virginia.edu

Morality and Intuition 2 Julie and Mark are brother and sister. They are traveling together in France on summer vacation from college. One night they are staying alone in a cabin near the beach. They decide that it would be interesting and fun if they tried making love. At very least it would be a new experience for each of them. Julie was already taking birth control pills, but Mark uses a condom too, just to be safe. They both enjoy making love, but they decide not to do it again. They keep that night as a special secret, which makes them feel even closer to each other. What do you think about that, was it OK for them to make love? Most people who hear the above story immediately say that it was wrong for the siblings to make love, and they then set about searching for reasons (Haidt, Bjorklund, & Murphy, 2000). They point out the dangers of inbreeding, only to remember that Julie and Mark used two forms of birth control. They argue that Julie and Mark will be hurt, perhaps emotionally, even though the story makes it clear that no harm befell them. Eventually, many people say something like I dont know, I cant explain it, I just know its wrong. But what model of moral judgment allows a person to know that something was wrong, without knowing why? Moral psychology has long been dominated by rationalist models of moral judgment (Figure 1). Rationalist approaches in philosophy stress the power of a priori reason to grasp substantial truths about the world (Williams, 1967, p.69 ). Rationalist approaches in moral psychology, by extension, say that moral knowledge and moral judgment are reached primarily by a process of reasoning and reflection (Kohlberg, 1969; Piaget, 1932/1965; Turiel, 1983). Moral emotions such as sympathy may sometimes be inputs to the reasoning process, but moral emotions are not the direct causes of moral judgments. In rationalist models one briefly becomes a judge, weighing issues of harm, rights, justice, and fairness, before passing judgment on Julie and Mark. If no condemning evidence is found, no condemnation is issued. This paper reviews evidence against rationalist models and proposes an alternative: the social intuitionist model (Figure 2). Intuitionism in philosophy refers to the view that there are moral truths, and that when people grasp these truths they do so not by a process of ratiocination and reflection, but rather by a process more akin to perception, in which one just sees without argument that they are and must be true (Harrison, 1967, p. 72). Thomas Jeffersons declaration that certain truths are self-evident is an example of ethical intuitionism. Intuitionist approaches in moral psychology, by extension, say that moral intuitions (including moral emotions) come first and directly cause moral judgments (Haidt, in press; Kagan, 1984; Shweder & Haidt, 1993; J. Wilson, 1993). Moral intuition is a kind of cognition, but it is not a kind of reasoning. The social part of the social intuitionist model proposes that moral judgment should be studied as an interpersonal process. Moral reasoning is usually an ex-post facto process used to influence the intuitions (and hence judgments) of other people. In the social intuitionist model one feels a quick flash of revulsion at the thought of incest and one knows intuitively that something is wrong. Then, when faced with a social demand for a verbal justification, one becomes a lawyer trying to build a case, rather than a judge searching for the truth. One puts forth argument after argument, never wavering in the conviction that Julie and Mark were wrong, even after ones last argument has been shot down. In the social intuitionist model it becomes plausible to say I dont know, I cant explain it, I just know its wrong. The paper begins with a brief review of the history of rationalism in philosophy and psychology. It then describes the social intuitionist model and recent relevant findings from a variety of fields. These findings offer four reasons for doubting the causality of reasoning in moral judgment: 1) there are two cognitive processes at work -- reasoning and intuition -- and the reasoning process has been overemphasized; 2) reasoning is often motivated; 3) the reasoning process constructs post-hoc justifications, yet we experience the illusion of objective reasoning; and 4) moral action covaries with moral emotion more than with moral reasoning. Since much of this evidence is drawn from research outside of the domain of moral judgment, the social intuitionist model is presented only as a plausible alternative approach to moral psychology, not as an

Morality and Intuition 3 established fact. The paper therefore concludes with suggestions for future research, and for ways of integrating the findings and insights of rationalism and intuitionism. It must be stressed at the outset that the social intuitionist model is an anti-rationalist model only in one limited sense: it says that moral reasoning is rarely the direct cause of moral judgment. That is a descriptive claim, about how moral judgments are actually made. It is not a normative or prescriptive claim, about how moral judgments ought to be made. Baron (1998) has demonstrated that people following their moral intuitions often bring about non-optimal or even disastrous consequences in matters of public policy, public health, and the tort system. A correct understanding of the intuitive basis of moral judgment may therefore be useful in helping decision makers avoid mistakes, and in helping educators design programs (and environments) to improve the quality of moral judgment and behavior. Philosophy and the Worship of Reason Philosophers have frequently written about the conflict between reason and emotion as a conflict between divinity and animality. Platos Timaeus presents a charming myth in which the gods first created human heads, with their divine cargo of reason, and then found themselves forced to create seething, passionate bodies to help the heads move around in the world. The drama of human moral life was the struggle of the heads to control the bodies by channeling their passions towards virtuous ends. The stoic philosophers took an even dimmer view of the emotions, seeing them as conceptual errors that bound one to the material world and therefore to a life of misery (Solomon, 1993). Medieval Christian philosophers similarly denigrated the emotions because of their link to desire, and hence sin. The 17th centurys continental rationalists (e.g., Leibniz, Descartes) worshiped reason as much as had Plato, hoping to model all of philosophy on the deductive method developed by Euclid. But in the 18th century, English and Scottish philosophers (e.g., Shaftesbury, Hutcheson, Hume, and Smith) began discussing alternatives to rationalism. They argued that people have a built-in moral sense which creates pleasurable feelings of approval toward benevolent acts and corresponding feelings of disapproval towards evil and vice. David Hume in particular proposed that moral judgments are similar in form to aesthetic judgments: they are derived from sentiment, not reason, and we attain moral knowledge by an immediate feeling and finer internal sense, not by a chain of argument and induction (Hume, 1776/1965, p.24). His most radical statement of this position was that we speak not strictly and philosophically when we talk of the combat of passion and of reason. Reason is, and ought only to be the slave of the passions, and can never pretend to any other office than to serve and obey them1 " (Hume, 1739/1969, p. 462). The thrust of Humes attack on rationalism was that reason alone cannot accomplish the magnificent role it has been given since Plato. Hume saw reason as a tool used by the mind to obtain and process information about events in the world, or about relations among objects. Reason can let us infer that a particular action will lead to the death of many innocent people, but unless we care about those people, unless we have some sentiment that values human life, reason alone cannot advise against taking the action. Hume argued that a person in full possession of reason yet lacking moral sentiment would have difficulty choosing any ends or goals to pursue, and would look like what we now call a psychopath (Hume, 1776/1965; Cleckley, 1955). Humes emotivist approach to ethics was not well received by philosophers. Kants (1785/1959) rationalist ethical theory2 was created as an attempt to refute Hume, and Kant has had a much larger impact than Hume on modern moral philosophers (e.g., Hare, 1981; Rawls, 1971), many of whom have followed Kant in attempting to deduce a foundation for ethics from the meaning of rationality itself. Psychology and the Focus on Reasoning Psychologists, however, freed themselves from the worship of reason in the late 19th century, when they abandoned the armchair and went into the laboratory. Until the cognitive revolution of the 1960's, the major schools of psychology did not see reason as the master of anything, and their views on morality were compatible with Humes emphasis on emotions. Freud (1900/1976) saw peoples

Morality and Intuition 4 judgments as driven by unconscious motives and feelings, which are then rationalized with publicly acceptable reasons. The behaviorists also saw moral reasoning as epiphenomenal in the production of moral behavior, explaining morality as the acts that a society happens to reward or punish (Skinner, 1971). Kohlberg and the Cognitive Revolution But then came Lawrence Kohlberg. Kohlbergs work was a sustained attack on irrational emotive theories (1969, p. 188), and his cognitivedevelopmental theory was an important part of the cognitive revolution. Kohlberg built on Piagets (1932/1965) pioneering work, developing an interviewing method that was suitable for use with adults as well as children. Kohlberg presented participants with dilemmas in which moral and nonmoral claims were present on both sides, and he then looked to see how people resolved the conflict. In his best known dilemma a man named Heinz must decide whether he should break into a druggists shop to steal a drug that may save the life of his dying wife. Kohlberg found a six level progression of increasing sophistication in how people handled such dilemmas. He claimed that children start as egoists, judging actions by the good or bad consequences they bring to the self. But as childrens cognitive abilities expand they develop the ability to role-take, or see a situation from other peoples perspectives. The experience of role-taking drives the child on to the less egocentric and more adequate conventional and then post-conventional levels of moral reasoning. Kohlbergs focus is on development, but in each of his major papers he addresses the question of mechanism. He consistently endorses a rationalist and somewhat Platonic model in which affect may be taken into account by reason (as in Figure 1), but it is reasoning that ultimately makes the decisions: We are claiming... that the moral force in personality is cognitive. Affective forces are involved in moral decisions, but affect is neither moral nor immoral. When the affective arousal is channeled into moral directions, it is moral; when it is not so channeled, it is not. The moral channeling mechanisms themselves are cognitive (1971, p.230-231). Kohlberg is quite explicit that the cognitive mechanisms he discusses involve conscious, language based thinking. He is interested in the phenomenology of moral reasoning, and he describes one of the pillars of his approach as the assumption that moral reasoning is the conscious process of using ordinary moral language (Kohlberg, Levine, & Hewer, 1983 p. 69). After Kohlberg Kohlberg trained or inspired most of the leading researchers in moral psychology today (see chapters in Kurtines & Gewirtz, 1991; and Lapsley, 1996). Rationalism still rules, and there appears to be a consensus that morality lives within the individual mind as a trait-like cognitive attainment, a set of knowledge structures about moral standards that children create for themselves in the course of their everyday reasoning (see Darley, 1993). The social-interactionist perspective (Nucci & Turiel, 1978; Turiel, 1983, 1998; Turiel, Killen, & Helwig, 1987) is one of the most widely used approaches at present, and can serve as an illustrative model. This research is based on a method developed by Nucci and Turiel (1978) in which children are interviewed about rule violations. After giving an initial judgment, the child is asked to respond to a series of probe questions designed to assess how the child thinks about the rule in question (e.g., if there was no rule, would the action be OK? Could the rule be changed?). Participants are also asked to provide justifications of their judgments. In the social-interactionist model, people are said to think about the consequences of an action before determining whether the action is a moral violation. Actions that lead to injustice, to harm, or to the violation of rights are recognized as falling within the moral domain, and are treated differently from other kinds of rule violations. Rules prohibiting moral violations are judged, even by young children, to be universally applicable and unalterable. Actions that involve no injustice, harm, or rights violations are treated as violations of social conventions (involving locally agreed-upon uniformities of behavior within social systems) or as personal issues (areas of individual prerogative). Researchers in this tradition are sensitive to the ways that moral development occurs in a social context, driven forward by childrens interactions

Morality and Intuition 5 with peers in such contexts as turn-taking, sharing, harming, and responding to harm. This emphasis on social interaction is in harmony with the social part of the social intuitionist model, and is not a source of contention in the present paper. The central source of contention, and the focus of the present paper, concerns the causal role of reflective, conscious reasoning. Questioning the Causality of Reasoning It is undeniable that people engage in moral reasoning. But does the evidence really show that such reasoning is the cause of moral judgment, rather than the consequence? Turiel, Hildebrandt, and Wainryb (1991) examined young adults reasoning about issues of abortion, homosexuality, pornography, and incest. They found that people who judged the actions to be moral violations also talked about harmful consequences, whereas people who thought the actions were not wrong generally cited no harmful consequences. Turiel et al. interpret these findings as showing the importance of informational assumptions, for example: people who thought that life begins at conception were generally opposed to abortion, whereas people who thought that life begins later were generally not opposed to abortion. But in making this interpretation Turiel et al. made a jump from correlation to causation. The correlation between judgment and supporting belief does not show that the belief caused the judgment. An intuitionist interpretation is just as plausible: the anti-abortion judgment (a gut feeling that abortion is bad) caused the belief that life begins at conception (an ex post facto rationalization of the gut feeling). Haidt, Koller, and Dias (1993) found evidence for such an intuitionist interpretation. They examined American and Brazilian responses to actions that were offensive yet harmless, such as eating ones dead pet dog, cleaning ones toilet with the national flag, or eating a chicken carcass one has just used for masturbation. The stories were carefully constructed so that no plausible harm could be found, and most participants directly stated that nobody was hurt by the actions in question. Yet participants still usually said the actions were wrong, and universally wrong. They frequently made statements such as its just wrong to have sex with a chicken. Furthermore, their affective reactions to the stories (statements that it would bother them to witness the action) were better predictors of their moral judgment than were their claims about harmful consequences. Haidt and Hersh (in press) found the same thing when they interviewed conservatives and liberals about sexual morality issues, including homosexuality, incest, and unusual forms of masturbation. For both groups, affective reactions were good predictors of judgment, while perceptions of harmfulness were not. Haidt and Hersh also found that participants were often morally dumbfounded(Haidt, Bjorklund, & Murphy, 2000), that is, they would stutter, laugh, and express surprise at their inability to find supporting reasons, yet they would not change their initial judgments of condemnation. It seems, then, that for affectively charged events such as incest and other taboo violations, an intuitionist model may be more plausible than a rationalist model. But can an intuitionist model handle the entire range of moral judgment? Can it accommodate the findings from rationalist research programs while also explaining new phenomena, and leading to new and testable predictions? The social intuitionist model may be able to do so. The Social Intuitionist Model The central claim of the social intuitionist model is that moral judgment is caused by quick moral intuitions, and is followed (when needed) by slow, ex-post facto moral reasoning. Clear definitions of moral judgment, moral intuition, and moral reasoning are therefore needed. Moral Judgment Moral philosophers have long struggled to distinguish moral judgments from other kinds of judgments (e.g., aesthetics, skill, or personal taste). Rather than seeking a formal definition that lists the necessary and sufficient features of a moral judgment, the present paper takes a more empirical approach, starting from a behavioral fact about human beings: that in every society, people talk about and evaluate the actions of other people, and these evaluations have consequences for future interactions (Boehm, 1999). Many of these evaluations occur against the backdrop of specific cultural practices, in which one praises or criticizes

Morality and Intuition 6 the skills or talents of an individual (e.g., she is a daring chef). But an important subset of these evaluations are made with respect to virtues or goods that are applied to everyone in the society (e.g., fairness, honesty, or piety in some cultures), or to everyone in a certain social category (e.g., chastity for young women in some cultures, or generosity for lineage heads). These virtues are obligatory in that everyone (within the relevant categories) is expected to strive to attain them. People who fail to embody these virtues, or whose actions betray a lack of respect for them, are subject to criticism, ostracism, or some other punishment. It is this subset of evaluations that is at issue in the present paper. (For more on moral goods see Ross, 1930; Shweder & Haidt, 1993.) Moral judgments are therefore defined as: evaluations (good versus bad) of the actions or character of a person that are made with respect to a set of virtues held by a culture or subculture to be obligatory. The definition is intentionally left broad to allow a large gray area of marginally moral judgments. For example, eating a low fat diet may not qualify as a moral virtue for most philosophers, yet in health-conscious subcultures, people who eat cheeseburgers and milkshakes are seen as morally inferior to those who eat salad and chicken (Stein & Nemeroff, 1995). Moral Reasoning Everyday moral reasoners are sometimes said to be like scientists, who learn by forming and testing hypotheses, who build working models of the social world as they interact with it, and who consult these models when making moral judgments (Turiel, 1983). A key feature of the scientist metaphor is that judgment is a kind of inference, made in several steps. The reasoner searches for relevant evidence, weighs evidence, coordinates evidence with theories, and reaches a decision (Kuhn, 1989; Nisbett & Ross, 1980). Some of these steps may be performed unconsciously, and any of the steps may be subject to biases and errors, but a key part of the definition of reasoning is that it has steps, at least a few of which are performed consciously. Galotti (1989, p.333), in her definition of everyday reasoning, specifically excludes any one-step mental processes such as sudden flashes of insight, gut reactions, and other forms of momentary intuitive response. Building on Galotti (1989), moral reasoning can now be defined as: conscious mental activity that consists of transforming given information about people in order to reach a moral judgment. To say that moral reasoning is a conscious process means that the process is intentional, effortful, controllable, and that the reasoner is aware that it is going on (Bargh, 1994). Moral Intuition Commentators on intuition have generally stressed the fact that a judgment, solution, or other conclusion appears suddenly and effortlessly in consciousness, without any awareness of the mental processes that led to the outcome (Bastick, 1982; Simon, 1992). Bruner (1960 p.57) said that intuition does not advance in careful steps; rather, it involves manoeuvers based seemingly on an implicit perception of the total problem. The thinker arrives at an answer, which may be right or wrong, with little if any awareness of the process by which he reached it. It must be stressed that the contrast of intuition and reasoning is not the contrast of emotion and cognition. Intuition, reasoning, and the appraisals contained in emotions (Frijda, 1986; Lazarus, 1991) are all forms of cognition. Rather, the words intuition and reasoning are intended to capture the contrast made by dozens of philosophers and psychologists between two kinds of cognition. The most important distinctions (see Table 1) are that intuition occurs quickly, effortlessly, and automatically, such that the outcome but not the process is accessible to consciousness, while reasoning occurs more slowly, requires some effort, and involves at least some steps that are accessible to consciousness. Building on Bastick, Bruner, Simon, and others, moral intuition can be defined as: the sudden appearance in consciousness of a moral judgment, including an affective valence (good-bad, likedislike), without any conscious awareness of having gone through steps of search, weighing evidence, or inferring a conclusion. Moral intuition is therefore the psychological process that the Scottish philosophers talked about, a process akin to aesthetic judgment: one sees or hears about a social

Morality and Intuition 7 event and one instantly feels approval or disapproval. The Links in the Model The social intuitionist model is composed of four principal links or processes, shown as solid arrows in Figure 2. The existence of each link is well established by prior research in some domains of judgment, although not necessarily in the domain of moral judgment. The model is therefore presented as a proposal to spur thinking and new research on moral judgment. 1) The intuitive judgment link. The model proposes that moral judgments appear in consciousness automatically and effortlessly as the result of moral intuitions. Examples of this link in non-moral cognition include Zajoncs (1980) demonstrations that affectively valenced evaluations are made ubiquitously and rapidly, before any conscious processing has taken place. More recent examples include findings that much of social cognition operates automatically and implicitly (Bargh & Chartrand, 1999; Greenwald & Banaji, 1995). 2) The post-hoc reasoning link. The model proposes that moral reasoning is an effortful process, engaged in after a moral judgment is made, in which a person searches for arguments that will support an already-made judgment. Nisbett and Wilson (1977) demonstrated such post-hoc reasoning for causal explanations. Kuhn (1991), Kunda (1990), and Perkins, Farady, and Bushey (1991) found that everyday reasoning is heavily marred by the biased search only for reasons that support ones already-stated hypothesis. 3) The reasoned persuasion link. The model proposes that moral reasoning is produced and sent forth verbally in order to justify ones already-made moral judgment to others. Such reasoning can sometimes affect other people, although moral discussions and arguments are notorious for the rarity with which persuasion takes place. Since moral positions always have an affective component to them, it is hypothesized that reasoned persuasion works not by providing logically compelling arguments, but by triggering new affectively valenced intuitions in the listener. The importance of using affective persuasion to change affectively based attitudes has been demonstrated by Edwards and von Hippel (1995), and by Shavitt (1990). 4) The social persuasion link. Because people are highly attuned to the emergence of group norms, the model proposes that the mere fact that friends, allies, and acquaintances have made a moral judgment exerts a direct influence on others, even if no reasoned persuasion is used. Such social forces may elicit only outward conformity (Asch, 1956), but in many cases peoples privately held judgments are directly shaped by the judgments of others (Berger & Luckman, 1967; Newcomb, 1943; Sherif, 1935). These four links form the core of the social intuitionist model. The core of the model gives moral reasoning a causal role in moral judgment, but only when reasoning runs through other people. It is hypothesized that people rarely override their initial intuitive judgments just by reasoning privately to themselves, because reasoning is rarely used to question ones own attitudes or beliefs (see the motivated reasoning problem, below). However, people are capable of engaging in private moral reasoning, and many people can point to times in their lives when they changed their minds on a moral issue just from mulling the matter over by themselves. While some of these cases may be illusions (see below, the post-hoc reasoning problem), other cases may be real, particularly among philosophers, one of the few groups that has been found to reason well (Kuhn, 1991). The full social intuitionist model therefore includes two ways in which private reasoning can shape moral judgments. 5) The reasoned judgment link. People may at times reason their way to a judgment by sheer force of logic, overriding their initial intuition. In such cases reasoning truly is causal, and cannot be said to be the slave of the passions. However such reasoning is hypothesized to be rare, occurring primarily in cases in which the initial intuition is weak and processing capacity is high. In cases where the reasoned judgment conflicts with a strong intuitive judgment a person will have a dual attitude (Wilson, Lindsey, & Schooler, 2000) in which the reasoned judgment may be expressed verbally, yet the intuitive judgment continues to exist under the surface. 6) The private reflection link. In the course of thinking about a situation a person may

Morality and Intuition 8 spontaneously activate a new intuition that contradicts the initial intuitive judgment. The most widely discussed method of triggering new intuitions is role taking (Selman, 1971). Simply by putting oneself into the shoes of another person one may instantly feel pain, sympathy, or other vicarious emotional responses. This is one of the principle pathways of moral reflection according to Piaget, Kohlberg, and other cognitive developmentalists. A person comes to see an issue or dilemma from more than one side and thereby experiences multiple competing intuitions. The final judgment may be determined either by going with the strongest intuition, or by allowing reason to choose among the alternatives based on the conscious application of a rule or principle. This pathway amounts to having an inner dialogue with oneself (Tappan, 1997), obviating the need for a discourse partner. Rationalist models focus on links 5 and 6. The social intuitionist model, in contrast, says that moral judgment consists primarily of links 1 through 4, although it allows that links 5 and 6 may sometimes contribute (such as during a formal moral judgment interview). The next section of this paper reviews four problems for rationalist models. For each problem, a social intuitionist re-interpretation of the evidence is offered, relying primarily on links 1 through 4. Four Reasons to Doubt the Causal Importance of Reason 1) The Dual Process Problem: There Is a Ubiquitous and Understudied Intuitive Process at Work It is now widely accepted in social and cognitive psychology that there are often two processing systems at work when a person makes judgments or solves problems (see Table 1, and the edited volume by Chaiken & Trope, 1999). Because these two systems typically run in parallel and are capable of reaching differing conclusions, these models are usually called dual process models. Dual process models have thus far had little impact on moral judgment research, as most researchers have focused their efforts on understanding the reasoning process (but see Gibbs, 1991; and Eisenberg, Shea, Carlo, & Knight, 1991). Yet there is evidence that moral judgment works like other kinds of judgment, in which most of the action is in the intuitive process. Automatic evaluation. Affective evaluation occurs so quickly, automatically, and pervasively that it is generally thought to be an integral part of perception. Zajonc (1980) synthesized findings from a variety of fields to create a modern version of Wundts (1907) affective primacy theory, in which he argued that feeling and thought are to some extent separate systems with separate biological bases. The affective system has primacy in every sense: it came first in phylogeny, it emerges first in ontogeny, it is triggered more quickly in real-time judgments, and it is more powerful and irrevocable when the two systems yield conflicting judgments (see also Reber, 1993). Research on the automatic evaluation effect confirms that very brief or even subliminal presentations of affectively valenced words (Bargh, Chaiken, Raymond, & Hymes, 1996; Fazio, Sanbonmatsu, Powell, & Kardes, 1986), facial expressions (Murphy & Zajonc, 1993) and photographs of people and animals (Hermans, de Houwer, & Eelen, 1994) alter the time it takes to evaluate a target object presented immediately afterwards, indicating that affective processing is at work within a quarter second of stimulus presentation. Automatic moral judgment. Moral judgments typically involve more complex social stimuli than the simple words and visual objects used in automatic evaluation studies. Could moral judgments be made automatically as well? The emerging view in social cognition is that most of our behaviors and judgments are in fact made automatically (i.e., without intention, effort, or awareness of process; Bargh, 1994; Bargh & Chartrand, 1999; Greenwald & Banaji, 1995). The literature most relevant to moral judgment is the literature on attitudes, where a central question has been: how do people form attitudes about other people? The evidence indicates that attitude formation is better described as a set of automatic processes than as a process of deliberation and reflection about the traits of a person. People form first impressions at first sight (Albright, Kenny, & Malloy, 1988), and the impressions that they form from observing a thin slice of behavior (as little as 5 seconds) are almost identical to the impressions they form from much longer and more leisurely observation and deliberation (Ambady & Rosenthal, 1992). These first impressions alter subsequent

Morality and Intuition 9 evaluations, creating a halo effect (Thorndike, 1920), in which positive evaluations of non-moral traits such as attractiveness lead to beliefs that a person possesses corresponding moral traits such as kindness and good character (Dion, Berscheid, & Walster, 1972). People also categorize other people instantly and automatically, applying stereotypes that often include morally evaluated traits (e.g., aggressiveness for African Americans, Devine, 1989). All of these findings illustrate the operation of the intuitive judgment link (link 1 in Figure 2), in which the perception of a person or an event leads instantly and automatically to a moral judgment without any conscious reflection or reasoning. Another illustration of automatic moral judgment can be seen in the literature on persuasion. Moral discourse in its natural setting is often a kind of persuasion, in which one person tells others about some event, and tries to recruit them to her reading of the event. According to Chaikens (1987) heuristic-systematic model of persuasion, people are guided in part by the "principle of least effort." Since people have limited cognitive resources, and since heuristic processing is easy and adequate for most tasks, heuristic processing (the intuitive process) is generally used unless there is a special need to engage in systematic processing (see also Simon, 1967). A particularly important heuristic for the study of moral judgment is the "I agree with people I like" heuristic (Chaiken, 1980). If your friend is telling you how Robert mistreated her, there is little need for you to think systematically about the good reasons Robert might have had. The mere fact that your friend has made a judgment affects your own intuitions directly, illustrating the social persuasion link (link 4). It is only if the agreement heuristic leads to other conflicts (e.g., if Robert is a friend of yours) that your sufficiency threshold will be raised above your actual level of confidence, triggering effortful systematic processing (links 5 and 6) to close the gap. However the social intuitionist model posits that moral reasoning is usually done interpersonally rather than privately. If Robert is in fact a friend of yours, then you and your friend might present arguments to each other (link 3, the reasoned persuasion link) in the hope of triggering new intuitions, getting the other to see Roberts actions in a better or worse light. Moral discussions can then be modeled as a repeated cycle through links 1, 2, and 3 in person A, then in person B, then in person A, etc. Link 4 would exert a constant pressure towards agreement if the two parties are friends, and a constant pressure against agreement if the two parties dislike each other. If at least one of the parties begins without a strong initial intuition then some degree of convergence is likely. However if both parties begin with strongly felt opposing intuitions (as in a debate over abortion), then reasoned persuasion is likely to have little effect, except that the post-hoc reasoning triggered in the other person could lead to even greater disagreement, a process labeled attitude polarization by Lord, Ross, and Lepper (1979). Petty and Cacioppos (1986) elaborationlikelihood model gives a similar reading of the standard moral judgment discussion. If you feel a strong identification with the source of the persuasive message (your friend) and you have no conflict motivating elaborated thinking, then a peripheral process is sufficient to lead to an attitude shift, a judgment that Robert is evil. However if the person talking to you is a stranger (a research psychologist) who challenges your judgment at every turn (what if Heinz didnt love his wife, should he still steal the drug?), then you will be forced to engage in extensive effortful, verbal, central processing. Standard moral judgment interviews may therefore create an unnaturally reasoned form of moral judgment, leading to the erroneous conclusion that moral judgment is primarily a reasoning process. And since forcing people to introspect to find reasons for their attitudes can change those attitudes temporarily (Wilson & J. Schooler, 1991; Wilson, Lindsey, & T. Schooler, 2000), standard moral judgment interviews might not even provide a valid measure of peoples real moral beliefs. (See also Schooler, Fiore, & Brandimonte, 1997, on the impairments caused by forcing people to verbalize what they know intuitively.) The social intuitionist solution. The social intuitionist model is fully compatible with modern dual process theories. Like those theories, the model posits that the intuitive process is the default process, handling everyday moral judgments in a rapid, easy, and holistic way. It is primarily when intuitions conflict, or when the social situation

Morality and Intuition 10 demands thorough examination of all facets of a scenario that the reasoning process is called upon. Reasoning can occur privately (links 5 and 6), and such solitary moral reasoning may be common among philosophers and among those high on need for cognition (Cacioppo & Petty, 1982). Yet ever since Plato wrote his Dialogues, philosophers have recognized that moral reasoning naturally occurs in a social setting, between people who can challenge each others arguments and trigger new intuitions (links 3 and 4). The social intuitionist model avoids the traditional focus on conscious private reasoning, and draws attention to the role of moral intuitions, and of other people, in shaping moral judgments. 2) The Motivated Reasoning Problem: The Reasoning Process Is More like a Lawyer Defending a Client than a Judge or Scientist Seeking Truth It appears, then, that a dual process model may be appropriate for a theory of moral judgment. If so, then the relationship between the two processes must be specified. Is the reasoning process the smarter but more cognitively expensive process, called in whenever the intuitive process is unable to solve a problem cheaply? Or is the relationship one of master and servant, as Hume suggested, in which reasons main job is to formulate arguments that support ones intuitive conclusions? Research on motivated reasoning and on everyday reasoning both suggest that the post-hoc reasoning link (link 2) is more important than the reasoned judgment and private reflection links (5 and 6). There are two major classes of motives that have been shown to bias and direct reasoning. The first class can be called relatedness motives, for it includes concerns about impression management and smooth interaction with other people. The second class can be called coherence motives, for it includes a variety of defensive mechanisms triggered by cognitive dissonance and threats to the validity of ones cultural worldview. Relatedness motives. From an evolutionary perspective it would be strange if our moral judgment machinery was designed principally for accuracy, with no concern for the disastrous effects of periodically siding with our enemies and against our friends. Studies of attitudes, person perception, and persuasion show that desires for harmony and agreement do indeed have strong biasing effects on judgments. Chaiken and her colleagues incorporated impression motivation into the heuristicsystematic model, which is described as the desire to hold attitudes and beliefs that will satisfy current social goals (Chen & Chaiken, 1999, p. 78). Chen, Shechter, and Chaiken (1996) found that people who expected to discuss an issue with a partner whose views were known expressed initial attitudes, before the interaction, that were shifted towards those of their anticipated partner. More broadly, Darley and Berscheid (1967) found that people rate a description of a persons personality as more likeable if they expect to interact with the person than if they do not expect to interact. The existence of motivations to agree with our friends and allies means that we can be directly affected by their judgments (the social persuasion link). The mere fact that your friend expresses a moral judgment against X is often sufficient to cause in you a critical attitude towards X. Such direct influence, circumventing reasoning entirely, fits with Chartrand and Barghs (1999) recent demonstration of the chameleon effect, in which people unconsciously mimic the postures, mannerisms, and facial expressions of their interaction partners. Chartrand and Bargh found that such automatic mimicry is socially adaptive, for people who are in sync with another person are liked better by that person. Coherence motives. Psychologists since Freud have argued that people construct views of themselves and of the world, and that they experience potentially crippling anxiety when these constructions are threatened (Moskowitz, Skurnik, & Galinsky, 1999). Research on cognitive dissonance (Festinger, 1957; Wicklund & Brehm, 1976) showed just how readily people change their thinking and beliefs to avoid the threat of internal contradictions. More recently, Chaiken, GinerSorolla, and Chen (1996, p.557) defined defense motivation as the desire to hold attitudes and beliefs that are congruent with existing self-definitional attitudes and beliefs. Self-definitional attitudes include values and moral commitments. When defense motivation is triggered, both heuristic and systematic thinking work to preserve self-definitional attitudes. The biasing effects of defense motivation can be seen in studies that challenge participants moral and

Morality and Intuition 11 political ideology. Lord, Ross, and Lepper (1979) found that students with strong opinions about the death penalty, when exposed to research evidence on both sides of the issue, accepted evidence supporting their prior belief uncritically, while subjecting opposing evidence to much greater scrutiny. Lerners (1965) just world hypothesis stated that people have a need to believe that they live in a world where people generally get what they deserve. People who suffer for no reason are a threat to this belief, so participants adjusted their moral judgments, derogating or blaming innocent victims (Lerner & Miller, 1978). Tetlock et al. (2000) found that peoples willingness to use relevant baserate information, or to engage in counterfactual thinking, depended on whether or not their sacred values were threatened by doing so. In all of these examples, reasoning is used to defend prior moral commitments. Moral judgments are also affected by the defensive motivations of terror management (Solomon, Greenberg, & Pyszczynski, 1991). When people are asked to think about their own deaths, they appear to suppress a generalized fear of mortality by clinging more tightly to their cultural world view. Death-primed participants then shift their moral judgments to defend that world view. They mete out harsher punishment to violators of cultural values, and they give bigger rewards to people who behaved morally (Rosenblatt et al., 1989). Death-primed participants have more negative attitudes towards those who do not fully share their world view (e.g., Jews; Greenberg et al., 1990). From a terror management perspective, moral judgment is a special kind of judgment, since moral judgments always implicate the cultural world view. It is plausible to say I dont like asparagus, but I dont care if you eat it. It is not plausible to say I think human life is sacred, but I dont care if you kill him. Mechanisms of bias. Studies of everyday reasoning reveal the mechanisms by which relatedness and coherence motivations make people act like lawyers. Kuhn (1991) found that most people have difficulty understanding what evidence is, and when pressed to give evidence in support of their theories they generally give anecdotes or illustrative examples instead. Furthermore, people show a strong tendency to search for anecdotes and other evidence exclusively on their preferred side of an issue, a pattern that has been called the myside bias (Baron, 1995; Perkins, Farady, & Bushey, 1991). Once people find supporting evidence, even a single piece of bad evidence, they often stop the search, since they have a makessense epistemology (Perkins, Allen, & Hafner, 1983) in which the goal of thinking is not to reach the most accurate conclusion; it is to find the first conclusion that hangs together well and that fits with ones important prior beliefs. Research in social cognition also indicates that people often behave like intuitive lawyers rather than intuitive scientists (Baumeister & Newman, 1994). Kundas (1990) review of motivated reasoning concludes that directional goals (motivations to reach a pre-ordained conclusion) work primarily by causing a biased search in memory for supporting evidence only. However Pyszczynski and Greenberg (1987) proposed a more comprehensive biased hypothesis testing model, in which self-serving motives bias each stage of the hypothesis testing sequence, including the selection of initial hypotheses, the generation of inferences, the search for evidence, the evaluation of evidence, and the amount of evidence needed before one is willing to make an inference. Research on the confirmatory bias (Snyder & Swan, 1978) shows that people do not always seek to confirm their initial hypothesis; sometimes they ask the right questions to get at the truth (Higgins & Bargh, 1987; Trope & Bassok, 1982). However such demonstrations of truth-seeking always involve hypotheses that the participant has no need to defend (e.g., the person you are about to meet is an extrovert). When hypotheses involve ones moral commitments (e.g., the death penalty does not deter murder), the empirical findings generally show bias and motivated reasoning (Kuhn, 1989; Lord, Ross, & Lepper, 1979). This review is not intended to imply that people are stupid or irrational. It is intended to demonstrate that the roots of human intelligence, rationality, and ethical sophistication should not be sought in our ability to search for and evaluate evidence in an open and unbiased way. Rather than following the ancient Greeks in worshiping reason, we should instead look for the roots of human intelligence, rationality, and virtue in what the mind does best: perception,

Morality and Intuition 12 intuition, and other mental operations that are quick, effortless, and generally quite accurate (Gigerenzer & Goldstein, 1996; Margolis, 1987). The social intuitionist solution. The reasoning process in moral judgment may be capable of working objectively under very limited circumstances: when the person has adequate time and processing capacity, a motivation to be accurate, no a priori judgment to defend or justify, and when no relatedness or coherence motivations are triggered (Forgas, 1995; Wegner & Bargh, 1998). Such circumstances may be found in moral judgment studies using hypothetical and unemotional dilemmas. Rationalist research methods may therefore create an unusual and non-representative kind of moral judgment. But in real judgment situations, such as when people are gossiping or arguing, relatedness motives are always at work. If more shocking or threatening issues are being judged, such as abortion, euthenasia, or consensual incest, then coherence motives will be at work too. Under these more realistic circumstances moral reasoning is not left free to search for truth, but is likely to be hired out like a lawyer by various motives, employed only to seek confirmation of preordained conclusions. 3) The Post-hoc Problem: the Reasoning Process Readily Constructs Justifications of Intuitive Judgments, Causing the Illusion of Objective Reasoning When people are asked to explain the causes of their judgments and actions they frequently cite factors that could not have mattered, and they fail to recognize factors that did matter. Nisbett and Schachter (1966), for example, asked participants to take electric shocks, either with or without a placebo pill that was said to produce the same symptoms as electric shock. Participants in the pill condition apparently attributed their heart palpitations and butterflies in the stomach to the pill, and were able to take four times as much shock as those who had no such misattribution available for their symptoms. Yet when the placebo condition participants were asked if they had made such an attribution, only 25% of them said that they had. The remaining participants denied that they had thought about the pill and instead made up a variety of explanations for their greater shock tolerance, e.g., Well, I used to build radios and stuff when I was 13 or 14, and maybe I got used to electric shock (Nisbett & Wilson, 1977, p.237). Nisbett and Wilson (1977) interpreted such causal explanations as post-hoc constructions. When asked to explain their behaviors, people engage in an effortful search that may feel like a kind of introspection. But what people are searching for is not a memory of the actual cognitive processes that caused their behaviors, for these processes are not accessible to consciousness. Rather, people are searching for plausible theories about why they might have done what they did. People turn first to a pool of culturally supplied explanations for behavior, which Nisbett and Wilson (1977, p.248) refer to as a priori causal theories. When asked why he enjoyed a party, a person turns first to his cultural knowledge about why people enjoy parties, chooses a reason, and then searches for evidence that the reason was applicable. The search is likely to be a one-sided search of memory for supporting evidence only (Kunda, 1990; Pyszczynski & Greenberg, 1987). Additional illustrations of post-hoc causal reasoning can be found in studies in which hypnosis (Zimbardo, LaBerge, & Butler, 1993) and subliminal presentation (Kunst-Wilson & Zajonc, 1980) were used to make people perform actions. When asked to explain their actions or choices, people readily made up reasons that sounded plausible but were false. Split brain patients show this effect in its most dramatic form. When the left hand, guided by the right brain, performs an action, the verbal centers in the left brain readily make up stories to explain it (Gazzaniga, Bogen, & Sperry, 1962). The language centers are so skilled at making up post-hoc causal explanations that Gazzaniga (1985) speaks of an interpreter module. He argues that behavior is usually produced by mental modules to which consciousness has no access, but the interpreter module provides a running commentary anyway, constantly generating hypotheses to explain why the self might have performed any particular behavior. Post-hoc moral reasoning. The idea that people generate causal explanations out of a priori causal theories is easily extended into the moral domain. In a moral judgment interview a participant is asked to decide if an action is right or wrong, and is then

Morality and Intuition 13 asked to explain why she thinks so. But if people have no access to the processes behind their automatic initial evaluations, then how do they go about providing justifications? By consulting their a priori moral theories. A priori moral theories can be defined as a pool of culturally supplied norms for evaluating and criticizing the behavior of others. A priori moral theories provide acceptable reasons for praise and blame (e.g., unprovoked harm is bad; people should strive to live up to Gods commandments). Since the justifications that people give are closely related to the moral judgments that they make, prior researchers have assumed that the justificatory reasons caused the judgments. But if people lack access to their automatic judgment processes then the reverse causal path becomes more plausible. If this reverse path is common, then the enormous literature on moral reasoning can be reinterpreted as a kind of ethnography of the a priori moral theories held by various communities3 and age groups. Kohlbergs (1969) studies demonstrate that young children in many cultures hold the a priori moral theory that acts that get punished are wrong; acts that get rewarded are good (stages 1 and 2), but they soon advance to the theory that acts that others approve of are good; acts that others condemn are bad (stage 3). If such statements were the rules that children really used to evaluate actions, then children at stages 1 and 2 would conclude that actions that are not punished must not be bad. Yet Turiel (1983) has shown that young children do not believe this. They say that harmful acts, such as hitting and pulling hair, are wrong whether they are punished or not. They even say that such acts would be wrong if adults ordered them to be done (Damon, 1977; Laupa & Turiel, 1986). So when a child offers the stage 1 statement that its wrong because shell get punished, the child is not introspecting on the reasoning that led to his condemnation; he is just giving a reason that sounds plausible, perhaps a reason he himself has heard from adults (if you do that, I will punish you). The illusions of moral judgment. If moral reasoning is generally a post-hoc construction intended to justify automatic moral intuitions, then our moral life is plagued by two illusions. The first illusion can be called the wag-the-dog illusion: we believe that our own moral judgment (the dog) is driven by our own moral reasoning (the tail). The second illusion can be called the wag-the-otherdogs-tail illusion: in a moral argument, we expect the successful rebuttal of an opponents arguments to change the opponents mind. Such a belief is like thinking that forcing a dogs tail to wag by moving it with your hand should make the dog happy. The wag-the-dog illusion follows directly from the mechanics of the reasoning process described above. Pyszczynski and Greenberg (1987) point out that by going through all the steps of hypothesis testing, even though every step can be biased by selfserving motivations, people can maintain an illusion of objectivity about the way they think. The wag-the-dog illusion may therefore be one of the mechanisms underlying naive realism (Griffin & Ross, 1991; Robinson, Keltner, Ward, & Ross, 1995), the finding that people think that they see the world as it is, while their opponents in a moral dispute are biased by ideology and self-interest. The bitterness, futility, and self-righteousness of most moral arguments can now be explicated. In a debate on abortion, politics, consensual incest, or on what my friend did to your friend, both sides believe that their positions are based on reasoning about the facts and issues involved (the wag-the-dog illusion). Both sides present what they take to be excellent arguments in support of their positions. Both sides expect the other side to be responsive to such reasons (the wag-the-other-dogs-tail illusion). When the other side fails to be affected by such good reasons, each side concludes that the other side must be closed-minded or insincere. In this way the culture wars over issues such as homosexuality and abortion can generate morally motivated players on both sides who believe that their opponents are not morally motivated (Haidt & Hersh, in press; Robinson, Keltner, Ward, & Ross, 1995). The social intuitionist solution. People have quick and automatic moral intuitions, and when called upon to justify these intuitions they generate post-hoc justifications out of a priori moral theories. They do not realize that they are doing this, so they fall prey to two illusions. Moral arguments are therefore like shadow boxing matches: each contestant lands heavy blows to the opponents shadow, then wonders why he doesnt fall down. Moral reasoning may therefore have little persuasive

Morality and Intuition 14 power in conflict situations, but the social intuitionist model says that moral reasoning can be effective in influencing people before a conflict arises. Our words and ideas do affect our friends, allies, and even strangers, via the reasonedpersuasion link. If one can get the other person to see the issue in a new way, perhaps by re-framing a problem to trigger new intuitions, then one can influence others with ones words. Martin Luther King Jrs I Have a Dream speech was remarkably effective in this task, using metaphors and visual images more than propositional logic to get White Americans to see and thus feel that racial segregation was unjust, and un-American (see Lakoff, 1996 on the role of metaphor in political persuasion). 4) the Action Problem: Moral Action Covaries with Moral Emotion More than with Moral Reasoning The analysis thus far has focused on moral judgment, not moral behavior. But the debate between rationalism and intuitionism can also be carried out using moral action as the dependent variable. There is a literature that directly examines the relationship between moral reasoning and moral action, and there is a literature that examines what happens when moral reasoning and moral emotions become dissociated (in the case of psychopaths). The weak link between moral reasoning and moral action. In a major review of the literature on moral cognition and action, Blasi (1980) concluded that moral reasoning and moral action are statistically related (p.37). But what is the nature of this relationship? Blasi was careful to state that the connection between moral reasoning ability and moral behavior is only a correlation, but later authors in the cognitive developmental tradition read the relationship as causal, stating that higher levels of moral reasoning cause better moral behavior (e.g., Lapsley, 1996). Yet Blasis review raised the possibility that a third variable caused both better reasoning and better behavior: intelligence. Blasi found that IQ was consistently related to honesty, and he concluded that future investigators must do a better job of controlling for IQ. Kohlberg (1969) reported that scores on his moral judgment interviews correlated with measures of IQ in the .30 to .50 range. Rest (1979) reports correlations of .20 to .50 between IQ and his Defining Issues Test (DIT). Intelligence may also be related to better moral behavior by a pathway that does not run through better moral reasoning. Metcalfe and Mischel (1999) proposed a dual process model of willpower in which two separate but interacting systems govern human behavior in the face of temptation. The hot system is specialized for quick emotional processing, and it makes heavy use of amygdala-based memory. The cool system is specialized for complex spatiotemporal and episodic representation and thought. It relies on hippocampal memory and frontal lobe planning and inhibition areas. It can block the impulses of the hot system, but it develops later in life, making childhood and adolescence seem like a long struggle to overcome impulsiveness and gain self-control. This theory was proposed in part to explain the astonishing finding that the number of seconds pre-schoolers were able to delay choosing an immediate small reward (one marshmallow) in favor of a later, bigger reward (two marshmallows) was a powerful predictor of adolescent social and cognitive competence, including SAT scores and ability to exert self control in frustrating situations, measured about 13 years later (Shoda, Mischel, & Peake, 1990). The correlation that Blasi found between moral reasoning and moral behavior may therefore be explained by a third variable, the strength of the cool system. Children start off with limited ability to resist temptation, but as the hippocampus and frontal cortex finish their development, children become more able to inhibit impulsive behaviors. Some children start off with a more effective cool system (Kochanska et al., 1996), due to better or faster frontal cortex development. Frontal cortex development makes these children smarter, and they therefore perform better on measures of moral reasoning, but their improved moral behavior comes more from their greater self-regulatory abilities than from their greater moral reasoning abilities. The development of the cool system does not represent the triumph of reasoning over emotion; rather, Metcalfe and Mischel (1999, p.16) see the successful development and integration of the cool system as an essential feature of emotional intelligence.

Morality and Intuition 15 This re-interpretation is supported by the fact that moral reasoning ability, in Blasis (1980) review, was most predictive of negative morality -refraining from delinquent behavior. Criminologists have consistently found an inverse relationship between criminality and IQ. Even after correcting for SES, the difference between delinquent and nondelinquent adolescent populations is approximately eight IQ points (Hirschi & Hindelang, 1977). However the story for positive morality -- directly helping others -- is less clear. Blasi found some support for the claim that high scorers on Kohlbergs and Rests scales were more likely to help other people, but more recent studies have raised doubts. Hart and Fegley (1995), and Colby and Damon (1992) both compared highly pro-social moral exemplars to non-exemplars, and found that the groups did not differ in their moral reasoning ability assessed by Kohlbergian techniques. A recent review of evidence supporting the utility of the Defining Issues Test described three studies that showed a relationship between DIT scores and negative moral behaviors, but none showing a relationship between DIT scores and positive morality (Thoma, Narvaez, Rest, & Derryberry, 1999). The relationship between moral reasoning ability and moral behavior therefore appears to be weak and inconsistent, once intelligence is partialled out. Emotional and self-regulatory factors seem to be more powerful determinants of actual behavior (Mischel & Mischel, 1976). The strong link between moral emotions and moral action. Further evidence that moral reasoning matters less than moral emotions comes from the study of psychopaths. Cleckleys (1955) case studies present chilling portraits of people in whom reasoning has become dissociated from moral emotions. Cleckley characterizes psychopaths as having good intelligence and a lack of delusions or irrational thinking. Psychopaths know the rules of social behavior and they understand the harmful consequences of their actions for others. They simply do not care about those consequences. Cleckleys psychopaths show a general poverty of major affective reactions, particularly those that would be triggered by the suffering of others (remorse, sympathy), condemnation by others (shame, embarrassment), or attachment to others (love, grief). (See Hare, 1993, for a more recent discussion of the emotional deficit.) Psychopaths can steal from their friends, dismember live animals, and even murder their parents to collect insurance benefits, without showing any trace of remorse, or of shame when caught. The very existence of the psychopath illustrates Humes statement that tis not contrary to reason to prefer the destruction of the whole world to the scratching of my little finger (1739/1969, p.461). It is not contrary to reason to kill your parents for money, unless it is also contrary to sentiment. Several lines of research are converging on the conclusion that psychopaths and people with antisocial personality disorder differ from normal people in the operation of the frontal cortex. Mednick et al. (1982) reviewed studies of EEG differences between criminals and non-criminals, and concluded that the bulk of the research points to differences in the anterior regions of the brain. More recent studies using PET techniques have narrowed the location of interest to the pre-frontal cortex (Raine, 1997). Samples of aggressive offenders show reduced metabolic activity in this area, relative to controls (Raine et al., 1994). The importance of the prefrontal cortex for moral behavior has been most fully explored by Damasio and his colleagues, who have found a consistent pattern of changes associated with damage to the ventro-medial area of the pre-frontal cortex (VMPFC, the area behind the bridge of the nose). Patients with damage restricted to the VMPFC show no reduction in their reasoning abilities. They retain full knowledge of moral rules and social conventions, and they show normal abilities to solve logic problems, financial problems, and even hypothetical moral dilemmas (Damasio, 1994). Yet when faced with real decisions, they perform disastrously, showing poor judgment, indecisiveness, and what appears to be irrational behavior. Damasio and his colleagues have demonstrated that the central deficit resulting from destruction of the VMPFC is the loss of emotional responsiveness to the world in general, and to one's behavioral choices in particular. When shown pictures that arouse strong skin conductance responses in normal people (nudity, mutilation, people dying) these patients show no response (Damasio, Tranel, & Damasio, 1990), mirroring the lack of autonomic

Morality and Intuition 16 responsiveness of psychopaths (Hare & Quinn, 1971). The patients know that the images should affect them, but they report feeling nothing. Damasio refers to this pattern of affect loss combined with intact reasoning as acquired sociopathy. Patients with acquired sociopathy do not generally become moral monsters, perhaps since they have a lifetime of normal emotional learning and habit formation behind them, although they do become much less concerned with following social norms, and they sometimes show outrageous and antisocial behavior, as in the case of Phineas Gage (Damasio, 1994). If we imagine a child growing up without a normal VMPFC, who never in his life felt the stings of shame and embarrassment, or the pain of emotional loss or empathic distress, then it becomes almost possible to understand the otherwise incomprehensible behavior of Cleckleys psychopaths. With no moral sentiments to motivate and constrain them, they simply do not care about the pain they cause and the lives they ruin4 . Emotions lead to altruism. If reasoning ability is not sufficient to motivate moral action, then what is? Batson and his colleagues have developed the empathy-altruism hypothesis, which states that empathy aroused by the perception of someones suffering evokes an altruistic motivation directed toward the ultimate goal of reducing the suffering (Batson, 1987; see also Hoffman, 1982). Batson, O'Quinn, Fulty, Vanderplass, and Isen (1983) found that participants who experienced empathy while watching a woman receiving (fake) electric shocks generally volunteered to take the shocks in her place, even when they were given the option of leaving the scene. Participants who experienced only nonempathic personal distress about the womans plight volunteered to trade places with her only when they thought they would have to continue watching the woman receive the shocks. Participants in the first group seemed to be genuinely motivated to help the distressed woman, not to relieve their own distress. Cialdini and his colleagues have challenged the empathy-altruism hypothesis, using a variety of experimental designs to show that other motives can often explain seemingly altruistic behavior (Cialdini et al., 1987). But throughout this long debate, both sides have consistently agreed that people are often motivated to help others, and that the mechanisms involved in this helping are primarily affective, including empathy as well as reflexive distress, sadness, guilt and shame (Cialdini, 1991). The social intuitionist solution. It is easier to study verbal reasoning than it is to study emotions and intuitions, but reasoning may be the tail wagged by the dog. The dog itself may turn out to be moral intuitions and emotions such as empathy and love (for positive morality) and shame, guilt, and remorse, along with emotional self-regulation abilities (for negative morality; see Haidt, in press, for a review and taxonomy of the moral emotions). A dogs tail is worth studying because dogs use their tails so frequently for communication. Similarly, moral reasoning is worth studying because people use moral reasoning so frequently for communication. But to really understand how human morality works it may be advisable to shift attention away from the study of moral reasoning and towards the study of intuitive and emotional processes. This concludes the presentation of the four reasons to doubt the importance of reason, and to look more closely at an alternative model, such as the social intuitionist model. The Mechanism of Intuition Because intuition is the heart of the social intuitionist model, more must be said about exactly how intuitive moral judgments are made (link 1). Recent work on the importance of bodily experience, as represented in the mind, makes such an account possible. Gut Feelings in the Mind The somatic marker hypothesis (Damasio, 1994) states that experiences in the world normally trigger emotional experiences that involve bodily changes and feelings. Once the brain is properly tuned up by repeated experiences of such emotional conditioning (e.g., Pavlov, 1927), the brain areas that monitor these bodily changes begin to respond whenever a similar situation arises. It is then no longer necessary for the rest of the body to be involved. At that point, the mere thought of a particular action becomes sufficient to trigger an as if response in the brain, in which the person experiences in a weaker form the same bodily feelings that she would experience if she performed the action. The critical job of the VMPFC is to integrate these feelings, or somatic markers,

Morality and Intuition 17 with the persons other knowledge and planning functions, in order to decide quickly upon a response. Damasios work fits well with research in social psychology on the affect as information hypothesis, which demonstrates that people frequently rely on their moods and momentary flashes of feeling as guides when making judgments and decisions (Clore, Schwartz, & Conway, 1994; Loewenstein, Weber, Hsee, & Welch, 1999; Schwarz & Clore, 1983; see also an fMRI finding that such flashes help explain peoples varying responses to philosophical moral dilemmas in Greene, Sommerville, Nystrom, Darley, & Cohen, 2000). Two recent studies have directly manipulated moral judgments by manipulating somatic markers. Batson, Engel, and Fridell (1999) used false physiological feedback to tell participants about their emotional reactions when listening to stories in which the values of either freedom or equality were threatened. When later asked to choose which value should be selected as a theme for a week-long program of events at their university, participants were more likely to choose the value for which they thought they had shown a stronger visceral reaction. Wheatley and Haidt (2000) manipulated somatic markers even more directly. Highly hypnotizable participants were given the suggestion, under hypnosis, that they would feel a pang of disgust when they saw either the word take or the word often. Participants were then asked to read and make moral judgments about six stories that were designed to elicit mild to moderate disgust, each of which contained either the word take or the word often. Participants made higher ratings of both disgust and moral condemnation about the stories containing their hypnotic disgust word. This study was designed to directly manipulate the intuitive judgment link (link 1), and it demonstrates that artificially increasing the strength of a gut feeling increases the strength of the resulting moral judgment. Metaphor and Embodiment While Damasio focuses on the role of the autonomic nervous system in thinking, Lakoff and Johnson (1999; Lakoff, 1987) have shown how the entire range of physical and emotional experience may underlie our embodied cognition. By analyzing how people think and talk about love, politics, morality, and other issues they have shown that nearly all complex thought relies on metaphors, drawn mostly from our experience as physical creatures. For example, because we all have experience with foods that are easily contaminated, we come to equate purity and cleanliness with goodness in the physical domain. We learn from experience that pure substances are quickly contaminated (e.g., by mold, dust, insects) when not guarded, and that once contaminated, it is often difficult to purify them again. These experiences in the physical world then form the basis (in many cultures) of conceptual schemes about moral purity, e.g., that children start off in a state of purity and innocence, but that they can be corrupted by a single exposure to sex, violence, drugs, homosexuality, or the devil (Haidt, Rozin, McCauley, & Imada, 1997; Rozin, Haidt, & McCauley, 2000). Some losses of purity can be rectified with great difficulty (e.g., exorcism after exposure to the devil), while others cannot be rectified at all (e.g., the loss of virginity). Moral intuition, then, appears to be the automatic output of an underlying, largely unconscious set of interlinked moral concepts. These concepts may have some innate basis (see below), which is then built up largely by metaphorical extensions from physical experience. Metaphors have entailments, and much of moral argument and persuasion involves trying to get the other person to apply the right metaphor. If Saddam Hussein is like Hitler, it follows that he must be stopped. But if Iraq is like Viet Nam, it follows that the United States should not get involved (Spellman & Holyoak, 1992). Such arguments are indeed a form of reasoning, but they are reasons designed to trigger intuitions in the listener. The Origin of Intuitions Perhaps because moral norms vary by culture, class, and historical era, psychologists have generally assumed that morality is learned in childhood, and they have set out to discover how morality gets from outside the child to inside. The social intuitionist model takes a different view. It proposes that morality, like language, is a major evolutionary adaptation for an intensely social species, built into multiple regions of the brain and

Morality and Intuition 18 body, which is better described as emergent than as learned, yet which requires input and shaping from a particular culture. Moral intuitions are therefore both innate and enculturated. The present section describes the ways in which moral intuitions are innate, while the next section describes the ways in which they are shaped by culture during development. Primate Proto-Morality Darwin (1874/1998) believed that the human moral sense grew out of the social instincts of other animals, and modern primatological research supports him. While all species can be said to follow descriptive rules for behavior with conspecifics, it is primarily the primates that show signs of prescriptive rules, which de Waal (1991) defines as rules that individuals have learned to respect because of active reinforcement by others (p. 338). Chimpanzee groups develop and enforce norms for mating, for playing with or touching infants, and for many other forms of interaction. When one individual violates these norms, others will sometimes look to or even get the attention of the individual whose interests have been violated, who may then take action to punish the transgressor (de Waal, 1991). De Waals work indicates that prescriptive behavioral norms can emerge and be understood and enforced by chimpanzees, without the benefit of language or language-based reasoning. Language may greatly increase the human use of norms, but the cognitive and emotional machinery of norm creation and norm enforcement was available long before language existed. It appears, furthermore, that this machinery has been carried forward into the human mind. Alan Fiske (1991, 1992) has identified four underlying models of social cognition that seem to be at work in all human cultures. His first three models fit closely with descriptions of other primates. Fiskes first model, communal sharing involves the linkage of kindness, kinship, and empathic concern for close others that de Waal describes both for chimpanzees (de Waal, 1996) and for bonobos (de Waal & Lanting, 1997). Fiskes second model, authority ranking, describes the ways that power and rank regulate access to resources, but also obligate superiors to protect their subordinates. Such mutual obligations are clear among chimpanzees (de Waal, 1982; Goodall, 1986). Fiskes third model, equality matching, involves the double-edged reciprocal altruism first described by Trivers (1971). Most apes and many monkeys seem remarkably adept at remembering and repaying both favors and slights (de Waal, 1982; 1996). It is only Fiskes fourth model, market pricing, in which ratio values of goods and services must be computed and aggregated across transactions, which seems to be uniquely human (Haslam, 1997). Given so many close parallels between the social lives of humans and chimpanzees, the burden of proof must fall on those who want to argue for discontinuity, i.e., that human morality arose ex nihilo when we developed the ability to speak and reason. The above considerations are not meant to imply that chimpanzees have morality, nor that humans are just chimps with post-hoc reasoning skills. There is indeed a moral Rubicon that only Homo Sapiens appears to have crossed: widespread third party norm enforcement. Chimpanzee norms generally work at the level of private relationships, where the individual that has been harmed is the one that takes punitive action. Yet human societies are marked by a constant and vigorous discussion of norms and norm violators, and by a willingness to expend individual or community resources to inflict punishment, even by those who were not harmed by the violator (Boehm, 1999). Dunbar (1996) has even proposed that language evolved primarily to fulfill the need for gossip. Only with language is it possible to keep track of who did what to whom, who is in, who is out, who can be trusted, and who should be avoided. While the evolution of language and intelligence may have been driven by the Machiavellian benefits they gave to individuals (Byrne & Whiten, 1988), the combination of language and a full theory of mind (Premack & Premack, 1995) made it possible for large groups of non-kin to reap the benefits of cooperation by monitoring each others behavior (via gossip), shunning or punishing cheaters, and rewarding team players. The social intuitionist model fits with this view of the functions of language by including two interpersonal links. Once morality is located in a groups efforts to solve cooperation and commitment problems (Darwin, 1874/1998; Frank, 1988), it becomes clear that individuals must simultaneously use language to influence others, while being at least

Morality and Intuition 19 somewhat open to interpersonal influence as specific norms, values, or judgments spread through a community. A group of judges independently seeking truth is unlikely to reach an effective consensus. But a group of people linked together in a large web of mutual influence (an extension of Figure 2 to multiple parties) may eventually settle into a stable configuration, in the same way that a connectionist network reaches a stable equilibrium after several iterations. The Externalization of Intuitions If many moral intuitions (e.g., sympathy, reciprocity, loyalty) are partially built in by evolution then the most important developmental question is not how do they get in to the child but rather how do they get out? Fiske (1991) argues that social development should be thought of partly as a process of externalization, in which innate cognitive models manifest themselves as a part of normal maturation. He reviews evidence (e.g., Damon, 1975) showing that the four models emerge during development in an invariant sequence: Communal Sharing in infancy, Authority Ranking by age three, Equality Matching around age four, and Market Pricing during middle or late childhood. This is the same sequence in which the models appear to have emerged phylogenetically in the mammalian and primate lineages. The contrast between internalization and externalization is particularly clear for Equality Matching. Western parents often try to get their young children to share and to play fairly. If moral development was a matter of gradual internalization, or even of reward and punishment, then childrens adherence to principles of fairness would show a gradual increase throughout early childhood. Instead, Fiske argues that children seem relatively insensitive to issues of fairness until around the age of four, at which point concerns about fairness burst forth and are overgeneralized to social situations in which they were never encouraged, and in which they are often inappropriate. This pattern of sudden similarly timed emergence with overgeneralization suggests the maturation of an endogenous ability, rather than the learning of a set of cultural norms. Only after the cognitive model has externalized itself can it be shaped and refined by cultural norms about when and how it should be used. The Development of Intuitions Even if moral intuitions are partially innate, children somehow end up with a morality that is unique to their culture or group. There are at least three related processes by which cultures modify, enhance, or suppress the emergence of moral intuitions to create a specific morality: by selective loss, by immersion in custom complexes, and by peer socialization. The Selective Loss of Intuitions The acquisition of phonology provides a useful analogy for the acquisition of morality. Children are born with the ability to distinguish among hundreds of phonemes, but after a few years of exposure to a specific language they lose the ability to make some unexercised phoneme contrasts (Werker & Tees, 1984). Likewise, Ruth Benedict (1934/1959) suggested, we can imagine a great arc of culture on which are arrayed all the possible aspects of human functioning. A culture that capitalized even a considerable proportion of these would be as unintelligible as a language that used all the clicks, all the glottal stops, all the labials.... (p. 24). Similarly, a culture that emphasized all of the moral intuitions that the human mind is prepared to experience would risk paralysis, as every action triggered multiple conflicting intuitions. Cultures seem instead to specialize in a subset of human moral potential. For example, Shweders theory of the big three moral ethics (Shweder, Much, Mahapatra, & Park, 1997; see also Jensen, 1997) proposes that moral goods (i.e., culturally shared beliefs about what is morally good and valuable) generally cluster into three complexes, or ethics, which cultures embrace to varying degrees: the ethic of autonomy (focusing on goods that protect the autonomous individual, such as rights, freedom of choice, and personal welfare); the ethic of community (focusing on goods that protect families, nations, and other collectivities, such as loyalty, duty, honor, respectfulness, modesty, and selfcontrol); and the ethic of divinity (focusing on goods that protect the spiritual self, such as piety, and physical and mental purity). A child is born prepared to develop moral intuitions in all three ethics, but her local cultural environment generally stresses only one or two of the ethics. Intuitions within culturally

Morality and Intuition 20 supported ethics become sharper and more chronically accessible (Higgins, 1996), while intuitions within unsupported ethics become weaker and less accessible. Such maintenance-loss models have been documented in other areas of human higher cognition. It seems to be a design feature of mammalian brains that much of neural development is experience expectant (Black, Jones, Nelson, & Greenough, 1998). That is, there are developmentally timed periods of high neural plasticity, as though the brain expected certain types of experience to be present at a certain time to guide its final wiring. Such sensitive periods are well documented in the development of sensory systems (Hubel & Wiesel, 1970), and language (Johnson & Newport, 1989). Huttenlocher (1994) reports that most synapse selection and elimination in the human cerebral cortex occurs in the first few years, but that in the pre-frontal cortex the period of plasticity is greatly delayed. Synapse selection in the pre-frontal cortex starts later, accelerates in late childhood, and then tails off in adolescence (see also Spear, 2000). Since the pre-frontal cortex is the brain area most frequently implicated in moral judgment and behavior (Damasio, Tranel, & Damasio, 1990; Raine, 1997), this suggests that if there is a sensitive period for moral learning it is likely to be later in childhood than psychoanalysts and most American parents suppose. But how exactly does a culture choose and emphasize a subset of the available intuitions? Immersion in Custom Complexes The custom complex has recently been proposed as the key construct for understanding development within a cultural context (Shweder, Goodnow, Hatano, LeVine, Markus, & Miller, 1998). The custom complex was originally defined by Whiting and Child (1953, p.27) as consisting of a customary practice and ... the beliefs, values, sanctions, rules, motives and satisfactions associated with it. The custom complex captures the idea that cultural knowledge is far more than a set of inherited beliefs about the right and wrong ways of doing things. Cultural knowledge is a complex web of explicit and implicit, sensory and propositional, affective, cognitive, and motoric knowledge (DAndrade, 1984; Shore, 1996). Custom complexes are easily found in the moral socialization of children. For example in Orissa, India, many spaces and objects are structured by rules of purity and pollution. Foreigners and dogs may be allowed near the entrance to a temple complex, but only worshipers who have properly bathed may be allowed into the central courtyard (Mahapatra, 1981). In the inner sanctum, where the deity sits, only the Brahmin priest is permitted to enter. Private homes have a similar structure with zones of high purity (the kitchen and the room where the household deity is kept), and lower purity. The human body has a similar structure, in which the head is the zone of highest purity, while the feet are highly polluting. Children in Orissa constantly encounter spaces and bodies structured by purity, and they learn to respect the dividing lines. They learn when to remove their shoes, and how to use their heads and feet in a symbolic language of deference (as when one touches ones head to the feet of a highly respected person). They develop an intuitive sense that purity and impurity must be kept apart. By participating in these interlinked custom complexes, an Oriya childs physical embodiment comes to include experiences of purity and pollution. When such children later encounter the intellectual content of the ethics of divinity (e.g., ideas of sacredness, asceticism, and transcendence), their minds and bodies are already prepared to accept these ideas, and their truth feels self-evident (see Lakoff, 1987). American children, in contrast, are immersed in a different set of practices regarding space and the body, supported by a different ideology. When an American adult later travels in Orissa, he may know how rules of purity and pollution govern the use of space, but he knows these things only in a shallow, factual, consciously accessible way; he does not know these things in the deep cognitive/affective/motoric way that a properly enculturated Oriya knows them. Fiske (1999) reviews evidence in anthropology that children are taught surprisingly little in most cultures, and that they acquire most of their cultural knowledge and expertise by observing and then imitating the practices of older children and adults. (See also Bandura & Walters, 1963, on imitation and social learning.) Fiske argues that anthropologists have generally underestimated the

Morality and Intuition 21 importance of motor schemas and implicit knowledge, relying instead on the verbal reports of informants as their primary source of ethnographic data. In other words, there is an asymmetry between how culture gets in to children, and how it gets out to anthropologists. Cultural knowledge gets in largely through non-verbal and non-conscious means, but it gets out through conscious verbal communication. This asymmetry brings the Nisbett and Wilson (1977) problem straight into the heart of anthropology: Informants pressed to explain practices that they themselves learned by observation, imitation, and participation generally have to make up concepts that have very tenuous, often imaginary relations with the manner in which the informants themselves actually acquired or generate the actions in question" (Fiske, 1999, p. 1; emphasis added). The importance of practice, repetition, and physical movement for the tuning up of cultural intuitions is further demonstrated by Liebermans (2000) recent review of the neural substrates of intuition. Lieberman finds that social learning uses some of the same circuits in the basal ganglia that motoric learning does, causing many social skills to become rapid and automatic, like well-learned motor sequences. Social skills and judgmental processes that are learned gradually and implicitly then operate unconsciously, projecting their results into consciousness, where they are experienced as intuitions arising from nowhere. (See also Reber, 1993 on implicit learning, and Clark, 1999, on the underestimated role of the body in cognition.) The implication of these findings for moral psychology is that moral intuitions are developed and shaped as children behave, imitate, and otherwise take part in the practices and custom complexes of their culture. Participation in custom complexes in this way provides a cultural front end for Damasios (1994) somatic marker h