---
title: "Behavioral Finance: Psychology, Manipulation, and Integrity in Markets"
author: "Robert Shiller · Yale Open Courses"
description: "I want to talk today about behavioral finance — about psychology in finance. This is a long-standing interest of mine; I’ve been involved with it for over 20 years. It is not really emphasized in..."
source: "https://adhenawer.net/posts/original/behavioral-finance-psychology-manipulation-integrity-markets.html"
lang: en
---

TL;DR · key points

1. I want to talk today about behavioral finance — about psychology in finance.
 2. Behavioral finance, or behavioral economics more broadly, is a kind of revolution that has occurred in finance and economics over the last 20 or 30 years.
 3. The real problem is that people are complex.

## WHY BEHAVIORAL FINANCE MATTERS 0:00

I want to talk today about behavioral finance — about psychology in finance. This is a long-standing interest of mine; I’ve been involved with it for over 20 years. It is not really emphasized in your textbook. Fabozzi and his co-authors discuss many things in the financial world, but they don’t really address the underlying human behavior.

Behavioral finance, or behavioral economics more broadly, is a kind of revolution that has occurred in finance and economics over the last 20 or 30 years. It remains somewhat controversial. I don’t quite fully understand why people polarize as much as they do, but some people simply don’t like it. Others — and I think we are becoming the majority — now regard behavioral finance as an important element of finance.

The real problem is that people are complex. Our financial institutions are designed for real people, and their functioning depends on the behavior of real people. It’s not as simple as we’d like. Another revolution is occurring in parallel, of even bigger significance: a revolution in neuroscience about the human brain. The human brain is a very complicated organ.

Economists have liked to invoke the principle of rationality as an underlying component of their theory. That has been useful, and it is not a very complicated argument, but it is of limited use, because people aren’t entirely rational. They are often rational, but not completely. Very often, people behave stupidly. That includes everyone, including me. We are human, and we have limits.

## HUMAN WEAKNESS AND THE IMPULSE TO EXPLOIT 2:18

One thing about the human species is that we are aware of other people’s weaknesses and have an impulse to exploit them. When you see other people behaving stupidly, sometimes you think, “Maybe I can turn this to my advantage.” That becomes a problem. The history of humankind is, in part, a history of exploitation of one person by another — not entirely, but it is an important element.

So I’m going to talk about these human failings. It’s not to say people are stupid. They’re people. We’re all imperfect. We’re smart in some dimensions, but we can also make important mistakes.

Before I start, I want to put this into perspective and begin on an upbeat note. I’ll talk about all kinds of human errors, but I believe the business world generally doesn’t exploit people terribly. Characteristically successful businesses in finance and elsewhere consider their long-term advantage and the reputation they have. Doing something blatantly exploitative of human weaknesses works against that long-term advantage. So we see a lot of human failings, but we don’t see people cashing in on them as often as you’d think.

Another aspect of human behavior is morality. Evolutionary biologists think morality evolved along with our other traits — we have an impulse to be moral. In the long run, you might not gain much satisfaction from exploiting other people’s mistakes, so you don’t necessarily do that.

## ADAM SMITH: FROM PRAISE TO PRAISEWORTHINESS 5:56

I have chapters from my forthcoming book assigned for this course, and I keep thinking the one on behavioral finance wasn’t quite ready. I start that chapter by talking about Adam Smith and his book “The Theory of Moral Sentiments.”

Adam Smith was a professor in Scotland. In 1759 he was a professor of moral philosophy, because there were no professors of economics in those days. In 1759 he wrote “The Theory of Moral Sentiments,” and in 1776 he wrote the more famous “Wealth of Nations.” He is probably the most important figure in the history of economic thought. “The Wealth of Nations” is considered the first real treatise on economics — a wonderful book, still very readable today. His “Theory of Moral Sentiments” is not as widely read, but it’s a book about psychology and morality, and I find it very good even 250 years later. He went through many editions of it, perhaps because he thought it was his most important work.

The book starts with selfishness and altruism. The real question, which Smith thinks defines economics, is whether people are really completely selfish — sometimes it seems that their presumed benevolence is just an artifice for their own benefit. He wonders how an economy could work if everyone is totally selfish, and he concludes that they are not.

What I emphasize from the beginning is that people inherently love praise. We crave the approval of other people. But Smith reflected further and asked: do people really want praise itself, or something else? Suppose people made a big mistake and thought you had accomplished something when in fact it was someone else. You find lots of people praising you, and you even know they will never find out you didn’t do it. Would that really be pleasurable? Smith said, probably not. Internally you’re thinking, “I’m getting all this praise, but I know I don’t deserve it.”

Then he said that, especially among more mature people, adults make a transition from a desire for praise to a desire for praiseworthiness. “I want to know that I am the kind of person who will be praised, and I don’t need to get the praise.” Ultimately, it is that tendency that makes an economy work: people don’t care just about praise. He gives the example of mathematicians. He had known many, and found that they are almost all obscure — the public doesn’t know about them, couldn’t explain what they do, and they don’t seem to care. A mathematician can do great work completely unknown. It is praiseworthiness that drives these people.

I think the finance profession, like other mature professions, is really dominated by people who have reached this desire for praiseworthiness. So they’re not going to exploit people extravagantly. Why would they? It just isn’t a good thing. They wouldn’t feel good about it — though some people will.

## PERSONALITY DISORDERS IN THE FINANCE PROFESSION 10:39

Not everyone reaches the mature state Adam Smith describes, and that is one of the complexities of human society. There is a whole branch of psychology called personality psychology that categorizes people by their personality. We are not all the same. A successful society promotes people who have the praiseworthiness desire, putting people of character into important positions — with not complete success.

The “Diagnostic and Statistical Manual,” DSM-IV, published by the American Psychiatric Association, is a kind of household word in my house because my wife is a psychologist. It is controversial among psychiatrists because it is too cut and dried for some of them. It tries to identify mental illnesses and personality types in a quantifiable, reproducible way, using checklists.

One disorder is Antisocial Personality Disorder (APD). One kind of APD is psychopathy; another is sociopathy. According to DSM-IV, 3% of the male population and 1% of the female population are APD. A simple definition for APD is: a jerk. There are more male jerks than female jerks, apparently — and females have characteristically different personality disorders, so it is much more than 3% of the population that would be diagnosed with one disorder or another.

Characteristics of APD include: lack of remorse, frequent lying, lack of empathy, superficial charm, shallow emotions, distorted sense of self, and a constant search for new sensations. Have you met someone like that? You probably have — that’s 3% of the population. An APD person is manipulative and feigns affectionate or warm feelings without feeling them, while trying to deceive you.

Once a student came to my office and asked to sign up as my research assistant. I later read about him in the Yale Daily News: he was an imposter student — not actually enrolled at Yale, and he had been around to three different campuses. Then he came later and asked me for a recommendation letter. I couldn’t believe it.

Someone did a study of APD at a prison, using DSM-IV standards, and found that 40% of the prisoners had APD. Neuroscientists have also found differences in the prefrontal cortex correlated with APD. Some people seem to have a brain structure that’s a little different, which may make it difficult for them to behave well. Adam Smith’s book still rings true: you have to judge people and learn their character, and you have to be a person of character, because in the long run that is what you want.

## MANIPULATION IN A COMPETITIVE ECONOMY 16:40

People are manipulable — unfortunately true. And unfortunately we live in a world where it is hard to avoid manipulating people at all, because we have a free enterprise system that encourages competition. If the competition is manipulating people, how do you stay completely clear of it? This is one of the contradictions of our society.

A simple and obvious manipulation is putting a price on something like $9.99. Why not just say $10? It’s called “pricing points” in marketing. $9.99 sounds a lot less psychologically than $10. Almost everyone does it. Is that bad? It’s manipulative. I’m annoyed by it. But if you were in business, you might feel you have to do it. It’s a harmless sort of manipulation.

Financial institutions are often manipulative in that sense. It’s similar to politics: if you want to be a member of Congress, you can’t say what you really believe — you wouldn’t get elected. You have to doctor your opinions to public opinion. But you might have a moral purpose underlying it all, because you want to get elected to do good things. It is hard to judge people good or bad; it’s an overall sense you get of someone’s character over time. Ultimately, our society — within limits — rewards people who show character through the confusing details.

## PROSPECT THEORY: THE VALUE FUNCTION 20:44

The most famous element of behavioral economics is prospect theory, invented by the psychologists Daniel Kahneman and Amos Tversky in the late 20th century. They called it prospect theory because it was a theory of how people form decisions about prospects — a prospect being a gamble, a decision under uncertainty. The theory has two parts: a value function and a weighting function.

The value function describes how people value gains and losses. The horizontal axis is wealth or money, with zero in the middle; the vertical axis is value, something like utility. People do not weigh gains and losses linearly. In the positive quadrant there is diminishing value — the curve is concave down, like diminishing marginal utility in standard economic theory. For losses, however, the curve is concave up. There is a kink at the origin.

The origin is the reference point — the zero from which you measure gains and losses. It is psychological and subject to manipulation. Normally the reference point is today’s wealth, but it can shift depending on how something is presented to you. “Framing,” according to Kahneman and Tversky, is presentation: the same prospect worded differently suggests a different reference point, and that changes people’s behavior.

The kink means people are very conscious of small changes in wealth, and are spooked by them. If you lost $5 this morning on the way to this lecture, you would feel exaggeratedly bad. You should really regard $5 as nothing — the present value of your lifetime income is in the millions — but you don’t think that way. You are deterred by small losses and less encouraged by small gains.

This allows exploitation. If people are so focused on small changes, business is encouraged to pick out things people are paying attention to and sell insurance policies on them. Insurance should be concentrated on the big things, like life insurance — the fact that a parent could die and leave the family without income. But it may not work to sell that; it is easier to sell something small. The classic example is funeral insurance, sold for thousands of years — they sold it in ancient Rome. Tell people that a family death means the expense of a proper burial, and they will insure that one little thing. Another example is airline flight insurance — insurance for a single flight. I heard an ad for funeral insurance recently. It is manipulative. Another is diamond ring insurance, for $5,000 — not essential. If you are insuring that and not other things, you are making a mistake.

Fortunately, the insurance industry has come around toward things that matter and are big. The Kahneman–Tversky value function represents an error people are prone to make, but the error is not total. Ultimately, people stop going to insurance companies that manipulate them. Word gets around. While manipulation exists, it is limited.

## PROSPECT THEORY: THE WEIGHTING FUNCTION 29:28

The weighting function describes how people psychologically think about probabilities. A probability is a number between 0 and 1, but people cannot accept a continuum psychologically. For very low probabilities, people round them to 0. For very high probabilities, they round them to 1. If they decide not to round, they exaggerate the difference between 0 and 1. The result looks like a broken line segment.

So when you get on an airplane and the probability of crashing is something like 1 in 10 million, most of us just mentally round to 0 — we are not going to worry about it. But some people don’t round to 0, and they blow the probability out of all proportion. In our minds, it is as if there are only three probabilities: can’t happen, maybe, and will happen. In some of the most primitive languages in the world, there are only a couple of numbers — 0, 1, maybe 2 or 3, and then no more. Our minds are still very primitive in dealing with probabilities.

Airline flight insurance is an example of trying to manipulate this characteristic. They capture all the people who exaggerate the probability. Vending machines outside airlines once encouraged you to buy insurance just for this flight, placed right there when you are most nervous. Most people don’t buy, but they don’t have to sell to everybody — they just sell to the exaggerators, at an outrageous price. Interestingly, I haven’t seen those vending machines anymore. Economists wrote about them 30 or 40 years ago, and they used to be everywhere. They just disappeared. Somehow we get past things like that. Kahneman and Tversky errors are not immutable.

Gerd Gigerenzer, at the Max Planck Institute in Berlin, has taken on Kahneman and Tversky, saying they are right that people show these error tendencies, but that he can train people out of them: he tells them there is an error, teaches them, and they stop doing it. Nick Barberis here ran a study using Caltech students, testing their ability to avoid such errors. Even these young math geniuses made the errors horribly. About a third got everything right. By the time they mature, they will eventually be trained out of these errors, but as undergraduates they behave just as behavioral finance predicts.

## REGRET, GAMBLING, AND SENSATION SEEKING 35:57

Regret theory, related to Kahneman and Tversky, says people fear the pain of regret. There’s an old expression: “I was kicking myself because I made a bad decision.” That’s a painful experience. Regret theory says there is actually a painful emotion — you are wired not to like having made a mistake. So you design your life around avoiding anything you might regret later, and that can create problems. You may make bad decisions because you are overly worried about regret.

Anthropologists report that gambling occurs in every human society. It is a human universal — not that everyone gambles, but in every society some people do. A 1974 study found that 61% of U.S. adults gambled at least once for money that year. I bet it has gone up; there are more opportunities. 1.1% of men are compulsive gamblers, versus 0.5% of women — another male-skewed trait, though only two to one.

Compulsive gambling is an addiction that distorts thinking. We have an organization, Gamblers Anonymous, that helps people with it. It ruins lives. People end up divorced because you can’t stay married to someone squandering the family money, sneaking around to gamble the way drinkers sneak around for the next drink. Gambling is psychologically associated with self-image, a sense of competence. Most gamblers play something they think reveals their competence, and they identify with one form of gambling and avoid the others.

Gambling behavior is part of what goes on in the stock market. People with a gambling personality find that a life in finance gives them the stimulation they seek. Gambling is almost like a drug addiction: depressed people may go to a casino to escape depression. They say that when they walk in, their troubles are gone — they feel invigorated and alive. It is almost like a hormonal change they seek, like injecting themselves with something. It is a very hard thing to conquer.

When New York State in 1811 created the first corporate law, which produced many questionable companies, people said it was just gambling — it was bad. But the other side is that gambling behavior is not usually a pathology. It is an aspect of human sensation seeking, part of what drives us. The stock market, in a sense, channels this behavior into something productive. Markets make it clear: this is about business; this is productive. The same emotional patterns underlie traits that work out well.

## OVERCONFIDENCE 40:38

Psychologists have found a human tendency to overestimate one’s own abilities. Most of us think we are above average; some of us think we are way above average. This has been revealed in many experiments. I’ll try one on you.

I will ask three questions, and I want you to write down a 90% confidence interval for each. A 90% confidence interval is a range of values such that you are 90% sure the true value lies within it. If I asked for the population of New Haven, you might say 50,000 to 150,000 — meaning you are 90% sure it falls in that range. If I ask for 90% confidence intervals, you should be right nine times out of ten. You could game me by giving an interval of zero to infinity, so I have to appeal to your honesty.

Question 1: the world population as of 8 a.m. this morning, per the U.S. Census World Population Clock. Question 2: the mass of the Earth, in kilograms (or tons). Question 3: the number of languages in the world, as defined by ethnologue.com.

The answers: world population was 6,901,330,581. About 80% of the class got it in their interval — not quite 90%, but close. The Earth weighs 5.974 × 10^24 kilograms, or about 5,974 billion billion tons. Here far fewer were in range. Ethnologue lists 6,909 languages — again far fewer than 90% captured it. Why did you do so much better on world population? Because you feel you understand the world, and feel less sure about astronomy or linguistics.

Why are people overconfident? Psychologists find that people have a sense that they understand the world better than they really do. It is an illusion — the world is infinitely complicated, full of surprises. When thinking about a question, there are many perspectives you could take; if you thought more, your imagination might widen your interval. But you can’t hold all perspectives at once, so you gravitate to the first one that comes to mind and underestimate your uncertainty. That is overconfidence. It is slightly higher in males than in females, but really everyone is overconfident — there is no important sex difference.

Overconfidence extends beyond yourself to your friends. We think we have very smart friends. When I was an undergraduate at the University of Michigan in the honors program, we thought we were pretty smart. There was one student we called “young Jack Kennedy.” Most of my friends ended up in good careers, but nothing spectacular. One of them, Bruce Wasserstein, really was a standout — he founded the investment bank Wasserstein Perella and later bought an interest in Lazard Frères. I met him again about ten years ago; then he died of a heart attack. But I was probably right about only one of them. The rest were not geniuses. Our ego extends to those we associate with — we respect the head of our central bank but have no respect for the head of a foreign central bank. That ego involvement produces overconfidence.

Rakesh Khurana, a professor at Harvard Business School, wrote “Searching for a Corporate Savior,” arguing that people tend to think CEOs are geniuses. Companies then seek out a “genius CEO” as a kind of stock-market manipulation: “If we get someone who has run other companies successfully, he must be a genius — our stock price will go up, and we can exercise our options.” Khurana says most are just lucky. Put such a person into a new company he knows nothing about, and he has to justify himself — so he lays off a lot of people, shuffles things around, and destroys the company. Nassim Taleb’s “Fooled by Randomness” makes the related point that most of what happens in life is chance. When it happens to us, we ascribe it to our own genius; when it happens to a friend, we think “I have genius friends.”

## COGNITIVE DISSONANCE 57:16

Cognitive dissonance is another psychological principle. The term was coined by the sociologist Leon Festinger in the 1950s. I actually met him — one nice thing about academia is meeting the great names if you stay long enough. The idea, perhaps oversimplified, is that people don’t want to admit they were wrong. It’s painful to think you believed something and it was wrong. People cling to old beliefs and look for evidence supporting them, because they have an ego involvement with the belief.

The famous experiment: psychologists got a list of people who had just bought a car, including the make, from car dealers. They called the buyers and invited them to a “marketing experiment,” without revealing that they knew which car had been bought. They went through magazines and asked which ads the buyers remembered reading. People remembered the ads for the car they had just bought, and avoided the ads for cars they had considered but didn’t buy. After buying a car, you want to confirm your belief in it, so you selectively gather information that supports the choice.

The error is not inviolable. People make it, then learn from their mistakes and correct it. It’s a recurring error, not a permanent state.

Will Goetzmann, a professor at the Yale School of Management, and co-authors found that when a mutual fund performs very badly, most investors sell — but some hang on. They interviewed those holdouts and found that many did not even know how badly the fund had done. They had blocked it out and held an exaggerated impression of success — characteristic cognitive dissonance. You forget contrary evidence and assemble supporting evidence.

Sendhil Mullainathan, at the Harvard Economics Department, studied financial advisors with co-authors in a clever experiment. They hired actors to visit financial advisors and say, for example, “My portfolio is almost entirely in money market funds,” or “almost entirely in tech stocks,” or “almost entirely in options,” or “almost entirely in my own company’s stock.” A truly professional advisor should question these assumptions. Many did, but often they did not — assuming the client was risk-averse, or wanted exactly what they had. They sent the client home with a slightly different mix, without challenging the underlying belief. Even when actors claimed to hold almost everything in their own company’s stock, only 40% of advisors told them to diversify out of it. The 60% left them mostly in their own company’s stock.

This reminds me of my Uncle Ralph, who worked for Ford Motor Company and put all his life savings into Ford stock. I told him, “Uncle Ralph, you shouldn’t do that — it’s your job and your life savings. What if something happened to Ford?” He said, “I’ve worked at Ford all my life. They treat me well. I believe in them. I’m not going to sell.” Mullainathan’s interpretation: advisors know about cognitive dissonance and are afraid of driving a new client away, so they don’t challenge deep beliefs. They become yes-men, hoping to gradually tilt clients toward a responsible portfolio. In the real world, I am not sure these advisors are doing the wrong thing — if they eventually move clients the right way, they can’t afford to drive them away.

## ANCHORING AND REPRESENTATIVENESS 1:05:37

Anchoring is the tendency to anchor your opinions on something that captures your attention. The famous anchoring experiment is again Kahneman and Tversky. They used a wheel of fortune that produced a random number between 0 and 100. They asked a question whose answer lay in that range — for example, what percent of nations in the world belong to the United Nations. They spun the wheel, showed the subject a random number, then asked for the answer. People gave answers close to the number that had just come up. Totally irrational — the wheel has nothing to do with the answer — yet they were influenced. When asked why, subjects said, “Just coincidence, I wasn’t influenced.” But statistically, they were. Anchoring means that when you face real ambiguity and must come up with a decision, you are swayed by silly and random things.

The representativeness heuristic, also from Kahneman and Tversky, says people overemphasize patterns they think are representative of what they have seen before. Certain rare patterns in the stock market, if they attract attention, get looked for again and again, and seen too often. An example is the head-and-shoulders pattern. McGee, the technical analyst, claimed it preceded crashes; but it’s hard to find, and looking for representative patterns is not the right way to process data. It invites manipulation. If people believe head-and-shoulders predicts a decline, I could take some thinly traded stock, trade back and forth with a friend to deliberately create the pattern, then short the stock massively at the sell signal, and make tons of money. Why don’t we? Because it is illegal manipulation.

## SOCIAL CONTAGION AND THE ZEITGEIST 1:09:35

I want to conclude with social contagion, because it is so important. This is really social psychology. It reflects the fact that people are interdependent — what I think is affected by what others think. There is something called herd behavior, the tendency for people to move with the herd. Not consciously; they don’t think they are moving with the herd.

A little sociology: Émile Durkheim, one of the founders of sociology, a French scholar of the late 19th and early 20th centuries, used the term “collective consciousness.” Our opinions about what is happening are formed by a collective understanding. We tend to think of ourselves as rational, with views coming from common-sense processing of facts. But we underappreciate how much our views are a little arbitrary and shared by millions of others. You live at a certain point in time in history, and certain facts, ideas, and anecdotes are circulating. There is another term: zeitgeist, German for “spirit of the times.” Durkheim and other sociologists helped us understand that the zeitgeist is determined by a collective memory, a collective set of facts we operate on. Herd behavior creates big swings in the stock market and has huge financial impact.

## TOWARD BETTER FINANCIAL INSTITUTIONS 1:11:56

I have listed a number of behavioral finance principles coming from psychology, and some tests and examples of their importance in finance. What do we conclude? My conclusion is that we are evolving toward better and better financial institutions. There is a lot of manipulation and exploitation, but as a society we have outlawed a lot of things. Manipulating a stock to create a head-and-shoulders pattern, for example, is an offense. I will talk about this in the next lecture, on regulation. But it is also people’s moral judgments — the people who become important in finance tend to have an internal compass, a desire for praiseworthiness that eliminates the worst behavior.

In the current issue of the Harvard Business Review, Michael Porter and co-author Mark Kramer argue for a principle they call “shared value.” A company’s manager should not underestimate the importance of shared value creation with society, with other people. We are all in this together. A mature manager recognizes that you don’t want to be exploitative, don’t want to upset the local community, don’t want the labor force to become disenchanted. Porter is not really writing about morals; he frames it in terms of long-term value. But morals creep into the same judgment — mature business people see shared value. Financial theory had leaned too much toward thinking a manager should selfishly pursue a narrow focus, like maximizing the short-run value of the shares.

Another example is a recent book by Anna Bernasek, a journalist: “The Economics of Integrity.” Her point is that a sense of personal integrity has dominated what people do in the business world much more than we thought. There has been too much disregard of the fact that people do things because they are right. She gives the example of milk. You drink milk regularly, I hope — it is good for you. But it could poison you. People used to get sick from contaminated milk. You don’t hear of that anymore. Why? We have government regulation of milk production, laws about purity. But when she looked into it, she concluded it is mostly not the regulation. You are safe drinking milk because of the integrity of the people. Go to a milk company and talk to the employees; they may not even know the regulations, but if you ask whether they are careful to keep the milk clean, they’ll say, “Someone is going to drink this — obviously it’s common sense. I’ll keep it clean.” It is the integrity of the people that makes the economic system work as well as it has. I have emphasized both sides — human failings, and people exploiting these failings, and people with antisocial personalities — but integrity, and the pursuit of praiseworthiness, remain the backbone of a functioning financial system.