Putting Our Assumptions to the Test

The transcript below may be for an earlier version of this episode. Our transcripts are provided by various partners and may contain errors or deviate slightly from the audio.

Shankar Vedantam: This is Hidden Brain, I'm Shankar Vedantam. Human beings are always trying to make sense of the world. When things happen to us or to our communities or in our nations, we understand those events through the lens of culture, through ideology or through the prism of our own perspectives. You see a homeless man on the street or a tycoon getting on a yacht, why is one so poor and the other so rich? Many of us have ready answers to these questions because we've spent years or decades perfecting our preferred stories, but what would happen if we just stopped? If we told ourselves, I might not actually have a very good handle on the world. My theories are just that... Theories. This week on Hidden Brain, the story of a kid from Calcutta who became a Nobel Prize winner by asking a deceptively simple question, "How do you know that's true?"

-music break-

In 1937 Abhijit Banerjee's grandfather struck a real estate deal in Calcutta, it turned out to be a very bad deal.

Abhijit Banerjee: Oh, my grandfather thought he was being very clever when he built his house. He thought, well, I'm getting this plot cheap, not quite getting the idea that there might be a reason why he was getting it cheap.

Shankar Vedantam: The plot of land turned out to be right on the edge of a slum. But as we will see throughout this episode, things that sound like bad ideas can sometimes turn out well, and seemingly good ideas can turn out poorly. Often, the only way to really find out how things will unfold is to watch and see what happens.

Abhijit Banerjee: Yeah. I think my grandfather, he loved the idea of doing something dramatic and with a flourish. And this was one of his flourishes and it turned out, in many ways it was a great idea, but not for the reasons he thought.

Shankar Vedantam: Now, I can't say for sure that Abhijit Banerjee would not have won the 2019 Nobel Prize in Economics, if his grandfather had not bought that plot of land next to a slum, but it definitely helped. Here's why, when he was a child, Abhijit would go out onto the flat roof of the house, he discovered two things. One, it was the perfect place to launch a kite.

Abhijit Banerjee: Because we literally were the last house. And the sidelines were for a mile. There was nothing blocking us. For flying a kite, it was the best place in the world. It was open, there was no place your kite would get stuck.

Shankar Vedantam: The second gift of the house was that precisely, because it was an ideal location to launch a kite. It became the neighborhood hangout for kids. And not just the kids who lived in the houses up the street, but the kids who lived in the tenements, down in the slum.

Abhijit Banerjee: They were better at flying kites. We would fly kites together on our terrace and they were better at it. And since flying kites is something that is very competitive and you would really want a strong team, it was nice to know them.

Shankar Vedantam: To the middle class families who lived in the houses, the people who lived in the slum might have sparked sympathy or perhaps judgment, but Abhijit, he regularly found himself in awe of his poorer playmates.

Abhijit Banerjee: I was scared of playing marbles with them. I love playing marbles. In front of our house, actually, you would dig a little hole in the ground and then you would play marbles. And they were so much better at it than me. For me, it was always a little bit, like I was half admiring of their savvy. They were more savvy than me. And they would use swear words much more. I mean, I have a very developed vocabulary in that domain, as it turns out, being coached by experts. But these kids, they would use words that I wouldn't dare use.

Shankar Vedantam: Seeing that cool, that bravado, Abhijit experienced a strange emotion.

Abhijit Banerjee: This was the Huckleberry Finn moment for me. I mean, absolutely, there is a wisdom that, for me, was very clear that I didn't have. I was also, for me, I played cricket with them and they were better at cricket than me. In fact, it was very interesting, they had somehow the idea that as someone from a middle class family, I should be better at cricket and I was clearly palpably worse and they found that a little baffling, in fact.

Shankar Vedantam: It's not like Abhijit was unaware that friends from the slum struggled with hardship, the contrast between his life and theirs was obvious.

Abhijit Banerjee: And it was very clear to me and my brother that there was this other life where the kids didn't go to school, where you might, were wearing a pant where it was kind of tied together with a rope and a shirt that was stitched together with safety pins. And, it sort of created a sense of somehow there was another world out there. And it's not that we were 10 year old social scientists, we weren't. I think we took it as a given that poor people lived that way.

Shankar Vedantam: But his close ties with these kids meant he did not see them as objects of pity or scorn. In the years that followed, these experiences as a child would help shape the work that led up to his Nobel Prize. Another early insight that shaped his perspective on the world came from an incident that unfolded at his school. And what will surely be heartening news to any student with less than stellar grades, our future Nobel Laureate turns out to have been a really bad student.

Abhijit Banerjee: So I did badly, in I would say everything, was a uniformity. I was simply not, I mean, I was so not engaged with the whole school business. For several years, I was made to sit at the teacher's table because otherwise I would be looking out of the window and paying no attention. It was really, it was an extreme case of someone who had no interest in the school system.

Shankar Vedantam: Abhijit's teachers were puzzled. He was the son of two accomplished economists. What could explain his underperformance?

Abhijit Banerjee: If you think this is a child of two professors, he has to be talented. Therefore, the explanation for my deficiency was talent, that I'm so bored with what's being taught in class that I needed to be promoted. So instead of demoting me, I was promoted one grade. In the middle of the year, I was taken from one grade and put in the next higher grade with the theory that once I'm stimulated enough, I'll do well. There was no reason to think that, but given that my parents were academics, their theory was that I must be able to do better. And so I was placed in this more advanced class. I did equally badly there that in a sense they were right. It was almost unrelated to whichever class they put me in, I would do equally badly.

Shankar Vedantam: The teachers who thought Abhijit should be doing better academically were doing something we all do all the time. When we look out at the world, we draw inferences from the events we see. We connect dots that may or may not be connected. We tell stories. These stories are powerful because they're often generated unconsciously. The fact that the kid of two smart parents was doing badly in school, didn't make sense to Abhijit's teachers. So they developed a theory to get the pieces to fit together. It was a story that satisfied Abhijit's parents, even as it did very little to change the trajectory of his schooling. In time, Abhijit came to sense that such stories obscure the truth, even as they reassure us that we understand the world. This pension for connecting the dots, for storytelling, abounded among the well-educated friends who came by his parents' home. Calcutta, or Kolkata as it's called today, is in a region of India called West Bengal. It's known for the passion with which people debate big ideas.

Abhijit Banerjee: Bengal is, at least in that era, was famous for armchair theorizing. And I often would have this reaction, "Yeah, but how do you know that?" I mean, maybe just being surrounded by people who are extremely generous with their intuitions and conclusions was a way to be a little more skeptical. I think it was more mostly reacting to my very voluble, emphatic environment I grew up in, people were very talkative and they all had strong views.

Shankar Vedantam: The Calcutta of Abhijit's youth was a hotbed of political arguments. By the time he was a teenager, Abhijit found himself pushing back against the sweeping theories of his friends and neighbors.

Abhijit Banerjee: I think the one strong influence on me was actually a reaction to, I think, left-wing ideology of a particularly Marxist kind. And I hit that very early. This is Calcutta in the 1960s and seventies, Marxist ideologies everywhere. And it's sort of the dominant ideological framework. And people had such quick answers to questions. And I must say, I think of myself very much as a leftist, but I did find that the speed at which people jumped to conclusions and sort of assumed that something was the result of some capitalist conspiracy or something, I found that very unsatisfying.

Shankar Vedantam: When his fellow Bengalis came up with stories about why the world was the way it was or why things worked the way they did, Abhijit often asked them to slow down. How did they know their theories were true? What was the evidence that backed up their claims?

Abhijit Banerjee: I think that was one thing that made me more empirical, I would say. I liked the idea of how things actually work at a more granular level, I think is also a good antidote against very broad sweep theorizing. So I remember arguing with people when I was 15 or 16 and we were discussing social issues and, for me, I understand things and yes, you tell me this happened, but how did that capitalist make that conspiracy work? How did they implement it?

Shankar Vedantam: Later in his career, after he became an economist at MIT, Abhijit would describe what he came to do as the academic equivalent of plumbing. Where an architect might be interested in the broad sweep of a building, how the overall design comes together, a plumber is interested in whether the pipes are connected, and the joints have leaks. Abhijit was always less interested in the big conclusion than in how things worked at a granular level. As a teenager, he came to be known among his friends as the guy who asked irritating questions.

Abhijit Banerjee: It was a very common trope of our conversations, me saying, "Why do you assume there is a reason for everything?" I remember saying that sentence a lot. The fact that things happened doesn't mean there's a good reason for it.

Shankar Vedantam: And were these, sort of, interjections received warmly, or were you sort of seen as being the fly in the ointment here, of basically asking people, how they came to the conclusions they did?

Abhijit Banerjee: I think I was seen as being slightly annoying. Sometimes people would have this exasperated look of he's making that point again.

Shankar Vedantam: Abhijit decided to follow his parents' career path and became an economist. As he pursued his training, he found that many people in the field shared the same tendencies as the voluble Bengali neighbors of his youth. People spent a lot of time arguing about different theories. Liberal economists clashed with conservative economists. Labor economists had models of why the economy worked the way it did, and they clashed with other economists whose frame was the market, our monetary policy. Experts disagreed about how the world worked and what policies made the most sense.

Abhijit Banerjee: Theory was so powerful. Even when I was a graduate student, theory was key, this is in the 1980s. And, I indeed followed the herd and studied theory as a result of that. It was very much the idea that, we know what the fundamentals of human behavior are, so if you put them together you get the answer. The empirical work was often shallow and shallow for the reason that doing experiments was seen as too challenging, but also because people thought, look, it's a waste of time. I think there was a lot of confidence in the theory.

Shankar Vedantam: Despite his natural predilection to be wary of sweeping theories, Abhijit fell in line with his peers. For years, he followed the norms of his field. He too spent time coming up with intricate models of how the world worked, models that involved complex mathematics and very little by way of field experiments.

Male Announcer: The Royal Swedish Academy of Sciences has today decided to award the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, for 2019, jointly, to Abhijit Banerjee, Esther Duflo, and Michael Kremer for their experimental approach to alleviating global poverty.

Shankar Vedantam: When we come back, how Abhijit found his way back to being the annoying teenager of his youth and discovered his real calling. You're listening to Hidden Brain, I'm Shankar Vedantam.

-music break-

Shankar Vedantam: This is Hidden Brain, I'm Shankar Vedantam. In the field of economics, as in many other academic disciplines, researchers spend a lot of time theorizing about why the world works the way it does. A libertarian economist, for example, might locate the roots of poverty in excessive government regulations that shackle entrepreneurs. A Marxist economist on the other hand, might locate the roots of poverty in an unequal capitalist playing field. If you set the libertarian and the Marxist economist down in a room to chat, they'll argue for hours about which theory is right. You see the same love of theorizing and argument outside of academia, too. If you turn on cable TV any night of the week, you'll see progressive and conservative commentators explaining why the world works the way it does and how their model offers the best path to fixing problems. Abhijit Banerjee began his career as an economist, developing and refining intricate theories, that's what you did if you were smart, job security and accolades usually followed. Right from the start though, Abhijit worried that many competing theories were just stories. They connected the dots often in superficial ways, much like his elementary school teachers who assumed he was failing as a student because he was actually an excellent student. His frustrations were particularly acute when it came to questions of poverty. Why are some people poor while others are not?

Abhijit Banerjee: I never found it, honestly, very easy to have a clear cut theory of poverty. So it was always a little bit opaque to me and that's part of why maybe I found the economics, particularly in that domain, particularly unsatisfying. Because it seemed to me that it was a theory that really had very little in it. There was just, either it was the kind theory which is that the poor don't have enough to eat, so they are handicapped. And we should think of them as being handicap people who therefore won't be able to do very much for themselves. And the less kind theory, which is that people get what they deserve and unproductive people are poor. And that's a very straightforward application of economics 101, which is that that's why they're poor. I never found that particularly satisfying. And I figured that the idea that it should come down to either not enough food or bad genes or whatever, it seems to me to be extraordinarily shallow. There's a sense in which growing up, observing the talents of this set of people who I was playing with, did inform me.

Shankar Vedantam: Abhijit kept asking, quietly at first, and then increasingly more loudly, the question he had asked his Bengali neighbors as a youth, "How do you know that's true?" Neither liberal nor conservative theory swirling among economists capture the complexity of the kids' Abhijit knew from the slum.

Abhijit Banerjee: I mean, I did feel that they were often smart, interesting, funny people with gifts of their own. And I think that sense maybe is important.

Shankar Vedantam: As Abhijit pondered the disconnect between theories of poverty and his own experiences in Calcutta, he reflected that there might be lessons to learn from another academic discipline. Medicine had experienced something of a revolution over the previous 150 years. For a long time, medicine had been the domain of theoreticians. People in different countries developed theories about why people felt sick and how to cure illness. Aristotle came up with a notion that the body had four humors or bodily fluids that needed to be kept in balance, when they were out of balance, people felt sick.

Abhijit Banerjee: The way medicine was often made was, people would have a broad theory of how the body functions and the, for example, humors, and then we are going to counteract the humor. If it's heat, I'm going to counteract the heat. And so it was often based on very analogic reasoning and the theory was often pretty shallow, but the driver of a lot of medical care was often not particularly well founded theory of how the body works and therefore how one counteracts it. And I think the big revolution comes, basically, people start to do observational studies. I think the famous one is cholera. And you start to see that in figuring out the roots of cholera or malaria, both of those, to figure out that malaria is caused by mosquitoes and cholera is caused by water. Both of those were, I think, deep insights. Ronald Ross won the Nobel Prize for figuring that out. He did it in Calcutta, and figured out how malaria gets transmitted. Those were still observational studies, but they meant that you started to keep track of data. And you tried to make sure that the data wasn't contaminated by other things.

Shankar Vedantam: About 100 years ago, some medical scientists came up with an even more radical idea, instead of arguing over which theory of disease was correct, why not simply run experiments to determine the truth?

Abhijit Banerjee: And starting in the 1920s, basically, there's a set of biostatisticians who start to argue that you can just do experiments. If you want to see whether the medicine works or not, pick people at random, give some the medicine, others not the medicine, and you'll see whether it works or not. As simple as that. Once that idea was there, it was hard to challenge it. And it changes the nature of medicine.

Shankar Vedantam: Take, for example, the development of an effective treatment for AIDS. It involved combining several different drugs into a single cocktail.

Abhijit Banerjee: The AIDS cocktail was in a sense just off-the-cuff reasoning. It was not deeply theorized. It was like a set of things that might work. People put them together and they tried it and it worked, and it saved millions of lives. The idea that we won't be able to theorize something, but we might be able to solve the problem nonetheless, became very powerful in medicine. And that's just the idea that we try, it works, it's great, if it doesn't, we'll try something else. And it's often guided by relatively little theory.

Shankar Vedantam: Where doctors with different models of human illness may once have argued for years about which theory was right, the new approach said, spend less time arguing about why things work the way they do and more time studying the granular details of how things work. Less architecting, more plumbing. You want to know if vaccine A works? Test it. You want to find out if acupuncture is effective against diabetes? Run an experiment. Instead of arguing about whether bad parenting causes mental illness, give people different forms of psychotherapy or medication and compare which one makes patients better.

Shankar Vedantam: Abhijit started to ask himself what would happen if he were to bring the same insight to economics. Instead of putting grand theories on a pedestal, what if he ran controlled experiments and looked at the results? He found a partner, an NGO working in a remote district in Northern India. And he decided to study a question that had an obviously correct answer. The goal in this first experiment was not to generate any new insights, but to simply understand how to run a field experiment.

Abhijit Banerjee: And so I wasn't trying to do anything deep. The idea was take the obvious. The obvious is the educational theory was very clear. Teacher, student ratios matter. This was how schools advertise themselves, we have seven students per teacher or whatever. The talk of not just economics, education specialists was very much that. So we were all completely confident that if you double the number of teachers, we'll get better test scores.

Shankar Vedantam: Abhijit found the simplest setting for the experiment. In a rural area, in the state of Rajasthan, many poor schools had just one teacher. Abhijit took a set of 40 schools, and 20 of them picked at random. He doubled the number of teachers to two, and the other 20, nothing was changed.

Abhijit Banerjee: We went from one teacher school to two teacher schools. If you increase the number of teachers by 10%, that's a big intervention. It's expensive. We had doubled it, so we thought this would be a slam dunk.

Shankar Vedantam: Then the results came in. In schools with two teachers, test scores of students changed by nada, nothing. There was zero effect.

Abhijit Banerjee: I think our first reaction was, we are doing something wrong. Let's go measure better. And so we did, I think, an extremely expensive and painful measurement, and found nothing. And at that point, both for the NGO and us, it was really this aha moment. Okay. Something strange is happening, we have to sort of go back to the drawing board, understand what's missing in that story? Because it was a common sense of both economics and education that this will work. If anything works, this will work. And so it really changed my life because it, in fact, there were nuances there, none of which matter. There was an attempt to hire a woman teacher, the idea was that that would be, the girls would benefit more. And so we looked for all those things. I mean, we had the theory, which is that women teachers are good for girls. We had all the nice hypotheses, which were laid out and then it was just that.

Shankar Vedantam: Did you find out sort of what actually was happening? Why? I mean, even now I'm sort of gobsmacked that this didn't have the effect that it should have had. I mean, doubling the number of teachers in a school by definition almost, I think, should make sense. I mean, even if you ask me today, I would tell you, of course it's going to improve the test scores of students. Did you ever figure out why the result turned out the way it did?

Abhijit Banerjee: We did the same thing in Mumbai, in a later experiment, in the early 2000's, where we cut the class size from 40 to 20 and it had no effects on the test score. So I think we were pretty convinced it was right. And then, eventually, I think we came up with a story. I can't tell you that it's necessarily right, but it fits with many other things. And the story is very simple. The way the Indian education system is constructed, the idea is there is content that needs to be delivered to the child every day. There's a syllabus, you follow that. And then, that's the best you can do. The fact that the child may or may not get it, these are among the poorest people in India, these are, female literacy in this area was 2%. So this is really an extremely underprivileged area. Parents were in no position to support their children often. So if the children fall behind, they fall behind. And so I think what's happening is that the teacher was just teaching any child who managed to stay with the teacher, but the rest of the children were behind. And it really wasn't, there was no mechanism for integrating them. Today's material is, we'll learn these kinds of words, if you don't follow that's too bad for you. We have actually lots of evidence suggesting that that's the right explanation.

Abhijit Banerjee: So therefore it doesn't matter how many, that if you double the number of teachers... There are only two or three children in that whole group who were able to follow, the rest of them were lost in any case, they were bewildered, looking at their hands. I could see that, physically I could see what was wrong, which is that the kids were often just so shell shocked by the whole process that they weren't really even trying to engage. They were just staring into space.

Shankar Vedantam: I mean, one of the things that sort of jumps out at me is of course, when we have these models in our heads about how the world is supposed to work, and then somebody comes along and provides you with data, showing that the world doesn't work that way, our first reaction might be to dismiss the data. But our second reaction would actually be to double down on the model. I mean, in some ways, when I'm thinking about the experience that you had as a small child, where you did badly in class and you got promoted, one conclusion you could draw from that is, all right, so he's still disengaged in this class, this higher class, that must mean that he is still bored because this new class is also not up to par with how smart young Abhijit is, let's promote him to the next class. So in other words, when you rely on sort of your intuitions about how the world works, you can make the same mistake over and over again, because you can interpret the results you're seeing in a way that conforms to your model.

Abhijit Banerjee: I think you're exactly right. People could have said that this just means that these kids are unteachable. I remember being in one conversation, I won't name the organization, a different one, where they said that, basically, these kids are so unteachable that you have to start at the beginning and reconstruct them, in a sense, to be able to learn. And I remember saying no, these kids are just being taught badly and if they were taught better, they'll learn. And I think the experience of the next 20 years of our work on education has been entirely that, which is that if you actually focus on teaching the kids that want to learn, in two months they make more progress than they make over the previous two years. And that's very much the conclusion, but you could have easily taken an essentialist understanding, which is, these kids are hopeless.

Shankar Vedantam: Yeah. Or you could have said, yes, we increased the size of the class, the teachers by a hundred percent, but what you really had to do was you actually had to increase it by 200% or 400% or 800%, that you could have doubled down on the same approach, because your fundamental model is that model is correct.

Abhijit Banerjee: And that's also true. But that was too expensive so that one didn't come up. The one that did come up is, "How do you know these kids can't be taught?" Maybe they are so malnutreated that their brain has collapsed or something. This is something that keeps coming back, sadly.

Shankar Vedantam: Mm-hmm (affirmative). So in another set of studies that you did, you looked at the effectiveness of different strategies in getting small children vaccinated. And if I remember correctly, in this particular area that you were working in, the vaccination rates were very low. They were like 2% or something. And they were sort of competing theories about why the vaccination rates were so low and the NGOs and the government had sort of different theories about what had happened. Paint me a picture of where this was, when this was, and what the different theories were for the very low vaccination rate.

Abhijit Banerjee: This was also in a rural district, the NGO we were working with was the same one. When the vaccination conversation started, they convened a meeting of local NGOs. And it was interesting, the local NGOs had the view that this is because the government supply system doesn't work. The government is incompetent and when you want to get vaccinated, you can't get vaccinated. The government, of course, had the opposite view, which is that these people, they're primitive. They have traditional beliefs and therefore you can't get them to get vaccinated because they don't want to get vaccinated. And so the NGO's interest was in a sense, the one we were working with, they were also interested primarily in showing that if you could have a reliable supply of vaccines, that you can get vaccinated in a predictable way, then that's going to solve the problem. So the first intervention we had was exactly that, this NGO worked with the government, the government gave them the vaccines, and they announced a day on which they would show up in each village. And on that day, they did arrive extremely punctually and the vaccines got delivered. Anybody who came could get vaccinated. So everything on the supply side worked. That raised it to, eventually it was 6% in the villages, which had the traditional, the status quo. It went to 17%, not to hundred. Now what we had done, partly just out of, again, just to see, let's see if it works. Not particularly because I had a strong intuition, is to say, okay, why don't we give them a small reward for getting vaccinated? And it was really one kilo of lentils, two and a half pounds of lentils. And that was, people's view was this is useless, but it was really very much the idea that this is some strange thing that academics think of, but turned out that that raised the vaccination rate to 36%. So, that had an enormous effect. And I think we've repeated that particular exercise many times. And every time we find the same thing, which is a small payment that makes the occasion memorable, is very important in getting people to get vaccinated. Otherwise they just get drowned in, there's so many things happening in their lives. There's so many challenges in being poor, it's easy to get distracted and forget to get vaccinated.

Shankar Vedantam: So one of the points that you've made in the study is that, you can call it an incentive, you can call it a bribe, you can call it a nudge. You're basically giving people something that's unrelated to the vaccine, the benefit of the vaccine, you're giving them something unrelated, to get them to take the vaccine. And you've made the point that in some ways, the reason that this is actually attractive to people, given that the gift is actually so small, is it changes what people are thinking about perhaps on the day of the vaccination. What do you mean by that?

Abhijit Banerjee: You could always say, "I'll do it next month." You are busy, you have another child, he's four and he needs to be taken care of. And if you want to go get the younger child vaccinated, you have to bring him. And he's going to not particularly want to be sitting there, he's going to run around and you have to chase after him. There's just all kinds of good reasons to want to postpone. What the lentil does is it just says, okay, fine, but if I get it done today, I'm going to get that nice thing. And that's going to be a little bit of sweetness in my life. I really think that that's the right psychology. I mean, I talked to some of these people who came for it and it's not that they will get rich because they got the lentils, it's just that it sort of makes that moment, when you are deciding between many compulsions, it's one compulsion that is very well defined. It's, I could go next month, but next month maybe I wouldn't get this, right now I'm going to get the lentils.

Shankar Vedantam: Notice that the intervention that had the biggest effect on vaccination rates was not shaped by some grand theory about why poor people don't get vaccinated. It wasn't driven by the belief that incompetent government officials and unreliable supply chains were the source of the problem. It also wasn't connected with the even broader theory that poor people in Rural Udaipur District had such primitive beliefs, that they would refuse to get vaccinated. It was just an experiment. Let's try this and see if it works. It was plumbing. The faucet has a leak. Let's install a new washer and see if it fixes the problem. In other work along the same line, for example, Abhijit and his colleagues have found that cash transfers to poor people do not prompt them to become lazy or drop out of the workforce, as many models in economics might predict. Many people use the money to live more comfortably, but Abhijit has also found that when you focus on the mundane and the practical, something surprising happens. Once you discover that a new washer fixes the leak in the faucet, the solution tells you what your problem was. Once you see that increasing the number of teachers doesn't automatically increase test scores, it allows you to come up with a new theory. Some kids are being left behind in class, and to help them, you have to change the way you teach. Once you see that two pounds of lentils can dramatically increase your vaccination rate, you're less likely to reach for part explanations about incompetent governments and primitive villagers. This is exactly what has happened in medicine. The experimental evidence of what works and what doesn't work has turned out to be a powerful engine to improve our understanding of how the body functions. The experiments improve the theories. When we come back, more of Abhijit's forays into controlled experiments, and what the debate between theory and experiment can teach us about the human capacity for insight and humility. You're listening to Hidden Brain, I'm Shankar Vedantam.

-music break-

Shankar Vedantam: This is Hidden Brain, I'm Shankar Vedantam. In 2019, Abhijit Banerjee, Esther Duflo and Michael Kremer were awarded the Nobel Prize in Economics for their experimental approach to alleviating global poverty. Over the past two decades, they have not only come up with a number of discoveries, they have revolutionized the practice of economics itself. Abhijit now works with teams in dozens of countries. He and his colleagues have now run hundreds of experiments and inspired thousands more. Increasingly, policy makers are taking a leaf from Abhijit's book.

Shankar Vedantam: Instead of falling back on path ideological answers to questions of public policy, smart leaders are saying, "Want to know which policy is the best policy in a given state or country? Don't argue about it on cable TV, just run an experiment and find out what works best." Take the question of how to fight malaria in poor countries. Everyone knew that when people in areas infested with malaria use bednets, fewer people fall sick, fewer kids die. The question policymakers confronted was, should you just give away bednets for free? Would people value the nets less if they were free? Would they start using bednets as fishing nets?

Abhijit Banerjee: The debate was in some ways classic economics in the sense that on one side, yes, there were the people who are saying, look, if you signal to people that these things are free, they won't value them. So even if you think it's entirely appropriate to give them free, don't give it free. So it was a debate between people. Nobody really wanted to charge the full price. I think there were crazies who did, but this was a debate among people who were all willing to subsidize it. But just, some people thought that if you make it fully free, then people won't value it.

Shankar Vedantam: Now you could spend hours arguing the question. You could make a case for why charging a nominal amount for bednets would actually increase usage and save lives. You could also make a case for why giving them away for free would save the most lives. Some of our biggest colleagues ran an experiment where different groups of people received either free bednets or cheap bednets, the researchers then carefully tracked their use. The result, there was no correlation between how much someone paid for the net and whether they used it.

Abhijit Banerjee: And then, I think now essentially everybody's giving away bednets free. And I think that's a great thing, but I think till that moment where the evidence showed that the usage was unaffected by the price, I think everybody had that doubt. There were all these stories about condoms that were used as balloons and the bednets being used as fishing nets and those stories keep coming back. I think anecdotes were trumping evidence till that point. The evidence really shut down the anecdotes. I think that was critical.

Shankar Vedantam: So I just want to stay with this idea for a second, because of course, when we see anecdotes in the world, when we come by information about somebody who's using their malaria-fighting bednet as a fishing net, or using a condom as a balloon, it does lend itself to your coming up with a story. I mean, this is what we do. When we look at events in the real world, we're constantly coming up with stories to explain why the events are the way they are. So it's not as if the people coming up with the stories are necessarily malicious or they're trying to draw the wrong conclusions about the world, many of them in fact are deeply well intentioned and want the best things for everyone else. It's just that the act of storytelling itself runs the risk that you are extrapolating from too little data, to sweeping a conclusion.

Abhijit Banerjee: Oh, it's even worse than that, I think. I think the act of storytelling really pivots on narratives like these. They're wonderful, when you say, there was this program and then kids were playing with these strange balloons, and when you looked at it, they were just condoms, little kids playing with condoms, filled with water. That just sounds like such a great story. So in fact, it's even better, even more seductive than just, we need to have a narrative. These are wonderful narratives and therefore they're particularly compelling. We keep passing them on because after all, if the story was that they didn't play with the condom, that's not an interesting story to tell. So, in a sense, these extreme examples are very, very seductive and they tend to over inform us all the time.

Shankar Vedantam: You know, I'm thinking about a sort of an unrelated study for a second, just to compliment what you are saying. The state of California at one point, decided to make salaries of state employees public. And this was partly designed with the view of basically saying sunshine and transparency are always good things. And one of the things they were trying to combat was the gender wage gap. And they basically said by making salaries public and transparent, we can reduce pay disparities between men and women. This is a fundamentally good thing to do. The economist Emmanuel Saez basically analyzed employees at the University of California system. And he found that as a result of this pay transparency, some 20 to 40% of people working for the university of California system were now thinking of leaving their jobs. Because now they looked over their shoulders, they saw that people who were doing work that was similar to theirs were now being paid more than them. And now, instead of being an engine for equity and transparency, it became an engine for resentment and frustration and people wanting to leave. And again, I think it's an example of how the stories that we tell about the world sometimes have unintended consequences and sometimes no matter how many unintended consequences we see, we can't help but constantly keep coming up with stories.

Abhijit Banerjee: Again, I think that's exactly right, but in particular, I think the words like transparency, how could you be against it? It's transparent. It's clear. There is a valorization that's built into these words and I think that's extremely compelling, often. And so I think to say that I am against transparency makes me sound sleazy, but in fact, I am against transparency and that's a critical problem in bureaucracy. People want to have a paper trail and that's often the reason why we are happy to say bureaucracies are inefficient. But in fact, bureaucracies are inefficient precisely because we want them to be transparent. And that turns out to be one of the drivers of a lot of red tape. So I'm often skeptical of transparency.

Shankar Vedantam: Our values can be a powerful driver of storytelling that obscures the truth. Of course, increasing the ratio of teachers to students is going to improve test scores. Of course, transparency is always a good thing. Another driver of the sort of storytelling, partisanship. It's so easy for us to see that the theories of our opponents are fanciful and misguided. It's so much harder to be skeptical of our own stories.

Abhijit Banerjee: I'm as guilty as anyone, I'm seduced by my own stories. That's why I think the methodologies are useful. They're useful because in some sense, while privately I hold onto many theories, I think the fact that we are now in a place where people are able to challenge you and say, "Do you have any evidence for it?" I think, does change it. I don't know that we'll ever be not seduced by stories, but I think the insistence of what's sometimes called the Credibility Revolution in economics, I think that's the discipline of saying, well, you have no evidence for it. I think that does help the public conversation. I think we are in a better place now.

Shankar Vedantam: So let's just stay with this idea for just a second, because it's an important point. It is the case that randomized control trials can tell you something about the world that you didn't know. It is true that the data can challenge your preconceived notions. But as you point out, there is a way of doing randomized control trials, and in some ways, predetermines the answers that you actually want to get, or in some ways, figures out how you're conducting the experiment to guide the experiment in a certain direction.

Shankar Vedantam: So the point that I'm trying to make is that the mere act of conducting an experiment in some ways does not produce, necessarily or automatically produce, the skepticism that you need to combat theories. In some ways it requires a certain honesty in terms of actually approaching the experiment with an open mind, with some humility, in order to be able to generate those answers. Because if you actually, again, if you allow your preconceived notions and theories to guide how you're setting up the experiment in the first place, you could very easily set up the experiments so that you always find eventually what you want to find.

Abhijit Banerjee: Or at least find the wrong answer. Even if you don't find the right... You may limit the set of possibilities so that you still learn something, but you'll often learn in a very limited way. And I don't think there is any obvious good antidote, as the one you said, which is ask yourself, "Why do you believe that this is the right set of possibilities?" And I think we try, but it's hard because again, it's easy to be seduced by your own narratives, as you said before.

Shankar Vedantam: Mm-hmm (affirmative). I'm wondering as someone who's won the Nobel Prize in Economics, you are widely seen as an expert and people must defer to your opinions in a variety of settings, academic settings, social settings, community settings. Do you sometimes feel like you are at risk yourself of falling prey to some of the models that you have challenged? Because now people sort of say, "Well, Abhijit Banerjee, Nobel Prize winner, clearly he must know the correct answer."

Abhijit Banerjee: Oh yeah. I call it the oracular status. You are the Oracle, now you can just speak. You know, absolutely, it's extremely dangerous and tempting sometimes because in the end, on some things I feel that, okay, I have an opinion, I might as well say it, and I can't say that I always resist. I do try very hard to tell myself I need to resist, shut up, shut up, shut up, shut up. But do I always manage? No.

Shankar Vedantam: We've sort of circled around the same question, I think multiple times, which is sort of the importance of sort of humility, sort of what I think I'm hearing over and over again, and sort of this trajectory of your life's work. Richard Feynman said, "The first rule is that you must not fool yourself and you are the easiest person to fool." So, I mean, we've known this for a very long time, but it feels like in practice, this is actually really hard to do.

Abhijit Banerjee: And it is really hard to do, partly because I think the idea that most answers come with enormous uncertainty, is very hard to convey. Uncertainty is very hard to convey. You can say, I think my specific belief is seven, but it could be 11 or four or three. And then people don't hear the 11 or three. They just hear the seven. It's very hard to convey uncertainty. When I speak, should I be silent, therefore? That's a very hard dilemma. I don't know that any of us know our way around it, because we know that it'll be over interpreted. If I say anything is going to be over interpreted, now that I have a Nobel Prize, even more over interpreted. So you sort of hold back and say, well, should I shut up? But then is it okay to shut up in a context where lives are at stake? And I don't know that there is a good resolution to that, but you are right in saying that it's extremely fundamental tension in any economist's life, I think, but in mine, especially.

Shankar Vedantam: Abhijit Banerjee is an economist at MIT. Along with Esther Duflo and Michael Kremer, he was awarded the 2019 Nobel Prize in Economics. Abhijit, thank you for joining me today on Hidden Brain.

Abhijit Banerjee: Thank you very much for having me.

Shankar Vedantam: Hidden Brain is produced by Hidden Brain media. Our production team includes Brigid McCarthy, Annie Murphy Paul, Kristin Wong, Laura Kwerel, Ryan Katz, Autumn Barnes, and Andrew Chadwick. Tara Boyle is our executive producer. I'm Hidden Brains executive editor.

Shankar Vedantam: Our unsung hero this week is Sarretta McDonough. Sarretta is one of the listeners who's made a financial contribution to Hidden Brain. Sarretta, thank you so much for your signal of support for our work. It truly provides an emotional boost to all of us on the team. If you'd like to join Sarretta in supporting the show, you can do so at support.hiddenbrain.org.

-music break-

Shankar Vedantam: We're working on an episode, and we're wondering if you had a story that we could feature on the show. Can you think of a time when an enemy became a friend? What caused you to clash with the other person in the first place? And what was the moment that changed your relationship? If you have a personal story you are willing to share with the Hidden Brain audience, record a voice memo and email it to us at [email protected] Use the subject line, enemies. That email again is [email protected] I'm Shankar Vedantam. See you soon.