SHANKAR VEDANTAM, HOST: If you're lucky, summer gives you time to breathe, maybe go on vacation, slow down. You finally have a chance to get to that long list of things you've been putting off - books to read, exercise, spending some time on yourself. The episode you're about to hear is part of our summer series about rebooting and reimagining our lives. We're calling it You 2.0. This week, we bring you a conversation from the fall of 2016. It explores the idea that the chaos of your regular life might have some secret benefits.(SOUNDBITE OF ARCHIVED BROADCAST)VEDANTAM: If you're a parent, you've probably told your kids a thousand times, your room is a mess, clean it up. We hear the same message in other contexts. Managers at companies around the world tell their employees to get organized. Here in the United States, we elect presidents to go clean up Washington.This is HIDDEN BRAIN. I'm Shankar Vedantam. My guest today has a radical idea. In many cases, he says, it's disorder rather than order that's our friend. Tim Harford is the author of "Messy: The Power Of Disorder To Transform Our Lives." He joined us recently for a live taping just blocks from the White House at NPR's Weekend in Washington. It's an annual event that brings together public radio fans and supporters.Tim Harford, Welcome to HIDDEN BRAIN.TIM HARFORD: Thank you very much, Shankar.VEDANTAM: I want to begin by playing you a piece of music, Tim.(SOUNDBITE OF KEITH JARRETT COMPOSITION)VEDANTAM: How did that piece of music come about? What's the story?HARFORD: It's beautiful, isn't it? My - two of my children were born to that. It was played by Keith Jarrett, great jazz pianist in 1975 in Cologne. And if you had been there two or three hours before the concert, you would not have expected things to go well. Keith Jarrett had just refused to play. And the reason he'd refused to play is because he had arrived on stage, met the piano. He was supposed to be - completely improvised, by the way - the whole thing was going to be improvised. Met this piano and realized that there'd been a mistake. And the thing was literally unplayable - without a tune, black keys were sticking. The pedals didn't work. The upper register of the keyboard was harsh and tinny 'cause all the felt had worn away. And, most importantly, it was too small. So it didn't have enough volume to actually reach the back of the Cologne auditorium. And so, of course, he refused to play.And the organizer of the concert, who was this young German girl - she's just 17 years old - desperately tried to fix the piano, to replace the piano. She managed to get it somewhat in tune. But, basically, she couldn't really improve on it. And she couldn't get it replaced. And so the only option she had was to beg Keith Jarrett to play. And of course he saw this teenager, thought of the 1,400 people who were about to show up and listen to the concert and said to her, never forget, only for you. And he went on stage. And he and his producer recorded the concert because they wanted documentary evidence of what a musical catastrophe sounds like so they could play it to future promoters to tell them to do a good job and get a proper piano. But, in fact, it's a masterpiece. It's Keith Jarrett's most popular work. In fact, it's the most - the best-selling jazz album in history - solo jazz album in history and the best-selling piano album in history. It's "The Koln Concert."And the surprising thing is that all the adjustments that Jarrett had to make to cope with this bad piano made the music better. So he avoided the harsh upper registers. He stuck to the middle of the keyboard. That makes it sound very soothing. But he also had to compensate for the fact that the piano was so quiet. So he had these rolling repetitive riffs in the bass to try to get some resonance. And he also just stood up and pounded down on the keys.(LAUGHTER)HARFORD: So you can hear him moaning in frustration during the concert. He's hating it. But it is amazing.(SOUNDBITE OF KEITH JARRETT COMPOSITION)HARFORD: And that combination of the peacefulness and the dynamism makes this electrifying piece of music. So he did not expect that a bad piano would produce a great concert. But, in fact, it did. And the argument in my book is, very often, we're faced with the unplayable piano. And, actually, we produce something great out of it.VEDANTAM: You have a number of examples besides Keith Jarrett about how disruptions and inconveniences and surprises can sometimes have paradoxically good effects on us even though we all try to avoid those disruptions. You talk about commuters, for example, who suddenly have to find a new way to work or scientists who are working on so many projects that they are discombobulated because they don't know what they're working on. In many of these cases, you find that disorder and confusion could actually be helpful.HARFORD: Yes. So in the case of commuters, there was a transport strike in London a couple years ago - Shut down not all but most of the London underground stations for 48 hours. So not a long strike. But most commuters in London use public transport. It's not really set up for the car. But you could travel on the bus. There are over-ground trains where you could use some of the tube lines that were not shut down. They have various other options. And the economists who studied this got hold of the data as to the trips that people were making. They identified people who took the same route to work every single day. Well, there's a technical term for this, it's called commuters.(LAUGHTER)HARFORD: You would think that these are the people who have absolutely perfected their route to work. Surely, commuters - they know every step. They know, you know, where to stand on the platform, which track. And yet the economists found, looking at the people who took the same route to work every day, who then changed their route during the strike - thousands of them, about 5 percent, thousands upon thousands - stayed with the new route. So these are people - this is the optimum population for having honed their route to perfection. And yet, 1 in 20 of them discovered that when they were forced to find a new way, they found a better way.And I think that that's quite a common situation. We get into particular habits, whether it's creative or whether it's just trying to get to work. We get into a particular habit. We're knocked off course. And very often, of course, our original habit was the right way to do things and we go back. But often enough to be useful, we find that the new way of doing things is actually better all along. And if only we'd been forced into it, we would have discovered that years ago.VEDANTAM: One of the things I want to do in this conversation is try and tease apart when messiness can be our friend and when it can be our enemy. There's a reason many of us try and avoid messiness. It's because messiness causes delays. It causes disruptions. You're ending up - you end up being late for some things. And I'm wondering, as I read some of the examples in your book, whether it was the case that messiness helps people who actually have a base of preparation.So in other words, Keith Jarrett - he's a professional musician. He's very gifted. He's practiced, and he's learned the rules of how to play a proper piano for years and years and years. You place me before that piano, and what you're going to hear is not genius. What you're going to hear is just a mess. And the reason Keith Jarrett could adapt was because he had the base of knowing the rules to now say, I can throw out the rule book and create something brilliant.HARFORD: Yeah. I think that's absolutely right, Shankar. And is it OK - can I open a big bag of complexity science on you? Is that all right?VEDANTAM: Absolutely.(LAUGHTER)HARFORD: OK. So, basically, if you are trying to solve a very complex problem, we have various algorithms that try to solve this. And we - I'll come to the question of skill in a second. And the algorithms - computer algorithms searching for solutions in a very, very complex problem space, one possible approach is to look for step-by-step improvements. So that's the equivalent of skill. You are looking for step-by-step improvements to get from a bad solution to a better solution and a better one and a better one and a better one. And, eventually, you find the perfect solution.It turns out that way of looking at complex problems doesn't usually work. And the reason is the algorithm, as it looks for these small improvements, gets stuck. So these algorithms that search for solutions - they work much better if you add random shocks. Lots of different ways to do that. But, basically, for most of this kind of problem I'm discussing, a computer scientist would approach it by saying, we're going to have the step-by-step quest for improvements. But we're also going to have lots of randomness, lots of shocks. Just keep knocking the algorithm off course because then it won't get stuck.So now we come back to Keith Jarrett. He's like an algorithm that's searching for step-by-step improvements. He's a great musician. He's always looking for the perfect concert. Clearly, he's a very creative person 'cause he's improvising every night. But he always starts with a good piano - always. And then this one night where they say, you know what, Keith? Twelve of these notes don't work, and you should probably avoid most of that part. And that thing with the pedals - don't do that either. And - but then he sits down at the piano and then his skill kicks in, and he's looking for the incremental improvements. He's looking for a path to find great music. And he finds it. So, yes, you need skill. But the random shocks help.VEDANTAM: So for all the children who are listening to this podcast, I want to make clear that you're not saying that every time your parent tells you to clean up the room, you should just say, Tim Harford says I shouldn't.(LAUGHTER)HARFORD: Well, I have to say since writing the book, I have stopped asking my children to clear up their room.(LAUGHTER)HARFORD: And the reason is I read an amazing study that had been conducted by two British psychologists - Alex Haslam and Craig Knight. And what they did was - it wasn't anything to do with children, but it was to do with clean desk policies. They got people to sit down and work in artificial office environments. You know, we've created an office for you. Here are some tasks. Sort this paperwork. Do these puzzles. Handle these emails. Regular sort of knowledge, worker tasks, standard stuff. And they measured how well people did. And there were four different arms to this experiment. People were randomized into different arms.So in one case, the office was very, very minimalist. Just a chair, you know, pencil, paper, no distractions. People did fine in that. They didn't really like it on average - suits some people. The second approach - there were more decorations in the office. There were pot plants. There were posters, you know, some distractions. But it felt friendlier. People liked that more. But here's the real kicker to this research. The other two treatments, the other two parts of the research - first, the researchers said, you can organize this office how you like. Bring in the posters, send away the posters, put the pot plants where you like. Or if you want, we'll take the pot plants away for you - however you want. People loved that. They got so much more done. They felt happier. They were more productive - significantly more productive.In the fourth part of the trial, they did that. They said, arrange it however you like. And then the experimenter came in and said, oh, I'm sorry, this isn't appropriate for the experiment, and rearranged everything, put it back the way it was in the second part of the trial where people had been perfectly happy to work in these decorated offices. What they weren't happy with was having someone come in, override their decisions, take away their control, take away their autonomy. They felt physically sick. And they hated everything about it. They hated the work. They hated the space. They hated the company that was hosting the space. And they absolutely hated the experimenter.And after that, I made two vows to myself. Number one, if I'm ever a boss - nobody should make me a boss, by the way - if I'm ever a boss, I will never impose a clean desk policy. And number two, me and my wife - we kind of are the boss in our house. And I no longer ask my children to tidy their rooms. And you know what? We don't argue about it as much. And the rooms are kind of as messy as they ever were. It hasn't made any difference to the physical state of the rooms. There's just a little bit more peace.VEDANTAM: Tim Harford comes from Britain. We're having this chat in the United States. In both countries, there have been enormous political upheavals recently. When we come back, I'm going to ask Tim about messiness in public life and in democracy. Stay with us.(SOUNDBITE OF MUSIC)VEDANTAM: This is HIDDEN BRAIN. I'm Shankar Vedantam. With Brexit and the election of Donald Trump, Tim, we've had populist revolutions on both sides of the Atlantic that reject the advice of scientists and economists, experts in general. Democracy, of course, is famously messy. Could you remind people why it's such a good thing?HARFORD: Yes. I think, in both cases, I try to remind myself of Keith Jarrett's unplayable piano, actually. To me, the Brexit vote felt like an unplayable piano. The expert consensus in the U.K. was that this was not a very good idea. The British people disagreed with the experts. And so they voted to leave. And so I thought to myself, OK, well, this is going to pose all kinds of obstacles and all kinds of difficulties. In what ways might they actually turn to our advantage?So not to try to be foolishly optimistic in the face of disappointment, or what was disappointment for a lot of people, but to say, well, there is an opportunity in everything. What are the solutions that we look for going forward? And I think that's an important thing to bear in mind whenever we are faced with an obstacle. It's easy to get frustrated. But we should then say, OK, how is this going to make me stronger? How is this going to make my society stronger? What are the solutions that are going to come out of disappointment?VEDANTAM: Donald Trump ran a campaign that many people thought was extremely disordered. He didn't listen to the experts. He went off script all the time. He said things that people said would be sure to make him lose. Now, your book came out only a few weeks ago, but clearly he must have read an early draft.(LAUGHTER)HARFORD: I deny leaking the book to Donald Trump. However, you're absolutely right. There is a section in the book about Donald Trump's campaigning style that was written before he won the election. And it was published before he won the election. And I said, he is a master of using chaos to his advantage.One of the interesting parallels between Trump's campaign and the Brexit vote in the U.K. is that the Brexit vote in the U.K. - there were two different leave campaigns, and they hated each other. They were taking legal action against each other. They actively contradicted each other. This may sound familiar - I don't know. You would think that that's a disadvantage. Turns out to be an advantage because support - they were able to coalesce two different groups of supporters who wanted different things. And they just tuned in to the things they liked to hear.Meanwhile, the opposition - the remain vote in the U.K., the Democrats in the United States - they weren't sure what to criticize 'cause there was, you know, no coherence on the other side. So there was no target to aim at. Trump, by the way, is not the only person to use this. The use of chaos as a weapon works perfectly well in wartime. I talk about Erwin Rommel, the great German tank commander. It works in business. Jeff Bezos at Amazon explicitly targeted - we're going to move fast. We're going to make a mess. We don't understand what's going on. But at least Barnes & Noble are more confused than we are.(LAUGHTER)HARFORD: And as long as Barnes & Noble are more confused than we are, we're doing OK. He was very, very clear about this. And it has proved very effective.VEDANTAM: I want to talk about another example of messiness in public life. And I want you to watch this video which describes a very famous speech. And I want to ask the question right after it.(SOUNDBITE OF SPEECH, "I HAVE A DREAM")MARTIN LUTHER KING JR: We can never be satisfied as long as our children are stripped of their selfhood and robbed of their dignity by signs stating for whites only.(APPLAUSE)KING JR: We cannot be satisfied as long as a Negro in Mississippi cannot vote and a Negro in New York believes he has nothing for which to vote.VEDANTAM: So as you can see from that speech, Tim, Martin Luther King is reading from a script. And he has some - a nice line there, but it's a writerly line. But at a certain point in the speech, he decides to leave the script behind and turn his eyes to the audience. And here's what we see after that.(SOUNDBITE OF SPEECH, "I HAVE A DREAM")KING JR: I still have a dream. It is a dream deeply rooted in the American dream. I have a dream that one day this nation will rise up, live out the true meaning of its creed. We hold these truths to be self-evident that all men are created equal.(APPLAUSE)VEDANTAM: Talk about this for a second because you talk about the speech in the book. And you talk about this moment when King left his notes behind and essentially stepped off the ledge of what he knew and what he was certain about and just rode the wave of what the crowd was telling him and what he was feeling.HARFORD: He had prepared that speech meticulously the night before. He'd stayed up very late. He was operating on a number of constraints. It was quite a tight time schedule. The eyes of the world were on him - he knew that. There were also various political constraints about what was his response going to be to the civil rights legislation. Would he embrace it? Would he criticize it as not going far enough? Other civil rights leaders were making different claims. He had to balance all of those.So he took great care. The result was a speech that was well-crafted but a little bit lifeless. And so six or seven minutes into the speech, he looks down at the script and there's a line there that's terrible. And it must have been terrible when he wrote it. But he looked, and he realized it's not working. Now let us go back to our communities as members for the society for creative dissatisfaction. He doesn't say it.(LAUGHTER)HARFORD: He says instead, go back to Alabama.(SOUNDBITE OF SPEECH, "I HAVE A DREAM")KING JR: Go back to Alabama. Go back to South Carolina. Go back to Georgia. Go back to Louisiana. Go back to the slums and ghettos of our northern cities.HARFORD: And there's another pause. And the people behind him on the stage know he's not on the script anymore. And Mahalia Jackson, great gospel singer, yells out to him because she's heard him talking in church about this dream. She yells out, tell them about the dream, Martin. And then he starts to improvise. And that is when he touches the crowd.VEDANTAM: I understand that after writing this book, every time you find yourself in a mess, and you turn to your wife for comfort, she now reminds you that being in a mess is a good thing.HARFORD: Yeah.(LAUGHTER)HARFORD: Actually, just two days ago, Shankar, I was trying to check into a hotel at Dublin Airport at midnight, knowing that I was going to have to get up very early in the morning and fly and then come over to Washington to speak to you. You know, I - just desperate to get that little bit of sleep that I would be able to get.And when I showed up at the airport hotel, they didn't have my reservation. And they were full, so nowhere to sleep. And as I toured Dublin in a taxi looking for hotels that were open, I could just hear my wife's voice in my head saying, this is making you smarter, Tim. This is what you say. This is making you more creative.(LAUGHTER)HARFORD: So, you know, I've really sort of created a rod to beat myself with.(LAUGHTER)HARFORD: I think it is - it's important. Look, sometimes, bad news is just bad news. Sometimes, an obstacle is just an obstacle, and there is no good side to be found. But very often when things go wrong, we need to stop and say, well, what could come out of this? Brian Eno, a musician who I interviewed for the book - he worked with David Bowie, he worked with U2, Coldplay, an amazing producer - he described his creative strategy as - it's a little bit similar to being in an accident.When you're in an accident, you get that sudden feeling of you paying attention to everything because there's this tremendous sense of threat. Well, but that attention - the threat is bad. The accident is bad. But the attention is good. And so he puts the musicians he works with in stressful situations so they feel almost like they've been in an accident. And, suddenly, he has their attention.And I think it's worth bearing in mind even if the accident is a problem, even if the obstacle is a problem, the sudden attention, the sudden feeling of being present may lead us to some kind of solution. And that's worth hanging onto. When everything is perfect, when everything is tidy, we're on autopilot. And we're not necessarily living in the moment, we're not necessarily paying attention. And that's a problem for us.VEDANTAM: Tim Harford, thank you for joining me today.HARFORD: Thanks, Shankar.(APPLAUSE)VEDANTAM: HIDDEN BRAIN is produced by Maggie Penman, Jennifer Schmidt, Rhaina Cohen, Lucy Perkins, Parth Shah and Renee Klahr. Our supervising producer is Tara Boyle. Our unsung hero this week is an institution. We want to thank the NPR Foundation board of trustees, many of whom joined us for this taping for Tim Harford. They played a vital role in getting HIDDEN BRAIN off the ground - not just financial support but guidance and emotional support, as well. I'm so grateful. Next week, we'll continue our You 2.0 series with an episode about how to achieve your dreams - really. I'm Shankar Vedantam, and this is NPR. Transcript provided by NPR, Copyright NPR.