Inbox scrap 50

CHAPTER 1

The Cult of the Head Start

When I asked Garry Kasparov, perhaps the greatest chess player in history, to explain his decision process for a move, he told me, “I see a move, a combination, almost instantly,” based on patterns he has seen before. Kasparov said he would bet that grandmasters usually make the move that springs to mind in the first few seconds of thought. Klein studied firefighting commanders and estimated that around 80 percent of their decisions are also made instinctively and in seconds. After years of firefighting, they recognize repeating patterns in the behavior of flames and of burning buildings on the verge of collapse. When he studied nonwartime naval commanders who were trying to avoid disasters, like mistaking a commercial flight for an enemy and shooting it down, he saw that they very quickly discerned potential threats. Ninety-five percent of the time, the commanders recognized a common pattern and chose a common course of action that was the first to come to mind.

One of Klein’s colleagues, psychologist Daniel Kahneman, studied human decision making from the “heuristics and biases” model of human judgment. His findings could hardly have been more different from Klein’s. When Kahneman probed the judgments of highly trained experts, he often found that experience had not helped at all. Even worse, it frequently bred confidence but not skill.

Kahneman included himself in that critique. He first began to doubt the link between experience and expertise in 1955, as a young lieutenant in the psychology unit of the Israel Defense Forces. One of his duties was to assess officer candidates through tests adapted from the British army. In one exercise, teams of eight had to get themselves and a length of telephone pole over a six-foot wall without letting the pole touch the ground, and without any of the soldiers or the pole touching the wall.* The difference in individuals’ performances were so stark, with clear leaders, followers, braggarts, and wimps naturally emerging under the stress of the task, that Kahneman and his fellow evaluators grew confident they could analyze the candidates’ leadership qualities and identify how they would perform in officer training and in combat. They were completely mistaken. Every few months, they had a “statistics day” where they got feedback on how accurate their predictions had been. Every time, they learned they had done barely better than blind guessing. Every time, they gained experience and gave confident judgments. And every time, they did not improve. Kahneman marveled at the “complete lack of connection between the statistical information and the compelling experience of insight.” Around that same time, an influential book on expert judgment was published that Kahneman told me impressed him “enormously.” It was a wide-ranging review of research that rocked psychology because it showed experience simply did not create skill in a wide range of real-world scenarios, from college administrators assessing student potential to psychiatrists predicting patient performance to human resources professionals deciding who will succeed in job training. In those domains, which involved human behavior and where patterns did not clearly repeat, repetition did not cause learning. Chess, golf, and firefighting are exceptions, not the rule.

The difference between what Klein and Kahneman documented in experienced professionals comprised a profound conundrum: Do specialists get better with experience, or not?

In 2009, Kahneman and Klein took the unusual step of coauthoring a paper in which they laid out their views and sought common ground. And they found it. Whether or not experience inevitably led to expertise, they agreed, depended entirely on the domain in question. Narrow experience made for better chess and poker players and firefighters, but not for better predictors of financial or political trends, or of how employees or patients would perform. The domains Klein studied, in which instinctive pattern recognition worked powerfully, are what psychologist Robin Hogarth termed “kind” learning environments. Patterns repeat over and over, and feedback is extremely accurate and usually very rapid. In golf or chess, a ball or piece is moved according to rules and within defined boundaries, a consequence is quickly apparent, and similar challenges occur repeatedly. Drive a golf ball, and it either goes too far or not far enough; it slices, hooks, or flies straight. The player observes what happened, attempts to correct the error, tries again, and repeats for years. That is the very definition of deliberate practice, the type identified with both the ten-thousand-hours rule and the rush to early specialization in technical training. The learning environment is kind because a learner improves simply by engaging in the activity and trying to do better. Kahneman was focused on the flip side of kind learning environments; Hogarth called them “wicked.”

In wicked domains, the rules of the game are often unclear or incomplete, there may or may not be repetitive patterns and they may not be obvious, and feedback is often delayed, inaccurate, or both.

In the most devilishly wicked learning environments, experience will reinforce the exact wrong lessons. Hogarth noted a famous New York City physician renowned for his skill as a diagnostician. The man’s particular specialty was typhoid fever, and he examined patients for it by feeling around their tongues with his hands. Again and again, his testing yielded a positive diagnosis before the patient displayed a single symptom. And over and over, his diagnosis turned out to be correct. As another physician later pointed out, “He was a more productive carrier, using only his hands, than Typhoid Mary.” Repetitive success, it turned out, taught him the worst possible lesson. Few learning environments are that wicked, but it doesn’t take much to throw experienced pros off course. Expert firefighters, when faced with a new situation, like a fire in a skyscraper, can find themselves suddenly deprived of the intuition formed in years of house fires, and prone to poor decisions. With a change of the status quo, chess masters too can find that the skill they took years to build is suddenly obsolete.

According to Kasparov, “Today the free chess app on your mobile phone is stronger than me.” He is not being rhetorical.

“Anything we can do, and we know how to do it, machines will do it better,” he said at a recent lecture. “If we can codify it, and pass it to computers, they will do it better.”

Still, losing to Deep Blue gave him an idea. In playing computers, he recognized what artificial intelligence scholars call Moravec’s paradox: machines and humans frequently have opposite strengths and weaknesses.

There is a saying that “chess is 99 percent tactics.” Tactics are short combinations of moves that players use to get an immediate advantage on the board. When players study all those patterns, they are mastering tactics. Bigger-picture planning in chess—how to manage the little battles to win the war—is called strategy. As Susan Polgar has written, “you can get a lot further by being very good in tactics”—that is, knowing a lot of patterns—“and have only a basic understanding of strategy.”

Thanks to their calculation power, computers are tactically flawless compared to humans. Grandmasters predict the near future, but computers do it better. What if, Kasparov wondered, computer tactical prowess were combined with human big-picture, strategic thinking?

In 1998, he helped organize the first “advanced chess” tournament, in which each human player, including Kasparov himself, paired with a computer. Years of pattern study were obviated. The machine partner could handle tactics so the human could focus on strategy. It was like Tiger Woods facing off in a golf video game against the best gamers. His years of repetition would be neutralized, and the contest would shift to one of strategy rather than tactical execution. In chess, it changed the pecking order instantly. “Human creativity was even more paramount under these conditions, not less,” according to Kasparov. Kasparov settled for a 3–3 draw with a player he had trounced four games to zero just a month earlier in a traditional match. “My advantage in calculating tactics had been nullified by the machine.” The primary benefit of years of experience with specialized training was outsourced, and in a contest where humans focused on strategy, he suddenly had peers.

A few years later, the first “freestyle chess” tournament was held. Teams could be made up of multiple humans and computers. The lifetime-of-specialized-practice advantage that had been diluted in advanced chess was obliterated in freestyle. A duo of amateur players with three normal computers not only destroyed Hydra, the best chess supercomputer, they also crushed teams of grandmasters using computers. Kasparov concluded that the humans on the winning team were the best at “coaching” multiple computers on what to examine, and then synthesizing that information for an overall strategy. Human/Computer combo teams—known as “centaurs”—were playing the highest level of chess ever seen. If Deep Blue’s victory over Kasparov signaled the transfer of chess power from humans to computers, the victory of centaurs over Hydra symbolized something more interesting still: humans empowered to do what they do best without the prerequisite of years of specialized pattern recognition.

In 2014, an Abu Dhabi–based chess site put up $20,000 in prize money for freestyle players to compete in a tournament that also included games in which chess programs played without human intervention. The winning team comprised four people and several computers. The captain and primary decision maker was Anson Williams, a British engineer with no official chess rating. His teammate, Nelson Hernandez, told me, “What people don’t understand is that freestyle involves an integrated set of skills that in some cases have nothing to do with playing chess.” In traditional chess, Williams was probably at the level of a decent amateur. But he was well versed in computers and adept at integrating streaming information for strategy decisions. As a teenager, he had been outstanding at the video game Command & Conquer, known as a “real time strategy” game because players move simultaneously. In freestyle chess, he had to consider advice from teammates and various chess programs and then very quickly direct the computers to examine particular possibilities in more depth. He was like an executive with a team of mega-grandmaster tactical advisers, deciding whose advice to probe more deeply and ultimately whose to heed. He played each game cautiously, expecting a draw, but trying to set up situations that could lull an opponent into a mistake.

In the end, Kasparov did figure out a way to beat the computer: by outsourcing tactics, the part of human expertise that is most easily replaced, the part that he and the Polgar prodigies spent years honing.

After Susan succeeded in her first test, National Geographic TV turned the truck around to show the other side, which had a diagram with pieces placed at random. When Susan saw that side, even though there were fewer pieces, she could barely re-create anything at all.

That test reenacted an experiment from 1973, in which two Carnegie Mellon University psychologists, William G. Chase and soon-to-be Nobel laureate Herbert A. Simon, repeated the De Groot exercise, but added a wrinkle. This time, the chess players were also given boards with the pieces in an arrangement that would never actually occur in a game. Suddenly, the experts performed just like the lesser players. The grandmasters never had photographic memories after all. Through repetitive study of game patterns, they had learned to do what Chase and Simon called “chunking.” Rather than struggling to remember the location of every individual pawn, bishop, and rook, the brains of elite players grouped pieces into a smaller number of meaningful chunks based on familiar patterns. Those patterns allow expert players to immediately assess the situation based on experience, which is why Garry Kasparov told me that grandmasters usually know their move within seconds. For Susan Polgar, when the van drove by the first time, the diagram was not twenty-eight items, but five different meaningful chunks that indicated how the game was progressing.

Chunking helps explain instances of apparently miraculous, domain-specific memory, from musicians playing long pieces by heart to quarterbacks recognizing patterns of players in a split second and making a decision to throw. The reason that elite athletes seem to have superhuman reflexes is that they recognize patterns of ball or body movements that tell them what’s coming before it happens. When tested outside of their sport context, their superhuman reactions disappear.

over the course of your life, you’ve learned patterns of words that allow you to instantly make sense of the second arrangement, and to remember it much more easily. Your restaurant server doesn’t just happen to have a miraculous memory; like musicians and quarterbacks, they’ve learned to group recurring information into chunks.

For more than fifty years, psychiatrist Darold Treffert studied savants, individuals with an insatiable drive to practice in one domain, and ability in that area that far outstrips their abilities in other areas. “Islands of genius,” Treffert calls it.* Treffert documented the almost unbelievable feats of savants like pianist Leslie Lemke, who can play thousands of songs from memory. Because Lemke and other savants have seemingly limitless retrieval capacity, Treffert initially attributed their abilities to perfect memories; they are human tape recorders. Except, when they are tested after hearing a piece of music for the first time, musical savants reproduce “tonal” music—the genre of nearly all pop and most classical music—more easily than “atonal” music, in which successive notes do not follow familiar harmonic structures. If savants were human tape recorders playing notes back, it would make no difference whether they were asked to re-create music that follows popular rules of composition or not. But in practice, it makes an enormous difference. In one study of a savant pianist, the researcher, who had heard the man play hundreds of songs flawlessly, was dumbstruck when the savant could not re-create an atonal piece even after a practice session with it. “What I heard seemed so unlikely that I felt obliged to check that the keyboard had not somehow slipped into transposing mode,” the researcher recorded. “But he really had made a mistake, and the errors continued.” Patterns and familiar structures were critical to the savant’s extraordinary recall ability. Similarly, when artistic savants are briefly shown pictures and asked to reproduce them, they do much better with images of real-life objects than with more abstract depictions.

It took Treffert decades to realize he had been wrong, and that savants have more in common with prodigies like the Polgar sisters than he thought. They do not merely regurgitate. Their brilliance, just like the Polgar brilliance, relies on repetitive structures, which is precisely what made the Polgars’ skill so easy to automate.

“There are so many layers of thinking,” he said. “We humans sort of suck at all of them individually, but we have some kind of very approximate idea about each of them and can combine them and be somewhat adaptive. That seems to be what the trick is.”

the bigger the picture, the more unique the potential human contribution. Our greatest strength is the exact opposite of narrow specialization. It is the ability to integrate broadly. According to Gary Marcus, a psychology and neural science professor who sold his machine learning company to Uber, “In narrow enough worlds, humans may not have much to contribute much longer. In more open-ended games, I think they certainly will. Not just games, in open ended real-world problems we’re still crushing the machines.”

The progress of AI in the closed and orderly world of chess, with instant feedback and bottomless data, has been exponential. In the rule-bound but messier world of driving, AI has made tremendous progress, but challenges remain. In a truly open-world problem devoid of rigid rules and reams of perfect historical data, AI has been disastrous. IBM’s Watson destroyed at Jeopardy! and was subsequently pitched as a revolution in cancer care, where it flopped so spectacularly that several AI experts told me they worried its reputation would taint AI research in health-related fields. As one oncologist put it, “The difference between winning at Jeopardy! and curing all cancer is that we know the answer to Jeopardy! questions.” With cancer, we’re still working on posing the right questions in the first place.

When we know the rules and answers, and they don’t change over time—chess, golf, playing classical music—an argument can be made for savant-like hyperspecialized practice from day one. But those are poor models of most things humans want to learn.

When narrow specialization is combined with an unkind domain, the human tendency to rely on experience of familiar patterns can backfire horribly—like the expert firefighters who suddenly make poor choices when faced with a fire in an unfamiliar structure.

Chris Argyris, who helped create the Yale School of Management, noted the danger of treating the wicked world as if it is kind. He studied high-powered consultants from top business schools for fifteen years, and saw that they did really well on business school problems that were well defined and quickly assessed. But they employed what Argyris called single-loop learning, the kind that favors the first familiar solution that comes to mind. Whenever those solutions went wrong, the consultant usually got defensive. Argyris found their “brittle personalities” particularly surprising given that “the essence of their job is to teach others how to do things differently.”

Psychologist Barry Schwartz demonstrated a similar, learned inflexibility among experienced practitioners when he gave college students a logic puzzle that involved hitting switches to turn light bulbs on and off in sequence, and that they could play over and over. It could be solved in seventy different ways, with a tiny money reward for each success. The students were not given any rules, and so had to proceed by trial and error.* If a student found a solution, they repeated it over and over to get more money, even if they had no idea why it worked. Later on, new students were added, and all were now asked to discover the general rule of all solutions. Incredibly, every student who was brand-new to the puzzle discovered the rule for all seventy solutions, while only one of the students who had been getting rewarded for a single solution did. The subtitle of Schwartz’s paper: “How Not to Teach People to Discover Rules”—that is, by providing rewards for repetitive short-term success with a narrow range of solutions.

doctors and nurses do not automatically find out what happens to a patient after their encounter. They have to find ways to learn beyond practice, and to assimilate lessons that might even contradict their direct experience.

If the amount of early, specialized practice in a narrow area were the key to innovative performance, savants would dominate every domain they touched, and child prodigies would always go on to adult eminence. As psychologist Ellen Winner, one of the foremost authorities on gifted children, noted, no savant has ever been known to become a “Big-C creator,” who changed their field.

There are domains beyond chess in which massive amounts of narrow practice make for grandmaster-like intuition. Like golfers, surgeons improve with repetition of the same procedure. Accountants and bridge and poker players develop accurate intuition through repetitive experience. Kahneman pointed to those domains’ “robust statistical regularities.” But when the rules are altered just slightly, it makes experts appear to have traded flexibility for narrow skill. In research in the game of bridge where the order of play was altered, experts had a more difficult time adapting to new rules than did nonexperts. When experienced accountants were asked in a study to use a new tax law for deductions that replaced a previous one, they did worse than novices. Erik Dane, a Rice University professor who studies organizational behavior, calls this phenomenon “cognitive entrenchment.” His suggestions for avoiding it are about the polar opposite of the strict version of the ten-thousand-hours school of thought: vary challenges within a domain drastically, and, as a fellow researcher put it, insist on “having one foot outside your world.”

Scientists and members of the general public are about equally likely to have artistic hobbies, but scientists inducted into the highest national academies are much more likely to have avocations outside of their vocation. And those who have won the Nobel Prize are more likely still. Compared to other scientists, Nobel laureates are at least twenty-two times more likely to partake as an amateur actor, dancer, magician, or other type of performer. Nationally recognized scientists are much more likely than other scientists to be musicians, sculptors, painters, printmakers, woodworkers, mechanics, electronics tinkerers, glassblowers, poets, or writers, of both fiction and nonfiction. And, again, Nobel laureates are far more likely still. The most successful experts also belong to the wider world. “To him who observes them from afar,” said Spanish Nobel laureate Santiago Ramón y Cajal, the father of modern neuroscience, “it appears as though they are scattering and dissipating their energies, while in reality they are channeling and strengthening them.” The main conclusion of work that took years of studying scientists and engineers, all of whom were regarded by peers as true technical experts, was that those who did not make a creative contribution to their field lacked aesthetic interests outside their narrow area. As psychologist and prominent creativity researcher Dean Keith Simonton observed, “rather than obsessively focus[ing] on a narrow topic,” creative achievers tend to have broad interests. “This breadth often supports insights that cannot be attributed to domain-specific expertise alone.”

Those findings are reminiscent of a speech Steve Jobs gave, in which he famously recounted the importance of a calligraphy class to his design aesthetics. “When we were designing the first Macintosh computer, it all came back to me,” he said. “If I had never dropped in on that single course in college, the Mac would have never had multiple typefaces or proportionally spaced fonts.” Or electrical engineer Claude Shannon, who launched the Information Age thanks to a philosophy course he took to fulfill a requirement at the University of Michigan. In it, he was exposed to the work of self-taught nineteenth-century English logician George Boole, who assigned a value of 1 to true statements and 0 to false statements and showed that logic problems could be solved like math equations. It resulted in absolutely nothing of practical importance until seventy years after Boole passed away, when Shannon did a summer internship at AT&T’s Bell Labs research facility. There he recognized that he could combine telephone call-routing technology with Boole’s logic system to encode and transmit any type of information electronically. It was the fundamental insight on which computers rely. “It just happened that no one else was familiar with both those fields at the same time,” Shannon said.

In 1979, Christopher Connolly cofounded a psychology consultancy in the United Kingdom to help high achievers (initially athletes, but then others) perform at their best. Over the years, Connolly became curious about why some professionals floundered outside a narrow expertise, while others were remarkably adept at expanding their careers—moving from playing in a world-class orchestra, for example, to running one. Thirty years after he started, Connolly returned to school to do a PhD investigating that very question, under Fernand Gobet, the psychologist and chess international master. Connolly’s primary finding was that early in their careers, those who later made successful transitions had broader training and kept multiple “career streams” open even as they pursued a primary specialty. They “traveled on an eight-lane highway,” he wrote, rather than down a single-lane one-way street. They had range. The successful adapters were excellent at taking knowledge from one pursuit and applying it creatively to another, and at avoiding cognitive entrenchment. They employed what Hogarth called a “circuit breaker.” They drew on outside experiences and analogies to interrupt their inclination toward a previous solution that may no longer work. Their skill was in avoiding the same old patterns. In the wicked world, with ill-defined challenges and few rigid rules, range can be a life hack.

Pretending the world is like golf and chess is comforting. It makes for a tidy kind-world message, and some very compelling books.

🔮 The Cosmos

Graph View

Backlinks