The Voltage Effect with John List

Silver Bullets and Voltage Drops

“A voltage drop is something I think of as the first law of scaling, which is that your result at scale will be significantly less than your initial result that you found in your small-scale research study. So, essentially, it's turning a mountain into a molehill.”

Avoiding the Most Common Mistake in Scaling

“Make sure your idea has voltage to begin with. And what I say is to replicate it two or three or four times. And when you replicate two, three or four times, what you find is the truth in the end. A false positive won't hold up to that kind of scrutiny.”

How Behavioral Forces Influence Our Ability to Scale Ideas

“Once somebody has an idea or a notion that their innovation will work, I think confirmation bias kicks in then. And they think, “Well, I'm going to look around every corner. And if I see something I like that confirms my belief, I'm going to grab onto it. If I see something that is at odds with it, I'm going to say, "Well, that's just an oddity, or that's not the truth.”” And that tends to exacerbate this idea, like the bandwagon effect or confirmation bias tends to roll us down the wrong path in many cases.”

You’ve Got to Know When to Fold ‘Em…

“I think there are many ideas that we should leave or at least leave for a short time and maybe come back when you have a different innovation that you can put on top of that idea. But in the end, if you get dataset after dataset after dataset, that points to looking at the idea as a turkey, it's not going to turn into anything good just because you wish it to.”

Bigger Doesn’t Mean Better

“There are many cases where diseconomies of scale will prohibit an idea from taking off. What's really interesting is that the very innovative and really keen people know it, and they don't choose ideas that have diseconomies of scale, because they know they can't be scaled any larger than some very small Petri-dish setting.”

Brooke: Hello, everyone, and welcome to the podcast of The Decision Lab, a socially-conscious applied research firm that uses behavioral science to improve outcomes for all of society. My name is Brooke Struck, Research Director at TDL. And I'll be your host for the discussion. My guest today is John List, Distinguished Professor of Economics at the University of Chicago, Chief Economist at Lyft, and author of The Voltage Effect. In today's episode, we'll be talking about scaling, why companies succeed or fail, and growing; early signals to watch for, and how to ride the wave of promising ideas. John, thanks for joining us.

John: Thanks so much for having me.

Brooke: Please tell us a bit about yourself and why you've written this book. Why is scaling keeping people up at night these days?

John: Sure, absolutely. So, my scaling story starts back in 2010, when I started a pre-K program in Chicago Heights, which is a suburb just south of Chicago. And I started it with Roland Fryer and Steven Levitt, a few people who your audience might know. And the general idea was, can we create a pre-K program for three, four and five-year olds to not only understand the education production function, but also create a program that can be used worldwide.

So, when you start up your own pre-K program, there's a lot of work to it. From soup to nuts, I was on the ground for years, setting up the teachers, setting up the classroom, setting up the building, etc. We started the program about a decade ago. And after the first two years, the results came back. And when I say results, I mean things like standardized test scores, measures of executive function skill enhancement, etc. And the results came back great. You can imagine that when you create a program that you think can change the world and the initial results come back looking so marvelous, you want to run out and tell everyone about your shiny toy. That's exactly what I did. I ran out and talked to policymakers and other academics about how great our new program was. That's what happened then.

Brooke: Yeah.

John: That's what happened then.

Brooke: I gather there's a somewhat ominous end to this tale.

John: There is a somewhat ominous answer. I was confronted immediately with the notion that, "Look, John, you found a great result in the petri dish, but that will never scale." And I thought, "Wait a second here. I did everything right. I hired teachers, like the school district would typically hire teachers. I had a hands-off approach. We randomly chose students from the Chicago Heights community to be part of our group. Why do you think my program won't scale?"

The typical response from the experts was, "It doesn't have the silver bullet." And then, I would push a little bit and I would say, "What do you mean by silver bullet?" And there would be some humming and hawing. And there never really was a consistent response, short of, "John, you will have a voltage drop."

Brooke: All right, that's a great pivot here to getting into the real substance of the book. And thank you for opening with that story. One of the things that I appreciated in reading the book is the number of stories that you've got throughout it that really put flesh on the bones. It's really, really helpful in that respect. So, you talked about the science of scaling and throughout, you used this analogy of voltage, and you used that to help readers think through which ideas will scale and identify what key markers they should be looking for in this kind of thing.

So, I'd like to start with this idea of a voltage drop, which you just talked about in the story. What is a voltage drop?

John: Sure. In short, a voltage drop is turning a mountain into a molehill. So, what do I mean by that? If I have a mentoring program, for example, that I pay mentors $100 per student to, let's say, train students on enhancing their brain activity toward taking an IQ test. That petri dish test might lead to something like five increased IQ points from that, say, three-month long program.

If I then wanted to scale that up, and do it for not only, let's say, 100 or 200 kids, but do it for 100,000 or 200,000 kids, a voltage drop is something I think of as the first law of scaling, which is your result at scale will be significantly less than your initial result that you found in your small-scale research study. So, essentially, it's turning a mountain into a molehill.

Brooke: Does that mean that the second law of scaling is, don't talk about voltage drops?

John: Exactly. The second law of scaling is, if you want your idea to get adopted and scaled, don't talk about it.

Brooke: Got you.

John: Because everyone believes that, "Well, that looks like a great idea." 

Now in my research, I was really taken aback by the response from people. So, I wanted to explore deeper about this thing I call now the ‘science of using science’. In  the early part of my career, I developed field experiments to explore and test theories in the field to explore, through an economic lens, why do people behave the way they do in the real world? Now, for me, this research agenda represented a significant pivot.

Because now I'm exploring - after you find, say, a great idea or a great policy innovation, what is the science behind scaling that idea? Is it the silver bullet, like the experts argued it should be? In the book, I argued it's quite different from that. And the differences stem from a bunch of empirical work that I've done in the last decade along with a bunch of theoretical work that we've done. And from all of that, I think we have a much stronger sense of what the science of using science is about.

Brooke: So, in the first part of your book, you dived into some of the early signs and signals that you've identified through the research you were just talking about, that an idea might be particularly susceptible to a voltage drop at scale. What are some of the signs that we should be looking for?

John: Absolutely. So, I want people to think about this problem not as a silver bullet problem, but more like an Anna Karenina problem. You might remember that the very famous first line in Anna Karenina, went  something like “Happy families are all alike, each unhappy family is unhappy in its own way.”

So, I want you to think now about scaling. Every scalable idea is alike, every unscalable idea is unscalable in its own way. But there are five vital signs that we should look for whenever we want to think about scaling an idea or a policy. And vital sign number one is what I would say, is a false positive or it never had voltage in the first place. So, here, I want to turn back the clock to the mid-1980s here in the United States. I was in high school. And a group of outsiders came into our high school with a program called, ‘Just Say No’. Now what they were trying to do is take on the rising drug epidemic in America, which was teens were using drugs. This was something that the First Lady Nancy Reagan used as her hallmark. This was going to be her mark on helping society, to stop teens from using drugs. I can still remember when they came in, the officials came in and gave the education program. I looked at my teacher and said, "Look, I don't do drugs. I have some friends who do, and there's no way that this program will work. This is crazy." And my teacher said, "Well, there's actually research behind it. And the research suggests that it works." Now, through my interest now in scaling, I went back to the original research that Nancy Reagan was using, it turned out to be a study in Honolulu. That was with roughly 1,700 kids that reported a result that showed that the ‘Just Say No’ educational program worked. It worked to curb drug use.

Now, the federal government used that, spent millions and millions of dollars and millions of millions of hours pushing this program. In the end, it turned out to just be one fat false positive, because that initial result was just an outlier. It was a statistical error that we took to be the truth. But in the end, it really was simply a false positive, or the idea never had voltage to begin with.

Now, when you look in policy circles and in businesses, there is a wealth of examples where people just scale too quickly, because what happens is, they read an academic paper, they think that's the truth. They don't really understand that there's a statistical model behind it. And 5% of the time, it's actually going to be an error. And it might be a small sample. So, it might even be a higher false positive rate than 5%. In many cases, it might be 30% or 40%. But people read an academic journal article and say, "It was peer reviewed, so it must be the truth." So, that's vital sign number one, is make sure your idea has voltage to begin with. And what I say in the book is to replicate it two or three or four times. And when you replicate two, three or four times, what you find is the truth in the end. A false positive won't hold up to that kind of scrutiny.

Brooke: Yeah, I like that you also got some percentages in there. Let's talk a bit about the numbers. So, if your threshold for statistical significance is 0.05, that sounds like boring stuff, but let's put that into really concrete terms. That means that out of... as a venture capitalist, let's say, if people are using that confidence interval or that confidence threshold, then for that venture capitalist, that means one out of 20 ideas coming across your desk is just a false positive.

In the optimistic scenario, as you mentioned, a lot of times the studies are not even demonstrating statistical significance at that level, it can be a much higher percent that are these spurious false positives that are coming across the desk that are making their way out as a pitch.

John: Exactly.

Brooke: Beyond false positives... sorry, go ahead.

John: Yeah, that's exactly right, Brooke. And another thing is, that once somebody has an idea or a notion that their innovation will work, I think confirmation bias kicks in then. And they think, "Well, I'm going to look around every corner. And if I see something I like that confirms my belief, I'm going to grab onto it. If I see something that is at odds with it, I'm going to say, "Well, that's just an oddity, or that's not the truth." And that tends to exacerbate this idea, like the bandwagon effect or confirmation bias tends to roll us down the wrong path in many cases.

Brooke: So, if we do find signs of trouble and if, for instance, we look at one initial study in the petri dish, things look really exciting, and we replicate that. And the second time, things don't look nearly as exciting. As you mentioned, confirmation bias kicks in in those instances and so do bandwagon effects where we say, "Well, the second study is probably spurious, right?” That's the outlier, the one we don't like. But if we find signs of trouble, what is it that we should do?

John: Yeah, I think part of it is you have to think about tweaking your idea and exploring how variations of your idea might work. There are other cases where you just need to quit. In many cases  I talked about in the second half of the book, society tells us that winners never quit. And I think that's a big mistake. I think there are many ideas that we should leave or at least leave for a short time and maybe come back when you have maybe a different innovation that you can put on top of that idea. But in the end, if you get dataset after dataset after dataset, that points to looking at the idea as a turkey, it's not going to turn into anything good just because you wish it to.

Brooke: Let's shift to another behavioral aspect here. When you've confronted companies that you've worked with and organizations that you've worked with, with the signs of trouble, how have they actually reacted?

John: Now that's a good question. So, when I think about other vital signs after a false positive, there might be spillovers in the marketplace. So, that's what I call vital sign number four in the first part of the book. And essentially, what that means is that when you put forward the innovation or policy or idea, there are actually market effects that occur that can potentially undo your initial idea. And this happened to me actually at Uber.

So, I used to be the Chief Economist at Uber. And there we had an idea that we should try to raise the pay for drivers, and that's through the rate card. So, what you do is you change the amount of money that each driver is paid per mile per minute. And some of my colleagues at Uber ended up looking at this data and wrote an academic paper on it. Essentially, what happened is, even though Uber raised the rate card, drivers ended up responding by both driving more and new drivers came in because they saw the higher rate card.

Those two features ended up making the initial rate card increase neutered, essentially, the wages did not increase at all for drivers. So, that particular idea, Uber was trying to do a good thing. They were trying to raise the wages for drivers, but the market dynamics worked out to where the market undid it. And in that case, the executive team at Uber said, "Look, that's not a particularly good way to try to raise wages. So, we're going to pull back and try something new." 

Now, there are a lot of examples like that, where a really good executive or a really good leader will see that the signs really aren't very good anymore, and they will end up pivoting. Another example that I used in the book is trying to figure out if your idea is scalable, or is it revolving around a human who can't scale. 

And this is a chapter that I call, is it the chef or the ingredients? So, there I talked about Jamie Oliver and his famous restaurant, it scaled very quickly. But it ended up failing in the end because the magic was really a few people within the restaurant. And what I've learned in my research is that humans don't scale very well. So, if you want to scale a restaurant, for example, and the chef is the secret sauce, that's not going to scale. But if the ingredients are the secret sauce, think about Domino's in America, we have the Domino's Pizza, where it's the ingredients, the ingredients will scale.

Now, something in between there that you can think about, well, sometimes scale or sometimes not scale, could be say, a wood oven for pizza. Sometimes you have this magical oven that creates a great tasting pizza. If that oven can't be transported to other kitchens in its same form, it won't scale. So, the idea is to figure out what is the secret sauce behind your innovation and essentially, what are the negotiables and non-negotiables, and can we have those at scale in the same way we have them in the Petri dish? If we can, then this particular vital sign ends up working because it's scalable. If you think about Chicago Heights now, my program that I'm very proud of that we talked about earlier, it's one thing to hire 20 teachers, but if you have to hire 20,000 really good teachers in Chicago, that's a whole different thing. So, if teachers end up being the secret sauce for why my program in Chicago Heights worked, then we need to be very careful about telling people to scale it unless we think we can hire 20,000 other really good teachers like we had in Chicago Heights.

Brooke: Right. So, for instance, if we think about teachers who are potentially going above and beyond just their formal responsibilities in the classroom, these are people who are probably more mission-driven than job security-driven, or there are just lots of different reasons that people get into teaching. It might be that, of the pool of available teachers out there, there are only so many who are driven by the right things that when they're put into the propitious environment, they will exhibit the behaviors we want them to exhibit.

But once you try to scale from 200 to 20,000, all of a sudden, you don't have as much of a rich supply anymore. You've got teachers who have different philosophies, different priorities, these kinds of things, who aren't as compatible with ecosystems, is that right?

John: No, you're 100% right. And you bring up a great point. So, if you want to scale this type of program, first of all, you need to make sure you understand the non-negotiables. And let's talk about teachers, like you mentioned. There's only a certain size pool of teachers. And if the pool isn't 20,000 teachers, what do you have to do? You either have to get lower quality teachers, which will lead to a voltage drop in the classroom, or you need to expand your budget and you need to enhance wages.

Maybe instead of paying teachers $40,000 or $50,000 a year, you have to pay them $200,000 per year to bring a teacher from Wall Street, for example, into the classroom. Now, if you do that, you might not get a voltage drop on the benefit side because you're still hiring great teachers. But guess what? There's still a voltage effect because of the cost side. And that's something that we end up not paying careful attention to, in many cases, is that to maintain high voltage on the benefit side, if we have to pay increasing costs, or what economists called, it has ‘diseconomies of scale’. So, each additional teacher we have to pay a little bit more for, that's not an idea that's scalable now, because it's not scalable. Because we should care both about benefits and costs, or "What are the net benefits of my innovation?" So, we have to pay attention to both sides of that equation.

Brooke: It's interesting that you bring up this term diseconomies of scale. Of course, economies of scale, that's in the popular lexicon. That's the jargon, you can toss that around. And people know what you're talking about, even if they don't really understand the concept all that well. But diseconomies of scale, all of a sudden, this seems like you're speaking a foreign language. You're coming from a completely different perspective. How much does that reflect the inherent value that we put on scaling for its own sake in our society?

John: No, I think that's a good point. When I look at some of the most innovative and profitable companies in the world, one thread that connects them all is that at one point in time, they have taken advantage of economies of scale. And think about Amazon. Once they reached this  economies of scale idea and understood it,  they got really, really low on the average total cost function.

But Amazon now reaches an average cost per product that is much lower than many other firms have a chance of doing. You look at what Elon Musk is doing with SpaceX and with Tesla, it's all built on economies of scale. And you're right, the lexicon is, "Let's leverage economies of scale and grow, grow, grow, grow, grow." But there are many cases where diseconomies of scale will prohibit an idea from taking off. What's really interesting is that, the very innovative and really keen people know that, and they don't choose ideas that have diseconomies of scale because they know they can't be scaled any larger than some very small petri dish setting.

Brooke: That raises a really interesting thought for me about climate change, actually. One of the challenges that we're facing is that coal and gasoline and other fossil fuels transport really, really well over space and they store really well through time, which allows us to really benefit from economies of scale. You can build the same car engine. You can build the same power plants anywhere in the world. And if they're fueled by fossil fuels, you basically get the same results. But electricity is not like that. Electricity is terrible to transport.

You think about these high voltage lines from massive hydroelectric dams, certainly here in the north of Quebec. The lousiness of that system is terrible, so much of the electricity just evaporates into the air before it ever gets anywhere that we can use it. And perhaps more visible than our current social conversations, is around the challenges of batteries. That electricity also doesn't store very well through time. And so, what we're seeing is the diseconomies of scale of electricity. That means, potentially, we need to think about different models of innovation for climate change than we've used before. This idea that like, one or two moonshot innovations are going to go viral, and they will solve the same problem all around the world, is maybe the wrong way to think about how we're going to tackle climate change, at least, the portion that has to do with electrification.

That may be because those innovations are not portable, they will suffer from these voltage drops, and I use that pun very intentionally in this instance. Electricity suffers from diseconomies of scale where fossil fuels got to enjoy those economies of scale. So, in that instance, we would need to think about much more local forms of innovation, need to think about much more local production of electricity, this kind of thing. Thank you for providing me with the conceptual tools to come to grips with a problem that's been in my mind, but that I couldn't express clearly until now.

So, we've talked about some of these signs of trouble, the vital signs that you've been talking about. And there are a couple that... or a few that you've talked about so far today. There were five total in the book. If we check all five of those vital signs, and we're good along those five dimensions, does that mean we're in the clear?

John: Not necessarily. So, I think about it in two steps. I think the first stage or first step is to say, "Do I have an idea that can scale?" And that's the five vital signs. Does it pass the five hurdles? Okay, let's say it does. And now I have this idea that has the features or the signatures of something that's going to be great at scale and change the world.

The way I think about it is, once you've launched that rocket ship, so you've launched the correct rocket ship from the first part of the book, the second part of the book is all about, "Can you maintain high voltage around your ideas?" In there, I do something called the four little secrets of high voltage. So, I begin by talking about incentives. When an economist says incentives, typically people roll their eyes and say, "Here he goes again, he's going to say, 'Pay people more money and the world would be a better place.'" That's actually not a very scalable incentive, cash. Something that is scalable is to think about non-pecuniary rewards, things like following social norms, things like humans like losses, or they actually hate losses much more than they like gains. This is what Kahneman and Tversky taught us years ago. So, my research around incentives has been going on for 25 years or so.

And this first chapter about maintaining voltage - a little secret to maintaining high voltage really stems from my work that shows, for example, you just mentioned the environment and how we can do well by the environment. I did a large-scale door-to-door field experiment where I sent canvassers to thousands and thousands of doors to try to get households to adopt CFLs. So, we're trying to get them to shift out of incandescent to CFLs. And what we found there was something really, really instructive. We found that to convince a person to buy the first package of CFLs, what worked was a social norm. So, a simple statement like, "Do you know 70% of your neighbors already have a CFL in their house? Maybe you should want one too."

That was very, very effective to get a household to adopt the new technology. Now, to get them to make deeper purchases, the social norm no longer works. That's where good hard cash did work. But my point here is that, if we think of innovative ways to use both pecuniary and non-pecuniary incentives, it will be that combination, especially when they have synergistic effects, like in my example, that will lead to really, really high voltage at scale.

Brooke: I think that there's also an oversimplification often when we talk about monetary incentives, that more equals better. The structure of monetary incentives is also really important. So, you talked earlier about understanding what the secret sauce is of your idea, understanding what it is that really makes your idea work, and what the negotiables and non-negotiables are.

That's really important as well, when it comes to the conversation about incentives, because you’ve got to be really loading up those incentives on the stuff that actually creates that emergent value.

John: No, that's 100%. It's not only about the level and type of incentive, but it's also about the institutional features and the structure around that incentive. You're 100% correct.

Brooke: Are there some ideas that you've seen that showed early on that they would scale, but then they tailed off later? Not necessarily because they eventually fell into one of those five pitfalls, right, that those early vital signs turned out to be a false reading, but that sustaining the rocket ship in flight tailed off and that the initial high voltage was a real reading, but that it wasn't able to sustain us at scale.

John: No, absolutely. So, one example is what happens at Lyft every day. So there, what we're after is to... well, in today's market, what we're after is to try to induce more drivers to come online because we have a supply problem. A lot of drivers are very apprehensive to come back in this time of COVID. So, we have a program that attempts to acquire new drivers.

Now, what you see in that program is advertisements on Facebook, advertisements on Google, etc.. What you find is that the initial dollars into those advertisements end up having pretty high voltage. But then, there is a serious drop that economists call ‘diminishing marginal returns’. And in many cases, when you have investments or ideas, you have to take great care to think through, "Is this a case that has increasing marginal returns or diminishing marginal returns?"

Now, in that case, then it behooves you to do something which I call, thinking on the margin. And the example is just crystal clear with Facebook and Google. When I looked at our data the first time at Lyft, what I saw was that we were spending three times more on Facebook ads to get the last driver to come in than we were spending on Google ads. So, you say, "Well, wait a second. Does this make sense?" And there's a rule in economics that says, "On the margin, the last dollar you spend on every input should lead to the same level of output." That's called optimization on the margin. Nearly every firm or organization that I've been a part of, all the way back to the White House, where I noticed that we weren't thinking on the margin in our multibillion-dollar programs that we were rolling out, we were not thinking on the margin. I saw it right in front of me here at Lyft, where if we simply moved some of the ad money from Facebook ads to Google ads, we would; a)get more drivers and b) we'd spend less money.

So, I think it's those cases in many of our ideas that we don't realize. Our minds think in a linear way that everything's going to be linear, but we don't realize that there are significant nonlinearities in many things we do. And in many cases, there might be a very large, diminishing return on some of our investments. And I think those are the cases that you need to pivot, think about a different way of doing it. Or in the Lyft and Uber case, just simply shift money from Facebook ads to Google ads.

Brooke: Sounds interesting, that makes me think of this dystopian situation where it's like, you start innovating on one thing, you have that early voltage in the petri dish, you're able to create and capture some value there. Once the voltage starts dropping, you just like, you jump to the next thing. You say, "Okay, well, I've tapped out that market, now I need to go to a different platform and put ads there because 'll be able to skim the cream off of a new container in this second social network, and once I've skimmed that cream, well, then I'll just jump to another one."

So then, you put ads on Instagram and here and there and everywhere. Is that a pathological approach to innovation? That we're so addicted to those early hits of the voltage in the pPetri dish that we refuse to stay put anywhere, and we were just always jumping to new pPetri dishes?

John: I think that when you look at ideas or innovations, or products, that end up making it big, very, very rarely will that be the initial innovation that the entrepreneur brought forward. Very rarely. What it ends up being is some major adjustment or major movement to a new type of product or a new type of space than the entrepreneur originally thought. I

n any ecosystem, you're going to have this jumping around in the spontaneity that you mentioned. And that's what's going to lead to innovation. It's certainly not going to be, "I'm going to stick with my original idea, and to hell or high water, there will be no perturbations." I think we’d have very little innovation, if we had this world of, "I'm just never going to quit on my original idea." And thank God, people understand the opportunity cost of time, because it's those people who understand and don't neglect the opportunity cost of time and are willing to give up on that idea. They're willing to not recognize the sunk cost and say, "I'm going to stick with that to try something new because it tends to be the third, fourth, fifth or sixth iteration." Look at Thomas Edison, is a good example, or any major innovator. It's the fifth, sixth, seventh, eighth, 10th, 20th iteration that ends up hitting. And then, that's the one that explodes.

Brooke: So, that's interesting. So, earlier in our conversation, we talked about winners quitting. Like, winners don't just stick around and bull through some idea that is clearly not going anywhere. But at the other end of the spectrum, you've got people who just don't stick around long enough for the thing to get through some of its growing pains. So, I really like these five pitfalls that these vital signs can help us to identify like, when should we bail?

But what about, when should we stay? What are the kinds of obstacles that we identify where we say, "Okay, well, this is a voltage drop that we're seeing, but it's only local, it's only temporary, we should push through here because there's something that's still going to be interesting on the other side"?

John: Yeah, absolutely. Great question. So, I think the two stories of my life helped to illuminate my thoughts on that very good question, Brooke. So, my dream as a high schooler was to be a professional golfer. And really the only reason I went to college was because I went to play golf. My brother is a truck driver. My dad's a truck driver. My grandpa was a truck driver. I was supposed to be a truck driver. Now, I had something else in mind. So, as a first gen kid, I convinced my parents to let me go to college. And I went on a partial golf scholarship.

And I realized about two months into that fall season in my freshman year that I was not good enough. And I talked about this in the book. I talked about how I came to the realization that my dream was over. So, I decided not to quit the team, of course. I fulfilled my obligations on the golf team for four years. But my dream pivoted to becoming an economist. And I always loved econ. I've always thought in ‘Economese’, which is what I call the language of economics. It was secondhand to me from the beginning. I realized I had a comparative advantage at doing economics. 

I had some hard times.I went to the University of Wyoming as a PhD student, I went on the job market in 1996. I applied to 150 schools, and only one school gave me an interview. I got a lot of rejections early on, people telling me that my field experiments didn't make sense. "Don't do experiments in the field, you should be doing them in the lab,." That's what people told me in the early '90s. I was one of the first to do field experiments in economics. So, I didn't quit. And I didn't quit because I realized that my comparative advantage was doing economics.

And I think once we have the mindset of, "I realize this is what I'm good at and this is a skill that I have that others don't have. It's my unique talent," that's something I want to stick with. When I realized that golf wasn't that for me, there were probably 150 million people in the world who were better than me at golf back then when I was 18, you realize quickly, "No matter how hard I practice, it's not going to work." So, the idea is in which I have a comparative advantage, I want to stick with those a little bit more.

Now, what the research tells us, however, is whether it's a job, an apartment, in education, a partnership, we stick with those too long. The evidence shows that if we would pivot sooner, we would be happier individuals. And I talked about that in the chapter called  ‘Winners Learn to Quit.’ But you're right, they don't jump and jump and jump. Because if you jump every 10 seconds like a gnat, as a human, you're not going to win. So, that's not what I'm proposing here. I'm proposing a rule about optimal quitting, not quitting at every moment.

Brooke: So, for someone who's been listening to this conversation and just totally drinking it in, smitten by all the ideas that we've been discussing. What can they start doing tomorrow morning to put this into practice and say like, "Thank you, John List. I am now going to lead my life differently"?

John: Yeah, absolutely. I think there's a little something in the book for everyone so I like that you asked the question. So, if I'm a policymaker, I'm going to read the book and I'm going to say, "You know what? Everyone was telling me that we should be doing evidence-based policymaking. But when I read what John is writing, and I hear what John is saying, he said we need to reverse that idea and we need to have policy-based evidence.

We need to think about what the policy will look like at scale, and bring that back to the petri dish and make sure that it's that policy that works with all of the flaws and warts and constraints that we have at scale. Now, I'm going to be a better policymaker because I'm going to have the mindset of policy-based evidence." 

Now if I'm a firm, I'm going to say, "Look, we're all in on this idea that we've been growing and growing and growing, we have a comparative advantage at it. What I'm going to do is I'm going to be a better marginal thinker.” This is what we do at Lyft now,every investment on the margin has to be more equalized. And it has to be done in a way that's rational and makes sense.

If I'm a new entrepreneur, I'm going to read the culture chapter and say, "You know what? From the very beginning, I need to build a culture of inclusion and a culture of equality, and that starts from the very first job ad that I put out." Because the words in that job ad, whether you want to be a CSR firm, whether you want wages to be negotiable, these things send signals to the types of workers who you want. And from the very beginning, you want to build a culture that is sustainable at scale. I lived in a culture that was not sustainable at scale with Uber. Now as the individual, the truck driver, my dad, my brother. What I want them to see from this is, "You know what? I'm one man and one truck in my company." It's called List Trucking in Sunbury, Wisconsin. Basically, one man, one truck. The key ingredient there is my brother. He's vivacious and great, he doesn't scale. So, guess what? He's going to do the best he can with what he has, and it's going to be a good life. You don't need to scale to billions and billions of people to be a success in life, you need to understand which of your ideas can make it huge and which are in the lower 10th. And if you're happy with that lower 10th, great. You have the correct expectations. 

I think in many parts of the book, you'll see items like,”This is a way you should think about real life problems, whether it's an idea or innovation that's great, but also just your real life thinking.How I think through problems." I think the book brings many of these out to the individual on the street as well.

Brooke: I really like that example you shared about your brother and his trucking company. I can hear in that story this Wall Street intensity of like, "But if it's not getting bigger, it's bad. It's broken, it's deficient." But that's really not what you're advocating here. When you talk about policy-based evidence, or what I would think of as policy-based experimentation, because policy-based evidence for me has a negative Boogeyman conversation.

But when you talk about policy-based experimentation, the first step there is like, "What does this look like at scale?" And implicit in that question is, "What is the scale at which we will consider this to be fully developed?" And that strikes me as a really, really powerful observation and points to reflecting here is that we tend to think about scaling in infinite terms.

Scaling is something that should always be continuing. But actually, there are lots of instances where there is an appropriate scale that we reach and then sustain. And this is not a deficiency, this is not some shortcoming. That's just a different model of what we might be after. It's a different set of values about what ideas are for and what organizations and companies are for.

And that really helps to displace this infinite growth mindset. And I think that for me, that's really the core takeaway. It's like, the first question you need to ask yourself is, "What do you mean by ‘at scale’?" And that then leads to, "Well, what will the idea need to look like at that scale, which then drives the further cascade that you're articulating before? What do the experiments need to look like in order to find something that can even reach that point?"

John: No, I think that's exactly right. I think about these things as features. "I have an idea, and what are the features of that idea and what is the expected scale?" I think it's very important to understand that from the outset. But many ideas are great that don't take over the world, that ended up being small scale but by design. And it's important that you understand the features of your idea before you begin to invest in it because by design is important.

You invest in different ways if the scale is to serve 1,000 people versus to serve 7.5 billion people. It's a different process of investment. Now, the back nine of the book, so to speak, is for any organization. They choose, let's call it the minimum efficient scale, that they want. And then, the back half of the book is about, regardless of what scale you choose, these four little secrets are how you should think about running your organization, because these little secrets work for any size organization.

Brooke: All right, John, this has been wonderful. Thank you so much for your time, your insights, your humor and your stories. Really appreciate that.

John: Brooke, thanks so much for having me. And I really appreciate you highlighting The Voltage Effect.

Brooke: My pleasure. And we hope to talk with you soon.

John: Take care, Brooke. Have a great day.

We want to hear from you! If you are enjoying these podcasts, please let us know. Email our editor with your comments, suggestions, recommendations, and thoughts about the discussion.