A large group of lumberjacks are cutting down trees in a forest and doing a really good job of it. Trees are falling left right and center, everyone is working with focus and tenacity. During a lull, one of the lumberjacks climbs up to the top of the biggest tree still standing and sees that in all the confusion to get the work started, they’ve made a bit of a mistake and should infact have been cutting down the forest next door to the one they’re in. So, he yells down to the rest of the team:
Hey guys, I think we’re in the wrong forest!
To which the foreman yells back:
Shut up! We’re making progress!
I don’t remember where I first saw this story (_I guess that’s the one downside to reading a lot of books :)_). But, this story (or one very similar) is actually a commonly used leadership metaphor. It is often used to demonstrate the difference between leadership and management. As interesting as that may be (and it is actually interesting enough that I might look at it further at a later date), to me it also demonstrates another common phenomenon – that of the sunk cost fallacy. You see the foreman, wasn’t really an idiot, he knew the climbing lumberjack was right. But the team had invested so much time and effort already, it would be a shame to waste it. Then there is explaining the mistake to upper management – could get awkward, plus think of all the paperwork. No, it is way too embarrassing and costly to admit the mistake, better to pretend that this is the way things were meant to happen all along, hopefully it might all turn out for the best and we’ll be able to salvage the current situation.
The Sunk Cost Fallacy
The sunk cost fallacy, also known as the Concorde fallacy, is a very interesting phenomenon. What it basically boils down to is the fact that it is human nature to throw good money after bad. The more resources (time, money) we invest in something, the more likely we are to stick with it despite all the indicators of our venture being a failure (I am not going to give generic examples; you’re welcome to check out the links above). It is therefore no big surprise that when it comes to software development, we’re not immune. Infact we take the sunk cost fallacy to new heights of awesome :).
In the world of finance, the sunk cost is accepted. If the money is spent and can’t be recouped, it can no longer influence any further decisions. On the other hand, in software, once we have spent any kind of effort/money on a feature/project, we just can’t let go. We would much rather delude ourselves and everyone around us that we can still turn everything around and make it all come out for the better. Developers do it, managers do it, it’s an industry wide trend. No matter how flawed the product vision turns out to be, we are much more likely to try to adapt the whole ecosystem to the flawed product/feature rather than starting over from scratch and building something that will better fit the ecosystem we have. More than that, we will go to great lengths, bring in extra people, do overtime, whatever it takes, as if we can make an incorrect decision right by sheer force of will and sweat. I am not just talking at the project level, even at the code level we (developers) will often stick with a technology/library choice through thick and thin long past the time we should have abandoned it and found something that fits our needs better. There are always, good reasons to justify all this, but what it comes down to in the end is self-delusion – the sunk cost fallacy at work.
Lalalala, I Don’t Want To Hear It
I once worked on a product that started out as an offshoot of a bigger project. Initially it was only adapted to work as part of that bigger project. But, after a while a decision was made to turn it into a framework. There was nothing wrong with the idea. This would have been a perfect point to abandon the existing code; it was unwieldy and difficult to work with. It would have made sense to merge it into the bigger project and start from scratch. But noone was willing to make that decision, money had been spent on development, it was practically a framework already, just needed a little extra effort and then we’d be able to use it on all the other projects in the company and save all kinds of money. A business proposition too good to refuse! Except, in trying to munge the existing code into a framework, everything became even more unwieldy, no project really wanted to use it. But, it was decided that the situation could still be salvaged, all we needed to do was turn the framework into a full-blown product. Once again not a bad idea on its own, and had we started from scratch here, we may have come up with some cool stuff, but we couldn’t just abandon all that ‘good’ code. So, we went on another retrofitting exercise, building more code on top of ever shakier foundations. At this point the ‘product’ had a terrible reputation within the company, noone except management wanted to have anything to do with it, but that just meant it was pushed even harder.
The company wasn’t willing to make the necessary decision to either abandon the project all together, or to start over and build something solid. The money that had already been invested was guiding all the decisions. Recouping losses was the word of the hour and in pursuit of that goal even more money was sunk into a venture that didn’t deserve it. If this were an isolated incident it wouldn’t be so bad, but this story probably sounds awfully familiar to many and there are more than a few horror stories that are way worse.
Delusion Industry Wide
How about an example of sunk cost self-delusion at an industry wide level? You need go no further than JavaScript. I don’t want to bad-mouth the language it has done some decent service and I won’t deny that it has some good parts. But as that photo I saw a few weeks ago clearly demonstrates, there are good parts and then there is the rest.
People tolerate JavaScript but most developers don’t really love it. It can be unwieldy and hard to use, some things just don’t make sense and yet it has become the de-facto language of web 2.0. It’s not as if there are no better scripting languages around, and even if there weren’t, it’s not like we can’t come up with anything better if we tried. But, all the browsers already support it (well, kinda, none in quite the same way) and there are all the frameworks that make everything much better. No, there is way too much invested in it, I am sure we can make it perfect eventually. In the meantime it makes a lot more sense and is infact much easier, to put millions of developers through daily pain, rather than phasing out JavaScript in favour of something better (anything would probably do). Seems like sunk cost fallacy at its finest. Admittedly though, I am no JavaScript guru, so if you are, please come along and tear my argument apart.
End of the day, there may not be much that each one of us, can do to affect grand change at an industry-wide level, but we CAN influence the projects we work on. If you see the beginnings of the sunk cost fallacy rearing its ugly head, don’t stay silent. It is never too late to remove a technology or library if it was clearly the wrong choice. Be ruthless, if it doesn’t do the job the way it should, get rid of it, find something better, there are always alternatives. For the management the advice is the same, just at a higher level. Don’t keep throwing good money after bad. Remember if all the other lumberjacks (including the foreman) had just stopped and taken notice when the first one told everyone of their error (regardless of the embarrassment and inconvenience), they could have avoided cutting down the rest of the magical woods – home of the unicorn (cause that was what they were chopping). Instead they could have started chopping the dark dreary forest, like they should have been doing in the first place and thereby made life better for everyone, including themselves.
Image by amateur_photo_bore