Jason Cohen


Recent Posts

Releasing Early Is Not Always Good? Heresy!

By Jason Cohen on December 21, 2009

The following is a guest post by Jason Cohen. Jason's got a knack for stirring pots and getting us thinking a bit.  And, thinking is a good thing.   Even when the thinking makes me awfully uncomfortable and goes against my strongly held beliefs.  I couldn't be a stronger advocate of "Release Early, Release Often", so it took a bit of effort to post this article "as is".  But, as I hope you'll see, Jason does make some pretty good points.  Well worth debating -Dharmesh

[UPDATE: I originally stated that Eric Ries was a proponent of "Release Early," but I was completely wrong.] OnStartups No Hear

You can't throw a stone in the blogosphere without hitting someone arguing in favor of releasing software as early as possible. The idea is that it's best to push new software onto potential customers before it would be traditionally considered "ready."

Here's the general line of reasoning: In the beginning, you have a theory about who your customers are, what their pain is, what your product needs to do, and how it will be used. But it's just a theory, and it will always be wrong. That's OK! It's life.

Since you're wrong, your v1.0 is really just a foil to get people talking. Therefore:

  • Don't add a lot of features, because you don't yet know which features are actually needed.
  • Don't fix every bug, because half of this code might not be here in a month.
  • Don't pretty up the user interface, because it won't look anything like this in a month.
  • Don't obsess over the customer workflow, because it will change completely in a month.
  • Don't worry about scaling, because you won't have that many customers for a while.
  • Don't worry about architecture/extensibility/documentation because most of your code will be different in a month.

Now in general I agree with this line of reasoning, but I don't like how one-sided this discussion is. Even a casual glance through this discussion of the subject shows that people -- me included! -- are recommending this strategy as a knee-jerk reaction.

So for the sake of thoughtful debate, I'd like to present the case against.

The best ideas aren't built by consensus

Would the iPod have been invented if it were built by iterative customer feedback? I doubt it:

  • Do you want a portable music device without a radio? No.
  • Do you want a battery-powered device in which you can't change the battery? No.
  • Do you want a portable music device without Bluetooth? No.
  • Do you want a wheel-based user interface which you don't already understand and which makes certain operations confusing? No.
  • Do you want it to take 5 clicks to toggle "shuffle," one of the most-used functions? No.

Apple doesn't ask customers what they want -- they just go invent awesome stuff. People complain, sure, but in the end the success of the iPod and iPhone is undeniable.

In general, disruptive products by definition cannot be built by consensus. In fact, it's well-known that "design by committee" is a sure-fire way to get mediocre design.

Small, incremental changes pulled by customers blind you to bigger, better ideas.

You're misinterpreting the 80/20 rule

The typical 80/20 rule is: 80% of your customers use just 20% of your features.

The "release early" folks take this to mean: Just implement 20% of the features you think you need, because if that's good enough to get 80% of your sales, this is a much simpler, efficient, and therefore profitable way to operate a software company.

But this interpretation is wrong! To spell out the 80/20 rule more accurately: 80% of your customers use just 20% of your features, but each customer uses a different 20%. That implies you need more features, not fewer, otherwise there won't be enough for the various use-cases for your software.

Take Microsoft Excel. Every person I talk to uses a different subset of functionality. Some people are experts at PivotTables whereas others have no idea how they work. Some people can't live without the database connectivity stuff whereas others don't know what a database is. Some people drive key business data using conditional formatting, some people know just enough Visual Basic to save themselves hours of manual labor, and some are experts with the charting system.

The point is that you can't just remove 80% of the features from Excel and expect 80% of the people to still be happy.

Mock-ups are faster than code iterations, without some of the drawbacks

There's a variety of software and techniques for mocking up applications, both web-based and desktop. Mockups can typically be built in a matter of days and put in front of customers for feedback. Iterating with fake screenshots is always faster than actually messing with HTML, CSS, and back-end code.

Once the mockups for v1.0 are set, actually writing v1.0 is fast because you know the goal ahead of time. The complete cycle is faster than if you started coding in the first place, and the code is cleaner because you don't have traces of major false-starts. (Think database migrations, CSS styles, extra AJAX routines, unused dialogs.)

Releasing too early can ruin your reputation

When your first version is sub-par, you'd better hope very few people find out about it.

Back to Apple: The iPod worked on day one, but back in the 90s the Newton didn't. The Newton (the world's first PDA) was hailed as a revolutionary technology (just as the iPod and iPhone would be), but it didn't work well. In particular, the handwriting recognition sucked and there wasn't a lot of apps.

It never recovered from its early reptutation as "doesn't do a lot, and what it does do doesn't work well." Even when that was remedied, it was too late.

The typical counter-argument to this is that "release early" doesn't mean "release with lots of known bugs," but rather "release with fewer features." But we all know the difficulty in separating a "feature request" from a "bug report" -- what a customer sees as a devistating lack of functionality (bug) you say is missing by design (feature).

Ignoring architecture creates waste

Software architecture is something the end-user never sees, and is therefore usually cast aside in the arguments for releasing early and collecting feedback. But incorrect choices in architecture can bite you later on, causing an immense amount of waste, possibly enough to sink the company.

It's true that most companies don't survive, and therefore having a growing user base with a problematic architecture is "a good problem to have." But it's also true that an ounce of prevention is worth a pound of cure.

Take Twitter. Their scalability problems are legendary. Suppose they didn't have billions of dollars in funding to throw expensive people and hardware at the problem? It's not hard to believe that continuing scalability problems would have sunk their ship.

Or take Netscape. The architecture problems with their browser were so severe it required a rewrite, which took so much time and effort the product (and company) died.

Of course this doesn't mean you should dwell on architecture for six months before working on features, but it does mean you shouldn't just ignore future maintenance issues for the sake of releasing a month sooner.

Untrue: "The worst thing you can do is built an unnecessary feature."

This is frequently used as an argument for releasing early. The logic goes: Every feature is effort. Not just in creating it in the first place, but debugging it, testing it, training customers on it, integrating it into all other features, supporting it in the user interface, covering it in the user's manual, and keeping it even as the product's focus changes. Therefore, never add a feature unless it's absolutely necessary.

I agree unneeded features are a major expense, provided you retain them in the product. But having incorrect architecture is also expensive, yet the "release early" folks tell us that expense is OK!

The fact is, you can change or even completely remove features that have become albatrosses around the neck of your tech support and product evolution. The "release early" argument also includes frequent iterateration, morphing the software as evidence suggests. Changing and removing features is simply a part of that, and doesn't imply that releasing early is smart.

In fact, adding features is one of the few ways to test what customers actually want, because:

Customers are notoriously bad at providing feedback

Sometimes users will tell you that they want a time/date-based voting widget they can send by email, when what they really mean is that they need to coordinate schedules.

In my experience customers are terrible at deciding what features they need, which features they use, or how features should be altered. They're much better at describing what's difficult in their life, what frustrates them, or what takes up a lot of their time.

Customers aren't even good at explaining why they did or didn't buy your product. From Jo Owen:

My research showed customers thought they were rational purchasers of video cameras: it was all about price and performance.

But when I interviewed them as they left a store, they were often unclear about how much they had actually paid for the model after splashing out for warranties, batteries and accessories and sorting out a financing plan. They were also very confused about the relative performance of different models.

Of course it's necessary to gather as much feedback as possible from both customers and lost-sales! But it's not clear how accurate the feedback is or how to weigh it against your own vision and goals for the product.

Relying on unreliable information as the primary driver of product decisions is unwise.

What do you think?

In the end, of course it's better to have more feedback than less, better to be more agile than less, and better to have technical debt with a successful product than a failed product. However, it's just not fair to present only one side of the argument!

Do you have more arguments either way? Do you agree we're taking "release early" a little too far? Leave a comment and join the conversation.  Or, if you've got a question, you can post it on Answers.OnStartups.com

Continue Reading

Answers OnStartups Roundup 1 - March of the parentheticals

By Jason Cohen on November 20, 2009

By now you've probably heard of Answers OnStartups, the new Q&A community for any topic about startups and entrepreneurship. If not, here's Dharmesh's announcement.

I have the good fortuneonstartups questions answers of being asked to co-moderate the site with Dharmesh, probably as a result of the popularity of a few provocative but useful guest-posts here on OnStartups. (Note to those of you hoping to expand your influence in the blogosphere and beyond -- take the advice from Leo and Darren and Mary and, I suppose, me: Guest-posting is a good way to do it!)

What does "moderation" mean, you ask? I took it mean "Get the CSS in order, make sure there are no JavaScript errors, and correct everyone's spelling errors. Oh, and be the #1 highest reputation person on the site." Durn, that last one actually took some effort... (Yes I see you gaining on me Alex!)

But, you know, it's not a competition. Yeah, right. That's why there's voting and reputation, 'cause it's not a competition. Riiight.

As some of you know, there was an unexpected problem with the site at the beginning. See, Answers OnStartups is built on the same technology as StackOverflow, through Joel Spolsky's StackExchange program. (Yup that's our Dharmesh on the front page! Which reminds me, that's a good lesson in marketing. Q: How do you get your name and URL on the front page of a high-traffic, highly regarded, high-SEO website like StackExchange? A: Write a fantastic (but honest) review of the product and give it to the owners. Make it so awesome that nothing they could say about themselves would be as good or believable as your words. Then they'd be foolish not to!)

(And since I'm on the subject, and since I've already interrupted myself with this many parentheticals, you might be interested to learn that, technically, StackExchange is FogCreek but StackOverflow is Jeff Atwood.)

Anyway. Where was I? Oh yeah, problems bootstrapping the site.

See, Answers OnStartups has an automatic permissions system where you have to have a reputation of at least 15 to vote, and 50 to leave a comment. Even more for things like inventing new tags. It's smart because it prevents people from (too easily) gaming the system by creating accounts and tearing up the place anonymously.

That works well at StackOverflow because they get a million hits per day (literally), so if you start contributing properly you get up-voted fast (sometimes in a matter of minutes). Very quickly an earnest contributor is granted these permissions.

But at Answers we had a bootstrapping problem. All these kind and wonderful folks came to use the site... and essentially couldn't do anything. The only people with rights to up-vote were Dharmesh and me (because we are blessed moderators, pronounced "bless-head"), and even then we were limited to 30 per day.

So all the initial members had to run around and up-vote each other for about a week, and newbies would come in an invariably ask questions about why no one is voting, which, ironically and usefully, got them a lot of votes.

The lessons from this little mishap:

  1. When you use pre-packaged software, you get the limitations along with the upsides. Just remember though, had you rolled your own you would have a different class of bug, probably worse.
  2. It didn't matter in the end, because what we're really doing is building a friendly community for startup discussions, and little problems like that aren't going to matter assuming you have intelligent members. And if you don't have intelligent members it's not going to work anyway. In other words, don't sweat the small stuff.
  3. Had we built our own, others would have beat us to the punch, and we would have lost the war.

As proof of that last point, there's now another StackExchange-based startups Q&A site! I'm not going to link to it because I don't want to give them the credit -- which I admit is kind of crappy of me, but in my defense I am writing in the throes of passion and cannot be responsible for my actions. Not buying it? Fine, but I'm linking to our question about it.

Anyway Answers OnStartups is better because we have a totally killer user base. Active users you've probably heard of include:

(I linked to their Answers user profiles too so you can see they're actually active, not just name-dropping.)

(I love the phrase "name-dropping." It sounds like you're taking a dump on the conversation which, often, you are.)

But perhaps even more fun is meeting all sorts of folks who aren't already famous but who are running interesting startups and have terrific, new things to add to the conversation.

Over the next few weeks, watch this space for highlights of these users, as well as highlights from some of the more action-packed discussions that are going on.

To whet your appetite, one of the most popular questions is: How do you pick a platform for a Web 2.0 startup company?

See that link for the full discussions, but here are some highlights:

  • The platform is not as important as other factors. There are hugely successful companies based on any technology you can name: Java, .NET, PHP, Ruby, Python, Perl.
  • A major factor is existing competence. If you know a platform/framework well already, that trumps other considerations because it means you can get started fastest. Speed to beta is part of success.
  • If you have no existing competence, it's most important to find/hire the best possible developers. Because it's always hard to find awesome talent, whatever platform they love is the right choice.
  • It depends on whether you need to get something running ASAP or whether long-term maintainability is necessary right from the start. If the former, Ruby and PHP generally get applications going fastest. If the latter, Java and C#/.NET generally have the best tools for that sort of thing.
  • Both Sun Startup Essentials and Microsoft BizSpark make certain application stacks particularly attractive. If you like the other benefits afforded by those programs, either are a good choice.
  • Pick a framework that will stick around for the long haul. Ruby on Rails isn't going to be ditched. ASP.NET won't be ditched but Microsoft is a moving target. PHP itself is stoic but any particular framework is more questionable. Which community do you think has longevity?
  • If your startup fails, which technology will leave you best positioned for future projects?

As with many questions on the site, there's usually not one right answer. As it should be.

So join the conversation! Come visit Answers.OnStartups.com and see what all the fuss is about. It's fun.

Continue Reading

Community

Let's Connect

And, you can find me on Google+, Twitter, and Linkedin.

Recent Posts

Chat with GrowthBot

It's a bot to help you with your marketing and grow. You can research your competitors, improve your SEO and a lot more. http:/GrowthBot.org