Combating the Innovators Dilemma – HubSpot’s Experiments Framework

About This Blog

This site is for  entrepreneurs.  A full RSS feed to the articles is available.  Please subscribe so we know you're out there.  If you need more convincing, learn more about the site.



And, you can find me on Google+

Connect on Twitter

Get Articles By Email

Your email:


Blog Navigator

Navigate By : 
[Article Index]

Questions about startups?

If you have questions about startups, you can find me and a bunch of other startup fanatics on the free Q&A website:

Subscribe to Updates


30,000+ subscribers can't all be wrong.  Subscribe to the RSS feed.

Follow me on LinkedIn


Current Articles | RSS Feed RSS Feed

Combating the Innovators Dilemma – HubSpot’s Experiments Framework


The following is a guest post by Brad Coffey, an early employee at HubSpot.  You can follow Brad on twitter @BradfordCoffey 

This week HubSpot was lucky enough to be included in the Inc. 500 list of fastest growing private companies.  It’s a great honor, we're really excited (and humbled) to be listed next to so many great companies.  In an adjoinging article in Inc. Magazine, our CEO Brian Halligan discusses a key part of our success (and looks impressive in a full-page photo too).  He talks about our approach to experimentation and our methodology for incubating new ideas.   What Brian describes is three tiered approach to promoting and funding unconventional projects.  It’s a methodology that served us well thus far and helped create an innovation pipeline that offsets our traditional disciplined focus on the core business.

The foundation of this framework is based heavily on Clay Christensen’s work in The Innovators Dilemma.  We’re huge Clay Christensen fans at HubSpot (even have a conference room named after him) and have been life-long students of his work.  In his work Clay asks a very straightforward question without an obvious solution - specifically: Why do well managed, successful companies repeatedly fail to create new disruptive innovations?

This framework was developed fundamentally to combat that challenge and create a lasting culture of entrepreneurial exploration.

HubSpot Experiment Framework

HubSpot’s Experimentation Framework

The framework has 3 stages, each with a distinct goal and approach.

AlphaLowering barriers to experimentation

No bureaucracy, no red tape, full access to information.  This stage is simply focused on enabling anyone with energy and an idea to try a new solution.  Tests are run by everyone and anyone – but are generally done in spare time (nights and weekends) and with few resources. You don’t need to ask permission to run these tests - and by design no one ever knows all the alpha stage experiments actively being pursued.  It's open and distributed.

Beta - Determining proper funding

When an experiment reaches Beta stage the ‘founders’ are fired from their day job and work on the experiment full time.  While founders determine their own goals and metrics – these leaders are encouraged to be patient for growth but impatient for profitable economics. Like many founders these people also report to a 'board' regularly and are subject to evaluation on future funding.  At its core this stage is about providing access to funding for entreprenurial folks with new ideas and transparency/accountability into the success of those early tests.

v1Scaling successful experiments

v1 projects have proven economics and now are looking to scale the success.  Often this requires growing the team beyond the founders, building dedicated systems and developing regular tracking of core metrics.  Founders with experiments graduated to v1 are now considered 'mini-CEO’s' and are tasked with running their project as a start-up within HubSpot.   

We established this framework in the hope of driving innovation and empowering the entrepreneurial edges of our organization to create change.  It seems to be working - we’ve had several successful founders graduate from the program (Pete Caputa with VAR program, Jordyne Wu with the Services Marketplace) and we created a culture to be proud of.   It's enabled us to focus on the core business without foregoing the entreprenurial engery and creativity of our team.

But is this enough? What do you think? Will this framework sustainably create the type of disruptive innovation required to create a lasting company?  Any other ideas or tactics we might want to consider?

Posted by Brad Coffey on Tue, Aug 23, 2011


Great Post, Brad. 
As someone who has been a part of many experiments at HubSpot and 2 of which that went from Alpha to Beta to v1, I believe in this process as I have seen it work :)

posted on Tuesday, August 23, 2011 at 8:41 AM by Ryan Beale

Congratulations on listing in Inc. 500. I used the website grader tool from OnStartups and very pleased with the accuracy of results.

posted on Tuesday, August 23, 2011 at 8:46 AM by Sofia Khatoon

I've love a lot of the articles from onstartups/Dharmesh (though I have to say you've faltered recently) so don't take the following too harshly! 
The two results you've listed - VAR/Partner program and the Services Marketplace can hardly be described as disruptive innovations IMHO. They are obvious steps a company like yours would pursue - lots of other companies have similar programs. 
Of course I can't see what's in your alpha/beta stages and perhaps that's where the future gems lie but I do wonder what's going on here - are you blowing your own trumpet, believing your own PR or perhaps its part of your ongoing drive to attract great employees?

posted on Tuesday, August 23, 2011 at 9:33 AM by Ryan

Nice Blog. Not really a comment...Beta and V1 make sense...Questions regarding Alpha Stage - How do you enable testing, is there funding available for alpha testing? Do people work in teams (I would think yes)? What is the success criteria to move to Beta?

posted on Tuesday, August 23, 2011 at 9:42 AM by Vernon Tirey

Great article. Keeping an innovative spirit in companies is always a challenge. This post provides some good ideas in keeping that innovative spirit alive.

posted on Tuesday, August 23, 2011 at 4:32 PM by Don Tarinelli

It's great to have a structure, but driving innovation requires much more, most importantly a culture that motivates and rewards risk taking, and also an organization which does not protect the most successful products. Potential solutions would be to ensure rotation of management, and or talent and not letting anyone establish domination over others. Best of luck, many companies fail to keep the innovation moving. Also a fan, and student of innovation.

posted on Tuesday, August 23, 2011 at 6:41 PM by Steve Meier

Although I applaud your internal “project development” process and it's structure, it’s hardly revolutionary. I have to agree with Ryan above who makes the point that the examples proffered are hardly disruptive innovations. Rather, this is a very good example of how multiple projects within a company can compete for acceptance and funding. In order to accomplish truly disruptive innovation, one has to look at the increase in value (Delta-V) that the "innovation" offers over current solutions and whether it's sufficiently large to justify the switching cost AND provide sufficient additional compelling new value to disrupt the current market's value equation.  
The only way to accomplish this disruptive new value equation is by identifying the fundamental value enablers - new technologies and/or processes or new applications of existing technologies and processes, new value chains, and new business models - that enable the creation of dislocating new value equations. This process can only be accomplished by having a clear picture of the current value equation, its enablers and the job that the customer is trying to accomplish with it. Without this foundation all that you are practicing is either incremental innovation, or innovation for its own sake. 
This is the same problem that exists with lean startup methodology, where the goal is to begin by defining a minimum viable product (MVP) and then iterate until you "find" the whole product required to satisfy the customer. If the MVP does not focus from the get-go on creating that Delta-V in value that is compelling enough to be dislocating, than the best that you can hope for is to stumble over it while iterating, or create another incremental improvement from the start-ups and current solutions that have gone before. 
The current fascination with getting the product out the door fast and then iterating can be compared to "A Random Walk Down Wall Street" which basically posits that if you jump in and invest and iterate your investment strategy, it's possible to hit big returns, but only because you were lucky enough to stumble over the randomness pattern of the markets along the way. I don't think that it's either wise or beneficial to our economy to have so many wanderers pivoting to and fro while destroying capital and jobs. 

posted on Wednesday, August 24, 2011 at 1:24 AM by Greg Ruff

It's important that this concept is offered to employees and they're encouraged to try out things they like and want to work on, but I can't tell if this process you've described effectively solves what I see as the core problem: "entrepreneurial diffusion" -- big companies repel innovators/risk takers. In the startup, the concentration of innovators is high and everyone is hungry. But as it grows, inertia gets frustrating for people that like to see things move, and are looking to drive major innovation that is able to replace core revenue generators.  
I've spent a lot of time working with major pharma companies, some of the largest and most frustrating organizations out there. And I'm happy to see that there are some champion innovators still there. But they're the few and very frustrated.  
Personally, if I was working in HubSpot, the presence of this process is irrelevant. I can work on projects in my own time and do this testing phase regardless. If I find something that works, best believe that I'm going to kick Dharmesh and Brian's door down and sell them on it until they setup the right structure to drive the initiative forward.  
Or get sick of me and tell me to leave.... 
Or I get sick of the organization and leave... 
so I guess my challenge is, are you able to keep those people? And does this process clearly enable that?

posted on Wednesday, August 24, 2011 at 3:53 PM by Kevin Vogelsang

And please tell me someone is alpha testing a better commenting system. You guys really need to get it fixed. Just doing my part in providing user feedback.

posted on Wednesday, August 24, 2011 at 3:56 PM by Kevin Vogelsang

that offsets our traditional disciplined focus on the core business

posted on Wednesday, August 24, 2011 at 9:23 PM by jacketsale

I agree with Ryan. The examples here just point to new, logical business offerings. The key to what HubSpot has done with the process is provided an efficient way to iron out the economics of the new offering before they scale. I would say that this is more of a Lean Startup approach than a solution to the Innovator's Dilemma.  
It seems like the real danger the Innovator's Dilemma talks about is disruption to the business model. The question is, does this process protect HubSpot against business model innovations? Are employees encouraged to try experiments that would disrupt the current business model? 
I'm not saying I know how to effectively do that. The only company I am aware of that has done that consistently well is Apple. The iPhone is killing the iPod. The MacBook Air will eventually kill a good chunk of the MacBook Pro line and the iMac is killing the Mac Pro. In only one case (the iPhone) is a product being replaced with a higher priced option. But obviously Apple's profits aren't hurting at all. They don't leave any room for someone else to disrupt them since they keep disrupting themselves.

posted on Thursday, August 25, 2011 at 9:23 AM by Greg DeVore

I feel it’s a little extreme. There may be many possibilities in this condition.

posted on Friday, September 02, 2011 at 2:44 AM by Canada Goose

Wow, amazing weblog format! How long have you ever been blogging for? you made blogging look easy. The entire look of your website is magnificent, as smartly the content material!

posted on Friday, September 02, 2011 at 9:06 PM by North Face discount

Comments have been closed for this article.