OnStartups

Dharmesh Shah

Recent Posts

We Like Humans, But We LOVE Code (Machine Learning Case Study)

Posted by Dharmesh Shah on April 16, 2015 in guest development 6 Comments

The following is a guest post by Jonah Lopin (@jonahlopin), founder at Crayon.  Jonah is a HubSpot alumnus, and I'm an angel investor in the company.  

Here at Crayon, we love humans. 

In fact, all our moms and many of our best friends are humans! 

But when it comes to solving problems, we prefer code.

Humans are awesome at certain things, like closing enterprise sales deals, delivering rousing speeches, and comforting small children. For these tasks and lots of others, humans are far superior to code.

But in a business context, humans have some serious drawbacks:

1. They’re hard to recruit (Crayon is hiring, by the way)

2. They’re expensive (compared to servers)

3. They sometimes get sick and can’t come to work (☺)

For a product-driven software company, there’s something else to consider. It’s subtle, but important.

Software is better than humans at providing an elegant solution to complex problems at scale.

Tomasz from Redpoint put together some fascinating data back in 2012 that showed billion dollar public SaaS companies have revenue per employee around 200k/year. Jeremiah Owyang puts revenue per employee at Google & Facebook around $1m per employee, about 5x higher! Clearly, Facebook & Google solve more problems with code than a typical B2B software company.

The fact that Google & Facebook solve their primary problems with code rather than people doesn’t just impact their metrics and margins, it shapes the elegance and efficiency of the solutions they provide. The subtle advantage of solving problems with code rather than humans is that it tends to make your core product better over time.

Crayon is small, but we dream big. We solve problems with code, not humans, because we want to serve millions of users as elegantly as possible. It’s just down-right hard to do that with too many humans in the mix, especially if you want to scale quickly. (And especially if you’re not Uber.)

The rest of this post is about a big problem we had and how we solved it with code.

The Problem: Some Web Pages Don’t Look So Hot

Crayon is a visual inspiration platform. We help marketers get great ideas so they can do better marketing.

We’re programmatically adding about half a million pages to our system every day. As I write this article, we’ve got about 13 million designs in the system, and we’ll have close to 100 million by the end of the year.

We let users vote designs up and down, so most categories on Crayon like Startup Home Pages, B2B Pricing Pages and Landing Pages have always looked pretty good.

But we still had a problem in deeper categories: folks would be happily browsing, only to have the smooth inspirational vibe unceremoniously broken by a crappy looking page. How could we get the “best” designs to the top of the result set and the “least inspiring” designs to the bottom?

We Challenged Ourselves to Solve it With Code

Can code separate “inspiring” marketing designs from “crappy” ones?

Many said it couldn’t be done… but if they were right, would I be writing this article?

Here’s How We Solved It

Step 1: Pick Training Sets

We picked a set of 200 “inspiring” marketing designs and a set of 200 “uninteresting” marketing designs.

Yes, this is a bit of a fuzzy thing to do because it’s based on human judgment. But after running the services team at HubSpot for 5.5 years, and working closely with more than 8,000 professional marketers, I felt qualified to judge which marketing designs were likely to be interesting and instructive to other marketers. So sue me!

Step 2: Make Some Guesses About Why The Good Ones Are Good

This part is like picking a basketball team without being able to watch the candidates play basketball. What characteristics of the players would you look at? For instance, you might pick taller players or players wearing high-tops.

In the style of Jeff Foxworthy’s comedy routine You Might Be a Redneck:

If your Html is all based on tables… you might not have a great marketing design (tweet)

If you’ve got lots of inline styles… you might not have a great marketing design (tweet)

If you don’t use media queries in your CSS… you might not have a great marketing design (tweet)

You get the idea.

Step 3. Test Your Guesses

We wrote some code to test each “guess” from step 2 against each design in the training sets from step 1. We were hoping to find things that were “true” for the “inspiring” pages, and “untrue” for the “uninteresting pages”.

We looked at 45 discrete “guesses”, and found 25 of them were predictive of “inspiring” marketing designs. Success!

Some of the best factors were things like:

  • Setting the viewport meta tag
  • Including Facebook Open Graph tags
  • Using the Bootstrap framework
  • Specifying Apple touch icons

Note that these factors don’t directly predict which pages are “inspiring”. Rather, these factors indicate that someone clueful created the page, which is directly predictive of a page being “inspiring”.

Some of the things we thought might work, but weren’t predictive at all include:

  • Whether a page uses JQuery doesn’t matter… it’s just too ubiquitous
  • The total text on the page doesn't matter.  It turns out there are equally as many crappy short pages as there are long pages (tweet)

The final step was some mathematical mojo based on how strong the signals were in step 2 to come up with an overall “inspiringness score” for each page.

The Results

Pages like the chatterbox.co homepage did very well:

interesting-example

While our friends at biomarkerstrategies.com didn’t fare as well:

uninteresting-example

 

Overall, here’s what we got:

humans-vs-code-chart-chart

How awesome is that?

The top 10% of Crayon search results just went from 50% “inspiring” to a whopping 93%!

We have 13 million designs in the system today, and reviewing those manually would take a 10 person team about 5 years. Ouch! But we’ll have 100 million designs by the end of the year, and that’s just the beginning. If we didn’t solve this problem with code, we’d be in serious trouble. And no humans were involved… except for the one human writing this article.

Your thoughts?

Does your business have a tough problem you plan to solve with code rather than humans?

Have you used machine learning to deliver elegant solutions to customers at scale?

Please continue the converstaion in the comments.

Go solve something with code! (tweet)

Read More

5 Mistakes Founders Make When Hiring Their First Salesperson

Posted by Dharmesh Shah on February 25, 2015 13 Comments

Your MVP is complete. Some seed funding is in the bank, or maybe even a Series A. You’re anxious to see if your product flies off the shelf. It’s time to make that first sales hire.

This is a big step. The right call can establish the rocket-ship revenue chart every founder dreams of. The wrong call can be bring down the company.

At HubSpot, I hired over 200 salespeople. I made a lot of mistakes. I figured a few things out. Over the years, I have leveraged this experience to help dozens of startups with this critical first sales hire decision.

Here are five mistakes I see many founders make when hiring their first salesperson.sales-acceleration-book

Mistake #1: Hiring a senior leader

“Yes! We just recruited the director of sales at the public company we are looking to disrupt.”

No! Trouble awaits.

Often startups get greedy and go after the big name leader in their space. Here is the issue - this leader has not sold on the front-line in years. The first question she will ask when joining the company is “where is my assistant?”. You need someone who is closer to the front line and is willing to roll up their sleeves and get their hands dirty.

 

Mistake #2: Placing too much weight on industry experience

Me: “What is the top criteria you are looking for in your first sales hire?”

Startup Founder: “Experience in our industry”

I receive the above answer 99% of the time. It does not sit well with me. The answer sounds logical. Domain experience is certainly straight forward to assess. However, all too often, startups put too much emphasis on candidates’ industry experience and miss real weaknesses in their abilities.

Don’t get me wrong. We hired plenty of sales people at HubSpot from our industry. However, believe it or not, it was a minority case. Most folks came from outside of the marketing software space and even outside of software and tech altogether. I would rather have the below average performers stay with our competitors. Go find the rock star from a dying industry and bring them into your space.

Mistake #3: Placing too little weight on go-to-market strategy experience

When your first sales hire joins the company, the go-to-market strategy is typically not developed. Who should they call? Big companies? Small companies? Should they focus on closing the C-level decision maker or getting the end user into a free trial? Will they sell direct or through a partner channel? Will deals require an in-person visit or can they be completed over the phone? Will they work with inbound inquiries or outbound cold calls?

Most sales people and sales leaders will simply try to replicate the go-to-market strategy from their previous employer. It worked then. It should work again, right?

Not necessarily.

As a founder, consider your buyer, your product, and your company strategy when determining your company’s ideal go-to-market approach. Be sure your first sales hire has experience with that approach or is at least willing to adapt and learn.

Mistake #4: Overlooking sales process development experience

“Yes! We just hired the top sales person from our Fortune 500 competitor. This salesperson was ranked #1 out of 1800 sales people. She is going to crush it here.”

I agree. This salesperson is talented. However, these will be her first questions when she arrives at your company:

§ “Where is the pitch deck?”

§ “What is the sales process I should follow?”

§ “Where is the list of top 10 objections and how I should handle them?”

You’ll be scratching your head, “I thought that is why we hired you”.

When this top salesperson started at your Fortune 500 competitor, she attended a month of training, walking through the blueprint of success. She is great at following the blueprint. In fact, she is the best.

However, can she build the blueprint? Probably not. You need someone that can work in a far less structured environment and at least lay out the foundations of the company’s first sales process.

Mistake #5: Hiring a “product pitcher” rather than a “consultative seller”

Believe it or not, the most valuable result from your first thousand sales calls will not be the early customers or revenue these calls produce. Instead, it will be the plethora of feedback from the market, allowing your team to continue to understand your buyer persona, iterate on your product, and perfect your market message.

Unfortunately, many salespeople will approach these early calls as an opportunity to dump as much information about the product and its beautiful features on the prospect, an approach we call “show up and throw up” in the industry. Chances are, your features and message are not quite right. The salesperson will fail to engage the prospect and throw up his hands, “it’s not working”.

The key question is “why?”.

Alternatively, a consultative seller will leverage the first sales calls to learn about the prospective buyer. They will learn about their goals, learn about the strategy they have to pursue these goals, and learn about the challenges they are facing in these pursuits. A consultative seller will be able to come back to the team with the same “it’s not working” result but they will understand why. As a result, the organization can continue to refine its approach and tighten the product/market fit even further.

I hope these five mistakes help you hone your skills as you embark on the exciting phase of scaling sales. The next logical question is where can I find candidates that avoid these mistakes. Here are a few ideas.

1. A recently promoted sales manager at a reasonably sized business. They are not too far from the front-line but also have some experience in sales process development.

2. A failed entrepreneur with formal sales training in their past. They usually score high in the consultative selling arena and experience with unstructured environments.

3. A top-performing salesperson that was around in the early days of their previous company’s growth. They may not have built the process. However, they certainly watched someone do it and had a hand in it.

Any other ideas?

This is a guest post by Mark Roberge. If you liked this article, check out Mark Roberge’s new book that launched this week, “The Sales Acceleration Formula”, about his experience in building the HubSpot sales team.

Read More