OnStartups

Why Your Startup Should Ignore Your Onboarding Experience (For Now)

Posted by Dharmesh Shah on April 27, 2015 in support 9 Comments

This following article is a guest post by Jackson Noel. Jackson is the Co-Founder of Appcues, a SaaS product that allows you to create in-product experiences without changing any code.  You can follow him on twitter: @jacksonjonson.

Since I joined a startup that focuses on user onboarding, I’ve had half a dozen conversations just like this one:

Startup Friend: “Dude I need to talk to you about our user onboarding.”

Me: “Okay, what’s up?”

Startup Friend: “We’ve changed it 3 times now. Can’t figure out how to get users to engage.”

Me: “Gotcha. Well what are you doing to change it?”

Startup Friend: “I’ve gone through every one of Samuel Hulick’s teardowns like 10 times. We’re just trying to get ours to be more like Slack’s.”

Me: “Have you looked at any of your own data?”

Startup Friend: “Nah, it’s too early to have anything insightful in there. We only had like 250 users sign up last month.”

Friends in early stage startups have come out of the woodwork to talk about their user onboarding experiences. And every time I tell them the same thing: completely ignore your user onboarding experience (for now).

Why your startup shouldn’t focus on user onboarding

To my friends, this advice usually seems *counterintuitive*. After all, it’s coming from someone whose business model hinges on people focusing on onboarding. But bear with me for a moment…

Great user onboarding makes users say, “WOW, this is awesome,” and recognize that your product is a must have experience. But these WOW moments don’t come easy. And the mechanics by which you onboard users is just a small part of whether or not they fall in love with your product.

The more substantial part of the equation is the value your product delivers to your user: something in their life that must get easier, faster, cheaper, more productive, more fun, etc. because of using your product. Otherwise, why would they switch?

And that’s the difficult part to create. That’s the part that requires customer development and experimentation. It requires you to test your assumptions, to pivot, to try new things.

Once you get that part right, building a silky smooth onboarding experience to deliver value to your users quickly will become far easier. But if you focus on onboarding too early, two bad things can happen:

  1. Your team will have less time to build something that truly delivers value (which again, is the much more difficult and important part of scaling user growth)

  2. You’ll create “product baggage” that makes it harder to build and experiment. Now you have to worry about your onboarding flow with each product change you want to make

After I explain this to my startup friends, I usually get a response along the lines of, “but wait, if I don’t have a good onboarding flow how am I ever supposed to know whether my product is delivering enough value?”

Indeed, we have ourselves a classic chicken or the egg conundrum.

Instead, onboard your customers manually

In lieu of investing your development time in user onboarding, you should do whatever it takes to help your new users achieve success. Reach out to them personally via email, install a live chat product, set up demos, etc.

Yes, this approach doesn’t scale. But it will save your startup’s most scarce resource, developer time, and accumulate your startup’s second most scarce resource, information.

Here are 4 things you should do:

  1. Talk to each new user who signs up to learn about their goals and offer to help get their account set up successfully

  2. Reach out to all of your existing customers and find out what they love about your product

  3. Touch base with every single user who signed up but never adopted your product (usually an email after 60 days of zero activity works well)

  4. Track your usage data very closely (Mixpanel/KISSmetrics are great tools for event tracking, Inspectlet is great for recording user sessions). Make sure to cross-reference all your qualitative findings (from customer conversations) with quantitative data.

Bonus: Here are two google docs with email templates, 3rd party tool recommendations and more to help you onboard users manually.

This manual onboarding process is not easy. If your product is still in the early stages, you’ll probably see concerning data and get a lot of negative feedback. It can be demoralizing.

But it will make your team stronger and your product better. You will better understand the motivations and anxieties of your users, identify points of confusion, and figure out why unengaged users didn’t stick around.

This information is a gold mine for an early stage startup. It gives you the information you need to thoughtfully iterate and pivot your product to get more users to say, “this product improves my life”.

When is the right time to prioritize your onboarding experience?

Once you have a high degree of certainty that users are getting great value from your product, you need to fire yourself as onboarding czar and hire your product. Unfortunately, there’s no hard and fast rule about when this time is for every startup.

But there are generally two types of measures you should keep your eyes on:

  • Measures of product value. This could be your NPS score, churn, or Sean Ellis’ 40% rule.

  • Measures of volume. Try to avoid a vanity metric like total number of users. Use something more meaningful like # of highly engaged users or # of paying customers.

At Appcues, we’re waiting until we have over 100 paying customers with greater than a 40% score on the Sean Ellis question to fully optimize our user onboarding experience. But your criteria should be unique to your situation.

When the time is right, you’ll have all the right information to optimize the in-app path that leads to user success based on what you’ve learned by onboarding users manually. Take a look at the user onboarding academy to help you build it the right way, and make sure you’re ready to commit to it for the long term.

As Samuel Hulick points out, user onboarding isn’t a feature. You can’t set it and forget it. Onboarding requires the same thoughtful iteration and pivoting as does the rest of your product.

The bottom line

With more products attempting no- or low-touch sales funnels, user onboarding has become a hot topic over the past year. And for mature products that have validated their value proposition and customer demand, it is a hugely important part of your funnel to optimize.

But for early stage startups that still have much to prove, optimizing your onboarding experience can be a monumental mistake. It will slow you and your team down, distracting you from truly solving your customer problem.

So completely ignore your user onboarding experience (for now). Instead, replace it with yourself. You’ll learn, build and address your customer pain faster.


Read More

We Like Humans, But We LOVE Code (Machine Learning Case Study)

Posted by Dharmesh Shah on April 16, 2015 in guest development 7 Comments

The following is a guest post by Jonah Lopin (@jonahlopin), founder at Crayon.  Jonah is a HubSpot alumnus, and I'm an angel investor in the company.  

Here at Crayon, we love humans. 

In fact, all our moms and many of our best friends are humans! 

But when it comes to solving problems, we prefer code.

Humans are awesome at certain things, like closing enterprise sales deals, delivering rousing speeches, and comforting small children. For these tasks and lots of others, humans are far superior to code.

But in a business context, humans have some serious drawbacks:

1. They’re hard to recruit (Crayon is hiring, by the way)

2. They’re expensive (compared to servers)

3. They sometimes get sick and can’t come to work (☺)

For a product-driven software company, there’s something else to consider. It’s subtle, but important.

Software is better than humans at providing an elegant solution to complex problems at scale.

Tomasz from Redpoint put together some fascinating data back in 2012 that showed billion dollar public SaaS companies have revenue per employee around 200k/year. Jeremiah Owyang puts revenue per employee at Google & Facebook around $1m per employee, about 5x higher! Clearly, Facebook & Google solve more problems with code than a typical B2B software company.

The fact that Google & Facebook solve their primary problems with code rather than people doesn’t just impact their metrics and margins, it shapes the elegance and efficiency of the solutions they provide. The subtle advantage of solving problems with code rather than humans is that it tends to make your core product better over time.

Crayon is small, but we dream big. We solve problems with code, not humans, because we want to serve millions of users as elegantly as possible. It’s just down-right hard to do that with too many humans in the mix, especially if you want to scale quickly. (And especially if you’re not Uber.)

The rest of this post is about a big problem we had and how we solved it with code.

The Problem: Some Web Pages Don’t Look So Hot

Crayon is a visual inspiration platform. We help marketers get great ideas so they can do better marketing.

We’re programmatically adding about half a million pages to our system every day. As I write this article, we’ve got about 13 million designs in the system, and we’ll have close to 100 million by the end of the year.

We let users vote designs up and down, so most categories on Crayon like Startup Home Pages, B2B Pricing Pages and Landing Pages have always looked pretty good.

But we still had a problem in deeper categories: folks would be happily browsing, only to have the smooth inspirational vibe unceremoniously broken by a crappy looking page. How could we get the “best” designs to the top of the result set and the “least inspiring” designs to the bottom?

We Challenged Ourselves to Solve it With Code

Can code separate “inspiring” marketing designs from “crappy” ones?

Many said it couldn’t be done… but if they were right, would I be writing this article?

Here’s How We Solved It

Step 1: Pick Training Sets

We picked a set of 200 “inspiring” marketing designs and a set of 200 “uninteresting” marketing designs.

Yes, this is a bit of a fuzzy thing to do because it’s based on human judgment. But after running the services team at HubSpot for 5.5 years, and working closely with more than 8,000 professional marketers, I felt qualified to judge which marketing designs were likely to be interesting and instructive to other marketers. So sue me!

Step 2: Make Some Guesses About Why The Good Ones Are Good

This part is like picking a basketball team without being able to watch the candidates play basketball. What characteristics of the players would you look at? For instance, you might pick taller players or players wearing high-tops.

In the style of Jeff Foxworthy’s comedy routine You Might Be a Redneck:

If your Html is all based on tables… you might not have a great marketing design (tweet)

If you’ve got lots of inline styles… you might not have a great marketing design (tweet)

If you don’t use media queries in your CSS… you might not have a great marketing design (tweet)

You get the idea.

Step 3. Test Your Guesses

We wrote some code to test each “guess” from step 2 against each design in the training sets from step 1. We were hoping to find things that were “true” for the “inspiring” pages, and “untrue” for the “uninteresting pages”.

We looked at 45 discrete “guesses”, and found 25 of them were predictive of “inspiring” marketing designs. Success!

Some of the best factors were things like:

  • Setting the viewport meta tag
  • Including Facebook Open Graph tags
  • Using the Bootstrap framework
  • Specifying Apple touch icons

Note that these factors don’t directly predict which pages are “inspiring”. Rather, these factors indicate that someone clueful created the page, which is directly predictive of a page being “inspiring”.

Some of the things we thought might work, but weren’t predictive at all include:

  • Whether a page uses JQuery doesn’t matter… it’s just too ubiquitous
  • The total text on the page doesn't matter.  It turns out there are equally as many crappy short pages as there are long pages (tweet)

The final step was some mathematical mojo based on how strong the signals were in step 2 to come up with an overall “inspiringness score” for each page.

The Results

Pages like the chatterbox.co homepage did very well:

interesting-example

While our friends at biomarkerstrategies.com didn’t fare as well:

uninteresting-example

 

Overall, here’s what we got:

humans-vs-code-chart-chart

How awesome is that?

The top 10% of Crayon search results just went from 50% “inspiring” to a whopping 93%!

We have 13 million designs in the system today, and reviewing those manually would take a 10 person team about 5 years. Ouch! But we’ll have 100 million designs by the end of the year, and that’s just the beginning. If we didn’t solve this problem with code, we’d be in serious trouble. And no humans were involved… except for the one human writing this article.

Your thoughts?

Does your business have a tough problem you plan to solve with code rather than humans?

Have you used machine learning to deliver elegant solutions to customers at scale?

Please continue the converstaion in the comments.

Go solve something with code! (tweet)

Read More