OnStartups

At my startup, HubSpot, we have been working with the Google Search API to implement some of the features we think would help our customers.

The Search API is reasonably robust in that it supports the various features of the Google search engine (finding related links, approximating the number of results, etc.)

But, there is one critical feature that the brainiacs at Google either forgot to include (which is bad) or intentionally left out (which is really bad).

Outside of normal “search” type stuff, I think one of the most common reasons people would use the API is to answer one simple question:

Most common question:  For a particular search phrase, where does my site rank on Google?

The reason this question is common should not be surprising (most webmasters, bloggers and SEO consultants care about this issue).  It’s also difficult to answer this question via the regular search engine (without manually entering the search term, and paging through the results looking for a “match”.  There are web utilities out there (that let you enter your API key and run a query), but they’re just doing a brute-force iteration over the result set too.

Here are some thoughts on the topic:

1. As it stands, there is no way to answer the above simple question without making repeated calls to the Search API (basically retrieving a page at a time and checking the results until a match is found).

2. This is even more annoying because Google only allows you to retrieve 10 result items at a time.  So, to figure out if you are in the top 100 hits for a search phrase, you have to hit Google 10 times.

3. This is made yet more annoying because Google limits the number of calls you can make to their API to 1,000 (with no clear way of increasing this limit – even by paying money).

4. It seems (at least from my perspective), extremely easy to implement this feature.  All they would have to do is include a separate method call that took a search query and a site name as parameters and returned the position of the first “match”.  This way, I could figure out that when searching for “software startups”, that this site (OnStartups.com) is the #5 hit.

Given how smart the Google folks are and how common this particular need likely is, I have only two theories about why they left this feature out:

1. Google intentionally left this feature out for some “strategic” reason.

2. Google doesn’t realize how important this missing feature is.

For the Google API experts out there:  Am I missing something simple?  Is there a work-around to this, or have I stumbled into something that is already widely known and has already been discussed to death?  If you have insight, please leave a comment.  All help is appreciated.