Google is undertaking the most radical change to its search results ever, introducing a “Universal Search” system that will blend listings from its news, video, images, local and book search engines among those it gathers from crawling web pages. The new system officially rolls out today for anyone using Google.com and searching in English. Not everyone will see it […]

May 16, 2007 at 2:33 pm

Google is undertaking the most radical change to its search results ever, introducing a “Universal Search” system that will blend listings from its news, video, images, local and book search engines among those it gathers from crawling web pages.

The new system officially rolls out today for anyone using Google.com and searching in English. Not everyone will see it at first, but over the course of the next several days, Universal Search should be more, well, universal. A new navigational interface has also been unveiled for Google and is covered more in the companion piece to this article, Google’s New Navigational Links: An Illustrated Guide.

The move potentially should be a huge boon for searchers, while search marketers who have paid attention to the importance of specialized or vertical search will see new opportunities. To fully explain the importance to both groups, I’m going to work step-by-step through the concept of vertical search engines, how they’re often ignored by searchers and search marketers alike, then how Google is going to make this content more visible through Universal Search.

As this is a long article, here are jumps to specific sections, should you wish to skip ahead:

What The Hell Is Vertical Search?

I hate the term vertical search. I routinely ask audiences if they understand what it means, and significant numbers don’t. I prefer the term specialized search or specialty search, but the financial community in particular has popularized the use of “vertical” search. Let me explain the concept.

Regular search — when you go to Google, Yahoo, Microsoft, Ask or any general-purpose search engine — is a “horizontal” search in that you are searching across a wide spectrum of material. Information from sports sites, news sites, medical sites, shopping sites — the entire horizontal spectrum of topics is represented.

With vertical search, you slice down vertically through one topic area. You search only against the news sites or against the medical information, for example. This type of focus can make for more relevant results.

Need help on fixing windows in your house? Search on that horizontally, and information about the Windows operating system might dominate the listings, simply because there’s so much about Windows out there. Try using a vertical search engine that only has home improvement information, and information about repairing the type of windows you look out of should become much more visible and relevant.

Making Vertical Search Visible

Sadly, many people are unaware of the vertical search resources that are out there. Indeed, Google has long put links to its vertical search services above the search box on its home page. First they were in the form of “tabs,” then later as regular links that currently promote that you can vertically search for images, or video, or news, or maps or in many more specialized areas.

To really understand all the vertical searches Google itself runs, consider this list:

• Blog Search
• Book Search
• Catalogs
• Code Search
• Directory
• Finance
• Images
• Local/Maps
• News
• Patent Search
• Product Search
• Scholar
• Video
• Web Search

“You almost need a search engine for all our search engines,” said Google’s Marissa Mayer, vice president of search products and user experience, in talking with me about today’s Universal Search news. “We’re so excited about taking all these different silos of information and making them all into one.”

Certainly the home page links haven’t worked to expose the vertical richness. Few searchers use those links.
Test this yourself.
Ask someone if they know what those links do. Quite often, they’ll tell you no — in fact, that they’ve not seen them before. I’ve shown people how do to an image search using them and literally gotten gasps of astonishment in response. These people had no idea you could search and get back actual images.

I’ve long written about this problem and described the solution as “invisible tabs.” From my article on the topic back in December 2003:

Many people simply do not see or use tabs, just like they regularly ignore drop down boxes, radio buttons and any type of other option you put out. Search engines have told me this over the years, and I also see it first hand….

The solution to tab blindness is clearly for me and an army of other search educators to head out and teach people how to use tabs! Naturally, that’s not going to happen. No, the solution really is for the search engines to make use of “invisible tabs,” where they make the correct choice for the user, behind the scenes.

This is what Google is doing with Universal Search. Google’s hitting several of its vertical search services all at once, then bringing back those results to blend in with “regular” results.

More Than A OneBox To Me & Book Search Blending

Hasn’t Google been already making vertical results available through those OneBox displays that appear at the top of some search results? Yes, but Universal Search is going beyond that. Let’s look at Google Book Search to illustrate this.

Google Book Search is Google’s vertical search engine that lets you find matching pages within books that it has scanned. Curious about that part in Tom Sawyer where he was whitewashing the fence? Enter tom sawyer fence, and Google Book Search tells you it’s on page 22 of one edition of Mark Twain’s classic novel.

So, Google Book Search lets you look within the content of books, but only if you specifically know to use that search engine. Most people just turn to Google’s regular search. That’s why Google tries to remind you there’s more than web search using OneBox displays like this:

See how the arrow is pointing at book search matches? This little section is called a OneBox, and it sits above and separate from the 10 “regular” search results that come from Google’s crawling of the web. This is effectively Google saying “Hey! We’ve got a book search engine with matches for this. You might want to check the results there!”

If you did click through, you’d then search just against the book content and get back matching book pages, like this:

Some people skip past the OneBox prompts because they look weird compared to the “real” results. However, Mayer said the bigger issue was that OneBoxes were “clunky” and often not as relevant as the results coming below them.

For example, Mayer said that the first result in Google might get clicked on more than a third of the time out of all the clicks on that page. In fact, Google’s search quality team makes it a goal to try and make that first result as relevant as possible. In contrast, the OneBox insertions get clicked on less despite getting prime placement on the page, since they aren’t as relevant.

Universal Search aims to fix this through blending. With Universal Search, Google will hit a range of its vertical search engines, then decide if the relevancy of a result from book search is higher than a match from web page search. (FYI, Infoseek got a patent related to this type of blending back in 1997; See also: Google’s Universal Search Patent Application & Assigned Patents from Infoseek from SEO By The Sea).

If so, a web page result might get dropped and a book search result inserted. That means instead of a OneBox, book results will begin showing up in Google search like this:

See how this is the same page listed at the top of Google Book Search, when I did a specific search there. This screenshot from Google illustrates how in the future, that “book page” will just get mixed in among the web pages of regular Google searches.

Of course, Google already does this with some of its other properties. For example:

• Google Code (see juls sampler)
• Google Base (see bayside florals & plants)

So mixing in book search results isn’t quite as radical as it sounds on the face of it. On the other hand, listings from the services above are getting in because they’re part of the web page index, as best I can tell. Rather than replacing a web page listing, as Universal Search does with book search, they’ll be part of web search (and potentially one of the listings that might get kicked out to make room for material from a vertical resource).

FYI, the OneBoxes aren’t dead, not even for cases where Universal Search is in place. In other words, you might still see a news OneBox or a Books OneBox. Says Google:

While the OneBox hasn’t entirely gone away, it has been replaced in many cases by blended results in relevant positions. As we continue development, we’ll continue to evolve our use of both the OneBox and blended results to deliver the most relevant results.

News & Local Blending

I’ve covered how book search results will be blended. Let’s turn next to news and local result blending, since they work in similar ways.

Here’s how news results currently get inserted:

Google Universal Search: Old News OneBox

As Universal Search rolls out this week, that OneBox display will get replaced, with news material blended like this:

Google Universal Search: Blended News

Similarly, local search results are being integrated. Before Universal Search, a OneBox would display them like this:

With Universal Search, they’re blended like this:

FYI, news search results began being blended last month, for some people, as part of an official change. Local blending started getting spotted by other widely yesterday.

Single Listing Vs. Multiple Listings Replacement

With the news blending example above, Google dropped only one of its usual web page listings to insert either a news unit. That unit can move around, by the way. It might be at the top of the page, somewhere in the middle or even at the end, based on wherever Google determines it will be most relevant related to the other listings. But what it is NOT supposed to be is positioned outside the regular 10 results in the way the OneBoxes have historically been. Instead, these units take one of the 10 regular positions.

With the local blending example, you can see that three web page results have been removed to make room for the three local listings along with the map. But sometimes, only one local listing will show. Google told me:

For the time-being, three local results are taking the place of three web results for result sets that trigger a grouped local UI. This happens only to categorical use cases (e.g. restaurants mountain view, new york dental school), where a group is appropriate. When there’s just one navigational result, like siam royal palo alto, there’s only one local result displacing one web result. While there are no immediate plans to change this, it could happen over time.

Google’s not taking the more dramatic step of simply replacing the bulk of the 10 web page listings with news or local material, even if a query might indicate this makes sense. For example, a search for new york hotels is pretty local oriented. Why not shift over to showing a full page of local results? Or if there’s a breaking news event, say with the Virginia shooting, most people searching under that term probably would be better served by getting many more actual news results than shown in a single unit.

Video Blending & Video Meta Search

There’s nothing new about Google listing links to video content in its regular search results, including links to pages in Google-owned Google Video and YouTube. However, Universal Search is changing things. Instead of considering that content as part of its web pages index, Universal Search will run a video search query across an enhanced version of Google Video.

What’s enhanced? Google Video is morphing into a meta video search site, similar to how Blinkx operates. Eventually, Google Video will list content that’s hosted at Google Video, YouTube or anywhere on the web.

Today, Google Video itself will remain listing only Google Video and YouTube content. The enhanced version will only be accessible through Universal Search. In the near future, Google Video itself will shift over.

As for that video content, unlike with news and local, more than one blended results might appear. For video content outside of Google Video and YouTube, the display will be like this:

Google Universal Search: Blended Video

Content on Google’s own video hosting services will look like this:

Google Universal Search: Blended Video

See the little + Watch Video part in the listing? That’s a video Plus Box, which have been spotted in the past. Now they are being formalized. Clicking on that will open up the video and let you watch it within the search results, like this:

Google Universal Search: Blended Video

After years of people ignoring tabs and links designed to get them to do specialized searches, Google is taking the plunge to push the right buttons behind the scenes and make specialized or vertical search results part of the “normal” experience.

Image Blending & Multiple Blends

For some searches, matches from Google Image Search will appear. Unlike the other blending, these will NOT take the place of one of the 10 regular listings. They were found to be too disruptive that way, Mayer said. Instead, those deemed to be of high relevancy will appear at the top of results, while those that are important but perhaps not as relevant will be shown at the bottom.

It’s also important to note that blending may happen from one or more of the vertical search engines. For example, Mayer talked about a search for Steve Jobs where pictures of him would be on top, news in the middle and a video clip at the bottom.

And Web Search Is….?

One thing that’s slightly confusing is that Universal Search will continue to be labeled as “Web” on the search results page. OK, most of the content is coming from web-based sources (Book Search is the key exception). But aside from these vertical search databases, there also remains a WEB PAGE database, what Google originally started with and what remains the core to everything else. I almost wish Google had labeled this “All” or “Everything” search, so that Web search could be a specific drill-down option for those specifically after web page content.

Impact On Searchers & Search Marketers

Overall, I think this is a great move for Google to make, assuming the actual implementation works. Mayer was at pains to explain this is the start of Universal Search, and that Google expect to refine and adjust things going forward. It should be helpful to searchers who have simply continued to ignore vertical search properties and miss out on good content within them.

As for search marketers, I’ve written for years that there were to two major changes coming for search engines: personalization of results and the growth of vertical. From 2003:

As a search engine marketer, the implementation of invisible tabs means that traffic from organic web page listings will diminish over time. The more specialty databases are implemented, the less traffic will fall to web search results

That needn’t be a problem. The skilled search engine marketer’s most important asset is understanding how search engines get various types of listings, then helping their clients enter the appropriate databases. Think beyond web page databases, and you’ll be prepared for the future.

From last year:

Vertical search is important because it’s one of the two major things I’ve long talked about as being how search will advance. First generation search analyzed words on a page to rank content. Second generation search tapped into link analysis. Third generation search to me is looking at both user input (what we visit; what we click on; personalized results) and making search go more vertical.

Why the generational jumps at all? To improve relevancy. Each generation has fought the double challenge of being overwhelmed by increasing amounts of information to organize plus having ranking systems become more and more vulnerable to manipulation and outright spam.

Google’s now done both now within the space of weeks, moved forward on the personalized and vertical fronts.

Google Ramps Up Personalized Search and Google Search History Expands, Becomes Web History cover key things that both marketers and searchers need to understand, on the personalized from. As for verticals — search marketers must understand that the supremacy of web page search just took a major hit. Web pages will still draw traffic — still plenty of traffic. But vertical search databases offer what remains a largely unexplored and ignored frontier. Pay attention to them, at Google and elsewhere.