Wednesday, July 30, 2008

Cuil: When your best feature becomes your worst enemy

First Look - Product announcements are always tricky in a the way how you describe a products features in a flashy enough way to attract attention, but remain sufficiently careful to not overstate its features and benefits. Even a fantastic new product can easily be trashed in today’s Internet world if you go overboard: That may be the case with Cuil, an interesting new try to come up with a new type of search engine, which, according to its developers, carries the world’s largest search index. We have 12 hours of Cuil usage behind us to see what the engine offers. The result (surprise, surprise): Bigger isn’t necessarily better.

Image

When we first heard about Cuil for the first time we were intrigued. And apparently, we weren’t the only ones. Investors poured $33 million into the startup and Cuil clearly dominated tech headlines today. If you are investor these days with couple of millions to play with, you certainly would be looking into search engines - Silicon Valley's latest hype. Search start-ups are surfacing at a rapid pace, with the apparent hope that Google’s Cinderella story can be replicated. The latest would-be Google killer is called Cuil, pronounced cool.

Similarly to Powerset, Cuil is semantic-driven search engine. Instead of extracting text from web pages that is inserted into a database that can be searched, semantic technology (often dubbed as Web 3.0) aims to find relationships in a vast pool of seemingly unrelated online content. But bringing an understanding to this information cacophony has its disadvantages.

At first glance, you may be excited when you enter your search terms in Cuil. It does have a GUI that is much fancier than Google (at least if a black background combined with a simplistic layout can be considered as cool). The results page is nicely laid out and we imagine that some users may find it more difficult to get an overview of the results, while the page itself is definitely much more attractive than the Google results page. What you get looks very much like what you would expect to see in a traditional dictionary. But then, Google pages never have been attractive anyway and always were built with performance and usability in mind.

Cuil’s two or three column layouts (depending on your preference) display each result with the headline, a short paragraph and often an image. General search terms deliver a menu bar that categorizes the search result that enables a user to limit the scope of the search. “Honda”, for example, will categorize the result into certain Honda models and parts. There is also an "Explore by Category" section that leads to most probable categories related to your search, with each category collapsing into sub-categories. Yes, ask.com and live.com (as well as a Google Labs extension) have offered such features as well, but Cuil takes this feature to a more conclusive level, considering possible research intentions of a user.

While the concept is truly innovative and we have no doubt that it is the next stage for search engines (in a way, Google has been preparing such a step by combining its information “silos” of text, images and videos to collect usage data that may enable the evolution of Google page rank search into semantic search), a closer look reveals that Cuil has flaws that impact the relevancy search results.

If you test-drive Cuil with terms such as iPhone, you receive links leading to the iPhone section on various web sites, such as online shopping malls or iPhone sections on tech portals. The "Explore by Category" section displays various general smartphone links, a few sections on Apple's website and some (which means not all) competitor websites. Tabs at the top of the page suggest queries to expand the initial search term, such as "Free iPhone, "Apple iPhone" and "New iPhone." If you try another product-related search such as "Ferrari," you'll end up with a similar list of Ferrari sections on different web sites, but not a single article related to Ferrari.

It appears that the more scientific and the more specific a search term is, the more useful Cuil gets – which makes sense as this should reveal the advantage the search engine has over established services such as Google. “Photosynthesis”, for example, yields a higher and more reasonable results page than, for example, “notebook PC”. In this case, you will get a mixture of links you will not be able to tell how useful they are right off the bat. Also, this specific term appeared to have a strong connection to Dell and Hewlett-Packard. Other than that the first page results shows relatively useless results such as an opinion article titled “The Cheap Notebook PC: What Took Them So Long?” If you ask us, this search result is a bit too specific for the general search nature of the term.

Image

We then tried more complex searches, such as "US smartphone market share" or "steve ballmer leaked note” returned links to articles relevant to the search term, but most of them were way too outdated. Cuil does not provide enough transparency how and why it lists certain search results on the first page and others on the following pages.

While the results page may appeal to a mainstream Internet audience, all that flash cannot hide the current poor relevancy of the actual content listed. There are other issues, too. Images listed next to the search result often do not have anything in common with the article itself (we discovered this when searching for "TG Daily"). The search engine is not too intelligent either. If your complex search term consists of a well-known brand name, the results will be skewed towards the brand itself, instead to the whole search term. Other problems: We were unable to separate sponsored links from actual search results. It seems that Cuil does not run keyword-based ads at the moment but the way how certain brands and products were listed you just get the uncomfortable feeling about something fishy going on. Take the notebook example: Only Dell and HP notebooks were listed and categorized.

The search speed was always fast when the site was reachable.

Cuil has a “Preferences” section that leads to just two options at the moment: Safe Search filter and typing suggestions, but neither seems to work at this time. Yes, we are a spoiled bunch here, but if you list these features as being available then they should work.

Semantic search is all nice but it is seriously limited by the lack of a rich content index that its counterpart Google has been collecting over years to improve the relevancy of search results on Google.com. Cuil claims that its search index spans 120 billion web pages, apparently about three times the size of Google's index that was last reported to span 8.2 billion web pages three years ago. But even the biggest search index is nearly worthless, if you cannot determine the importance of a search result. There are problems with Google as well and website owners have learned to use Google’s Page Rank system to their advantage, but you still are able to find relatively quickly what you are looking for in most cases. Cuil’s search results appear to be arbitrary and won’t win any prizes in its current state.

Everything about Cuil appears to be work in progress. Both Powerset (which was recently acquired by Microsoft) and Cuil don't stand any chance to replace Google, Yahoo or Live Search anytime soon in our opinion. For instance, Powerset is good as a starting point for online research. It searches third-party knowledge sources such as Wikipedia so it can prove more quicker if you need information from a specific source. Cuil is more suited to become a web directory as it presents results that mostly lead to starting sections within a web site, rather than to the specific piece of content.

Cuil clearly looks like a possible acquisition target if it can figure out to solve its problems. Interestingly, its founders said that their search engine will not be up for a sale. The fact that it has been developed by former Google engineers surely makes some noise across the media, but we believe the announcement of the site was made too soon. There are just too many issues that need to be resolved before “cool” turns into “usability.”

At this time, Cuil seems not to be collapsing under the weight of its 120-billion page index and has no ability to showcase the benefits of this advantage over the competition.

No comments: