Internet search engines and artificial intelligence

By | July 10, 2007

The Search book coverA couple of days ago, I finished reading John Battelle’s The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture. The book is about the rise and fall of online search engines, a domain that for the last few years has been dominated by Google.

The book makes for some good reading and I highly recommend it to anyone who is interested in knowing how, in less than 10 years, companies such as Google and Yahoo became household names and an integral part of our lives.

But the reason I mention the book here is because Battelle mentions Artificial Intelligence and its relation to search in his book. In fact, he does so in the very first chapter in a section titled “Search as Artificial Intelligence?” As you may have guessed from the section’s title, he is a bit off when it comes to understanding the link between AI and search.

Battelle mostly focuses on Google’s efforts to create a natural language interface for their index,

“I would like to see the search engines become like the computers in Star Trek,” Google employee number one, Craig Silverstein, quips. “You talk to them and they understand what you’re asking.”

Other search engines, most notably Ask Jeeves, have also pursued this goal and continue to do so with little success. Natural language understanding is an important field within AI but search is far more integral to AI than this.

A.I. the movieThe major components of an AI system are a knowledge base (KB,) an Inference/Reasoning Engine and a language that we can use to communicate with the computer. The KB (basically a database) makes it possible for us to write down knowledge/facts such as “the sky is blue” or “cars have 3 or 4 wheels.” The inference engine allows us to derive new facts from those in the knowledge base, or in other words, it gives us the ability to compute answers to questions posed in the system’s language. Semantics is also a very important component but I won’t go into it now because it is usually hard to explain in a couple of sentences and I want to talk about search.

So, how does search fit into an AI system?

Well, search is the engine underneath the hood that makes the inference/reasoning engine work.

For any interesting domain, the size of the KB and the additional facts that can be inferred from it comprise a huge space. Efficient search algorithms are important for navigating this space in order to answer users’ queries. Algorithms such as Depth First Search (DFS,) Breadth First Search (BFS,) Heuristic Search and Stochastic Search are what make inference possible. Brute force methods such as DFS and BFS don’t really work for large problems and this is why stochastic and heuristic search algorithms are the focus of most current research. For those of you have an undergraduate degree in Computer Science, I hope that you now know why we insist on teaching you these search algorithms. They are extremely important!

So, how does Google fit into all this?

Google logeGoogle is on a mission to index all of the data that is available on the World Wide Web, our desktop computer and also in our printed books and press. Not forgetting of course satellite and street level images and video. I believe their corporate motto is “Don’t be evil” but their mission is to “organize the world’s information and make it universally accessible and useful.” Google’s index is in fact a huge KB and the well known PageRank (PR) value assigned to every page they index is a heuristic (one of many that Google uses) that makes it possible to efficiently search through this enormous KB. The language that we use to access the KB and the inference engine in order get our questions answered is the keyword-based interface on Google’s simplistic homepage.

In other words, Google is and it has always been an AI system. The natural language interface that Battelle talks about in his book would be a mostly welcomed upgrade to the system’s language, i.e., a more intuitive way for asking questions and probably for presenting answers back to us.

2 thoughts on “Internet search engines and artificial intelligence

  1. SEO Web Designer

    This is a book I’ve picked up a few times to buy, but always put it back down for some reason. After reading this, I’m much more inclined to walk it over to the checkout counter.

    After reading many claims that natural language search engine AIs were right around the corner, it is a bit disappointing that they haven’t lived up to most expectations yet. There are a lot of promising projects in the works, but I think it is going to take the money, man power and search engine knowledge that Google has acquired to provide the first engine that qualifies as a suitable beginning to natural language search.

    If you have time, what are your time estimates for a good start to a natural language search engine? What will you expect from a decent natural language search engine?

    In other words, I know some engines could claim to be natural language search engines, but what do you expect a good one to be able to do and when will the first arrive?

  2. Awesom-o

    Those are good questions. It is hard to say for sure what the future holds especially considering that AI has a track record of making huge promises and then failing to deliver. That said, let me think about this for a couple of days and then respond with another post.


Comments are closed.