Real time indexing has become the new holy grail in the search engine business. Pandia takes a look at how Google is doing.
The Twitter effect
It is Twitter that is the culprit. If you follow the debate that is taking place there on a specific topic, you may get the news as soon as they happen.
Google (NSDQ:GOOG) has to compete with that, but the traditional web search engines were not really built for that kind of immediate indexing.
First of all it takes time for the search engines to visit all the existing web sites (Google now has some 1 trillion web pages in its index), and if you want your search engine index to stay perfectly fresh, you would have to revisit all those pages once every ??? well, I don???t know ??? every five minutes, maybe?
That kind of indexing traffic would cause the Internet to kneel and webmasters to despair. It is not feasible.
Google makes a compromise. How often Google revisits a site is dependent on its popularity, authority (i.e. to what extent other ???good??? sites link to it) and its updating frequency.
