Know about Google bot sites
Search Engine Roundtable has published a list of bots that belongs to Google. Bots, or robots is a software agents to gather information. Bots and robots from Google are also called web crawlers or spiders in which they are web robots that recursively gather web-page information, as does the bot used by Google (“GoogleBot”).
According to the article, current Google robots that are active including the common GoogleBot (compatible; Googlebot/2.1; +http://www.google.com/bot.html) which is the main spider for the search engine, MediaBot (Mediapartners-Google/2.1) which analyzes web pages with Google AdSense ads to determine relevant ads, ImageBot (Googlebot-Image/1.0) which crawls for the Image Search, AdsBot (AdsBot-Google) which checks Google AdWords landing pages’ quality, Feedfetcher (Feedfetcher-Google) which retrieves rss feeds when users explicitly added them to their Google homepage or Google Reader, Generic Mobile Phone (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) which translates (or “transcodes”) these pages by analyzing the original HTML code and converting it to a mobile-ready format.
Other robot is Google WAP Proxy (Google WAP Proxy/1.0) which is a bridge for mobile phones to access and adapting to normal web pages via the wap protocol by converting .html to .wml.
0 Komentar