Give us a call: (800) 252-6164

How Many Googlebot Crawlers Are There? (New data for 2023)

September 13, 2023 | By admin | Filed in: web-crawler-development.

Wondering how many Googlebot crawlers there are?

Google has 19 Googlebot web crawlers.

What Are Google’s Web Crawlers?

Here’s an overview of what Google’s web crawlers are and what they do:

Crawler NameUse
Common Crawlers
Googlebot SmartphoneUsed for mobile search indexing
Googlebot DesktopUsed for desktop search indexing
Googlebot ImageCrawling image bytes for Google Images and products dependent on images
Googlebot NewsCrawling news articles
Googlebot VideoCrawling video bytes for Google Video and products dependent on videos
Google FaviconFetching favicons; might ignore robots.txt for user-initiated requests
Google StoreBotCrawling certain types of pages like product details, cart pages, etc.
Google-InspectionToolUsed by Search testing tools like the Rich Result Test and URL inspection in Search Console
GoogleOtherOne-off crawls for internal R&D; generic crawler
Special-case Crawlers
APIs-GoogleDelivering push notification messages for Google APIs
AdsBot Mobile Web AndroidChecks Android web page ad quality
AdsBot Mobile WebChecks iPhone web page ad quality
AdsBot DesktopChecks desktop web page ad quality
AdSenseDetermine site content for relevant ad provisioning
Mobile AdSenseDetermine mobile site content for relevant ad provisioning
User-triggered Fetchers
FeedfetcherCrawling RSS or Atom feeds for Google Podcasts, Google News, and PubSubHubbub
Google Publisher CenterProcesses feeds from the Google Publisher Center for Google News landing pages
Google Read AloudFetches and reads out web pages using text-to-speech (TTS)
Google Site VerifierFetches Search Console verification tokens upon user request

What Do The Googlebot User Agent Strings Look Like?

Wondering what the Googlebot useragent strings look like? Here are samples of each one:

Crawler NameSample User Agent
Common Crawlers
Googlebot SmartphoneMozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36
Googlebot DesktopMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36
Googlebot ImageGooglebot-Image/1.0
Googlebot NewsUses various Googlebot user agent strings (not one specific string mentioned)
Googlebot VideoGooglebot-Video/1.0
Google FaviconMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Google Favicon
Google StoreBotMozilla/5.0 (X11; Linux x86_64; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36
Google-InspectionToolMozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36
GoogleOtherGoogleOther
Special-case Crawlers
APIs-GoogleAPIs-Google (+https://developers.google.com/webmasters/APIs-Google.html)
AdsBot Mobile Web AndroidMozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36
AdsBot Mobile WebMozilla/5.0 (iPhone; CPU iPhone OS 14_7_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Mobile/15E148 Safari/604.1
AdsBot DesktopAdsBot-Google (+http://www.google.com/adsbot.html)
AdSenseMediapartners-Google
Mobile AdSenseVarious mobile device types (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)
User-triggered Fetchers
FeedfetcherFeedFetcher-Google; (+http://www.google.com/feedfetcher.html)
Google Publisher CenterGoogleProducer; (+http://goo.gl/7y4SX)
Google Read AloudMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36
Google Site VerifierMozilla/5.0 (compatible; Google-Site-Verification/1.0)

Source: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers


Tags:

Comments are closed here.

Scroll To Top