Crawler A program search engines use to crawl the web. Bots visit web pages to collect information and add or update a search engine’s index. Also known as: Bot, Spider, Web Crawler.