Spider

Software, part of a search engine. The spider, also crawler, browses the sites in the World Wide Web and collects information for a search engine database update. The large search engines have a few spider operating at he same time, but sometimes it could take up to five months for a page to be crawled.