How Google gets from your page to a search result
Google describes Search as three stages: crawling, indexing and serving results. It says not all pages make it through each one. Our audit goes in that order.
- Crawling. Google's crawler downloads the page. Server trouble and robots.txt rules can keep it out.
- Indexing. Google works out what the page is about, and picks one page to show from each group of near-duplicates.
- Serving. Google matches indexed pages to a search. Its guide names the searcher's location, language and device among the factors.
The guide also says Google doesn't accept payment to crawl a site more often or rank it higher, and that anyone who tells you otherwise is wrong.
