Contents
Let’s begin by exploring what search engines are, why they exist, and how they make money.
What are search engines?
Search engines are searchable databases of web content. They’re made up of two main parts:
- Search index. A digital library of information about webpages.
- Search algorithm(s). Computer program(s) tasked with matching results from the search index.
What is the aim of search engines?
Every search engine aims to provide the best, most relevant results for users. That’s partly how they gain market share.
How do search engines make money?
Search engines have two types of search results:
- Organic results from the search index. You can’t pay to be here.
- Paid results from advertisers. You can pay to be here.
Each time someone clicks a paid search result, the advertiser pays the search engine. This is known as pay-per-click (PPC) advertising, and it’s why market share matters. More users mean more ad clicks and more revenue.
Each search engine has its own process for building a search index. Below is a simplified version of the process Google uses.[1]
Let’s break it down.
URLs
Everything begins with a known list of URLs. Google discovers these in many ways, but the three most common are:
- From backlinks. Google has an index of hundreds of billions of webpages.[2] If someone links to a new page from a known page, Google can find it from there.
- From sitemaps. Sitemaps tell Google which pages and files you think are important on your site.[3]
- From URL submissions. Google lets site owners request crawling of individual URLs in Google Search Console.
Crawling
Crawling is where a computer bot called a spider visits and downloads known URLs. Google’s crawler is Googlebot.[4]
Processing and rendering
Processing is where Google works to understand and extract key information from crawled pages. To do this, it has to render the page, which is where it runs the page’s code to understand how it looks for users.
Nobody outside of Google knows every detail about this process. But it doesn’t matter. All we really need to know is that it involves extracting links and storing content for indexing.
Indexing
Indexing is where processed information from crawled pages gets added to the search index.
The search index is what you search when you use a search engine. That’s why getting indexed in major search engines like Google and Bing is so important. Users can’t find you unless you’re in the index.
Did you know?
Google owns 91.43% of the search engine market. It can send you more traffic than other search engines, as it’s the one most people use.[5]
Discovering, crawling, and indexing content only make up the first part of the puzzle. Search engines also need a way to rank matching results when a user performs a search. This is the job of search algorithms.
What are search algorithms?
Search algorithms are formulas that match and rank relevant results from the index. Google uses many factors in its algorithms.
Key Google ranking factors
Nobody knows every Google ranking factor because Google hasn’t disclosed them. But we do know some key ones. Let’s look at a few of them.
Backlinks
Backlinks are links from a page on one website to another. They’re one of Google’s strongest ranking factors.[6] This is probably why we saw a strong correlation between linking domains and organic traffic in our study of over a billion pages.[7]
It’s not all about quantity, though. Quality matters too. Pages with a few high-quality backlinks often outrank those with many low-quality backlinks.
Did you know?
You can check backlinks to your website for free in Ahrefs.
Sign up for a free Ahrefs Webmaster Tools account, enter your domain in Site Explorer, and go to the Backlinks report.
Our crawler is the fifth most active on the web,[8] so you’ll see a pretty complete view of your backlinks here.
Relevance
Relevance is the usefulness of a given result for the searcher. Google has many ways of determining this. At the most basic level, it looks for pages containing the same keywords as the search query. It also looks at interaction data to see if others found the result useful.[9]
Freshness
Freshness is a query-dependent ranking factor. It’s stronger for searches that call for fresh results.[9] That’s why you see a recently published top result for “new netflix series” but not “how to solve a rubik’s cube.”
Page speed
Page speed is a ranking factor on desktop and mobile.[10][11] But it’s more of a negative ranking factor than a positive one. This is because it negatively affects the slowest pages rather than positively affect lightning-fast pages.
Did you know?
You can check your page speed for free in Ahrefs.
Sign up for a free Ahrefs Webmaster Tools account, crawl your website with Ahrefs’ Site Audit, then go to the Performance report. In general, the less red you see, the better.
Mobile-friendliness
Mobile-friendliness has been a ranking factor on mobile and desktop since Google’s switch to mobile-first indexing in 2019.[12]
Google tailors search results for each user. It uses information such as your location, language, and search history to do this.[9] Let’s take a closer look at these things.
Location
Google uses your location to personalize the results for searches with local intent. That’s why all results for “italian restaurant” are from or about local restaurants. Google knows you’re unlikely to fly halfway around the world for lunch.
Language
Google knows there’s no point in showing English results to Spanish users. That’s why it ranks localized versions of content (if available) to users who speak different languages.
Search history
Google saves the things you do and places you go to give you a more personalized search experience.[13] You can opt out of this, but most people probably don’t.
Key takeaways
- A search engine consists of two main parts: index and algorithms.
- To build its index, it crawls known pages and follows links to find new ones.
- The aim of search algorithms is to return the best, most relevant results.
- Search result quality is important for building market share.
- Nobody knows all of Google’s ranking factors for organic results.
- Key ranking factors include backlinks, relevance, and freshness.
- Google personalizes its results based on your location, language, and search history.
References
- “Understand JavaScript SEO Basics”. Google. Retrieved August 16th 2022.
- “Organizing Information — How Google Search Works”. Google. Retrieved 16th August 2022
- “Learn about sitemaps”. Google. Retrieved 16th August 2022
- “Googlebot”. Google. Retrieved 16th August 2022
- “Search Engine Market Share Worldwide”. Statcounter. Retrieved 16th August 2022
- “Google Q&A+ #March”. YouTube. Retrieved 16th August 2022
- “90.63% of Content Gets No Traffic From Google. And How to Be in the Other 9.37%”. Ahrefs. January 31, 2020. Retrieved 16th August 2022
- “CloudFlare Radar”. CloudFlare. Retrieved 16th August 2022
- “Ranking Search Results — How Google Search Works”. Google. Retrieved 16th August 2022
- “Using site speed in web search ranking”. Google. Retrieved 16th August 2022
- “Using page speed in mobile search ranking”. Google. Retrieved 16th August 2022
- “Mobile-first indexing best practices”. Google. Retrieved 16th August 2022
- “Find & control your Web & App Activity”. Google. Retrieved 16th August 2022