In web development, design and functionality are the keys to a website’s success. However, neither of them would mean anything if the website could not be found. That is where search engines come in. The question is: what actually happens when the search button is pressed?
The way a search engine works is “bots” or “spiders” search the web based on a user query and then rank the results according to a ranking system (i.e. Google’s™ PageRank). When a user enters a search query into a search engine, they think they are searching “the web” but what they are actually searching is an index that represents the content on the web.
A bot is a computer program that performs repetitive tasks at nearly impossible speeds that no human can replicate. They constantly “crawl” the web looking for new or updated pages and then create a list of keywords based on those pages. Next, they index the pages they find and then encode them to save space and for easy user access. Once the user enters their query, the bot searches the index while updating and looking for new pages that match that query. Once they have a list of results, they rank them according to a ranking system and then display them to the user. All of this happens within a tenth of a second.
By Jason Ellis, Web Developer