How Search Engines Work

Millions of web searches are made every day but not many people understand how search engines really work. Search engines retrieve information but they also collect the information that forms your results in the first place. Although it seems that search engines such as Google, Yahoo and MSN Live are querying the World Wide Web to retrieve the information you are searching for, they are actually searching a huge database which contains details of the information that is on the web.

Types Of Search Engine

There are two types of search engine. A crawler search engine sends out an automatic program known as a spider to find a catalog the information available on web pages. Some crawlers collect page titles, while others catalog every word on a web page. This information is then stored in a large database called the index. When you do a search, you are really searching this database and the results of your search appear on another web page. The software that allows you to search the database is the third part of a search engine.

Web information is also stored in human edited directories. In most cases, the information is submitted manually by the owner of the web site, so only the information that person thinks is relevant is in the database. Once the information is in the database, you can search it in the same way as with another search engine.

Crawler search engines update their directory regularly and automatically find changes to web pages and update the search database. Human edited directories are unlikely to change once a listing has been added, which means that some of the information can be out of date.

The accuracy of the results you get on a search engine depends on how often the index is updated. As the web gets bigger, so does the task of indexing the pages, so you may end up with dead links from time to time.

Search Engine Results

While all search engines search a database, the results they produce can be very different. There are several reasons for this, but the main one is to do with the search algorithm. This means the method that search engines use to determine which web pages are relevant to the search terms you type in. With millions of pages out there, the search engines need a way to make sure you find what you are really searching for.

The search engine algorithms look at keywords, which can be any word on a page, to see if the terms you are searching for appear a few times on a particular page. This is a sign that this page may match your search criteria. However, search engines give this less weight than they did, mainly because some people have tried to manipulate the ranking of their web pages by keyword stuffing - artificially increasing the number of times particular keywords appear on a page.

The other part of a search engine algorithm is looking at the way pages link to other pages. If a site is deemed to be important and links to another site, then the second site gets some kudos from the first site. These two factors together determine the results that you see.

Finally, there is another type of search engine - a meta search engine such as Widow and Metacrawler. This type of search engine does not search the internet directly. Instead, it searches the indexes of other search engines all at once to get the results that are most relevant to your search.