Home

Site Submissions FAQ

How Search Engines Work:

Search engines actually do three things: a) scan (i.e. spider or crawl) the web for URLs and keywords, b) index (i.e. categorize and store) those URLs and keywords in a database, and c) allow visitors to search that database. Most people only know about the third part of the search engine, that is, the "front end" that lets you search the database. This is the only part that most web surfers will ever see. However, as a webmaster, it is helpful to understand the "back end" of the search engine. This is how you get your site listed and, ultimately, how web surfers will find your site.

In in order for a search engine to be useful, it needs to have a large database of URLs. To get those, it uses a "robot" program to constantly scan the web and index what it finds. Theoretically, a search engine robot will eventually scan the entire Internet. That means that it will eventually find your site on its own. In reality, though, it might takes years to get around to your site. To speed up the process, most search engines allow you to add your site to their "list of sites to be indexed." That means you can tell their robot program that you'd like it to scan and index your site as soon as possible. It will take most major search engine robots at least two weeks to get to your site.

When the robot program makes it's way to your site, it will scan your first page and record the title and keywords. (Each search engine has a slightly different way of determining the keywords) It will also scan for any links on your page. The robot will then visit each of the linked pages and scan those as well. It will scan for links on those pages and so on. It will eventually work it's way through your entire site. Robots can theoretically index your entire site in a matter of seconds. It is considered bad netiquette though, to index someone's site that quickly because of the stress it puts on your server. Each search engine has it's own policies and methods for spidering the web. Once your site has been indexed, it is in that search engine's database. The next time a web surfer uses that search engine, your site may show up in the results.

Link Exchange

In the early days, search engines ranked sites based only on the text content of the pages. As people got wise to this they began to create pages for the sole purpose of drawing in search engine traffic. People began overloading pages with keywords - even if they had nothing to do with the site - just to get more traffic.

The search engines got wise to these tricks fairly quickly and now use various methods to make sure that you get relevant results and not just a bunch of "decoy" pages when you submit a search query. One of the current methods that is used is ranking results based on "popularity." Popularity is typically determined by the number of links elsewhere on the web to your site.

What this means is that you need to get as many links from other sites to your site as possible. The way to do this is to contact other related sites and see if you can exchange links. Some sites may even have link-exchange bulletin boards where you can post your URL. Since it is mutually benificial, most people are pretty receptive to exchanging links.

TITLE, META and ALT Tags

HTML provides a few tags that you use to describe and categorize your site. Each search engine may or may not use this information. They generally do, though, and you should take the extra time to put the right information there.

The TITLE and META tags do not require that much explaination. They go at the top of the document. If you use a graphical editor, then you may not be familiar with the raw HTML code. Most HTML editors will have a properties tab or something where you can insert the page title, description and keywords. For those of us who do edit the raw code, these tags go in the <HEAD> section of your page like so:

<HEAD>
<TITLE>Your Site Title Goes Here: 69 Characters or Less</TITLE>
<META NAME="keywords" CONTENT="keywords,separated,by,commas,avoid,common,words">
<META NAME="description" CONTENT="Describe your site in 200 Characters or Less">
</HEAD>

Generally, the TITLE tag is the most important. This is usually what will appear on the search engine results screen. You want it to be something that accurately represents your site and is catchy as well.

ALT tags are used to describe a graphic image on your page. You may notice that when you hold your mouse over certain images, a text-tip pops up. This is the ALT tag. Since search engines only index text and not graphics, it is a good idea to put something in the ALT tags. This is especially true if your site consists of mostly graphics. The ALT tag goes inside the IMG tag like so:

<IMG SRC="mygraphic.gif" ALT="Description of the Graphic">

As a side note, it is also nice to put in ALT tags when you use graphics for your navigation buttons. Otherwise, it is impossible to navigate your site using a non-graphical browser (such as lynx).