How Search Engines Work:
Search engines actually do three things: a) scan (i.e. spider or crawl) the web for URLs and keywords, b) index (i.e. categorize and store) those URLs and keywords in a database, and c) allow visitors to search that database.
Most people only know about the third part of the search engine, that is, the "front end" that lets you search the database. This is the only part that most web surfers will ever see. However, as a webmaster, it is helpful to understand the "back end" of the search engine. This is how you get your site listed and, ultimately, how web surfers will find your site.
In in order for a search engine to be useful, it needs to have a large database of URLs. To get those, it uses a "robot" program to constantly scan the web and index what it finds.
Theoretically, a search engine robot will eventually scan the entire Internet. That means that it will eventually find your site on its own. In reality, though, it might takes years to get around to your site. To speed up the process, most search engines allow you to add your site to their "list of sites to be indexed." That means you can tell their robot program that you'd like it to scan and index your site as soon as possible. It will take most major search engine robots at least two weeks to get to your site.
When the robot program makes it's way to your site, it will scan your first page and record the title and keywords. (Each search engine has a slightly different way of determining the keywords) It will also scan for any links on your page. The robot will then visit each of the linked pages and scan those as well. It will scan for links on those pages and so on. It will eventually work it's way through your entire site.
Robots can theoretically index your entire site in a matter of seconds. It is considered bad netiquette though, to index someone's site that quickly because of the stress it puts on your server. Each search engine has it's own policies and methods for spidering the web.
Once your site has been indexed, it is in that search engine's database. The next time a web surfer uses that search engine, your site may show up in the results.
In the early days, search engines ranked sites based only on the text
content of the pages. As people got wise to this they began to create
pages for the sole purpose of drawing in search engine traffic. People
began overloading pages with keywords - even if they had nothing to do with
the site - just to get more traffic.
The search engines got wise to these tricks fairly quickly and now use
various methods to make sure that you get relevant results and not just a
bunch of "decoy" pages when you submit a search query. One of the current
methods that is used is ranking results based on "popularity." Popularity
is typically determined by the number of links elsewhere on the web to your
What this means is that you need to get as many links from other sites to
your site as possible. The way to do this is to contact other related
sites and see if you can exchange links. Some sites may even have
link-exchange bulletin boards where you can post your URL. Since it is
mutually benificial, most people are pretty receptive to exchanging links.
TITLE, META and ALT Tags
HTML provides a few tags that you use to describe and categorize your site. Each search engine may
or may not use this information. They generally do, though, and you should take the extra time
to put the right information there.
The TITLE and META tags do not require that much explaination. They go at the top of the document. If you use
a graphical editor, then you may not be familiar with the raw HTML code. Most HTML editors
will have a properties tab or something where you can insert the page title, description and keywords.
For those of us who do edit the raw code, these tags go in the <HEAD> section of your page like so:
<TITLE>Your Site Title Goes Here: 69 Characters or Less</TITLE>
<META NAME="keywords" CONTENT="keywords,separated,by,commas,avoid,common,words">
<META NAME="description" CONTENT="Describe your site in 200 Characters or Less">
Generally, the TITLE tag is the most important. This is usually what will appear on the search
engine results screen. You want it to be something that accurately represents your site and is
catchy as well.
ALT tags are used to describe a graphic image on your page. You may notice that when you hold
your mouse over certain images, a text-tip pops up. This is the ALT tag. Since search engines
only index text and not graphics, it is a good idea to put something in the ALT tags. This is
especially true if your site consists of mostly graphics. The ALT tag goes inside the IMG tag
<IMG SRC="mygraphic.gif" ALT="Description of the Graphic">
As a side note, it is also nice to put in ALT tags when you use graphics for your navigation
buttons. Otherwise, it is impossible to navigate your site using a non-graphical browser
(such as lynx).