Search Engines

Getting a first page listing in the major search engines is like the 'holy grail' of free advertising. It's an advertisers dream come true... Because it's free, however, there's a lot more competition - suddenly everyone can afford to advertise!

Fortunately for you, most webmaster's aren't very good at optimising their web pages, and it's still relatively easy to get a high ranking in the major engines. Search engine optimisation is big business!

See the the tutorial on doorway pages. for loads of info on search engines and how they work, and how to optimise your webpages. (I'm currently running an experiment to see whether pages which have been 'optimised' for the search engines (by WPG's Page Critic) do actually rank higher than the ones I wrote myself...)

Before you'll be able to tell how well optimised your pages are, you'll need to get your site listed in the search engines.

General consensus now has it that manual submission of your website to the most important search engines is the best way to get listed.

Automatic submission software/services like WebPosition Gold or the online submission services you see advertised may be useful for getting your site listed in the plethora of smaller and regional engines, but whether they get you much traffic is debateable...

Quite a few search engines are now operating a Cost-Per-Click (CPC) system where you simply won't be listed for free. See the Promotion/Advertising page for details of paid promotion / advertising and the Cost-Per-Click engines.

target="_top">SearchEngineWatch.com has been reporting since 1995 and has some great advice. Some websites are like institutions, this is one of them.

Books (from Amazon)
>Search Engine Visibility by Shari Thurow	Search Engine Visibility is not about merely obtaining top positions in search results. Rather, Search Engine Visibility is about designing, writing and creating a web site primarily for your sites visitors, and helping them find what they are searching via the major search engines, directories, and industry related sites. Most web developers/designers build a site first and worry about "searchability" later, which is a costly mistake.
>The Extreme Web Searcher's Guide to Web Seach Engines: a Handbook for the Serious Searcher by R Hock	Describes search strategies, explains how documents are pulled into engines's databases, and explores the special characteristics of such Internet search engines as AltaVista, Excite, Infoseek, Yahoo!, HotBot, and Lycos.

Online Resources There are some great online resources available to webmasters to help with Search Engine Optimisation:
Tutorial from WebMonkey	A great tutoriasl on how to optimise your pages for free.
target="_top">SearchEngineWatch.com	There's plenty of info freely available, and a members area with the more cutting edge stuff. Recommended!
Search Engines	A general article from webreference.com
hypermart.net	...give away this rankings checker from their site (as a marketing tool to pick up leads!). If you'd rather not give your email address try Keysub.com
Meta Medic	From northernwebs.com will provide basic checking of your meta tags.

How Much Disk Space?

Thousands of web pages are added to the Internet every day, there are an estimated 40 thousand million (a US billion) web pages out there (at time of writing), it's an amazing volume of information!

The basic problem that search engines face is that the amount of space required to store details about every page on the web is immense...

For example, if for each web page we store the absolute minimum data:

• URL - 100 characters
• Title - 100 characters
• Keywords - 100 characters
• Description - 200 characters
• Body Text - 500 characters.

That's just 1000 bytes (1 byte stores 1 character) per web page...

So a 10Gb disk could store info on 10,000,000,000 / 1000 = 10 million web pages

Google indexes (on 2/5/2003): 3,083,324,652 web pages.

This would be: 3,083,324,652,000 bytes = over 3000 Gigabytes!

In fact, this is a simplified example, it would actually take more space than this because the database indexes also occupy disk space!