Know Thy SEO Spider

NO I am not talking about the eight-legged hairy creatures of nightmares. These spiders are electronic, used by search engines, and if you ignore them they give you nightmares that will haunt you every waking moment

Search Engine Optimization (SEO) is all about common sense; it doesnt necessitate specific knowledge of algorithms, programming and classifications. All it requires is a basic understanding of how search engines spiders work. Taking care of the search engine spiders likes and dislikes can prove to be crucial for the health and success of your website.

The Spider at Work

They are called spiders because they crawl over the Web in search of content. Search engines gather data about a website by sending the spider or bots to read the site and copy its content. This content is stored in the search engines database. As they copy and absorb content from one document, they create record links and send other bots to make copies of content on those linked documents this process goes on and on. As of now the major search engines have established databases that measure their size in the tens of billions using this process.

Spiders are designed to read site content like a human. It will start at the top left hand corner, reading line by line. In case of columns spiders will read the left hand column before moving to central and right hand column. It will read and record the whole site to the very last word.

Likes of the Spider

Provide clear paths for spiders to follow, put easy to follow text links directed to the most important pages in the site at the bottom of each document. The sitemap is the most important link in the site as it directs spiders through the website.

Provide as much relevant key word rich text as possible. Search engine spiders read the text that is on your Web site to decide if your site is relevant or not. If they do not find the keyword being searched they will simply ignore your site and move to the next site. Also make sure the text is relevant and makes sense.

Provide clear identification between areas that to be used and areas that are off limit to the spiders.



Provide clear titles; after the URL of a site, the first information a spider records is the title of the site. Titles should have the most important keyword targets.

Provide a well written Description Meta tag. Search engines use Description Meta tag to collect information on the topic or theme of the site.

Dislikes of the Spider

Avoid Very large volume in content. Spiders can only download a fraction of the Web pages within a given time frame, so if you have too much it might skip the most important parts.

Avoid dynamic page generation, spiders generally tend to avoid pages having any resources that have a ? i.e. dynamic in nature. Stick to static generation spiders prefer them.

Avoid too many graphics. Spiders are not fond of graphics as they are not readable for them so keep it simple.

Avoid Key word misuse. Dont just string a lot of keywords together, remember the spiders are very smart and they will ban you for spamming.

About Author