Importance of Sitemaps for Google Success

There are several SEO tips and tricks that help in optimizing a site but one of those, the importance of which is sometimes underestimated is sitemaps. Sitemaps, as the name implies, are just a map of your website – i.e. on one single page you describe the structure of your site, its sections, the links between them, etc. Sitemaps make navigating your site easier and having an updated sitemap on your site is good both for your users and for search engines. Sitemaps are thus an important way of communication with search engines. While in robots.txt you tell search engines which parts of your site to exclude from indexing, in your site map you tell search engines where you’d like them to go. Sitemaps are not a new thing. They have always been part of best Web design practices but with the adoption of sitemaps by search engines, now they become even more important. However, it is necessary to make a clarification that if you are interested in sitemaps mainly from a SEO point of view, you can’t go on only with the conventional sitemap  (though currently Yahoo! and MSN still keep to the standard html format). For instance, Google Sitemaps uses a special (XML) format that is different from the ordinary html sitemap for human visitors.

One might ask why two different sitemaps are necessary. The answer is obvious – one is for humans, the other is for search engine spiders (for now mainly Googlebot but it is reasonable to expect that other crawlers will join soon). In this context, it is necessary to clarify that having two sitemaps is not regarded as duplicate content. In ‘Introduction to Sitemaps‘, Google explicitly states that using a sitemap will never lead to penalty for your site.

Why Use a Sitemap ?

Using sitemaps has many advantages, not only easier navigation and better visibility by search engines. Sitemaps offer the opportunity to inform search engines immediately about any changes on your site. Of course, you cannot expect that search engines will rush right away to index the changed pages in your site but certainly the changes will be indexed faster, compared to when you don’t have a sitemap.

Also, when you have a sitemap and submit it to the search engines, you rely less on external links that will bring search engines to your site. Sitemaps can even help with messy internal links – for instance if you by accident have orphaned pages or broken internal links   that cannot be reached in other way (however it is always better that you fix your errors than rely on a sitemap).

If your site is new, or if you have a significant number of new (or recently updated pages), then using a sitemap can be vital to your success. Although you can still go without a sitemap, it is very likely that soon sitemaps will become the standard way of submitting a web site to search engines. Though it is certain that spiders will continue to index the Web and sitemaps will not make the standard crawling procedures obsolete, it is logical to say that the importance of sitemaps will continue to increase.

Sitemaps also help in classifying your web site content, though search engines are by no means obliged to classify a page as belonging to a particular category or as matching a particular keyword only because you have told them so.

Having in mind that the sitemap programs of major search engines like Google are still in beta, using a sitemap might not generate huge advantages in the short run, but as search engines improve their sitemap indexing algorithms, it is expected that more and more sites will be indexed faster via sitemaps.

Generating and Submitting the Sitemap

The steps you need to perform in order to have a sitemap for your web site are simple. First, you need to generate it, then you upload it to your web site, and finally you notify Google about it.

Depending on your technical skills, there are broadly two ways to generate a sitemap – to download and install a sitemap generator or to use an online sitemap generation tool. The first is more difficult but you have more control over the output. You can download the Google sitemap generator from here. After you download the package, just follow the installation and configuration instructions in it. This generator is based on a Python script, so your Web server must have Python 2.2 or later installed, in order to run it.

The second way to generate a sitemap is easier. There are many free online tools that can do the job for you. For instance, have a look at this collection ofThird-party Sitemap tools. Although Google says explicitly that it has neither tested, nor verified them, this list will be useful because it includes links to downloadable sitemap generators, online sitemap enerators,  sitemap plugins for popular content-management systems, etc., so you will be able to find exactly what you need.

After you have created the sitemap, you need to upload it to your site (if it is not already there) and notify Google about its existence. Notifying Google includesadding the site to your Google Sitemaps account, so if you do not have an account with Google, it is high time to open one. Also it is useful to know in advance is that in order to add the sitemap to your account, you need to verify that you are the legitimate owner of the site.

Currently Yahoo and MSN do not support sitemaps, or at least not in the XML format, used by Google. Yahoo allows webmasters to submit “a text file with a list of URLs” (which can actually be a stripped-down version of a site map), while MSN does not offer even that but there are rumors that it is indexing sitemaps when they are available onsite. Most likely this situation will change in the near future and both Yahoo and MSN will catch with Google because user-submitted site maps are just a too powerful SEO tool and cannot be ignored.

So sitemaps are becoming increasingly important for your success in Google as well as other search engines.