Ken Moss, General Manager of Live Search,announced today that Microsoft, Google, and Yahoo! will all support the site maps protocol , a simple, xml based way for web site owners to share information with search engines without having to rely on crawl algorithms. Ken describes the work Microsoft is doing:

We are 100% behind this protocol – this kind of collaboration will help improve the search experience for all of our customers, and we are working hard to release full support in 2007.

Ken points to site maps. org., which has some details on the protocol:

Site maps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Site map is an XML file that lists URL for a site along with additional meta data about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URL in the site) so that search engines can more intelligently crawl the site.

Web crawlers usually discover pages from links within the site and from other sites. Site maps supplement this data to allow crawlers that support Site maps to pick up all URL in the Site map and learn about those URL s using the associated meta data. Using the Site map protocol does not guarantee that web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.