Optimizing internet searches
Version:
The topics in this section describe how to configure sitemaps and robots.txt files to help search crawlers index the pages on your website, or instruct search crawlers to exclude certain pages from indexing.