X-robots-tag is like the usual meta robots tags, however this tag provides instructions to crawlers on how to crawl and/or index web page content.