This page has moved to /guides/robots-txt-basics.html.