Robots.txt files can be centralized on CDNs, not just root domains. Websites can redirect robots.txt from main domain to CDN. This unorthodox approach complies with updated standards. Google's Gary ...