I want to keep dev.domain.com from being indexed as I’m working on a redesign and would like to keep the content from being indexed.
I currently have the dev.domain.com subdomain in the folder /htdocs/dev.
What is the best way to keep the entire dev subdomain (or the /htdocs/dev folder) from being indexed?
Put the following in /htdocs/dev/robots.txt
Done and Done!
The Robots.txt or meta Robots caches the site for SEO right?
It seems like it does that once in a while I am assuming based on traffic flow, my question is that can you cache it yourself without the search engine crawling for it?
You must be logged in to reply to this topic.