Grow your CSS skills. Land your dream job.

Robots.txt or <meta name="robots"> ?

  • # June 4, 2009 at 2:27 pm

    I want to keep dev.domain.com from being indexed as I’m working on a redesign and would like to keep the content from being indexed.

    I currently have the dev.domain.com subdomain in the folder /htdocs/dev.

    What is the best way to keep the entire dev subdomain (or the /htdocs/dev folder) from being indexed?

    ?" class="bbp-reply-permalink"># June 8, 2009 at 7:23 pm

    Put the following in /htdocs/dev/robots.txt

    Code:
    User-agent: *
    Disallow: /
    ?" class="bbp-reply-permalink"># June 9, 2009 at 11:07 am

    Thanks Dave.

    Done and Done!

    ?" class="bbp-reply-permalink"># June 17, 2009 at 12:18 pm

    The Robots.txt or meta Robots caches the site for SEO right?
    It seems like it does that once in a while I am assuming based on traffic flow, my question is that can you cache it yourself without the search engine crawling for it?

Viewing 4 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic.

*May or may not contain any actual "CSS" or "Tricks".