Grow your CSS skills. Land your dream job.

Robots.txt

  • # December 15, 2010 at 5:08 am

    May not be the best place for this question, but if I disallow a Robot from a file that is included in another file, will that affect the indexing?

    e.g. I have a PHP Header file, which I don’t want indexed on its own because it means nothing. If I set Robots.txt to Disallow: /Header.php will this cause problems when I include the file within the Default.php?

    # December 15, 2010 at 10:41 am

    I don’t think it should affect it at all. Google shouldn’t see header.php and index.php as having any relationship with eachother since google can’t see the php, only the html.

    User-agent: *
    Disallow: /staging-area/
    Disallow: /includes/
    Disallow: /styles/

    That’s within the robots.txt on one of the websites I work on and it seems to be doing fine on google.

    /includes/ contains header.php, footer.php, menu.php, etc.

    # December 16, 2010 at 8:41 pm

    Definitely won’t matter. But also most likely isn’t necessary. There probably isn’t any links to your header.php file so google doesn’t even know about it. And if it did, I would think it’s smart enough to understand what’s happening and not list it in any results.

Viewing 3 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

*May or may not contain any actual "CSS" or "Tricks".