The forums ran from 2008-2020 and are now closed and viewable here as an archive.

Home Forums Other DIW suggested to go to SEOtools Website – I have a question/potential problem

  • This topic is empty.
Viewing 4 posts - 1 through 4 (of 4 total)
  • Author
  • #29998

    Just finished Digging Into WordPress. Thanks for writing such a great book. I had one question though…

    In the sidebar on page 307, you link to the Spider Test Tool –

    I went there and put in my website – Everything checked out fine. Then I put in a blog post, again everything was fine. THEN I put in my blog URL – and nothing came up. I remember a while ago checking another tool regarding something else and it said ### Forbidden for my blog URL (pound signs mean numbers I don’t remember…).

    Why is my blog URL not showing anything in the Spider Test Tool and is this a problem that I need to fix? Or, is this a normal WP thing? Anybody else notice this problem?

    Thanks Chris! You rock :-)

    Chris Coyier

    I’d check that your robots.txt file doesn’t say anything about not indexing your blog. And also, check for any tags saying “noindex”. Probably not it, but a good double check just in case.


    Hi Chris, thanks for your suggestions. Here is my robots.txt:

    User-agent: *
    Disallow: /wp-*
    Disallow: *.php
    Disallow: */feed*
    Disallow: */trackback*
    Disallow: /20*

    My header does NOT contain anything regarding no-index. I only have 1 META tag but it’s just the default WP stuff…

    Any other thoughts?


    I think I may have found the problem. I recently redesigned my whole website from the inside out and left a folder called ‘blog’ as my last theme had the blog in a subdirectory. Now it doesn’t. Let’s see if that helps!

    Thanks for your previous tips Chris :-) Did I mention DIW rocks! lol.

Viewing 4 posts - 1 through 4 (of 4 total)
  • The forum ‘Other’ is closed to new topics and replies.