DIW suggested to go to SEOtools Website – I have a question/potential problem

[kdonnigan]

Just finished Digging Into WordPress. Thanks for writing such a great book. I had one question though…

In the sidebar on page 307, you link to the Spider Test Tool – http://tools.seobook.com/general/spider-test/

I went there and put in my website – http://www.anythinggraphic.net. Everything checked out fine. Then I put in a blog post, again everything was fine. THEN I put in my blog URL – http://www.anythinggraphic.net/blog/ and nothing came up. I remember a while ago checking another tool regarding something else and it said ### Forbidden for my blog URL (pound signs mean numbers I don’t remember…).

Why is my blog URL not showing anything in the Spider Test Tool and is this a problem that I need to fix? Or, is this a normal WP thing? Anybody else notice this problem?

Thanks Chris! You rock :-)

[Chris Coyier]

I’d check that your robots.txt file doesn’t say anything about not indexing your blog. And also, check for any tags saying “noindex”. Probably not it, but a good double check just in case.

[kdonnigan]

Hi Chris, thanks for your suggestions. Here is my robots.txt:

User-agent: *
Disallow: /wp-*
Disallow: *.php
Disallow: */feed*
Disallow: */trackback*
Disallow: /20*

My header does NOT contain anything regarding no-index. I only have 1 META tag but it’s just the default WP stuff…

Any other thoughts?

[kdonnigan]

I think I may have found the problem. I recently redesigned my whole website from the inside out and left a folder called ‘blog’ as my last theme had the blog in a subdirectory. Now it doesn’t. Let’s see if that helps!

Thanks for your previous tips Chris :-) Did I mention DIW rocks! lol.

[Author]

Posts