SEO question regarding .htaccess and robots.txt

[SnapshotMedia]

Hi there,

On my new business website I’m having a difficult time deciding what’s best for rewriting my URLs etc. and preventing Google from indexing my ‘bad’ URLs, whilst still telling Google to index my ‘pretty’ URLs.

If I’ve got this in my .htaccess:

Code:

RewriteRule ^([A-Za-z]+)$ /index.php?page=$1 [L,NC]

…it makes all my /index.php?page=home etc. turn into /home …great!

But… I want Google to index /home, /aboutus etc., and not /index.php?page=home, /index.php?page=aboutus etc.

Am I okay to simply add this to my robots.txt:

Code:

Disallow: /index.php?page=home
Disallow: /index.php?page=aboutus

…or will this stop Google from indexing my site altogether? I’ve included /home and /aboutus in my sitemap.xml and submitted it with Google webmaster tools.

Any advice would be really helpful on this one.

Cheers,

SM

[AshtonSanders]

There’s a couple things you could do.

Disallowing is not recommended.

I would always start with a canonical tag so any "accidental duplicate pages" can be remedied. This would require your page to know what page it’s supposed to be.

Otherwise, I would do a 301 redirect either manually through htaccess, or program it into your index.php page to make sure the address is correct.

[AshtonSanders]

You should probably create your own thread instead of posting your question as a reply to someone else’s thread. Esp since the two are not related in any way.

[Author]

Posts