To prevent all search bots from indexing a page:
<meta name="robots" content="noindex">
To prevent just Google:
<meta name="googlebot" content="noindex">
If you have control over it, you probably want to add nofollow to links that point to that page as well:
<a href="privatepage.html" rel="nofollow">Link to private page</a>
Theoretically that’s not needed for Google, which claims to drop all pages with noindex from their directory. But there are more search engines out there and it doesn’t hurt.
Also, you might want to add a robots.txt to prevent the bots from indexing your directories, it might look like this:
which would prevent all bots from indexing; of course you can also only block certain bots by their user agent; google it :-)
Hi
great page!!!
is there any way, one can add a css code for “no indexing” a certain part of a page…but index the rest so to speak, if one wants to no index certain phone number for instance?
Hi.. if use nofollow, that link will be index or not?
I’m not sure, but:
User-agent: *
Disallow: /
will do the job!
Very useful especially for admin pages.
Hello,
User-agent: *
Disallow: /
If you used this robot txt , than any search engine don’t index full your web site.
and
User-agent: *
Disallow:
Disallow: /your-folder/
Disallow: /page.html
This robot txt help search engine to index full your site but don’t index your-folder and page.html .
This Might Help ..
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://www.malikshahbaz.com/feeds/posts/default?orderby=UPDATED