Grow your CSS skills. Land your dream job.

Last updated on:

Meta Tag to Prevent Search Engine Bots

To prevent all search bots from indexing a page:

<meta name="robots" content="noindex">

To prevent just Google:

<meta name="googlebot" content="noindex">

If you have control over it, you probably want to add nofollow to links that point to that page as well:

<a href="privatepage.html" rel="nofollow">Link to private page</a>

Theoretically that's not needed for Google, which claims to drop all pages with noindex from their directory. But there are more search engines out there and it doesn't hurt.

Reference URL

Comments

  1. Also, you might want to add a robots.txt to prevent the bots from indexing your directories, it might look like this:

    User-agent: *
    Disallow: /

    which would prevent all bots from indexing; of course you can also only block certain bots by their user agent; google it :-)

  2. Hi.. if use nofollow, that link will be index or not?

  3. Very useful especially for admin pages.

  4. Permalink to comment#

    Hello,

    User-agent: *
    Disallow: /
    If you used this robot txt , than any search engine don’t index full your web site.

    and

    User-agent: *
    Disallow:
    Disallow: /your-folder/
    Disallow: /page.html

    This robot txt help search engine to index full your site but don’t index your-folder and page.html .

  5. Permalink to comment#

    This Might Help ..

    User-agent: Mediapartners-Google
    Disallow:

    User-agent: *
    Disallow: /search
    Allow: /

    Sitemap: http://www.malikshahbaz.com/feeds/posts/default?orderby=UPDATED

Leave a Comment

Posting Code

  • Use Markdown, and it will escape the code for you, like `<div class="cool">`.
  • Use triple-backticks for blocks of code.
    ``` 
    <div>
      <h1>multi-line block of code</h1>
      <span>be cool yo.</span>
    </div>
    ```
  • Otherwise, escape your code, like <code>&lt;div class="cool"&gt;</code>. Markdown is just easier though.

Current ye@r *

*May or may not contain any actual "CSS" or "Tricks".