Jump
to HelpDesk | Announcements
| FAQ | Webmail Login
How can I keep certain pages from being indexed by search engines and other spiders?
Two methods: You can insert a NO INDEX meta tag on specific pages, and you can also upload a robots.txt file that gives instructions to crawlers and refuses some malicious spiders (the kind that harvest email addresses and such). You can keep them away from a few pages, a directory, or a whole site.
You can google "noindex tag" to find more resources, but here is a good place to start:
http://www.searchengineworld.com/metatag/robots.htm
And here is a site all about robots.txt files:
http://www.robotcop.org/links.html
Caution: Malicious bots can ignore both methods. If there is something you truly wish to hide, do not post it on public pages that are not password-protected.
If there is a specific bot you wish to avoid, you can use .htacess, editable from the control panel, to block it the IP address.
Related Topics:
How can transfer large files to other people without using email? What perl modules are installed? What do you have for multiple domains? How do I get Dreamweaver to connect to my account? What do the different terms mean in the statistic programs? How can I help make the shared server more secure? How do I use the shared SSL certificate? Can I set up ODBC access to a database on the server? Why does my form redirect to a "contact support" message? How can I use mod_rewrite to make my URLs more search engine friendly? How can I see/edit my .htaccess file? Do you run SUexec? What are the options for sending an email list or newsletter? Is there telnet? Can I get shell access? Do you do Windows hosting? Do you support Cold Fusion? How can I manage downloads from my site? Do I need PHP? Should I learn it? Can I use Frontpage 2003? How do I install my own SSL certificate? I need to install a form on a website... what should I use? What port do I use to access shell via SSH2? Are your servers mirrored, or otherwise "bullet-proof?" I want to sell things from my site. What do I need? Can you recommend an SSH client? How can I install WordPress and get started using it? Can I use PHP and SSI together? Where can I find a good tell-a-friend referral script? How can I more easily move accounts from another host? How can I block certain IP addresses? My form is "breaking" my page layout. How do I use the Auto-Installer? How do I set up a database? I need help with SSI (Server Side Includes).
Developer Note: This is powered by SmartFAQ, a free cgi script.
|