Gryphyn Media web hosting - Philapdelphia PA
Hosting for people who know a thing or two.

Plans | Features | Network | Policies | Support | About | Contact

Jump to HelpDesk | Announcements | FAQ | Webmail Login


Match word(s).

How can I keep certain pages from being indexed by search engines and other spiders?

Two methods: You can insert a NO INDEX meta tag on specific pages, and you can also upload a robots.txt file that gives instructions to crawlers and refuses some malicious spiders (the kind that harvest email addresses and such). You can keep them away from a few pages, a directory, or a whole site.

You can google "noindex tag" to find more resources, but here is a good place to start:
http://www.searchengineworld.com/metatag/robots.htm

And here is a site all about robots.txt files:
http://www.robotcop.org/links.html

Caution: Malicious bots can ignore both methods. If there is something you truly wish to hide, do not post it on public pages that are not password-protected.

If there is a specific bot you wish to avoid, you can use .htacess, editable from the control panel, to block it the IP address.

Related Topics:
How can transfer large files to other people without using email?
What perl modules are installed?
What do you have for multiple domains?
How do I get Dreamweaver to connect to my account?
What do the different terms mean in the statistic programs?
How can I help make the shared server more secure?
How do I use the shared SSL certificate?
Can I set up ODBC access to a database on the server?
Why does my form redirect to a "contact support" message?
How can I use mod_rewrite to make my URLs more search engine friendly?
How can I see/edit my .htaccess file?
Do you run SUexec?
What are the options for sending an email list or newsletter?
Is there telnet?
Can I get shell access?
Do you do Windows hosting?
Do you support Cold Fusion?
How can I manage downloads from my site?
Do I need PHP? Should I learn it?
Can I use Frontpage 2003?
How do I install my own SSL certificate?
I need to install a form on a website... what should I use?
What port do I use to access shell via SSH2?
Are your servers mirrored, or otherwise "bullet-proof?"
I want to sell things from my site. What do I need?
Can you recommend an SSH client?
How can I install WordPress and get started using it?
Can I use PHP and SSI together?
Where can I find a good tell-a-friend referral script?
How can I more easily move accounts from another host?
How can I block certain IP addresses?
My form is "breaking" my page layout.
How do I use the Auto-Installer?
How do I set up a database?
I need help with SSI (Server Side Includes).

Developer Note: This is powered by SmartFAQ, a free cgi script.


©Gryphyn Media, Inc 1998-2003
about | policies | contact | philadelphia