|
REMOVE YOUR WEBSITE FROM THE SEARCH ENGINE |
If you wish to exclude your entire website or a specific section (directory) of your server from 's index, you can place a file at the root of your server called robots.txt.
To prevent and other search engines from crawling your site, place the following robots.txt file in your server root:
User-agent: *This is the standard protocol that most web crawlers observe for excluding a web server or directory from an index. More information on robots.txt is available here: http://www.robotstxt.org/wc/norobots.html.
|
Are Search Engines Legal? |
"Only last week, a California judge ruled, in
a case brought by Ticketmaster against Tickets.com, that it's not illegal for
one site to link to another. Among other things, that suit concerned "deep linking."
Ticketmaster alleged that by bypassing its home page and linking directly to
"inside" pages, Tickets.com violated its copyright. The judge, however, held
that " hyperlinking does not in itself involve a violation of the Copyright
Act."
http://www.bc.edu/bc_org/avp/law/st_org/iptf/headlines/content/2000040401.html
http://www.ip-surveys.com/_disc3/00000002.htm
"On March 27, 2000, the federal district court for the Central District of California ruled on a motion to dismiss that Tickets.com had not violated the Copyright Act simply by virtue of creating links between its web-site and the web-site of ticket-seller Ticketmaster. "
"The court also determined that there was no deception involved in terms of how the links operated: [T]he customer is automatically transferred to the particular genuine web-page of the original author."
- http://www.hollandhart.com/newsitem.cfm?ID=207
We operate a FULL BLOWN SEARCH ENGINE
Examples:
http://www./.asp?=Mcdonalds Playground Safety|
Remove individual pages |
If you want to prevent all robots from indexing individual pages on your site, then you can place the following meta tag element into the page's HTML code:
If you want to allow other robots to index individual pages on your site, preventing only 's robots from indexing the pages, use the following tag:
More information on this standard meta tag element is available here: http://www.robotstxt.org/wc/exclusion.html#meta.
|
|
Remove snippets |
A snippet is a text excerpt from the returned result page that has all query
terms bolded. The excerpt allows users to see the context in which search terms
appear on a web page, before clicking on the result. Users are more likely to
click on a search result if it has a corresponding snippet.
If you wish
to prevent from displaying snippets for your pages, use the following
tag:
Note: removing snippets also removes cached pages.
|
|
Remove cached pages |
keeps the text of the many documents it crawls available in a cache. This allows an archived, or "cached", version of a web page to be retrieved for your end users if the original page is ever unavailable (due to temporary failure of the page's web server). The cached page appears to users exactly as it looked when last crawled it. The cached page also includes a message (at the top of the page) to indicate that it's a cached version of the page.
If you want to prevent all robots from archiving content on your site, use
the NOARCHIVE meta tag. Place this tag in the <HEAD> section
of your documents as follows:
If you want to allow other indexing robots to archive your page's content, preventing only 's robots from caching the page, use the following tag:
Note: this tag only removes the "cached" link for the page. continues to index the page and display a snippet.
|
|
Remove an outdated ("dead") link |
updates its entire index automatically on a regular basis. When we crawl the web, we find new pages, discard dead links, and update links automatically. Links that are outdated now will most likely "fade out" of our index during our next crawl.
|