¿Cuáles son algunas malas web robots?
Esta lista de malas web robots fue originalmente elaborado por funender.
Algunos de estos robots están diseñados para copiar todo los sitios web, que puede utilizar una gran cantidad de ancho de banda. Otros cosecha direcciones de correo electrónico para los spammers. Mayor beneficio a sus propietarios, al tiempo que absolutamente ningún beneficio para el propietario del Web site. Usted debe decidir por sí mismo lo que califica como una mala red de robots.
Aquí hay una lista de robots que puede que desee excluir, en robots.txt formato:
User-agent: Alexibot
User-agent: Aqua_Products
User-agent: BackDoorBot
User-agent: BackDoorBot/1.0
User-agent: Black.Hole
User-agent: browser
User-agent: Blowfish
User-agent: BlowFish/1.0
User-agent: Guardar herramienta de búsqueda
User-agent: Bot mailto: craftbot@yahoo.com
User-agent: BotALot
User-agent: BotRightHere
User-agent: BuiltBotTough
User-agent: Bullseye
User-agent: Bullseye/1.0
User-agent: BunnySlippers
User-agent: Cegbfeieh
User-agent: CheeseBot
User-agent: CherryPicker
User-agent: CherryPickerElite/1.0
User-agent: CherryPickerSE/1.0
User-agent: ChinaClaw
User-agent: Copernic
User-agent: CopyRightCheck
User-agent: Media Luna
User-agent: Herramientas para la Media Luna de Internet HTTP OLE Control v.1.0
User-agent: Custo
User-agent: Disco
User-agent: Disco Bomba 3,0
User-agent: Disco Bomba 3,2
User-agent: DISCoFinder
User-agent: DittoSpyder
User-agent: Descargar Demonio
User-agent: Descargar Demon/3.2.0.8
User-agent: Descargar Demon/3.5.0.11
User-agent: EirGrabber
User-agent: EmailCollector
User-agent: EmailSiphon
User-agent: EmailWolf
User-agent: EroCrawler
User-agent: Express WebPictures
User-agent: Express WebPictures (www.express-soft.com)
User-agent: ExtractorPro
User-agent: EyeNetIE
User-agent: FairAd Cliente
User-agent: Flaming AttackBot
User-agent: FlashGet
User-agent: FlashGet WebWasher 3,2
User-agent: Foobot
User-agent: FrontPage
User-agent: FrontPage [NC, OR]
User-agent: Gaisbot
User-agent: GetRight
User-agent: GetRight/2.11
User-agent: GetRight/3.1
User-agent: GetRight/3.2
User-agent: GetRight/3.3
User-agent: GetRight/3.3.3
User-agent: GetRight/3.3.4
User-agent: GetRight/4.0.0
User-agent: GetRight/4.1.0
User-agent: GetRight/4.1.1
User-agent: GetRight/4.1.2
User-agent: GetRight/4.2
User-agent: GetRight/4.2b (Portuguxeas)
User-agent: GetRight/4.2c
User-agent: GetRight/4.3
User-agent: GetRight/4.5
User-agent: GetRight/4.5a
User-agent: GetRight/4.5b
User-agent: GetRight/4.5b1
User-agent: GetRight/4.5b2
User-agent: GetRight/4.5b3
User-agent: GetRight/4.5b6
User-agent: GetRight/4.5b7
User-agent: GetRight/4.5c
User-agent: GetRight/4.5d
User-agent: GetRight/4.5e
User-agent: GetRight/5.0beta1
User-agent: GetRight/5.0beta2
User-agent: GetWeb!
User-agent: Go! Zilla
User-agent: Go! Zilla (www.gozilla.com)
User-agent: Go! Zilla 3.3 (www.gozilla.com)
User-agent: Go! Zilla 3.5 (www.gozilla.com)
User-agent: Luz verde-got-Es
User-agent: Googlebot-Image
User-agent: GrabNet
User-agent: Grafula
User-agent: HMView
User-agent: HTTrack
User-agent: HTTrack 3.0
User-agent: HTTrack [NC, OR]
User-agent: Harvest
User-agent: Harvest/1.5
User-agent: Imagen Stripper
User-agent: Imagen sucker
User-agent: Indy Biblioteca
User-agent: Indy Library [NC, OR]
User-agent: InfoNaviRobot
User-agent: InterGET
User-agent: Internet Ninja
User-agent: Internet Ninja 4,0
User-agent: Internet Ninja 5,0
User-agent: Internet Ninja 6,0
User-agent: Iron33/1.0.2
User-agent: JOC Web Spider
User-agent: JennyBot
User-agent: JetCar
User-agent: Kenjin Spider
User-agent: Kenjin.Spider
User-agent: Palabra clave Density/0.9
User-agent: Keyword.Density
User-agent: LNSpiderguy
User-agent: LeechFTP
User-agent: LexiBot
User-agent: LinkScan/8.1a Unix
User-agent: LinkScan/8.1a.Unix
User-agent: LinkWalker
User-agent: LinkextractorPro
User-agent: MIDown herramienta
User-agent: MIIxpc
User-agent: MIIxpc/4.2
User-agent: MSIECrawler
User-agent: Masa Downloader
User-agent: Masa Downloader/2.2
User-agent: Mata Hari
User-agent: Mata.Hari
User-agent: Microsoft URL Control
User-agent: Microsoft URL Control - 5.01.4511
User-agent: Microsoft URL Control - 6.00.8169
User-agent: Microsoft.URL
User-agent: Mister Pix
User-agent: Mister Pix Version.dll
User-agent: Mister Pix II 2,01
User-agent: Mister Pix II 2.02a
User-agent: Mister.PiX
User-agent: NICErsPRO
User-agent: NPBot
User-agent: NPbot
User-agent: Navroad
User-agent: NearSite
User-agent: Red Vampiro
User-agent: Red Vampire/3.0
User-agent: NetAnts
User-agent: NetAnts/1.10
User-agent: NetAnts/1.23
User-agent: NetAnts/1.24
User-agent: NetAnts/1.25
User-agent: NetMechanic
User-agent: NetSpider
User-agent: NetZip
User-agent: NetZip Downloader 1.0 Win32 (12 de noviembre de 1998)
User-agent: NetZip-Downloader/1.0.62 (Win32; 7 de diciembre de 1998)
User-agent: NetZippy + (http://www.innerprise.net/usp-spider.asp)
User-agent: Pulpo
User-agent: Fuera de línea Explorer
User-agent: Fuera de línea Explorer/1.2
User-agent: Desconectado Explorer/1.4
User-agent: Fuera de línea Explorer/1.6
User-agent: Fuera de línea Explorer/1.7
User-agent: Fuera de línea Explorer/1.9
User-agent: Fuera de línea Explorer/2.0
User-agent: Fuera de línea Explorer/2.1
User-agent: Fuera de línea Explorer/2.3
User-agent: Fuera de línea Explorer/2.4
User-agent: Fuera de línea Explorer/2.5
User-agent: Fuera de línea Navigator
User-agent: Offline.Explorer
User-agent: Openbot
User-agent: Openfind
User-agent: Openfind recolectores de datos
User-agent: Oracle Ultra Búsqueda
User-agent: PageGrabber
User-agent: Foto de Papa
User-agent: PerMan
User-agent: ProPowerBot/2.14
User-agent: ProWebWalker
User-agent: Python-urllib
User-agent: QueryN Metabúsqueda
User-agent: QueryN.Metasearch
User-agent: RMA
User-agent: Radiación Retriever 1.1
User-agent: ReGet
User-agent: RealDownload
User-agent: RealDownload/4.0.0.40
User-agent: RealDownload/4.0.0.41
User-agent: RealDownload/4.0.0.42
User-agent: RepoMonkey
User-agent: RepoMonkey Carnada y Tackle/v1.01
User-agent: SiteSnagger
User-agent: SlySearch
User-agent: SmartDownload
User-agent: SmartDownload/1.2.76 (Win32; 1o de abril de 1999)
User-agent: SmartDownload/1.2.77 (Win32, 17 de agosto de 1999)
User-agent: SmartDownload/1.2.77 (Win32; 1ro de febrero de 2000)
User-agent: SmartDownload/1.2.77 (Win32, 19 de junio de 2001)
User-agent: SpankBot
User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux
User-agent: SuperBot
User-agent: SuperBot/3.0 (Win32)
User-agent: SuperBot/3.1 (Win32)
User-agent: SuperHTTP
User-agent: SuperHTTP/1.0
User-agent: Surfbot
User-agent: Szukacz/1.4
User-agent: Teleport
User-agent: Teleport Pro
User-agent: Teleport Pro/1.29
User-agent: Teleport Pro/1.29.1590
User-agent: Teleport Pro/1.29.1634
User-agent: Teleport Pro/1.29.1718
User-agent: Teleport Pro/1.29.1820
User-agent: Teleport Pro/1.29.1847
User-agent: TeleportPro
User-agent: Telesoft
User-agent: El Intraformant
User-agent: The.Intraformant
User-agent: TheNomad
User-agent: TightTwatBot
User-agent: Titan
User-agent: True_Robot
User-agent: True_Robot/1.0
User-agent: TurnitinBot
User-agent: TurnitinBot/1.5
User-agent: URL de Control
User-agent: URL_Spider_Pro
User-agent: URLy Advertencia
User-agent: URLy.Warning
User-agent: VCI
User-agent: VCI WebViewer VCI WebViewer Win32
User-agent: VoidEYE
User-agent: WWW-E-colector
User-agent: WWWOFFLE
User-agent: Web Image Colector
User-agent: Web sucker
User-agent: Web.Image.Collector
User-agent: WebAuto
User-agent: WebAuto/3.40 (Win98; I)
User-agent: WebBandit
User-agent: WebBandit/3.50
User-agent: WebCapture 2,0
User-agent: WebCopier
User-agent: WebCopier v.2.2
User-agent: WebCopier v2.5
User-agent: WebCopier v2.6
User-agent: WebCopier v2.7a
User-agent: WebCopier v2.8
User-agent: WebCopier v3.0
User-agent: WebCopier v3.0.1
User-agent: WebCopier v3.2
User-agent: WebCopier v3.2a
User-agent: WebEMailExtrac .*
User-agent: WebEnhancer
User-agent: WebFetch
User-agent: WebGo ES
User-agent: WebLeacher
User-agent: WebReaper
User-agent: WebReaper [info@webreaper.net]
User-agent: WebReaper [webreaper@otway.com]
User-agent: WebReaper v9.1 - www.otway.com / WebReaper
User-agent: WebReaper v9.7 - www.webreaper.net
User-agent: WebReaper v9.8 - www.webreaper.net
User-agent: WebReaper vWebReaper v7.3 - www, otway.com / WebReaper
User-agent: WebSauger
User-agent: WebSauger 1.20b
User-agent: WebSauger 1.20j
User-agent: WebSauger 1.20k
User-agent: WebStripper
User-agent: WebStripper/2.03
User-agent: WebStripper/2.10
User-agent: WebStripper/2.12
User-agent: WebStripper/2.13
User-agent: WebStripper/2.15
User-agent: WebStripper/2.16
User-agent: WebStripper/2.19
User-agent: WebWhacker
User-agent: WebZIP
User-agent: WebZIP/2.75 (http://www.spidersoft.com)
User-agent: WebZIP/3.65 (http://www.spidersoft.com)
User-agent: WebZIP/3.80 (http://www.spidersoft.com)
User-agent: WebZIP/4.0 (http://www.spidersoft.com)
User-agent: WebZIP/4.1 (http://www.spidersoft.com)
User-agent: WebZIP/4.21
User-agent: WebZIP/4.21 (http://www.spidersoft.com)
User-agent: WebZIP/5.0
User-agent: WebZIP/5.0 (http://www.spidersoft.com)
User-agent: WebZIP/5.0 PR1 (http://www.spidersoft.com)
User-agent: WebZIP
User-agent: WebZip/4.0
User-agent: WebmasterWorldForumBot
User-agent: Sitio web Quester
User-agent: Sitio web Quester - www.asona.org
User-agent: Sitio web Quester - www.esalesbiz.com/extra/
User-agent: Sitio web extractor
User-agent: Sitio Web Extractor (http://www.asona.org)
User-agent: Website.Quester
User-agent: Webster Pro
User-agent: Webster.Pro
User-agent: Wget
User-agent: Wget/1.5.2
User-agent: Wget/1.5.3
User-agent: Wget/1.6
User-agent: Wget/1.7
User-agent: Wget/1.8
User-agent: Wget/1.8.1
User-agent: Wget/1.8.1 + cvs
User-agent: Wget/1.8.2
User-agent: Wget/1.9-beta
User-agent: Viuda
User-agent: Xaldon WebSpider
User-agent: Xaldon WebSpider 2.5.b3
User-agent: Xenu la
User-agent: Xenu de Enlace Sleuth 1.1C
User-agent: Zeus
User-agent: Zeus 11389 Webster Pro V2.9 Win32
User-agent: Zeus 11652 Webster Pro V2.9 Win32
User-agent: Zeus 18018 Webster Pro V2.9 Win32
User-agent: Zeus 26378 Webster Pro V2.9 Win32
User-agent: Zeus 30747 Webster Pro V2.9 Win32
User-agent: Zeus 32297 Webster Pro V2.9 Win32
User-agent: Zeus 39206 Webster Pro V2.9 Win32
User-agent: Zeus 41641 Webster Pro V2.9 Win32
User-agent: Zeus 44238 Webster Pro V2.9 Win32
User-agent: Zeus 51070 Webster Pro V2.9 Win32
User-agent: Zeus 51674 Webster Pro V2.9 Win32
User-agent: Zeus 51837 Webster Pro V2.9 Win32
User-agent: Zeus 63567 Webster Pro V2.9 Win32
User-agent: Zeus 6694 Webster Pro V2.9 Win32
User-agent: Zeus 71129 Webster Pro V2.9 Win32
User-agent: Zeus 82016 Webster Pro V2.9 Win32
User-agent: Zeus 82900 Webster Pro V2.9 Win32
User-agent: Zeus 84842 Webster Pro V2.9 Win32
User-agent: Zeus 90872 Webster Pro V2.9 Win32
User-agent: Zeus 94934 Webster Pro V2.9 Win32
User-agent: Zeus 95245 Webster Pro V2.9 Win32
User-agent: Zeus 95351 Webster Pro V2.9 Win32
User-agent: Zeus 97371 Webster Pro V2.9 Win32
User-agent: Zeus Enlace Scout
User-agent: asterias
User-agent: b2w/0.1
User-agent: cosmos
User-agent: eCatch
User-agent: eCatch/3.0
User-agent: hloader
User-agent: httplib
User-agent: humanlinks
User-agent: ia_archiver
User-agent: larbin
User-agent: larbin (samualt9@bigfoot.com)
User-agent: larbin samualt9@bigfoot.com
User-agent: larbin_2.6.2 (kabura@sushi.com)
User-agent: larbin_2.6.2 (larbin2.6.2 @ unspecified.mail)
User-agent: larbin_2.6.2 (listonATccDOTgatechDOTedu)
User-agent: larbin_2.6.2 (vitalbox1@hotmail.com)
User-agent: larbin_2.6.2 kabura@sushi.com
User-agent: larbin_2.6.2 larbin2.6.2 @ unspecified.mail
User-agent: larbin_2.6.2 larbin@correa.org
User-agent: larbin_2.6.2 listonATccDOTgatechDOTedu
User-agent: larbin_2.6.2 vitalbox1@hotmail.com
User-agent: libWeb / clsHTTP
User-agent: lwp trivial
User-agent: lwp-trivial/1.34
User-agent: moget
User-agent: moget/2.1
User-agent: pavuk
User-agent: pcBrowser
User-agent: psbot
User-agent: searchpreview
User-agent: llave
User-agent: Suzuran
User-agent: takeout
User-agent: toCrawl / UrlDispatcher
User-agent: turingos
User-agent: webfetch/2.1.0
User-agent: wget
Disallow: /
 |
Guardar ¿Cuáles son algunas malas web robots?
