Help - Search - Members - Calendar
Full Version: How To Block Bad Bots On My Website .htaccess Code
Web Design Seo Forum > 3D Web Design English Forum > Htaccess Guides
Web Design Seo
On internet are many Bad bots that only post spam and load your site. How to block this type of bots?
Best way is by .htaccess.

Code for .htaccess to do this is:


Код
# Block Bad Bots
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule .* - [F]


Related searches:

htaccess Blocking php bots - sample code.
deny bots htaccess
block bots htaccess
Web Design Seo
You can try also this Blocking bots code for htaccess:

Код
# Block Bad Bots by User-Agent
SetEnvIfNoCase User-Agent ^$ bad_bot
SetEnvIfNoCase User-Agent "^AESOP_com_SpiderMan" bad_bot
SetEnvIfNoCase User-Agent "^Alexibot" bad_bot
SetEnvIfNoCase User-Agent "Anonymouse.org" bad_bot
SetEnvIfNoCase User-Agent "^asterias" bad_bot
SetEnvIfNoCase User-Agent "^attach" bad_bot
SetEnvIfNoCase User-Agent "^BackDoorBot" bad_bot
SetEnvIfNoCase User-Agent "^BackWeb" bad_bot
SetEnvIfNoCase User-Agent "Bandit" bad_bot
SetEnvIfNoCase User-Agent "^Baiduspider" bad_bot
SetEnvIfNoCase User-Agent "^BatchFTP" bad_bot
SetEnvIfNoCase User-Agent "^Bigfoot" bad_bot
SetEnvIfNoCase User-Agent "^Black.Hole" bad_bot
SetEnvIfNoCase User-Agent "^BlackWidow" bad_bot
SetEnvIfNoCase User-Agent "^BlowFish" bad_bot
SetEnvIfNoCase User-Agent "^Bot\ mailto:craftbot@yahoo.com" bad_bot
SetEnvIfNoCase User-Agent "^BotALot" bad_bot
SetEnvIfNoCase User-Agent "Buddy" bad_bot
SetEnvIfNoCase User-Agent "^BuiltBotTough" bad_bot
SetEnvIfNoCase User-Agent "^Bullseye" bad_bot
SetEnvIfNoCase User-Agent "^BunnySlippers" bad_bot
SetEnvIfNoCase User-Agent "^Cegbfeieh" bad_bot
SetEnvIfNoCase User-Agent "^CheeseBot" bad_bot
SetEnvIfNoCase User-Agent "^CherryPicker" bad_bot
SetEnvIfNoCase User-Agent "^ChinaClaw" bad_bot
SetEnvIfNoCase User-Agent "Collector" bad_bot
SetEnvIfNoCase User-Agent "Copier" bad_bot
SetEnvIfNoCase User-Agent "^CopyRightCheck" bad_bot
SetEnvIfNoCase User-Agent "^cosmos" bad_bot
SetEnvIfNoCase User-Agent "^Crescent" bad_bot
SetEnvIfNoCase User-Agent "^Curl" bad_bot
SetEnvIfNoCase User-Agent "^Custo" bad_bot
SetEnvIfNoCase User-Agent "^DA" bad_bot
SetEnvIfNoCase User-Agent "^DISCo" bad_bot
SetEnvIfNoCase User-Agent "^DIIbot" bad_bot
SetEnvIfNoCase User-Agent "^DittoSpyder" bad_bot
SetEnvIfNoCase User-Agent "^Download" bad_bot
SetEnvIfNoCase User-Agent "^Download\ Demon" bad_bot
SetEnvIfNoCase User-Agent "^Download\ Devil" bad_bot
SetEnvIfNoCase User-Agent "^Download\ Wonder" bad_bot
SetEnvIfNoCase User-Agent "Downloader" bad_bot
SetEnvIfNoCase User-Agent "^dragonfly" bad_bot
SetEnvIfNoCase User-Agent "^Drip" bad_bot
SetEnvIfNoCase User-Agent "^eCatch" bad_bot
SetEnvIfNoCase User-Agent "^EasyDL" bad_bot
SetEnvIfNoCase User-Agent "^ebingbong" bad_bot
SetEnvIfNoCase User-Agent "^EirGrabber" bad_bot
SetEnvIfNoCase User-Agent "^EmailCollector" bad_bot
SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot
SetEnvIfNoCase User-Agent "^EmailWolf" bad_bot
SetEnvIfNoCase User-Agent "^EroCrawler" bad_bot
SetEnvIfNoCase User-Agent "^Exabot" bad_bot
SetEnvIfNoCase User-Agent "^Express\ WebPictures" bad_bot
SetEnvIfNoCase User-Agent "Extractor" bad_bot
SetEnvIfNoCase User-Agent "^EyeNetIE" bad_bot
SetEnvIfNoCase User-Agent "^FileHound" bad_bot
SetEnvIfNoCase User-Agent "^FlashGet" bad_bot
SetEnvIfNoCase User-Agent "^Foobot" bad_bot
SetEnvIfNoCase User-Agent "^flunky" bad_bot
SetEnvIfNoCase User-Agent "^FrontPage" bad_bot
SetEnvIfNoCase User-Agent "^GetRight" bad_bot
SetEnvIfNoCase User-Agent "^GetSmart" bad_bot
SetEnvIfNoCase User-Agent "^GetWeb!" bad_bot
SetEnvIfNoCase User-Agent "^Go!Zilla" bad_bot
SetEnvIfNoCase User-Agent "Google\ Wireless\ Transcoder" bad_bot
SetEnvIfNoCase User-Agent "^Go-Ahead-Got-It" bad_bot
SetEnvIfNoCase User-Agent "^gotit" bad_bot
SetEnvIfNoCase User-Agent "Grabber" bad_bot
SetEnvIfNoCase User-Agent "^GrabNet" bad_bot
SetEnvIfNoCase User-Agent "^Grafula" bad_bot
SetEnvIfNoCase User-Agent "^Harvest" bad_bot
SetEnvIfNoCase User-Agent "^hloader" bad_bot
SetEnvIfNoCase User-Agent "^HMView" bad_bot
SetEnvIfNoCase User-Agent "^httplib" bad_bot
SetEnvIfNoCase User-Agent "^HTTrack" bad_bot
SetEnvIfNoCase User-Agent "^humanlinks" bad_bot
SetEnvIfNoCase User-Agent "^ia_archiver" bad_bot
SetEnvIfNoCase User-Agent "^IlseBot" bad_bot
SetEnvIfNoCase User-Agent "^Image\ Stripper" bad_bot
SetEnvIfNoCase User-Agent "^Image\ Sucker" bad_bot
SetEnvIfNoCase User-Agent "Indy\ Library" bad_bot
SetEnvIfNoCase User-Agent "^InfoNaviRobot" bad_bot
SetEnvIfNoCase User-Agent "^InfoTekies" bad_bot
SetEnvIfNoCase User-Agent "^Intelliseek" bad_bot
SetEnvIfNoCase User-Agent "^InterGET" bad_bot
SetEnvIfNoCase User-Agent "^Internet\ Ninja" bad_bot
SetEnvIfNoCase User-Agent "^Iria" bad_bot
SetEnvIfNoCase User-Agent "^Jakarta" bad_bot
SetEnvIfNoCase User-Agent "^JennyBot" bad_bot
SetEnvIfNoCase User-Agent "^JetCar" bad_bot
SetEnvIfNoCase User-Agent "^JOC" bad_bot
SetEnvIfNoCase User-Agent "^JustView" bad_bot
SetEnvIfNoCase User-Agent "^Jyxobot" bad_bot
SetEnvIfNoCase User-Agent "^Kenjin.Spider" bad_bot
SetEnvIfNoCase User-Agent "^Keyword.Density" bad_bot
SetEnvIfNoCase User-Agent "^larbin" bad_bot
SetEnvIfNoCase User-Agent "^LeechFTP" bad_bot
SetEnvIfNoCase User-Agent "^LexiBot" bad_bot
SetEnvIfNoCase User-Agent "^lftp" bad_bot
SetEnvIfNoCase User-Agent "^libWeb/clsHTTP" bad_bot
SetEnvIfNoCase User-Agent "^likse" bad_bot
SetEnvIfNoCase User-Agent "^LinkextractorPro" bad_bot
SetEnvIfNoCase User-Agent "^LinkScan/8.1a.Unix" bad_bo
SetEnvIfNoCase User-Agent "^LNSpiderguy" bad_bott
SetEnvIfNoCase User-Agent "^LinkWalker" bad_bot
SetEnvIfNoCase User-Agent "^lwp-trivial" bad_bot
SetEnvIfNoCase User-Agent "^LWP::Simple" bad_bot
SetEnvIfNoCase User-Agent "^Magnet" bad_bot
SetEnvIfNoCase User-Agent "^Mag-Net" bad_bot
SetEnvIfNoCase User-Agent "^MarkWatch" bad_bot
SetEnvIfNoCase User-Agent "^Mass\ Downloader" bad_bot
SetEnvIfNoCase User-Agent "^Mata.Hari" bad_bot
SetEnvIfNoCase User-Agent "^Memo" bad_bot
SetEnvIfNoCase User-Agent "^Microsoft.URL" bad_bot
SetEnvIfNoCase User-Agent "^Microsoft\ URL\ Control" bad_bot
SetEnvIfNoCase User-Agent "^MIDown\ tool" bad_bot
SetEnvIfNoCase User-Agent "^MIIxpc" bad_bot
SetEnvIfNoCase User-Agent "^Mirror" bad_bot
SetEnvIfNoCase User-Agent "^Missigua\ Locator" bad_bot
SetEnvIfNoCase User-Agent "^Mister\ PiX" bad_bot
SetEnvIfNoCase User-Agent "^moget" bad_bot
SetEnvIfNoCase User-Agent "^Mozilla/3.Mozilla/2.01" bad_bot
SetEnvIfNoCase User-Agent "^Mozilla.*NEWT" bad_bot
SetEnvIfNoCase User-Agent "^NAMEPROTECT" bad_bot
SetEnvIfNoCase User-Agent "^Navroad" bad_bot
SetEnvIfNoCase User-Agent "^NearSite" bad_bot
SetEnvIfNoCase User-Agent "^NetAnts" bad_bot
SetEnvIfNoCase User-Agent "^Netcraft" bad_bot
SetEnvIfNoCase User-Agent "^NetMechanic" bad_bot
SetEnvIfNoCase User-Agent "^NetSpider" bad_bot
SetEnvIfNoCase User-Agent "^Net\ Vampire" bad_bot
SetEnvIfNoCase User-Agent "^NetZIP" bad_bot
SetEnvIfNoCase User-Agent "^NextGenSearchBot" bad_bot
SetEnvIfNoCase User-Agent "^NG" bad_bot
SetEnvIfNoCase User-Agent "^NICErsPRO" bad_bot
SetEnvIfNoCase User-Agent "^NimbleCrawler" bad_bot
SetEnvIfNoCase User-Agent "^Ninja" bad_bot
SetEnvIfNoCase User-Agent "^NPbot" bad_bot
SetEnvIfNoCase User-Agent "^Octopus" bad_bot
SetEnvIfNoCase User-Agent "^Offline\ Explorer" bad_bot
SetEnvIfNoCase User-Agent "^Offline\ Navigator" bad_bot
SetEnvIfNoCase User-Agent "^Openfind" bad_bot
SetEnvIfNoCase User-Agent "^OutfoxBot" bad_bot
SetEnvIfNoCase User-Agent "^PageGrabber" bad_bot
SetEnvIfNoCase User-Agent "^Papa\ Foto" bad_bot
SetEnvIfNoCase User-Agent "^pavuk" bad_bot
SetEnvIfNoCase User-Agent "^pcBrowser" bad_bot
SetEnvIfNoCase User-Agent "^PHP\ version\ tracker" bad_bot
SetEnvIfNoCase User-Agent "^Pockey" bad_bot
SetEnvIfNoCase User-Agent "^ProPowerBot/2.14" bad_bot
SetEnvIfNoCase User-Agent "^ProWebWalker" bad_bot
SetEnvIfNoCase User-Agent "^psbot" bad_bot
SetEnvIfNoCase User-Agent "^Pump" bad_bot
SetEnvIfNoCase User-Agent "^QueryN.Metasearch" bad_bot
SetEnvIfNoCase User-Agent "^RealDownload" bad_bot
SetEnvIfNoCase User-Agent "Reaper" bad_bot
SetEnvIfNoCase User-Agent "Recorder" bad_bot
SetEnvIfNoCase User-Agent "^ReGet" bad_bot
SetEnvIfNoCase User-Agent "^RepoMonkey" bad_bot
SetEnvIfNoCase User-Agent "^RMA" bad_bot
SetEnvIfNoCase User-Agent "Siphon" bad_bot
SetEnvIfNoCase User-Agent "sitecheck.internetseer.com" bad_bot
SetEnvIfNoCase User-Agent "^SiteSnagger" bad_bot
SetEnvIfNoCase User-Agent "^SlySearch" bad_bot
SetEnvIfNoCase User-Agent "^SmartDownload" bad_bot
SetEnvIfNoCase User-Agent "^Snake" bad_bot
SetEnvIfNoCase User-Agent "^Snapbot" bad_bot
SetEnvIfNoCase User-Agent "^Snoopy" bad_bot
SetEnvIfNoCase User-Agent "^sogou" bad_bot
SetEnvIfNoCase User-Agent "^SpaceBison" bad_bot
SetEnvIfNoCase User-Agent "^SpankBot" bad_bot
SetEnvIfNoCase User-Agent "^spanner" bad_bot
SetEnvIfNoCase User-Agent "^Sqworm" bad_bot
SetEnvIfNoCase User-Agent "Stripper" bad_bot
SetEnvIfNoCase User-Agent "Sucker" bad_bot
SetEnvIfNoCase User-Agent "^SuperBot" bad_bot
SetEnvIfNoCase User-Agent "^SuperHTTP" bad_bot
SetEnvIfNoCase User-Agent "^Surfbot" bad_bot
SetEnvIfNoCase User-Agent "^suzuran" bad_bot
SetEnvIfNoCase User-Agent "^Szukacz/1.4" bad_bot
SetEnvIfNoCase User-Agent "^tAkeOut" bad_bot
SetEnvIfNoCase User-Agent "^Teleport" bad_bot
SetEnvIfNoCase User-Agent "^Telesoft" bad_bot
SetEnvIfNoCase User-Agent "^TurnitinBot/1.5" bad_bot
SetEnvIfNoCase User-Agent "^The.Intraformant" bad_bot
SetEnvIfNoCase User-Agent "^TheNomad" bad_bot
SetEnvIfNoCase User-Agent "^TightTwatBot" bad_bot
SetEnvIfNoCase User-Agent "^Titan" bad_bot
SetEnvIfNoCase User-Agent "^toCrawl/UrlDispatcher" bad_bot
SetEnvIfNoCase User-Agent "^True_Robot" bad_bot
SetEnvIfNoCase User-Agent "^turingos" bad_bot
SetEnvIfNoCase User-Agent "^TurnitinBot" bad_bot
SetEnvIfNoCase User-Agent "^URLy.Warning" bad_bot
SetEnvIfNoCase User-Agent "^Vacuum" bad_bot
SetEnvIfNoCase User-Agent "^VCI" bad_bot
SetEnvIfNoCase User-Agent "^VoidEYE" bad_bot
SetEnvIfNoCase User-Agent "^Web\ Image\ Collector" bad_bot
SetEnvIfNoCase User-Agent "^Web\ Sucker" bad_bot
SetEnvIfNoCase User-Agent "^WebAuto" bad_bot
SetEnvIfNoCase User-Agent "^WebBandit" bad_bot
SetEnvIfNoCase User-Agent "^Webclipping.com" bad_bot
SetEnvIfNoCase User-Agent "^WebCopier" bad_bot
SetEnvIfNoCase User-Agent "^WebEMailExtrac.*" bad_bot
SetEnvIfNoCase User-Agent "^WebEnhancer" bad_bot
SetEnvIfNoCase User-Agent "^WebFetch" bad_bot
SetEnvIfNoCase User-Agent "^WebGo\ IS" bad_bot
SetEnvIfNoCase User-Agent "^Web.Image.Collector" bad_bot
SetEnvIfNoCase User-Agent "^WebLeacher" bad_bot
SetEnvIfNoCase User-Agent "^WebmasterWorldForumBot" bad_bot
SetEnvIfNoCase User-Agent "^WebReaper" bad_bot
SetEnvIfNoCase User-Agent "^WebSauger" bad_bot
SetEnvIfNoCase User-Agent "^WebSite" bad_bot
SetEnvIfNoCase User-Agent "^Website\ eXtractor" bad_bot
SetEnvIfNoCase User-Agent "^Website\ Quester" bad_bot
SetEnvIfNoCase User-Agent "^Webster" bad_bot
SetEnvIfNoCase User-Agent "^WebStripper" bad_bot
SetEnvIfNoCase User-Agent "^WebWhacker" bad_bot
SetEnvIfNoCase User-Agent "^WebZIP" bad_bot
SetEnvIfNoCase User-Agent "^Wget" bad_bot
SetEnvIfNoCase User-Agent "Whacker" bad_bot
SetEnvIfNoCase User-Agent "^Widow" bad_bot
SetEnvIfNoCase User-Agent "^WISENutbot" bad_bot
SetEnvIfNoCase User-Agent "^WWWOFFLE" bad_bot
SetEnvIfNoCase User-Agent "^WWW-Collector-E" bad_bot
SetEnvIfNoCase User-Agent "^Xaldon" bad_bot
SetEnvIfNoCase User-Agent "^Xenu" bad_bot
SetEnvIfNoCase User-Agent "^Zeus" bad_bot
SetEnvIfNoCase User-Agent "^Zyborg" bad_bot

<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
Web Design Seo
Код
#bad bots start, public  version 2017.12
RewriteCond %{HTTP_USER_AGENT} \
12soso|\
192\.comagent|\
1noonbot|\
1on1searchbot|\
3de\_search2|\
3d\_search|\
3g\ bot|\
3gse|\
50\.nu|\
a1\ sitemap\ generator|\
a1\ website\ download|\
a6\-indexer|\
aasp|\
abachobot|\
abonti|\
abotemailsearch|\
aboundex|\
aboutusbot|\
accmonitor\ compliance\ server|\
accoon|\
achulkov\.net\ page\ walker|\
acme\.spider|\
acoonbot|\
acquia\-crawler|\
activetouristbot|\
ad\ muncher|\
adamm\ bot|\
adbeat\_bot|\
adminshop\.com|\
advanced\ email\ extractor|\
aesop\_com\_spiderman|\
aespider|\
af\ knowledge\ now\ verity\ spider|\
aggregator:vocus|\
ah\-ha\.com\ crawler|\
ahrefs|\
aibot|\
aidu|\
aihitbot|\
aipbot|\
aisiid|\
aitcsrobot/1\.1|\
ajsitemap|\
akamai\-sitesnapshot|\
alexawebsearchplatform|\
alexfdownload|\
alexibot|\
alkalinebot|\
all\ acronyms\ bot|\
alpha\ search\ agent|\
amerla\ search\ bot|\
amfibibot|\
ampmppc\.com|\
amznkassocbot|\
anemone|\
anonymous|\
anotherbot|\
answerbot|\
answerbus|\
answerchase\ prove|\
antbot|\
antibot|\
antisantyworm|\
antro\.net|\
aonde\-spider|\
aport|\
appengine\-google|\
appid\:\ s\~stremor\-crawler\-|\
aqua\_products|\
arabot|\
arachmo|\
arachnophilia|\
archive\.org\_bot|\
aria\ equalizer|\
arianna\.libero\.it|\
arikus\_spider|\
art\-online\.com|\
artavisbot|\
artera|\
asaha\ search\ engine\ turkey|\
ask|\
aspider|\
aspseek|\
asterias|\
astrofind|\
athenusbot|\
atlocalbot|\
atomic\_email\_hunter|\
attach|\
attrakt|\
attributor|\
augurfind|\
auresys|\
autobaron\ crawler|\
autoemailspider|\
autowebdir|\
avsearch\-|\
axfeedsbot|\
axonize\-bot|\
ayna|\
b2w|\
backdoorbot|\
backrub|\
backstreet\ browser|\
backweb|\
baidu|\
bandit|\
batchftp|\
baypup|\
bdfetch|\
becomebot|\
becomejpbot|\
beetlebot|\
bender|\
besserscheitern\-crawl|\
betabot|\
big\ brother|\
big\ data|\
bigado\.com|\
bigcliquebot|\
bigfoot|\
biglotron|\
bilbo|\
bilgibetabot|\
bilgibot|\
bintellibot|\
bitlybot|\
bitvouseragent|\
bizbot003|\
bizbot04|\
bizworks\ retriever|\
black\ hole|\
black\.hole|\
blackbird|\
blackmask\.net\ search\ engine|\
blackwidow|\
bladder\ fusion|\
blaiz\-bee|\
blexbot|\
blinkx|\
blitzbot|\
blog\ conversation\ project|\
blogmyway|\
blogpulselive|\
blogrefsbot|\
blogscope|\
blogslive|\
bloobybot|\
blowfish|\
blt|\
bnf\.fr\_bot|\
boaconstrictor|\
boardreader|\
boia\-scan\-agent|\
boia\.org|\
boitho|\
boi\_crawl\_00|\
bookmark\ buddy\ bookmark\ checker|\
bookmark\ search\ tool|\
bosug|\
bot\ apoena|\
botalot|\
botrighthere|\
botswana|\
bottybot|\
bpbot|\
braintime\_search|\
brokenlinkcheck\.com|\
browseremulator|\
browsermob|\
bruinbot|\
bsearchr&d|\
bspider|\
btbot|\
btsearch|\
bubing|\
buddy|\
buibui|\
buildcms\ crawler|\
builtbottough|\
bullseye|\
bumblebee|\
bunnyslippers|\
buscadorclarin|\
buscaplus\ robi|\
butterfly|\
buyhawaiibot|\
buzzbot|\
byindia|\
byspider|\
byteserver|\
bzbot|\
c\ r\ a\ w\ l\ 3\ r|\
cacheblaster|\
caddbot|\
cafi|\
camcrawler|\
camelstampede|\
canon\-webrecord|\
careerbot|\
cataguru|\
catchbot|\
cazoodle|\
ccbot|\
ccgcrawl|\
ccubee|\
cd\-preload|\
ce\-preload|\
cegbfeieh|\
cerberian\ drtrs|\
cert\ figleafbot|\
cfetch|\
cfnetwork|\
chameleon|\
charlotte|\
check&get|\
checkbot|\
checklinks|\
cheesebot|\
chemiede\-nodebot|\
cherrypicker|\
chilkat|\
chinaclaw|\
cipinetbot|\
cis455crawler|\
citeseerxbot|\
cizilla|\
clariabot|\
climate\ ark|\
climateark\ spider|\
clshttp|\
clushbot|\
coast\ scan\ engine|\
coast\ webmaster\ pro|\
coccoc|\
collapsarweb|\
collector|\
colocrossing|\
combine|\
connectsearch|\
conpilot|\
contentsmartz|\
contextad\ bot|\
contype|\
cookienet|\
coolbot|\
coolcheck|\
copernic|\
copier|\
copyrightcheck|\
core\-project|\
cosmos|\
covario\-ids|\
cowbot\-|\
cowdog\ bot|\
crabbybot|\
craftbot\@yahoo\.com|\
crawler\.kpricorn\.org|\
crawler43\.ejupiter\.com|\
crawler4j|\
crawler@|\
crawler\_for\_infomine|\
crawly|\
crawl\_application|\
creativecommons|\
crescent|\
cs\-crawler|\
cse\ html\ validator|\
cshttpclient|\
cuasarbot|\
culsearch|\
curl|\
custo|\
cvaulev|\
cyberdog|\
cybernavi\_webget|\
cyberpatrol\ sitecat\ webbot|\
cyberspyder|\
cydralspider|\
d1garabicengine|\
datacha0s|\
datafountains|\
dataparksearch|\
dataprovider\.com|\
datascape\ robot|\
dataspearspiderbot|\
dataspider|\
dattatec\.com|\
daumoa|\
dblbot|\
dcpbot|\
declumbot|\
deepindex|\
deepnet\ crawler|\
deeptrawl|\
dejan|\
del\.icio\.us\-thumbnails|\
deltascan|\
delvubot|\
der\ gro§e\ bildersauger|\
der\ große\ bildersauger|\
deusu|\
dfs\-fetch|\
diagem|\
diamond|\
dibot|\
didaxusbot|\
digext|\
digger|\
digi\-rssbot|\
digitalarchivesbot|\
digout4u|\
diibot|\
dillo|\
dir\_snatch\.exe|\
disco|\
distilled\-reputation\-monitor|\
djangotraineebot|\
dkimrepbot|\
dmoz\ downloader|\
docomo|\
dof\-verify|\
domaincrawler|\
domainscan|\
domainwatcher\ bot|\
dotbot|\
dotspotsbot|\
dow\ jones\ searchbot|\
download|\
doy|\
dragonfly|\
drip|\
drone|\
dtaagent|\
dtsearchspider|\
dumbot|\
dwaar|\
dxseeker|\
e\-societyrobot|\
eah|\
earth\ platform\ indexer|\
earth\ science\ educator\ \ robot|\
easydl|\
ebingbong|\
ec2linkfinder|\
ecairn\-grabber|\
ecatch|\
echoosebot|\
edisterbot|\
edugovsearch|\
egothor|\
eidetica\.com|\
eirgrabber|\
elblindo\ the\ blind\ bot|\
elisabot|\
ellerdalebot|\
email\ exractor|\
emailcollector|\
emailleach|\
emailsiphon|\
emailwolf|\
emeraldshield|\
empas\_robot|\
enabot|\
endeca|\
enigmabot|\
enswer\ neuro\ bot|\
enter\ user\-agent|\
entitycubebot|\
erocrawler|\
estylesearch|\
esyndicat\ bot|\
eurosoft\-bot|\
evaal|\
eventware|\
everest\-vulcan\ inc\.|\
exabot|\
exactsearch|\
exactseek|\
exooba|\
exploder|\
express\ webpictures|\
extractor|\
eyenetie|\
ez\-robot|\
ezooms|\
f\-bot\ test\ pilot|\
factbot|\
fairad\ client|\
falcon|\
fast\ data\ search\ document\ retriever|\
fast\ esp|\
fast\-search\-engine|\
fastbot\ crawler|\
fastbot\.de\ crawler|\
fatbot|\
favcollector|\
faviconizer|\
favorites\ sweeper|\
fdm|\
fdse\ robot|\
fedcontractorbot|\
fembot|\
fetch\ api\ request|\
fetch\_ici|\
fgcrawler|\
filangy|\
filehound|\
findanisp\.com\_isp\_finder|\
findlinks|\
findweb|\
firebat|\
firstgov\.gov\ search|\
flaming\ attackbot|\
flamingo\_searchengine|\
flashcapture|\
flashget|\
flickysearchbot|\
fluffy\ the\ spider|\
flunky|\
focused\_crawler|\
followsite|\
foobot|\
fooooo\_web\_video\_crawl|\
fopper|\
formulafinderbot|\
forschungsportal|\
francis|\
freewebmonitoring\ sitechecker|\
freshcrawler|\
freshdownload|\
freshlinks\.exe|\
friendfeedbot|\
frodo\.at|\
froggle|\
frontpage|\
froola\ bot|\
fr\_crawler|\
fu\-nbi|\
full\_breadth\_crawler|\
funnelback|\
furlbot|\
g10\-bot|\
gaisbot|\
galaxybot|\
gazz|\
gbplugin|\
generate\_infomine\_category\_classifiers|\
genevabot|\
geniebot|\
genieo|\
geomaxenginebot|\
geometabot|\
geonabot|\
geovisu|\
germcrawler\ |\
gethtmlcontents|\
getleft|\
getright|\
getsmart|\
geturl\.rexx|\
getweb!|\
giant|\
gigablastopensource|\
gigabot|\
girafabot|\
gleamebot|\
gnome\-vfs|\
go!zilla|\
go\-ahead\-got\-it|\
go\-http\-client|\
goforit\.com|\
goforitbot|\
gold\ crawler|\
goldfire\ server|\
golem|\
goodjelly|\
gordon\-college\-google\-mini|\
goroam|\
goseebot|\
gotit|\
govbot|\
gpu\ p2p\ crawler|\
grabber|\
grabnet|\
grafula|\
grapefx|\
grapeshot|\
grbot|\
greenyogi|\
gromit|\
grub|\
gsa|\
gslfbot|\
gulliver|\
gulperbot|\
gurujibot|\
gvc\ business\ crawler|\
gvc\ crawler|\
gvc\ search\ bot|\
gvc\ web\ crawler|\
gvc\ weblink\ crawler|\
gvc\ world\ links|\
gvcbot\.com|\
happyfunbot|\
harvest|\
hatena\ antenna|\
hawler|\
hcat|\
hclsreport\-crawler|\
hd\ nutch\ agent|\
header\_test\_client|\
healia\
[NC,OR]
#500 new rule
RewriteCond %{HTTP_USER_AGENT} \
helix|\
here\ will\ be\ link\ to\ crawler\ site|\
heritrix|\
hiscan|\
hisoftware\ accmonitor\ server|\
hisoftware\ accverify|\
hitcrawler|\
hivabot|\
hloader|\
hmsebot|\
hmview|\
hoge|\
holmes|\
homepagesearch|\
hooblybot\-image|\
hoowwwer|\
hostcrawler|\
hsft\ \\-\ link\ scanner|\
hsft\ \\-\ lvu\ scanner|\
hslide|\
ht://check|\
htdig|\
html\ link\ validator|\
htmlparser|\
httplib|\
httrack|\
huaweisymantecspider|\
hul\-wax|\
humanlinks|\
hyperestraier|\
hyperix|\
iaarchiver\-|\
ia\_archiver|\
ibuena|\
icab|\
icds\-ingestion|\
ichiro|\
icopyright\ conductor|\
ieautodiscovery|\
iecheck|\
ihwebchecker|\
iiitbot|\
iim\_405|\
ilsebot|\
iltrovatore|\
image\ stripper|\
image\ sucker|\
image\-fetcher|\
imagebot|\
imagefortress|\
imageshereimagesthereimageseverywhere|\
imagevisu|\
imds\_monitor|\
imo\-google\-robot\-intelink|\
inagist\.com\ url\ crawler|\
indexer|\
industry\ cortex\ webcrawler|\
indy\ library|\
indylabs\_marius|\
inelabot|\
inet32\ ctrl|\
inetbot|\
info\ seeker|\
infolink|\
infomine|\
infonavirobot|\
informant|\
infoseek\ sidewinder|\
infotekies|\
infousabot|\
ingrid|\
inktomi|\
insightscollector|\
insightsworksbot|\
inspirebot|\
insumascout|\
intelix|\
intelliseek|\
interget|\
internet\ ninja|\
internet\ radio\ crawler|\
internetlinkagent|\
interseek|\
ioi|\
ip\-web\-crawler\.com|\
ipadd\ bot|\
ipselonbot|\
ips\-agent|\
iria|\
irlbot|\
iron33|\
isara|\
isearch|\
isilox|\
istellabot|\
its\-learning\ crawler|\
iu\_csci\_b659\_class\_crawler|\
ivia|\
jadynave|\
java|\
jbot|\
jemmathetourist|\
jennybot|\
jetbot|\
jetbrains\ omea\ pro|\
jetcar|\
jim|\
jobo|\
jobspider\_ba|\
joc|\
joedog|\
joyscapebot|\
jspyda|\
junut\ bot|\
justview|\
jyxobot|\
k\.s\.bot|\
kakclebot|\
kalooga|\
katatudo\-spider|\
kbeta1|\
keepni\ web\ site\ monitor|\
kenjin\.spider|\
keybot\ translation\-search\-machine|\
keywenbot|\
keyword\ density|\
keyword\.density|\
kinjabot|\
kitenga\-crawler\-bot|\
kiwistatus|\
kmbot\-|\
kmccrew\ bot\ search|\
knight|\
knowitall|\
knowledge\ engine|\
knowledge\.com|\
koepabot|\
koninklijke|\
korniki|\
krowler|\
ksbot|\
kuloko\-bot|\
kulturarw3|\
kummhttp|\
kurzor|\
kyluka\ crawl|\
l\.webis|\
labhoo|\
labourunions411|\
lachesis|\
lament|\
lamerexterminator|\
lapozzbot|\
larbin|\
lbot|\
leaptag|\
leechftp|\
leechget|\
letscrawl\.com|\
lexibot|\
lexxebot|\
lftp|\
libcrawl|\
libiviacore|\
libw|\
likse|\
linguee\ bot|\
link\ checker|\
link\ validator|\
linkalarm|\
linkbot|\
linkcheck\ by\ siteimprove\.com|\
linkcheck\ scanner|\
linkchecker|\
linkdex\.com|\
linkextractorpro|\
linklint|\
linklooker|\
linkman|\
links\ sql|\
linkscan|\
linksmanager\.com\_bot|\
linksweeper|\
linkwalker|\
link\_checker|\
litefinder|\
litlrbot|\
little\ grabber\ at\ skanktale\.com|\
livelapbot|\
lm\ harvester|\
lmqueuebot|\
lnspiderguy|\
loadtimebot|\
localcombot|\
locust|\
lolongbot|\
lookbot|\
lsearch|\
lssbot|\
lt\ scotland\ checklink|\
ltx71.com|\
lwp|\
lycos\_spider|\
lydia\ entity\ spider|\
lynnbot|\
lytranslate|\
mag\-net|\
magnet|\
magpie\-crawler|\
magus\ bot|\
mail\.ru|\
mainseek\_bot|\
mammoth|\
map\ robot|\
markwatch|\
masagool|\
masidani\_bot\_|\
mass\ downloader|\
mata\ hari|\
mata\.hari|\
matentzn\ at\ cs\ dot\ man\ dot\ ac\ dot\ uk|\
maxamine\.com\-\-robot|\
maxamine\.com\-robot|\
maxomobot|\
mcbot|\
medrabbit|\
megite|\
memacbot|\
memo|\
mendeleybot|\
mercator\-|\
mercuryboard\_user\_agent\_sql\_injection\.nasl|\
metacarta|\
metaeuro\ web\ search|\
metager2|\
metagloss|\
metal\ crawler|\
metaquerier|\
metaspider|\
metaspinner|\
metauri|\
mfcrawler|\
mfhttpscan|\
midown\ tool|\
miixpc|\
mini\-robot|\
minibot|\
minirank|\
mirror|\
missigua\ locator|\
mister\ pix|\
mister\.pix|\
miva|\
mj12bot|\
mnogosearch|\
moduna\.com|\
mod\_accessibility|\
moget|\
mojeekbot|\
monkeycrawl|\
moses|\
mowserbot|\
mqbot|\
mse360|\
msindianwebcrawl|\
msmobot|\
msnptc|\
msrbot|\
mt\-soft|\
multitext|\
my\-heritrix\-crawler|\
myapp|\
mycompanybot|\
mycrawler|\
myengines\-us\-bot|\
myfamilybot|\
myra|\
my\_little\_searchengine\_project|\
nabot|\
najdi\.si|\
nambu|\
nameprotect|\
nasa\ search|\
natchcvs|\
natweb\-bad\-link\-mailer|\
naver|\
navroad|\
nearsite|\
nec\-meshexplorer|\
neosciocrawler|\
nerdbynature\.bot|\
nerdybot|\
nerima\-crawl-|\
nessus|\
nestreader|\
net\ vampire|\
net::trackback|\
netants|\
netcarta\ cyberpilot\ pro|\
netcraft|\
netexperts|\
netid\.com\ bot|\
netmechanic|\
netprospector|\
netresearchserver|\
netseer|\
netshift=|\
netsongbot|\
netsparker|\
netspider|\
netsrcherp|\
netzip|\
newmedhunt|\
news\ bot|\
newsgatherer|\
newsgroupreporter|\
newstrovebot|\
news\_search\_app|\
nextgensearchbot|\
nextthing\.org|\
nicebot|\
nicerspro|\
niki\-bot|\
nimblecrawler|\
nimbus\-1|\
ninetowns|\
ninja|\
njuicebot|\
nlese|\
nogate|\
norbert\ the\ spider|\
noteworthybot|\
npbot|\
nrcan\ intranet\ crawler|\
nsdl\_search\_bot|\
nuggetize\.com\ bot|\
nusearch\ spider|\
nutch|\
nu\_tch|\
nwspider|\
nymesis|\
nys\-crawler|\
objectssearch|\
obot|\
obvius\ external\ linkcheck|\
ocelli|\
octopus|\
odp\ entries\ t\_st|\
oegp|\
offline\ navigator|\
offline\.explorer|\
ogspider|\
omiexplorer\_bot|\
omniexplorer|\
omnifind|\
omniweb|\
onetszukaj|\
online\ link\ validator|\
oozbot|\
openbot|\
openfind|\
openintelligencedata|\
openisearch|\
openlink\ virtuoso\ rdf\ crawler|\
opensearchserver\_bot|\
opidig|\
optidiscover|\
oracle\ secure\ enterprise\ search|\
oracle\ ultra\ search|\
orangebot|\
orisbot|\
ornl\_crawler|\
ornl\_mercury|\
osis\-project\.jp|\
oso|\
outfoxbot|\
outfoxmelonbot|\
owler\-bot|\
owsbot|\
ozelot|\
p3p\ client|\
pagebiteshyperbot|\
pagebull|\
pagedown|\
pagefetcher|\
pagegrabber|\
pagepeeker|\
pagerank\ monitor|\
page\_verifier|\
pamsnbot\.htm|\
panopy\ bot|\
panscient\.com|\
pansophica|\
papa\ foto|\
paperlibot|\
parasite|\
parsijoo|\
pathtraq|\
pattern|\
patwebbot|\
pavuk|\
paxleframework|\
pbbot|\
pcbrowser|\
pcore\-http|\
pd\-crawler|\
penthesila|\
perform\_crawl|\
perman|\
personal\ ultimate\ crawler|\
php\ version\ tracker|\
phpcrawl|\
phpdig|\
picosearch|\
pieno\ robot|\
pipbot|\
pipeliner|\
pita|\
pixfinder|\
piyushbot|\
planetwork\ bot\ search|\
plucker|\
plukkie|\
plumtree|\
pockey|\
pocohttp|\
pogodak\.ba|\
pogodak\.co\.yu|\
poirot|\
polybot|\
pompos|\
poodle\ predictor|\
popscreenbot|\
postpost|\
privacyfinder|\
projectwf\-java\-test\-crawler|\
propowerbot|\
prowebwalker|\
proxem\ websearch|\
proximic|\
proxy\ crawler|\
psbot|\
pss\-bot|\
psycheclone|\
pub\-crawler|\
pucl|\
pulsebot|\
pump|\
pwebot|\
python|\
qeavis\ agent|\
qfkbot|\
qualidade|\
qualidator\.com\ bot|\
quepasacreep|\
queryn\ metasearch|\
queryn\.metasearch|\
quest\.durato|\
quintura\-crw|\
qunarbot|\
qwantify|\
qweerybot|\
qweery\_robot\.txt\_checkbot|\
r2ibot|\
r6\_commentreader|\
r6\_feedfetcher|\
r6\_votereader|\
rabot|\
radian6|\
radiation\ retriever|\
rampybot|\
rankivabot|\
rankur|\
rational\ sitecheck|\
rcstartbot|\
realdownload|\
reaper|\
rebi\-shoveler|\
recorder|\
redbot|\
redcarpet|\
reget|\
repomonkey|\
research\ robot|\
riddler|\
riight|\
risenetbot|\
riverglassscanner\
[NC,OR]

#1000 new rule
RewriteCond %{HTTP_USER_AGENT} \
robopal|\
robosourcer|\
robotek|\
robozilla|\
roger|\
rome\ client|\
rondello|\
rotondo|\
roverbot|\
rpt\-httpclient|\
rtgibot|\
rufusbot|\
runnk\ online\ rss\ reader|\
runnk\ rss\ aggregator|\
s2bot|\
safaribookmarkchecker|\
safednsbot|\
safetynet\ robot|\
saladspoon|\
sapienti|\
sapphireweb|\
sbider|\
sbl\-bot|\
scfcrawler|\
scich|\
scientificcommons\.org|\
scollspider|\
scooperbot|\
scooter|\
scoutjet|\
scrapebox|\
scrapy|\
scrawltest|\
screaming\ frog|\
scrubby|\
scspider|\
scumbot|\
search\ publisher|\
search\ x\-bot|\
search\-channel|\
search\-engine\-studio|\
search\.kumkie\.com|\
search\.updated\.com|\
search\.usgs\.gov|\
searcharoo\.net|\
searchblox|\
searchbot|\
searchengine|\
searchhippo\.com|\
searchit\-bot|\
searchmarking|\
searchmarks|\
searchmee!|\
searchmee\_v|\
searchmining|\
searchnowbot|\
searchpreview|\
searchspider\.com|\
searqubot|\
seb\ spider|\
seekbot|\
seeker\.lookseek\.com|\
seeqbot|\
seeqpod\-vertical\-crawler|\
selflinkchecker|\
semager|\
semanticdiscovery|\
semantifire|\
semisearch|\
semrushbot|\
seoengworldbot|\
seokicks|\
seznambot|\
shablastbot|\
shadowwebanalyzer|\
shareaza|\
shelob|\
sherlock|\
shim\-crawler|\
shopsalad|\
shopwiki|\
showlinks|\
showyoubot|\
siclab|\
silk|\
simplepie|\
siphon|\
sitebot|\
sitecheck|\
sitefinder|\
siteguardbot|\
siteorbiter|\
sitesnagger|\
sitesucker|\
sitesweeper|\
sitexpert|\
skimbot|\
skimwordsbot|\
skreemrbot|\
skywalker|\
sleipnir|\
slow\-crawler|\
slysearch|\
smart\-crawler|\
smartdownload|\
smarte\ bot|\
smartwit\.com|\
snake|\
snap\.com\ beta\ crawler|\
snapbot|\
snappreviewbot|\
snappy|\
snookit|\
snooper|\
snoopy|\
societyrobot|\
socscibot|\
soft411\ directory|\
sogou|\
sohu\ agent|\
sohu\-search|\
sokitomi\ crawl|\
solbot|\
sondeur|\
sootle|\
sosospider|\
space\ bison|\
space\ fung|\
spacebison|\
spankbot|\
spanner|\
spatineo\ monitor\ controller|\
spatineo\ serval\ controller|\
spatineo\ serval\ getmapbot|\
special\_archiver|\
speedy|\
sphere\ scout|\
sphider|\
spider\.terranautic\.net|\
spiderengine|\
spiderku|\
spiderman|\
spinn3r|\
spinne|\
sportcrew\-bot|\
sproose|\
spyder3\.microsys\.com|\
sq\ webscanner|\
sqlmap|\
squid\-prefetch|\
squidclamav\_redirector|\
sqworm|\
srevbot|\
sslbot|\
ssm\ agent|\
stackrambler|\
stardownloader|\
statbot|\
statcrawler|\
statedept\-crawler|\
steeler|\
stegmann\-bot|\
stero|\
stripper|\
stumbler|\
suchclip|\
sucker|\
sumeetbot|\
sumitbot|\
summizebot|\
summizefeedreader|\
sunrise\ xp|\
superbot|\
superhttp|\
superlumin\ downloader|\
superpagesbot|\
supremesearch\.net|\
supybot|\
surdotlybot|\
surf|\
surveybot|\
suzuran|\
swebot|\
swish\-e|\
sygolbot|\
synapticwalker|\
syntryx\ ant\ scout\ chassis\ pheromone|\
systemsearch\-robot|\
szukacz|\
s\~stremor\-crawler|\
t\-h\-u\-n\-d\-e\-r\-s\-t\-o\-n\-e|\
tailrank|\
takeout|\
talkro\ web\-shot|\
tamu\_crawler|\
tapuzbot|\
tarantula|\
targetblaster\.com|\
targetyournews\.com\ bot|\
tausdatabot|\
taxinomiabot|\
teamsoft\ wininet\ component|\
tecomi\ bot|\
teezirbot|\
teleport|\
telesoft|\
teradex\ mapper|\
teragram\_crawler|\
terrawizbot|\
testbot|\
testing\ of\ bot|\
textbot|\
thatrobotsite\.com|\
the\ dyslexalizer|\
the\ intraformant|\
the\.intraformant|\
thenomad|\
theophrastus|\
theusefulbot|\
thumbbot|\
thumbnail\.cz\ robot|\
thumbshots\-de\-bot|\
tigerbot|\
tighttwatbot|\
tineye|\
titan|\
to\-dress\_ru\_bot\_|\
to\-night\-bot|\
tocrawl|\
topicalizer|\
topicblogs|\
toplistbot|\
topserver\ php|\
topyx\-crawler|\
touche|\
tourlentascanner|\
tpsystem|\
traazi|\
transgenikbot|\
travel\-search|\
travelbot|\
travellazerbot|\
treezy|\
trendiction|\
trex|\
tridentspider|\
trovator|\
true\_robot|\
tscholarsbot|\
tsm\ translation\-search\-machine|\
tswebbot|\
tulipchain|\
turingos|\
turnitinbot|\
tutorgigbot|\
tweetedtimes\ bot|\
tweetmemebot|\
twengabot|\
twice|\
twikle|\
twinuffbot|\
twisted\ pagegetter|\
twitturls|\
twitturly|\
tygobot|\
tygoprowler|\
typhoeus|\
u\.s\.\ government\ printing\ office|\
uberbot|\
ucb\-nutch|\
udmsearch|\
ufam\-crawler\-|\
ultraseek|\
unchaos|\
unisterbot|\
unidentified|\
unitek\ uniengine|\
universalsearch|\
unwindfetchor|\
uoftdb\_experiment|\
updated|\
url\ control|\
url\-checker|\
urlappendbot|\
urlblaze|\
urlchecker|\
urlck|\
urldispatcher|\
urlspiderpro|\
urly\ warning|\
urly\.warning|\
url\_gather|\
usaf\ afkn\ k2spider|\
usasearch|\
uss\-cosmix|\
usyd\-nlp\-spider|\
vacobot|\
vacuum|\
vadixbot|\
vagabondo|\
validator|\
valkyrie|\
vbseo|\
vci\ webviewer\ vci\ webviewer\ win32|\
verbstarbot|\
vericitecrawler|\
verifactrola|\
verity\-url\-gateway|\
vermut|\
versus\ crawler|\
versus\.integis\.ch|\
viasarchivinginformation\.html|\
vipr|\
virus\-detector|\
virus\_detector|\
visbot|\
vishal\ for\ clia|\
visweb|\
vital\ search'n\ urchin|\
vlad|\
vlsearch|\
voilabot|\
vmbot|\
vocusbot|\
voideye|\
voil|\
vortex|\
voyager|\
vspider|\
w3c\-webcon|\
w3c\_unicorn|\
w3search|\
wacbot|\
wanadoo|\
wastrix|\
water\ conserve\ portal|\
water\ conserve\ spider|\
watzbot|\
wauuu|\
wavefire|\
waypath|\
wazzup|\
wbdbot|\
web\ ceo\ online\ robot|\
web\ crawler|\
web\ downloader|\
web\ image\ collector|\
web\ link\ validator|\
web\ magnet|\
web\ site\ downloader|\
web\ sucker|\
web\-agent|\
web\-sniffer|\
web\.image\.collector|\
webaltbot|\
webauto|\
webbot|\
webbul\-bot|\
webcapture|\
webcheck|\
webclipping\.com|\
webcollage|\
webcopier|\
webcopy|\
webcorp|\
webcrawl\.net|\
webcrawler|\
webdatacentrebot|\
webdownloader\ for\ x|\
webdup|\
webemailextrac|\
webenhancer|\
webfetch|\
webgather|\
webgo\ is|\
webgobbler|\
webimages|\
webinator\-search2|\
webinator\-wbi|\
webindex|\
weblayers|\
webleacher|\
weblexbot|\
weblinker|\
weblyzard|\
webmastercoffee|\
webmasterworld\ extractor|\
webmasterworldforumbot|\
webminer|\
webmoose|\
webot|\
webpix|\
webreaper|\
webripper|\
websauger|\
webscan|\
websearchbench|\
website|\
webspear|\
websphinx|\
webspider|\
webster|\
webstripper|\
webtrafficexpress|\
webtrends\ link\ analyzer|\
webvac|\
webwalk|\
webwasher|\
webwatch|\
webwhacker|\
webxm|\
webzip|\
weddings\.info|\
wenbin|\
wep\ search|\
wepa|\
werelatebot|\
wget|\
whacker|\
whirlpool\ web\ engine|\
whowhere\ robot|\
widow|\
wikiabot|\
wikio|\
wikiwix\-bot\-|\
winhttp|\
wire|\
wisebot|\
wisenutbot|\
wish\-la|\
wish\-project|\
wisponbot|\
wmcai\-robot|\
wminer|\
wmsbot|\
woriobot|\
worldshop|\
worqmada|\
wotbox|\
wume\_crawler|\
www\ collector|\
www\-collector\-e|\
www\-mechanize|\
wwwoffle|\
wwwrobot|\
wwwster|\
wwwwanderer|\
wwwxref|\
wysigot|\
x\-clawler|\
x\-crawler|\
xaldon|\
xenu|\
xerka\ metabot|\
xerka\ webbot|\
xget|\
xirq|\
xmarksfetch|\
xqrobot|\
y!j|\
yacy\.net|\
yacybot|\
yanga\ worldsearch\ bot|\
yarienavoir\.net|\
yasaklibot|\
yats\ crawler|\
ybot|\
yebolbot|\
yellowjacket|\
yeti|\
yolinkbot|\
yooglifetchagent|\
yoono|\
yottacars\_bot|\
yourls|\
z\-add\ link\ checker|\
zagrebin|\
zao|\
zedzo\.validate|\
zermelo|\
zeus|\
zibber\-v|\
zimeno|\
zing-bottabot|\
zipppbot|\
zongbot|\
zoomspider|\
zotag\ search|\
zsebot|\
zuibot|\
zyborg|\
zyte\
[NC]
RewriteRule .* - [F]
#bad bots end
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2019 Invision Power Services, Inc.

Web Analytics