Page 1 of 1

Hướng dẫn sử dụng chức năng "Block Bad Bots Scan Website"

PostPosted:27 Jul 2018 14:40
by nguyenoanh
Bình thường các bot scan tất cả các website trên server để lấy dữ liệu nhưng một số bot scan rất khó chịu, có khi ngốn băng thông rất lớn, thậm chí còn làm lag, chậm server.
Một số bạn không muốn các bộ máy tìm kiếm (không cần thiết) scan lấy dữ liệu từ website của mình thì có thể dùng chức năng này để chặn các bad bots này lại.

Đường dẫn chức năng: VPSSIM Menu ==> Bảo Mật Server & Website ==> Block Bad Bots Scan Website

Các Bad Bot được liệt kê trong : /etc/nginx/conf/blockbadbots.conf

Bạn có thể thêm hoặc bớt badbot trong file này.

Nội dung file này:
Code: Select all
if ($http_user_agent ~* (360Spider|80legs.com|Abonti|AcoonBot|Acunetix|adbeat_bot|AddThis.com|adidxbot|ADmantX|AhrefsBot|AngloINFO|Antelope|Applebot|BeetleBot|billigerbot|binlar|bitlybot|BlackWidow|BLP_bbot|BoardReader|Bolt\ 0|BOT\ for\ JCE|Bot\ mailto\:craftbot@yahoo\.com|casper|CazoodleBot|CCBot|checkprivacy|ChinaClaw|chromeframe|Clerkbot|Cliqzbot|clshttp|CommonCrawler|comodo|CPython|crawler4j|Crawlera|CRAZYWEBCRAWLER|Curious|Curl|Custo|CWS_proxy|Default\ Browser\ 0|diavol|DigExt|Digincore|DIIbot|discobot|DISCo|DoCoMo|DotBot|Download\ Demon|DTS.Agent|EasouSpider|eCatch|ecxi|EirGrabber|Elmer|EmailCollector|EmailSiphon|EmailWolf|Exabot|ExaleadCloudView|ExpertSearchSpider|ExpertSearch|Express\ WebPictures|ExtractorPro|extract|EyeNetIE|Ezooms|F2S|FastSeek|feedfinder|FeedlyBot|FHscan|finbot|Flamingo_SearchEngine|FlappyBot|FlashGet|flicky|Flipboard|g00g1e|Genieo|genieo|GetRight|GetWeb\!|GigablastOpenSource|GozaikBot|Go\!Zilla|Go\-Ahead\-Got\-It|GrabNet|grab|Grafula|GrapeshotCrawler|GTB5|GT\:\:WWW|Guzzle|harvest|heritrix|HMView|HomePageBot|HTTP\:\:Lite|HTTrack|HubSpot|ia_archiver|icarus6|IDBot|id\-search|IlseBot|Image\ Stripper|Image\ Sucker|Indigonet|Indy\ Library|integromedb|InterGET|InternetSeer\.com|Internet\ Ninja|IRLbot|ISC\ Systems\ iRc\ Search\ 2\.1|jakarta|Java|JetCar|JobdiggerSpider|JOC\ Web\ Spider|Jooblebot|kanagawa|KINGSpider|kmccrew|larbin|LeechFTP|libwww|Lingewoud|LinkChecker|linkdexbot|LinksCrawler|LinksManager\.com_bot|linkwalker|LinqiaRSSBot|LivelapBot|ltx71|LubbersBot|lwp\-trivial|Mail.RU_Bot|masscan|Mass\ Downloader|maverick|Maxthon$|Mediatoolkitbot|MegaIndex|MegaIndex|megaindex|MFC_Tear_Sample|Microsoft\ URL\ Control|microsoft\.url|MIDown\ tool|miner|Missigua\ Locator|Mister\ PiX|mj12bot|Mozilla.*Indy|Mozilla.*NEWT|MSFrontPage|msnbot|Navroad|NearSite|NetAnts|netEstate|NetSpider|NetZIP|Net\ Vampire|NextGenSearchBot|nutch|Octopus|Offline\ Explorer|Offline\ Navigator|OpenindexSpider|OpenWebSpider|OrangeBot|Owlin|PageGrabber|PagesInventory|panopta|panscient\.com|Papa\ Foto|pavuk|pcBrowser|PECL\:\:HTTP|PeoplePal|Photon|PHPCrawl|planetwork|PleaseCrawl|PNAMAIN.EXE|PodcastPartyBot|prijsbest|proximic|psbot|purebot|pycurl|QuerySeekerSpider|R6_CommentReader|R6_FeedFetcher|RealDownload|ReGet|Riddler|Rippers\ 0|rogerbot|RSSingBot|rv\:1.9.1|RyzeCrawler|SafeSearch|SBIder|Scrapy|Scrapy|Screaming|SeaMonkey$|search.goo.ne.jp|SearchmetricsBot|search_robot|SemrushBot|Semrush|SentiBot|SEOkicks|SeznamBot|ShowyouBot|SightupBot|SISTRIX|sitecheck\.internetseer\.com|siteexplorer.info|SiteSnagger|skygrid|Slackbot|Slurp|SmartDownload|Snoopy|Sogou|Sosospider|spaumbot|Steeler|sucker|SuperBot|Superfeedr|SuperHTTP|SurdotlyBot|Surfbot|tAkeOut|Teleport\ Pro|TinEye-bot|TinEye|Toata\ dragostea\ mea\ pentru\ diavola|Toplistbot|trendictionbot|TurnitinBot|turnit|Twitterbot|URI\:\:Fetch|urllib|Vagabondo|Vagabondo|vikspider|VoidEYE|VoilaBot|WBSearchBot|webalta|WebAuto|WebBandit|WebCollage|WebCopier|WebFetch|WebGo\ IS|WebLeacher|WebReaper|WebSauger|Website\ eXtractor|Website\ Quester|WebStripper|WebWhacker|WebZIP|Web\ Image\ Collector|Web\ Sucker|Wells\ Search\ II|WEP\ Search|WeSEE|Wget|Widow|WinInet|woobot|woopingbot|worldwebheritage.org|Wotbox|WPScan|WWWOFFLE|WWW\-Mechanize|Xaldon\ WebSpider|XoviBot|yacybot|YisouSpider|zermelo|Zeus|zh-CN|ZmEu|ZumBot|ZyBorg) ) {
    return 410;
}
#Yahoo|YandexBot|Yandex|BaiduSpider|
Cách sử dụng:
=========================================================================
                VPSSIM - Quan Ly VPS/Server by HostingAZ.VN
=========================================================================
                         Bao Mat Server & Website
=========================================================================

1) User & Password Mac Dinh	     8) Dat Mat Khau Bao Ve Website
2) Quan Ly CSF Firewall		     9) Block Exploits, SQL Injections
3) Quan Ly IPtables Firewall	    10) Block Bad Bots Scan Website
4) Linux Malware Detect & ClamAV    11) Run Script In Writable Folder
5) Check & Block IP DOS		    12) BAT/TAT Email Thong Bao Login
6) Thay Doi Port SSH Number	    13) Thay Password Account Root
7) Dat Mat Khau Bao Ve Folder
Lua chon cua ban (0-Thoat): 10
=========================================================================
Dung chuc nang nay de config Disabled bad bots (spiders) scan Website
-------------------------------------------------------------------------
Mac dinh, tat ca bot deu co the scan website. Su dung chuc nang nay de
-------------------------------------------------------------------------
config/block nhung bots xau ma ban khong muon no scan website cua ban.
-------------------------------------------------------------------------
Neu ban muon dua ve config mac dinh, chay chuc nang mot lan nua, nhap ten
-------------------------------------------------------------------------
website va chon DISABLE cau hinh config block badbots.
=========================================================================
Them hoac xoa badbots, spider trong: /etc/nginx/conf/blockbadbots.conf
=========================================================================
Ban muon xem danh sach website tren server ? [y/N] n
=========================================================================
Nhap ten website website [ENTER]: oanh.com
=========================================================================
oanh.com hien tai khong config block block badbots.
-------------------------------------------------------------------------
Ban muon BAT config nay cho website ?  [y/N] y
Kết quả:
=========================================================================
Da BAT config [Block Bad Bots] cho oanh.com !
=========================================================================
                VPSSIM - Quan Ly VPS/Server by HostingAZ.VN
=========================================================================
                         Bao Mat Server & Website
=========================================================================

1) User & Password Mac Dinh	     8) Dat Mat Khau Bao Ve Website
2) Quan Ly CSF Firewall		     9) Block Exploits, SQL Injections
3) Quan Ly IPtables Firewall	    10) Block Bad Bots Scan Website
4) Linux Malware Detect & ClamAV    11) Run Script In Writable Folder
5) Check & Block IP DOS		    12) BAT/TAT Email Thong Bao Login
6) Thay Doi Port SSH Number	    13) Thay Password Account Root
7) Dat Mat Khau Bao Ve Folder
Lua chon cua ban (0-Thoat):
Để tắt config block badbots cho domain, bạn chạy lại chức năng này và làm theo hướng dẫn của VPSSIM
=========================================================================
                VPSSIM - Quan Ly VPS/Server by HostingAZ.VN
=========================================================================
                         Bao Mat Server & Website
=========================================================================

1) User & Password Mac Dinh	     8) Dat Mat Khau Bao Ve Website
2) Quan Ly CSF Firewall		     9) Block Exploits, SQL Injections
3) Quan Ly IPtables Firewall	    10) Block Bad Bots Scan Website
4) Linux Malware Detect & ClamAV    11) Run Script In Writable Folder
5) Check & Block IP DOS		    12) BAT/TAT Email Thong Bao Login
6) Thay Doi Port SSH Number	    13) Thay Password Account Root
7) Dat Mat Khau Bao Ve Folder
Lua chon cua ban (0-Thoat): 10
=========================================================================
Dung chuc nang nay de config Disabled bad bots (spiders) scan Website
-------------------------------------------------------------------------
Mac dinh, tat ca bot deu co the scan website. Su dung chuc nang nay de
-------------------------------------------------------------------------
config/block nhung bots xau ma ban khong muon no scan website cua ban.
-------------------------------------------------------------------------
Neu ban muon dua ve config mac dinh, chay chuc nang mot lan nua, nhap ten
-------------------------------------------------------------------------
website va chon DISABLE cau hinh config block badbots.
=========================================================================
Them hoac xoa badbots, spider trong: /etc/nginx/conf/blockbadbots.conf
=========================================================================
Ban muon xem danh sach website tren server ? [y/N] n
=========================================================================
Nhap ten website website [ENTER]: oanh.com
=========================================================================
oanh.com hien tai dang duoc config block badbots.
-------------------------------------------------------------------------
Ban muon tat config nay cho website ?  [y/N] y
Kết quả:
=========================================================================
Da tat config [Block Bad Bots] cho oanh.com.
=========================================================================
                VPSSIM - Quan Ly VPS/Server by HostingAZ.VN
=========================================================================
                         Bao Mat Server & Website
=========================================================================

1) User & Password Mac Dinh	     8) Dat Mat Khau Bao Ve Website
2) Quan Ly CSF Firewall		     9) Block Exploits, SQL Injections
3) Quan Ly IPtables Firewall	    10) Block Bad Bots Scan Website
4) Linux Malware Detect & ClamAV    11) Run Script In Writable Folder
5) Check & Block IP DOS		    12) BAT/TAT Email Thong Bao Login
6) Thay Doi Port SSH Number	    13) Thay Password Account Root
7) Dat Mat Khau Bao Ve Folder
Lua chon cua ban (0-Thoat):