Anti Aspiration Site web

WRInaute discret
Bonsoir,

Je sais qu'il y a beaucoup de post à ce sujet mais je souhaite protéger mon site contre l'aspiration de site web

J'ai essayé plusieurs .htaccess aucun ne fonctionne, il y a peut être un conflit avec les autres redirections (phpbb annuaire ect..) Mais ou ?

Voici mon .htaccess

ErrorDocument 401 http://www.nova-star.fr/page-introuvable-401.htm
ErrorDocument 402 http://www.nova-star.fr/page-introuvable-402.htm
ErrorDocument 403 http://www.nova-star.fr/page-introuvable-403.htm
ErrorDocument 404 http://www.nova-star.fr/page-introuvable-404.htm

RewriteEngine on
RewriteRule ^ajouter-site-([0-9]+)\.html$ annuaire-internet/index.php?menu=submitsite&id=$1 [L]
RewriteRule ^nouveaux-sites-([0-9]+).html$ annuaire-internet/index.php?menu=newsites&page=$1 [L]
RewriteRule ^(.+)-([0-9]+)-([0-9]+)\.html$ annuaire-internet/index.php?id=$2&page=$3 [L]
RewriteRule ^v2/$ index.html [R=301,L]

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} .*Atomic\_Email\_Hunter* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*HTTrack* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Filangy* [OR]
RewriteCond %{HTTP_USER_AGENT} .*BackWeb* [OR]
RewriteCond %{HTTP_USER_AGENT} .*BackStreet* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Bandit* [OR]
RewriteCond %{HTTP_USER_AGENT} .*BatchFTP* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Bullseye* [OR]
RewriteCond %{HTTP_USER_AGENT} .*bumblebee* [OR]
RewriteCond %{HTTP_USER_AGENT} .*capture* [OR]
RewriteCond %{HTTP_USER_AGENT} .*CherryPicker* [OR]
RewriteCond %{HTTP_USER_AGENT} .*CherryPickrElite* [OR]
RewriteCond %{HTTP_USER_AGENT} .*CherryPickerSE* [OR]
RewriteCond %{HTTP_USER_AGENT} .*ChinaClaw* [OR]
RewriteCond %{HTTP_USER_AGENT} .*clipping* [OR]
RewriteCond %{HTTP_USER_AGENT} .*collage* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Collector* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Copier* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Crescent* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*Download* [OR]
RewriteCond %{HTTP_USER_AGENT} .*eCatch* [OR]
RewriteCond %{HTTP_USER_AGENT} .*EirGrabber* [OR]
RewriteCond %{HTTP_USER_AGENT} .*email* [OR]
RewriteCond %{HTTP_USER_AGENT} .*EmeraldShield* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*FlashGet* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*FlickBot* [OR]
RewriteCond %{HTTP_USER_AGENT} .*FrontPage* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*GetRight* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*GetSmart* [OR]
RewriteCond %{HTTP_USER_AGENT} .*GetWeb* [OR]
RewriteCond %{HTTP_USER_AGENT} .*GetWebPage* [OR]
RewriteCond %{HTTP_USER_AGENT} .*gigabaz* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Go!Zilla* [OR]
RewriteCond %{HTTP_USER_AGENT} .*GornKer* [OR]
RewriteCond %{HTTP_USER_AGENT} .*gotit* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Grabber* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*GrabNet* [OR]
RewriteCond %{HTTP_USER_AGENT} .*hloader* [OR]
RewriteCond %{HTTP_USER_AGENT} .*httpdown* [OR]
RewriteCond %{HTTP_USER_AGENT} .*InterGET* [OR]
RewriteCond %{HTTP_USER_AGENT} .*JustView* [OR]
RewriteCond %{HTTP_USER_AGENT} .*kapere* [OR]
RewriteCond %{HTTP_USER_AGENT} .*larbin* [OR]
RewriteCond %{HTTP_USER_AGENT} .*LeechFTP* [OR]
RewriteCond %{HTTP_USER_AGENT} .*LexiBot* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Missigua* [OR]
RewriteCond %{HTTP_USER_AGENT} .*MSIECrawler* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Vampire* [OR]
RewriteCond %{HTTP_USER_AGENT} .*NetAnts* [OR]
RewriteCond %{HTTP_USER_AGENT} .*NetMechanic* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Openfind* [OR]
RewriteCond %{HTTP_USER_AGENT} .*PageGrabber* [OR]
RewriteCond %{HTTP_USER_AGENT} .*pavuk* [OR]
RewriteCond %{HTTP_USER_AGENT} .*pcBrowser* [OR]
RewriteCond %{HTTP_USER_AGENT} .*PersonaPilot* [OR]
RewriteCond %{HTTP_USER_AGENT} .*PingALink* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Python-urllib* [OR]
RewriteCond %{HTTP_USER_AGENT} .*PycURL* [OR]
RewriteCond %{HTTP_USER_AGENT} .*RealDownload* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Reaper* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Recorder* [OR]
RewriteCond %{HTTP_USER_AGENT} .*ReGet* [OR]
RewriteCond %{HTTP_USER_AGENT} .*replacer* [OR]
RewriteCond %{HTTP_USER_AGENT} .*SearchExpress* [OR]
RewriteCond %{HTTP_USER_AGENT} .*SlySearch* [OR]
RewriteCond %{HTTP_USER_AGENT} .*SmartDownload* [OR]
RewriteCond %{HTTP_USER_AGENT} .*snagger* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Snake* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Stripper* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Sucker* [OR]
RewriteCond %{HTTP_USER_AGENT} .*SuperBot* [OR]
RewriteCond %{HTTP_USER_AGENT} .*SuperHTTP* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Surfbot* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Syntryx* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Teleport* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Telesoft* [OR]
RewriteCond %{HTTP_USER_AGENT} .*NetSpider* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebAuto* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebBandit* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebCapture* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Webclipping* [OR]
RewriteCond %{HTTP_USER_AGENT} .*webcollage* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebCopier* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebEMailExtrac* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebFetch* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebIndexer* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebLeacher* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebMiner* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebMirror* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebReaper* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebSauger* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Website* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Webster* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebStripper* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebWhacker* [OR]
RewriteCond %{HTTP_USER_AGENT} .*WebZIP* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Wget* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Whacker* [OR]
RewriteCond %{HTTP_USER_AGENT} .*whizbang* [OR]
RewriteCond %{HTTP_USER_AGENT} .*Xenu*
RewriteRule .* - [F]



# Lines That should already be in your .htacess
<Files "config.php">
Order Allow,Deny
Deny from All
</Files>
<Files "common.php">
Order Allow,Deny
Deny from All
</Files>

# You may need to un-comment the following line
# Options +FollowSymlinks
# REMEBER YOU ONLY NEED TO STARD MOD REWRITE ONCE
RewriteEngine On
# REWRITE BASE
RewriteBase /
# HERE IS A GOOD PLACE TO ADD THE WWW PREFIXE REDIRECTION

#####################################################
# PHPBB SEO REWRITE RULES - MIXED
#####################################################
# AUTHOR : dcz www.phpbb-seo.com
# STARTED : 01/2006
#################################
# FORUMS PAGES
###############
# FORUM INDEX REWRITERULE WOULD STAND HERE IF USED. 'forum' REQUIRES TO BE SET AS FORUM INDEX
# RewriteRule ^phpbb/forum\.html$ /phpbb/index.php [QSA,L,NC]
# FORUM
RewriteRule ^phpbb/[a-z0-9_-]*-f([0-9]+)/?(page([0-9]+)\.html)?$ /phpbb/viewforum.php?f=$1&start=$3 [QSA,L,NC]
# TOPIC WITH VIRTUAL FOLDER
RewriteRule ^phpbb/[a-z0-9_-]*-f([0-9]+)/topic([0-9]+)(-([0-9]+))?\.html$ /phpbb/viewtopic.php?f=$1&t=$2&start=$4 [QSA,L,NC]
# GLOBAL ANNOUNCES WITH VIRTUAL FOLDER
RewriteRule ^phpbb/annonces/topic([0-9]+)(-([0-9]+))?\.html$ /phpbb/viewtopic.php?t=$1&start=$3 [QSA,L,NC]
# TOPIC WITHOUT FORUM ID & DELIM
RewriteRule ^phpbb/([a-z0-9_-]*)/?topic([0-9]+)(-([0-9]+))?\.html$ /phpbb/viewtopic.php?forum_uri=$1&t=$2&start=$4 [QSA,L,NC]
# PROFILES THROUGH USERNAME
RewriteRule ^phpbb/membre/([^/]+)/?$ /phpbb/memberlist.php?mode=viewprofile&un=$1 [QSA,L,NC]
# USER MESSAGES THROUGH USERNAME
RewriteRule ^phpbb/membre/([^/]+)/(topics|posts)/?(page([0-9]+)\.html)?$ /phpbb/search.php?author=$1&sr=$2&start=$4 [QSA,L,NC]
# GROUPS ADVANCED
RewriteRule ^phpbb/[a-z0-9_-]*-g([0-9]+)(-([0-9]+))?\.html$ /phpbb/memberlist.php?mode=group&g=$1&start=$3 [QSA,L,NC]
# POST
RewriteRule ^phpbb/post([0-9]+)\.html$ /phpbb/viewtopic.php?p=$1 [QSA,L,NC]
# ACTIVE TOPICS
RewriteRule ^phpbb/sujets-actifs(-([0-9]+))?\.html$ /phpbb/search.php?search_id=active_topics&start=$2&sr=topics [QSA,L,NC]
# UNANSWERED TOPICS
RewriteRule ^phpbb/sans-reponses(-([0-9]+))?\.html$ /phpbb/search.php?search_id=unanswered&start=$2&sr=topics [QSA,L,NC]
# NEW POSTS
RewriteRule ^phpbb/nouveaux-messages(-([0-9]+))?\.html$ /phpbb/search.php?search_id=newposts&start=$2&sr=topics [QSA,L,NC]
# THE TEAM
RewriteRule ^phpbb/equipe\.html$ /phpbb/memberlist.php?mode=leaders [QSA,L,NC]
# HERE IS A GOOD PLACE TO ADD OTHER PHPBB RELATED REWRITERULES

# FORUM WITHOUT ID & DELIM
# THESE FOUR LINES MUST BE LOCATED AT THE END OF YOUR HTACCESS TO WORK PROPERLY
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-l
RewriteRule ^phpbb/([a-z0-9_-]+)/?(page([0-9]+)\.html)?$ /phpbb/viewforum.php?forum_uri=$1&start=$3 [QSA,L,NC]
# END PHPBB PAGES
#####################################################


Cordialement
 
WRInaute occasionnel
De plus, c'est tellement facile pour un aspirateur de site de se faire passer pour Mozilla ou un autre navigateur, que ça ne sert pas à grand chose.
 
WRInaute passionné
Le meiux restant contre les "aspirateurs de sites " est encore de mesurer le temps de navigation entre chaques page. Si il est inférieure ou égal à 1 seconde sur 5 pages, laisse tomber, c'est pas humain, et tu peux filtrer. :)
 
WRInaute accro
BadProcESs a dit:
Le meiux restant contre les "aspirateurs de sites " est encore de mesurer le temps de navigation entre chaques page. Si il est inférieure ou égal à 1 seconde sur 5 pages, laisse tomber, c'est pas humain, et tu peux filtrer. :)
Sauf que ca peut également être un bot de moteur de recherche..et alors là, plus de SERP.
 
WRInaute passionné
Heu... En général les bots des moteurs de recherche sont "propres", à savoir que leurs vitesse de crawl ne descend jamais en dessous d'une seconde par page pour ne pas faire "tomber" le site crawlé.
 
Discussions similaires
Haut