Nginx : How to block user agent ?
Problem : Looking for ways to configure Nginx to block certain user agents like crawlers, SEO tools and robots from molesting your website.
Solution : Configure nginx.conf file to instruct Nginx to return 403 status when certain user agents visit the website.
For example:
vi /usr/local/nginx/conf/nginx.conf
and add these lines under the server block. If you have additional server block for SSL, add the same lines for the server block as well.
server {
listen 80;
if ($http_user_agent ~* (wget|majestic|ahrefs) ) {
return 403;
}
Note : ~* means case insensitive blocking. For case sensitive, use ~ instead.
restart nginx
/usr/local/nginx/sbin/nginx -s reload
In this way, whenever one of these user agents visit your website, it will be given a 403 status.
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+4.8k Linux/Unix/PHP : Restart PHP-FPM
+15.4k Golang : Test floating point numbers not-a-number and infinite example
+7.6k Golang : Implementing class(object-oriented programming style)
+7.3k Golang : Append and add item in slice
+22.7k Golang : Check if element exist in map
+18.7k Golang : Convert(cast) bytes.Buffer or bytes.NewBuffer type to io.Reader
+39.8k Golang : How to count duplicate items in slice/array?
+14.3k Golang : How do I get the local IP (non-loopback) address ?
+4.9k Javascript : How to refresh page with JQuery ?
+29.3k Golang : Generate random string
+8.9k Golang : Sort and reverse sort a slice of floats
+4.6k Golang : Generate Interleaved 2 inch by 5 inch barcode