- Date: Wed, 15 May 2019 09:35:16 -0400
- From: Greg Wooledge <wooledg@xxxxxxxxxxx>
- Subject: Re: baido.com
On Wed, May 15, 2019 at 09:29:15AM -0400, Gene Heskett wrote:
> Greetings all;
> I just caught the baido-spider crawling my site for about 15 minutes.
> Is putting www.baido.com in my hosts.deny enough to shut that down? Or
> is there a deny function in apache2 I should be using instead?
In general, use the robots.txt file in your web server first and foremost.
If that doesn't work, THEN move on to heavier solutions like firewalls.