bot hit - server response time and overall page loadtime increase - aws, ec2, google analytic, cloudflare, ms sql cpu usage 100%
Dear All, I've recently facing website page load time and server response time issue, troubleshooting - 1. IIS log via - http log browser (tool to identify the bot hit) 2. disallow in robots.txt 3. article found and feature enabled in Cloudflare Bot Fight Mode Challenge requests matching patterns of known bots before they can access your site. Requests matching Cloudflare-identified, non-legitimate automated traffic patterns will be challenged and/or blocked by Cloudflare. And, JavaScript Detections Use lightweight, invisible JavaScript detections to improve Bot Management products. 4. AWS EC2 Add new security group for Cloudflare ips list Ref: https://dev.to/johnmccuk/automate-aws-security-group-with-cloudflare-ips-la6 And referred follow articles: https://www.imperva.com/products/advanced-bot-protection-management/ https://blog.apnic.net/2019/11/15/how-to-hide-aws-ec2-instances-from-network-scanning-bots-using-ipv6/ https://pawelurbanek.com/ec...