Skip to main content

bot hit - server response time and overall page loadtime increase - aws, ec2, google analytic, cloudflare, ms sql cpu usage 100%

 Dear All,

I've recently facing website page load time and server response time issue, 

troubleshooting - 

1. IIS log via - http log browser (tool to identify the bot hit)

2. disallow in robots.txt

3. article found and feature enabled in Cloudflare 

 Bot Fight Mode

Challenge requests matching patterns of known bots before they can access your site.

Requests matching Cloudflare-identified, non-legitimate automated traffic patterns will be challenged and/or blocked by Cloudflare.

And,

JavaScript Detections

Use lightweight, invisible JavaScript detections to improve Bot Management products.

4. AWS EC2 

Add new security group for Cloudflare ips list

Ref: https://dev.to/johnmccuk/automate-aws-security-group-with-cloudflare-ips-la6


And referred follow articles:

https://www.imperva.com/products/advanced-bot-protection-management/

https://blog.apnic.net/2019/11/15/how-to-hide-aws-ec2-instances-from-network-scanning-bots-using-ipv6/

https://pawelurbanek.com/ec2-ssh-dynamic-access





 

Comments

Post a Comment

Popular posts from this blog

IPv4 header to requests when a client is using IPv6

  What is Pseudo IPv4? As a stopgap to accelerate the adoption of IPv6, Cloudflare offers Pseudo IPv4 which supports IPv6 addresses in legacy applications expecting IPv4 addresses. The goal is to provide a nearly unique IPv4 address for each IPv6 address, using Class E IPv4 address space, which is designated as experimental and would not normally see traffic. To learn more see  here . Options Add header:  Add additional Cf-Pseudo-IPv4 header only Overwrite headers:  Overwrite the existing Cf-Connecting-IP and X-Forwarded-For headers with a pseudo IPv4 address Cloudflare
Post a Comment
Read more

Intrusion detection system

An   intrusion detection system   ( IDS ) is a device or   software application   that monitors network or system activities for malicious activities or policy violations and produces reports to a management station.  IDS come in a variety of “flavors” and approach the goal of detecting suspicious traffic in different ways. There are network based (NIDS) and host based (HIDS) intrusion detection systems. Some systems may attempt to stop an intrusion attempt but this is neither required nor expected of a monitoring system.  Intrusion detection and prevention systems (IDPS) are primarily focused on identifying possible incidents, logging information about them, and reporting attempts. In addition, organizations use IDPSes for other purposes, such as identifying problems with security policies, documenting existing threats and deterring individuals from violating security policies.  IDPSes have become a necessary addition to the security infrastru...
Post a Comment
Read more

IIS Manager - applocation pool restart C# code

using System; using System.Collections.Generic; using System.Text; using System.DirectoryServices; namespace RecycleApplicationPool { class Program { static void Main( string [] args) { string appPoolName = "DefaultAppPool" ; string machineName = "LOCALHOST" ; if (args.Length > 0) appPoolName = args[0]; if (args.Length > 1) machineName = args[1]; string error = null ; DirectoryEntry root = null ; try { Console .WriteLine( "Restarting Application Pool " + appPoolName + " on " + machineName + "..." ); root = new DirectoryEntry ( "IIS://" + machineName + "/W3SVC/AppPools/" +appPoolName); Console .WriteLine(root.InvokeGet( "Name" )); root.Invoke( "Recycle" ); ...
Post a Comment
Read more