We recently started seeing a very large amount of traffic to the kiwix mirrors. This seems to be duplicate requests from single ip’s many times a second, or at fast the connection allows. Blocking these does not work as they move onto other ip’s.
This has largely overpowered our mirror server and internet connection. This traffic might be legitimate, but user agents are not very descriptive. To make sure our other services remain usable, I have decided to put a rather large rate limit on /kiwix.
If this in any way hinders a legitimate usage of /kiwix, then please please reach out and help me understand what is going on. I will keep this rate limiting for a couple of days, and then check whether the traffic persists.