Rate limiting on mirrors.dotsrc.org/kiwix

We recently started seeing a very large amount of traffic to the kiwix mirrors. This seems to be duplicate requests from single ip’s many times a second, or at fast the connection allows. Blocking these does not work as they move onto other ip’s. This has largely overpowered our mirror server and internet connection. This traffic might be legitimate, but user agents are not very descriptive. To make sure our other services remain usable, I have decided to put a rather large rate limit on /kiwix. [Read More]