UFW Limit

· JB's Place

Using and modifying the UFW limit feature
#systems

# Unwanted traffic

In my spare time I run an AdFilter(ing) public DNS service that is available via DoT/DoH. While I've had reasonably little trouble, occasionally when running public services you can get unwanted traffic for whatever reason.

Recently I got alerts for resource usage on one of my servers and when checking the traffic graph for the server it looked like this.

Traffic flood

Over a period of a few hours Internet traffic to/from the server had increased to a level 10x the normal levels for this server. This traffic increase resulted in increased CPU and memory utilisation, which in turn caused alerts for me to investigate.

Using tcpdump I determined that the traffic increase (and now the vast majority of traffic to this server) was DoH at ~6000 packets per second from an Internet provider in Turkmenistan.

Given DoH is TCP port 443, I assume the traffic is somewhat legitimate as opposed to say a UDP DDOS flood, however I did note that that traffic was concentrated to a spread of IP addresses across a /22 network subnet.

This particular server was a smaller virtual machine, as the traffic continued to increase it started to cause CPU/RAM resource exhaustion on the server which resulted in impact to other users of the service.

# Rate limiting via UFW

I use UFW to manage rules to only allow traffic in/out from my servers that I want. UFW does have the ability to add rules to rate limit connections so I thought I would give it a try, it was as simple as:

sudo ufw limit from 95.x.x.0/22 to any port https

After implementing this I found that the default limit rule for UFW is 6 connections over a 30 second window, given I was applying this rule to a network subnet it was far too restrictive.

# Modifying the UFW default rate limit

I tried editing the full firewall rules as defined in /etc/ufw/user.rules to something more along the lines of what I wanted, however this appeared to be overwritten with the defaults when issuing a reload of UFW.

Given I wasn't using the UFW rate limit on this machine for anything else, I decided the best quick fix was to edit the script that builds the iptables rules for UFW.

UFW limit change

I changed the values for seconds and hitcount to something more appropriate for what I needed and issued sudo ufw reload, this rebuilt and applied the rules with the updated default values.

As the rate limit is tracking new connections (not packets per second or DNS queries), I had to iterate a few times to find the right set of values.

The desired outcome was to limit traffic enough to manage the server load at a point where the traffic was no longer impacting the server resources and other users.


© JB - If you don't already have my contact, you can reach me via email.