05 Sep 2013

In 2013 I had the pleasure of beginning to work with eQualitie on a few projects including their Deflect DDoS mitigation system. While working on analysing large attacks, we came to the realisation that the current method of banning using fail2ban was not performant enough, as adding new rules to the ban process lead to large CPU overhead that marred performance to an unacceptable degree. This is largely due to the special situation that Deflect edges find themselves in, with massive log files that Fail2Ban is unable to reprocess efficiently- adding new patterns that match new bots leads to reapplication of each regex, which leads to massive increases in resource consumption.

I spent some time working on a patch to fix this issue but soon lost motivation as I realised that fail2ban, while being really cool, was way overtooled for what I wanted to do. As Deflect moves into DDeflect, the need for decoupled components that do simple things in an isolated fashion becomes greater. When communicating with tools like the machine-learning banning system Learn2Ban, having a tool that can be fed information from multiple sources in a lightweight way becomes very useful.

So as an alternative to our existing Traffic Server/Fail2Ban system, the team started working on a series of components designed to quickly identify malicious bots, ban them and analyse their behaviour to stop similar attacks from other hosts. This broader project involved many components and as a result of this effort, Swabber was born.

Swabber uses ØMQ to create a publisher/subscriber system where the daemon will subscribe to a ban publisher, ban the IP address and then after a period of time, unban the host. Initially this was accomplished through storing bans in an SQL database as I was still trying to rid myself of my MySQL habit from previous years, but thankfully sense was seen and now the only moving parts outside of the codebase itself is iptables. Ban times are kept as timestamps in iptables comments. There are multiple backend options for banning hosts, with the easiest way being direct shell-out iptables commands, the most “correct” being the use of the python-iptables library and the fastest (but probably least supported unless you’re really into xinetd) being the /etc/hosts.deny file.

Published on 05 Sep 2013