Protecting your Rails application with fail2ban

The less sophisticated attacks

One of the characteristics of the more naive attacks are that they are usually started with a bulk scan of your server. This less sophisticated attackers don’t even bother fine-tuning their scanners either which results in lots of weird requests hitting your Rails app (e.g. for .aspx or .jsp pages). One of the very first things you do when putting an app out there is to have some sort of exception notification plugin that reports application-level error to either a central console or via email.

Around 02:00am on the 1st of January (a great way to start the new year) one of our servers was targeted. Nothing was broken, but thousands of exception notifications were received which is both time consuming, and unnecessary. We could have detected the request flood and prevented it further attempts, but we didn’t have the infrastructure in place. Today I decided doing something about it.

Fail2ban, exception_notification and Rails

I regularly use fail2ban to detect http 404 floods and protect the SSH service, but up until now I never linked it with an application-layer detection mechanism.

exception_notification is a nifty gem that you can use to create a chain of “notifiers” that will send exception notifications to different services.

I created a simple notifier that takes each exception and generates and dumps it in a new log file that you can easily parse with fail2ban, the meat of the code is below:

[warning, safest to use this gist for copy/pasting]

            def call(exception, options={})
            env = options[:env]
            request = ActionDispatch::Request.new(env)

            # {ip} : {exception class} : {method} {path} -- {params}
            msg = "%s : %s : %s %s -- %s" % [
            request.remote_ip,
            exception.class,
            request.request_method,
            env["PATH_INFO"],
            request.filtered_parameters.inspect
            ]
            @logger.error(msg)
            end
          

This defines the fail2ban jail (add to /etc/fail2ban/jail.local):

            # Custom Rails app jail. Add to /etc/fail2ban/jail.local
            [rails-app]
            enabled = true
            port = http,https
            filter = rails-app
            logpath = /path/to/app/log/fail2ban.log
            bantime = 3600
            findtime = 600
            maxretry = 10
          

And this defines the filter (add to /etc/fail2ban/filters.d/rails-app.conf):

            # Custom Rails app filter. Place in /etc/fail2ban/filter.d/
            [Definition]
            failregex = :  :
            ignoreregex =
          

Finally, configure the new notifier in your config/environments/production.rb file:

            config.middleware.use ExceptionNotification::Rack,
            :email => { ... },
            :fail2ban => {}
          

I’ve already submitted a pull request for this, so hopefully the next release of the gem will have the fail2ban notifier by default.

References

• exception_notification custom notifiers
• http://www.fail2ban.org/wiki/index.php/MANUAL_0_8