How To Firewall, Failover, Load Balance, And Throttle Multiple Network Interfaces In Linux

There is a plethora of documentation on how to configure individual aspects of Linux boxes used as gateways; however, what is needed is a single document that comprehensively details ALL aspects of a gateway's configuration, particularly when multiple public and private network interface cards (NICs) are employed. The aspects to which I'm referring are:

Combining the aspects is crucial for a Linux router and is why I wrote fifalobath and its associated scripts.

firewall + failover + load balance + throttle == fifalobath

Each of the aspects requires multiple degrees of configuration:

In the simplest case, that of a single public interface, fifalobath creates a highly customizable and easy to use firewall. In the most complex case, that of multiple public and private interfaces, fifalobath handles all the aspects and their degrees of configuration in a modular way. The modularity keeps things simple.

If you've already setup fifalobath and are having trouble then go to the Troubleshooting section.

Example

The following ASCII art represents a Linux router that has two internal (private) interfaces, eth0 and wlan0, in addition to two independent Internet service providers (ISPs) on eth1 and ppp0, respectively. It is this configuration that is used as an example.

                                                                      ________
        __                                     +------------+        /
    ___/  \_                                   |            |       |
  _/        \__                   +------------+ Provider 1 +-------
 /             \                  |            |            |     /
| Local network ----      +-------+--------+   +------------+    |
 \_           __/    \    |       eth1     |                    /
   \__     __/        ----+ eth0           |                   |
      \___/               |                |                   |
        __                |   Linux router |                   |     Internet
    ___/  \_              |                |                   |
  _/        \__       ----+ wlan0          |                   |
 /             \     /    |       ppp0     |                    \
| Local network ----      +-------+--------+   +------------+    |
 \_           __/                 |            |            |     \
   \__     __/                    +------------+ Provider 2 +-------
      \___/                                    |            |       |
                                               +------------+        \________


I'm most familiar with Fedora, so that's the context for the scripts. They can be applied to other distributions (Ubuntu, Suse, Gentoo, etc) by making appropriate file location/NIC configuration substitutions. Centos and RHEL should be good to go right out of the box. You need to be root to do all that is outlined in this document. I'll start with the basic configuration options and then move on to advanced options.

Setup

As root you can copy-and-paste these commands into a shell in order to complete the setup. Each command is preceded by a comment about what it does; you can cut-and-paste the comments - they will be ignored by the shell. For commands in "optional" stanzas you can either prefix them with "#" or not cut-and-paste them if you don't want their functionality.

## set the location of the fifalobath script
export FIFALOBATH=-scripts/fifalobath

## get it
 > "$FIFALOBATH"

## make it executable
chmod a+x "$FIFALOBATH"

## add fifalobath file locations to 
cat <<EOS >> 
# http://www.avaritia.com/linux_network_firewall_failover_load_balance_throttle
# firewall, failover, load balance, throttle: fifolobath
FIFALOBATH="$FIFALOBATH"
FIFALOBATH_DYN=/var/log/dynamic_nic_info.log # dynamic interface info log
EOS

## add NIC info to ; eg, for the ASCII art above
cat <<EOS >> 
EXTIFs="eth1 ppp0"  # external (public) interfaces; primary first
INTIFs="eth0 wlan0" # internal (private) interfaces
EOS

## optional customizations to ; comment them out if you don't want them
cat <<EOS >> 
RESOLV_SEARCH="search domain_with_dns_on_localhost.com some_other_domain.com"
RESOLV_DNS="nameserver 127.0.0.1\nnameserver 1.2.3.4" # \n delimited additions to DNSs
EOS

## optional logging of packets dropped by the firewall
cat <<EOS >> 
LOG_DROPPED=1       # log dropped packets
EOS

## optional common firewall options
cat <<EOS >> 
ALLOW_PING=1        # comment out to disallow pings on EXTIFs
ALLOW_SSH=1         # comment out to disallow ssh on EXTIFs - HIGHLY recommended to keep
ALLOW_DHCPD=1       # comment out ot disallow dhcpd request on INTIFs
EOS

## optional failover timeout
cat <<EOS >> 
ROUTE_TIMEOUT=3     # failover timeout in seconds
EOS

## optional flag for load balancing
cat <<EOS >> 
LOAD_BALANCE=1      # comment out for failover only, no load balancing
EOS

## throttling variables; if THROTTLE is set then ALLOT_REALTIME and ALLOT_P2P are mandatory
cat <<EOS >> 
THROTTLE=0.9        # maximum fraction of each external interface's UPRATE available to users
ALLOT_REALTIME=0.15 # dedicated fraction of bandwidth for skype, rtp, etc
ALLOT_P2P=0.05      # dedicated fraction of bandwidth for p2p
EOS

Alternatively, after you've obtained fifalobath, here's an example of that can be edited in place and then cut-and-paste into your .

If none of the optional configuration variables are set then the basic setup is a simple firewall that does network address translation (NAT) and forwards packets between internal interfaces. NAT allows machines on the local nets to access the Internet. The forwarding, for example, allows machines on the eth0 subnet to browse samba shares on the wlan0 subnet and vice versa - nice!

For the ASCII art example, we need to specify the load balancing weights and upload rates for eth1 and ppp0. Assuming eth1 gets 20Mbps down and 1Mbps up and ppp0 gets 3Mbps down and 512kbps up:

## specify eth1's load balancing weight and upload rate
cat <<EOS >> -scripts/ifcfg-eth1
WEIGHT=20
UPRATE=1mbps
EOS

## specify ppp0's load balancing weight and upload rate
cat <<EOS >> -scripts/ifcfg-ppp0
WEIGHT=3
UPRATE=512kbps
EOS

Now it's time to do make sure that all the ancillary stuff upon which fifalobath relies is available:

## check that the necessary load balancing/throttling variables are set
$FIFALOBATH ncheck # skip this if you just want a firewall

## check that the necessary entries are in /etc/ip_route2/rt_tables
$FIFALOBATH tcheck # skip this if you just want a firewall

## check that the necessary executables are available
$FIFALOBATH xcheck

## all together now
$FIFALOBATH ncheck && $FIFALOBATH tcheck && $FIFALOBATH xcheck

If you saw "all good" across the board then you're all good. Otherwise, follow the directions provided by the output to correct the problems. Re-run the checks until you get all the "all good"s.

If your Linux router is assigned IP addresses dynamically then you'll also need /etc/sysconfig/network-scripts/dhclient-exit-hooks:

## grab dhclient-exit-hooks
 > -scripts/dhclient-exit-hooks

So, in Fedora, the first thing to do is because NetworkManager sucks at handling sophisticated network configurations.

Next, since fifalobath will manage the firewall and uses it uses rsyslog for logging

service iptables stop && chkconfig iptables off

Network configuration information that fifalobath reads needs to be added to /etc/sysconfig/network like this:

- a lot of rules are standard; most rules are boilerplate; custum rules are custom
- rsyslog before network
- last step is cd /etc/dhcp/ && ln -s dh..exit* ; ( cd /sbin && ln -s fifalobath ifup-local && ln -s fifa ifdown-local )

Troubleshooting

If something goes wrong try

## "reset" the firewall and routing
-scripts/fifalobath bootstrap

Cutting-and-pasting the above commands into a shell will flush all the firewall and routing rules. If that doesn't work then manually edit $FIFALOBATH_DYN and insert the appropriate values for any interfaces with dynamic IP addresses.

Miscellaneous

Ethernet bonding has amazing failover capabilities but only works for certain NICs - search through the kernel source code to find them - and cannot be used if you use two independent ISPs.

Files & Links

/etc/sysconfig/network-scripts/dhclient-exit-hooks

/etc/sysconfig/network-scripts/fifalobath

/etc/sysconfig/network-scripts/fifalobath.private