Fallback routing with Linux

Some quick notes to document how I turned a Raspberry Pi into a wireless router. A wireless access point is created and its traffic is forwarded over ethernet if available, falling back to a tethered connection over my iPhone (via Bluetooth) if that's not available and lastly a Huawei E3131 3G dongle as a last resort.

The use-case here is a mobile router, say on a boat. Sometimes you're in port, connected to and routing traffic over the marina wireless network. Or you're at sea within sight of a mobile basestation and want the boat to connect over that. Or the boat is unoccupied with no marina wifi, but you want it to have basic network access (say to transmit a burglar alarm or high-wind alert, or to receive a remote request to power up the heating). Until I'm afloat it will serve that purpose as a burglar alarm in the shed.

Tested on Raspberry Pi hardware but will work on any Linux box (instructions for Debian/Ubuntu).

Step 1. Make Raspberry Pi into a Wireless Access point

1. I am using a generic 802.11g USB dongle. Install the required packages:

             # aptitude install hostapd udhcpd
           

2. Configure /etc/hostapd/hostapd.conf for your network. Not much to do here except set the ssid, hw_mode, wpa and wpa_passphrase parameters to appropriate values
3. Configure /etc/udhcpd.conf to dish out addresses in the correct LAN range (I use 192.168.5, which is not going to conflict with other LANs I use. In particular if you are using an E3131 modem don't use 192.168.1 as this is always used by that device. Also set the interface to wlan0 and the dns, subnet, router values to appropriate values (I use "8.8.8.8 8.8.4.4", "255.255.255.0", "192.168.5.1" respectively. I configured wlan0 in /ect/network/interfaces like so, to ensure that hostapd and udhcpd are started/stopped with the interface:

           allow-hotplug wlan0
           iface wlan0 inet static
                   address 192.168.5.1
                   netmask 255.255.255.0
                   up service hostapd start
                   up service udhcpd start
                   down service udhcpd stop
                   down service hostapd stop
           

4. Ensure DHCPD_ENABLED=no is commented out in /etc/default/udhcpd
5. Enable packet forwarding from wlan0 to eth0, to make the box a simple router for devices that join the Wireless network it's created. Edit /etc/sysctl.conf to uncomment net.ipv4.ip_forward=1, then run the following (as root, of course):

           # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
           # iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT
           # iptables-save > /etc/iptables
           

   And append the following to /etc/rc.local to start the firewall rules on reboot

           iptables-restore < /etc/iptables
           

6. Reboot. You should see your wireless network being advertised, be able to join it, and be able ssh to your router (at 192.168.5.1 in my case), and connect to the wider internet via the Raspberry Pi.

Step 2. Configure the Huawei E3131 dongle

1. Install the sg3_utils (edit 2016: sg3-utils) package, and create /etc/udev/rules.d/40-huawei.rules with the following content

           SUBSYSTEMS=="usb", ATTRS{modalias}=="usb:v12D1p1F01*", SYMLINK+="hwcdrom", RUN+="/usr/bin/sg_raw /dev/hwcdrom 11 06 20 00 00 00 00 00 01 00"
           

   (edit 2016: the device may not be called /dev/hwcdrom in recent releases - try /dev/sg0)
2. Plug in the dongle, it should create a new network device called eth1. Add the following to /etc/network/interfaces to bring the dongle up automatically

           allow-hotplug eth1
           iface eth1 inet dhcp
                   metric 1000
           

   The metric value controls the priority for routing - for me, routing traffic over this device is expensive so I don't want to do this unless no other interfaces are available. The metric of 1000 is higher than any other route metrics in my system, so it's the lowest priority route. If you unplug/replug the dongle and type route -n you should see the table look like this:

           Kernel IP routing table
           Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
           0.0.0.0         192.168.x.1     0.0.0.0         UG    0      0        0 eth0
           0.0.0.0         192.168.1.1     0.0.0.0         UG    1000   0        0 eth1
           192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
           192.168.x.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
           

   Which shows packets will be routed over eth0 by preference rather than eth1. Note the network used by this dongle is always 192.168.1.0/24 - there is no way to change it. If your other networks are on this range, renumber them.
3. Add packet forwarding for this device as well. It turns out this is as simple as just another two identical commands to iptables. The metric will ensure packets are forwarded over the LAN if available, the dongle if not.

           # iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
           # iptables -A FORWARD -i wlan0 -o eth1 -j ACCEPT
           # iptables-save > /etc/iptables
           

Step 3. Configure the iPhone Bluetooth tethering

Routing traffic over a tethered iPhone can be done in one of three ways - over wireless, over Bluetooth, or over USB. Over wireless is relatively simple - just configure the wireless network as any other. Over USB should be simple, but for me (iOS 6.1.3) was impossible as the ipheth driver (and it's associated ipheth-pair tool) did not work, filling my logs with "ipheth_tx_timeout". It doesn't appear to have been updated for years.

On to the Bluetooth stack, which is also currently in a poor state on Linux. Most of the documentation refers to the legacy 3.x release, and much as I dislike using legacy stuff, I could find no way to do this without them. The documentation is non-existent, the pand daemon is marked as deprecated but no replacement is specified, and the obvious tools that might (bluez-test-network and bluez-network) failed with errors. Linux at it's worst, IMHO. But I digress.

1. To add a tethered iPhone as a gateway:

           # aptitude install bluez bluez-compat
           # hcitool scan
               ... MAC address of your iPhone should be listed, assuming it's discoverable
           # Bluetooth-simple-agent hci0 xx:xx:xx:xx:xx:xx
               ... enter your pin to match
           # Bluetooth-test-device trusted xx:xx:xx:xx:xx:xx
               ... device will now be paired without prompting (after reboot) when it found
           

2. This should discover, pair and trust your iPhone device with the Bluetooth device on the Linux box. The next step is the personal-area network, and for that you need the pand daemon from the bluez-compat package. To cause it to pair with any trusted device that offers the network service, add this to the end of your /etc/rc.local:

           pand --search --persist --role PANU
           

   This will cause your Linux box to continually search for and connect to the iPhone you have previously paired - if the connection drops, it will be reestablished. For me, I had to unlock the phone to get a connection, even though the "Personal Hotspot" was enabled, and even then it took a few seconds (< 30s) for the iPhone to report "Personal Hotspot" was in use. Once the hotspot was established, it did not drop when phone went back to sleep.
3. The pand daemon adds a network device bnep0 to the kernel, and from there the setup is identical to the above devices to cause it to act as a router: add the following to /etc/network/interfaces

              allow-hotplug bnep0
              iface bnep0 inet dhcp
                  metric 50
           

   which gives it a lower priority than the ethernet connection, but higher than the 3G dongle. And to get it to route packets received over the wireless access point we created:

           iptables -t nat -A POSTROUTING -o bnep0 -j MASQUERADE
           iptables -A FORWARD -i wlan0 -o bnep0 -j ACCEPT
           iptables-save > /etc/iptables
           

Summary

With all three routes to the internet available (ethernet, iPhone via Bluetooth and 3G dongle) my configuration looks like this:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.2.1     0.0.0.0         UG    0      0        0 eth0
0.0.0.0         172.20.10.1     0.0.0.0         UG    50     0        0 bnep0
0.0.0.0         192.168.1.1     0.0.0.0         UG    1000   0        0 eth1
172.20.10.0     0.0.0.0         255.255.255.240 U     0      0        0 bnep0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.5.0     0.0.0.0         255.255.255.0   U     0      0        0 wlan0

# cat /etc/network/interfaces 
auto lo
iface lo inet loopback

iface eth0 inet dhcp

allow-hotplug wlan0
iface wlan0 inet static
        address 192.168.5.1
        netmask 255.255.255.0
        up service hostapd start
        up service udhcpd start
        down service udhcpd start
        down service hostapd stop

allow-hotplug eth1
iface eth1 inet dhcp
        metric 1000

allow-hotplug bnep0
iface bnep0 inet dhcp
        metric 50

# cat /etc/iptables 
*filter
:INPUT ACCEPT [38:2628]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [22:2200]
-A FORWARD -i wlan0 -o eth0 -j ACCEPT
-A FORWARD -i wlan0 -o bnep0 -j ACCEPT
-A FORWARD -i wlan0 -o eth1 -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [7:224]
:INPUT ACCEPT [7:224]
:OUTPUT ACCEPT [1:228]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o eth0 -j MASQUERADE
-A POSTROUTING -o bnep0 -j MASQUERADE
-A POSTROUTING -o eth1 -j MASQUERADE
COMMIT
    

Notes

1. My RaspberryPi with these devices (E3131, wifi adapter, bluetooth adapter) plugged in via a USB hub tends to hang. It's not the end of the world, but annoying. I usually suspect power draw in this situation, solution is to spend more than £6 on a USB hub.
2. There is no network bridging going on here - there are other guides on the net that cover this, and that will be useful if you want to have your laptop (connected to the Raspberry Pi over wireless) on the same network as your server (connected to the Raspberry Pi over ethernet). That's not my usecase, I'm using this as a server in a remote location with no other internet access so I haven't touched on it.
3. What I haven't found is a way to reliable way to limit internet traffic based on routing cost. For example, I don't want a nightly apt-get update being run when I am connected over 3G dongle. The analogy I am looking for is load-average: there are many scripts that check load average and choose not to run if it's too high. A similar system-wide "network traffic cost" value would be a nice to have, rather than having to hand-modify every cron-job. But if Linux has one I haven't found it.
4. The state of the Bluetooth stack is shocking. Was it created to scratch an itch or serve a purpose? If the latter, the tools badly need documentation, or even implementation - it appears the bluez team are developing an API but with no tools that use it, and have deprecated the only tools in existence that because they used the older API. Of course it's their API and they can do what they want, but as an end user it guarantees a shitty experience.

BeagleBone Notes

I've switched to a BeagleBone for this project, running Ubuntu 13.04, and had to make the following changes to the above.

1. The E3131 modem is configured as a network device on boot, but comes up as "rename3". So rather than the 40-huawei.rules file in /etc/udev/rules.d I had to create the file as 99-huawei.rules and give it the following content:

         SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="58:2c:80:13:XX:XX", ATTR{dev_id}=="0x0", ATTR{type}=="1", NAME="modem0"
         

   I also had to change the /etc/network/interfaces - allow_hotplug wasn't working (perhaps due to borked USB hotplug support in Beaglebone kernel 3.8.13-bone28?). Now looks like:

         allow-hotplug modem0
         allow-auto modem0
         iface modem0 inet dhcp
             metric 1000