Re: Home stretch on new network - if_bridge looking better

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:On 02/24/11 11:50, Matthew Dillon wrote:
:
:> 	http://apollo-vc.backplane.com/DFlyMisc/bridge1.txt
:> 	http://apollo-vc.backplane.com/DFlyMisc/bridge2.txt
:
:So - reading over this - is it correct that the setup is roughly like:
:
:- assign a local interface (lan0) to a network
:- add this network to the bridge
:- create openvpn 'bridged' mode tunnels
:- add these to the bridge

    In the case of my current setup, lan0, uverse0, comcast0, and
    aerio0 are all physical ethernet ports.  lan0 is the LAN, and
    the other three connect to the three different WAN services I
    have.

    Only lan0 and the tunnels (tap0, tap1, tap2) are associated with
    the bridge.

    The other physical ethernet ports (uverse0, comcast0, and aerio0)
    each have a different IP and a different default route and I use
    IPFW to associate packets sourced from the IP to the default route
    for each port.  Currently uverse0 and comcast0 are both dynamic
    while aerio0 is a static IP (the old DragonFly net /26).

    The OpenVPN tunnels are built using these IPs and back the tap devices.
    The tap devices are then associated with the bridge and the main LAN.
    The tap devices themselves, and the bridge, have *NO* IPs associated
    with them.  All the local IP spaces are on lan0, including some local
    NATted spaces (10.x.x.x).  The bridge code and the ARP code deal with
    the inconsistencies and provide a consistent ARP for the bridge members.

    Also, not shown here, is that I have a massive set of PF rules and ALTQs
    on each of the TAP interfaces (tap0, tap1, and tap2). In particular I'm
    running the ALTQs on the TAP devices with fair-share scheduling and
    tuned to the bandwidth of each WAN so ping times will be low no matter
    what topology the bridge is using.  (Of course I can't do fair-share
    scheduling on the WAN ports, uverse0, comcast0, and aerio0, because
    the only thing running over them is the OpenVPN UDP packets and it
    can't dig into them to see what they represent).

:so the L2 bridge / STP will 'map' according to the state of
:the ethernet bridging, which in turn relates to the openvpn tunnel
:state?

    Exactly.  The if_bridge module does its own 'pinging' using STP
    config packets so it can detect when a link goes down.  OpenVPN
    itself also has a ping/restart feature.  I use both.  OpenVPNs internal
    keepalive auto-restarts openvpn on failure, and the if_bridge's
    pinging is used to detect actual good flow across the link and controls
    the failover.

:Without diverging any security sensitive whatnot,
:Is the VPN tunnel created to the ISP or to say, the colo space?
:(I'd assume the latter)

    Yes, a colo space that the DragonFly project controls, provided by
    Peter Avalos.  OpenVPN itself is running encrypted UDP packets.
    Very easy to set up.  The colo has around 10 MBytes/sec of bandwidth
    which is plenty for our project.

:Have been working on my own openvpn (routing mode) fun to a pair
:of VPS's as well over the last few days so this is of interest :D
:
:also - I note in the "bridge2.txt" file you 'cd /usr/pkg/etc/openvpn'
:before running - is this so openvpn can find the config files?

    Yes, that's actually a bit broken.  I've since changed it to put a
    'cd' directive in the config file itself and then just run openvpn
    with the full path to the config file.  Openvpn has problems restarting
    itself if you don't do this (it winds up getting confused and not being
    able to find the key files if it restarts).

:if so - to note, you can add a 'cd /path/to/configdir' within the
:config files..

    Yah, found that :-)

:also - assuming you have statics on both end of the tunnels -
:why did you choose openvpn ethernet bridging over say IP layer + ipsec?
:(or even openvpn 'routing' mode) with something like OSPF or similar
:
:and - do you have hw crypto cards on either endpoint?

    I originally attempted to route a subnet but the problem is we have
    a full class C at the colo, but DragonFly isn't really designed to
    operate with two different subnets where one subnet overlaps the other.
    Ethernet switching turned out to be the better solution.  The colocated
    box itself is ON the class C, it doesn't have a separate IP outside
    the class C space.  So there was no easy way to swing a routed network.

    I wouldn't even consider something as complex as OSPF for a simple
    setup like this, even with a routed solution.

:(my soekris 486 gets a little bogged down by the crypto, which is why I ask)
:
:ok enough questions ;)
:
:its definitely fun trying to convert consumer internet into a 'real 
:connection' :D
:
:- Chris
:
:(from a gigabit LAN piggybacked on a sometimes 56k wifi link)

    OpenVPN has options to run in the clear after authentication is complete,
    I think, but I highly recommend using the crypto TLS support.  The
    instructions on setting up all the files are pretty clear (you can find
    them on OpenVPN's web site).

    The problem with running in the clear even after using crypto
    authentication is that you really have to use UDP piggybacking if you
    want the VPN to be reliable.  Using TCP for the VPN is a huge mistake.
    With UDP it is very easy for outside entities to inject data or otherwise
    mess up your VPN if it isn't 100% encrypted.  Encryption also prevents
    outside entities from doing deep-packet inspection of your VPN, and
    believe me they probably would if it weren't encrypted (though perhaps
    the deep packet inspection engines don't yet understand OpenVPN's
    protocol :-)).

    CPU-wise the encryption does take a bit more horsepower, a 486 equivalent
    won't be so happy, but any desktop built in the last 10 years ought to
    have plenty of horsepower for a few megabytes a second worth of link
    at least.

    I've also found that VPNing to a colo space is far more reliable then
    trying to run through a vendor-supplied static IP block.  The U-Verse
    static IP block service is a piece of crap.  It's unusable and it can't
    handle a lot of concurrent connections due to the stateful firewall
    it imposes even on the static IP block.  I don't know much about
    ComCast's static IP service but I suspect similar issues, particularly
    since ComCast is trying to work around net neutrality.  I just don't
    want them to be able to classify my traffic.  The VPN is the best
    solution for that.

					-Matt
					Matthew Dillon 
					<dillon@backplane.com>