An AWS VPN: Part III, Routing

The last two posts in this series (part I, IPSec and part II, L2TP) covered enough to get an L2TP/IPSec VPN connection up, to the point where arbitrary traffic can be exchanged between the client and the server.  But, there's a missing feature yet.  Remember this picture from part II?

If we assigned a third network for the VPN, how does the client know that the protected network is even back there to send traffic to?  It's not the remote end of the VPN link (172.16.0.1) so it'll get routed via the gateway and fail at some point.

The answer is that someone conveniently threw in an extra hack for us: the VPN client sends a DHCPINFORM message over the L2TP connection, and the server just has to respond to it with a few vital options.

An AWS VPN: Part II, L2TP/PPP

In part I of this series, we covered the IPSec layer.  With IPSec up and running, we can move on to configuring L2TP; routing will be covered in part III.

In the "usual" VPN setup, we'd cut a hole in the existing DHCP pool in the protected network to reserve as addresses to be given to VPN clients.  In a /24 subnet, network gear might get addresses up to 30, the DHCP pool could run from 31-150, and other static IPs (hi, printers!) might live at 200-254, with 151-199 open for VPN access.

Unfortunately, my protected network in this scenario is AWS, which I can't actually carve random IPs out of to assign to connecting clients.  I have to assign one in a different netblock and hope none of the three (the client, AWS, and my new L2TP-only address pool) conflict.

Luckily, there are three private network spaces defined, and everyone always forgets about the weird one.  Let's set up the connection something like this:

Of course I didn't actually use 172.16.0.x myself, that's asking for almost as much trouble as 192.168.0.x.  (Also, I guess I could have colored the NAT-to-server link green as well, because that's the same network space in my AWS setup.  Too late, we're going!)

An AWS VPN: Part I, IPSec

I recently set up an IPSec/L2TP VPN.  This post is about how I did it, and how I debugged each individual part.  In honor of how long it took me, this will be a three-part series, one for each day of work I spent on it.  Part II will cover L2TP/PPP, and part III will get into routing and DNS.

First things first: IP addresses have been changed.  Some details will apply to Linux, OpenSWAN, xl2tpd, pppd, iproute2, dnsmasq, or road-warrior configurations in particular, but the theory is applicable cross-platform.  I want to cover both, because the theory is invaluable for debugging the practice.

This also covers a few details about putting the VPN server on an AWS instance in EC2 Classic with an elastic IP.  But first, let's take a look at the general structure we have to work with:

A client connecting from a network we don't control, over networks that are possibly evil (hence the need for a VPN), to a server that provides access to vague "other resources" in non-globally-routable space behind that server.  We don't know where the client is going to be, and/or the corporate net is sufficiently large, that we can't use dirty NAT tricks to move each network "into" the other.  (But, spoiler, we will take some inspiration from there.)

Without further ado, here's part I: IPSec.  Part II and III are forthcoming.

VPN Terminology: PPTP and L2TP/IPSEC

A little reminder before I get started:

Do not use PPTP

Seriously, do not use it. It’s insecure and it occasionally has terrible performance due to its underlying design. Don’t use it.

Just don’t.

With that aside…

Remembering PPP

Back in the days of dialup, people used physical circuits like the telephone network to connect to the Internet. On that last hop between the home and the ISP, the protocol that ferried IP directly was PPP, the Point-to-Point Protocol.

PPP was designed for an ISP to serve dial-in lines, so it had a bit more to worry about than just ferrying IP packets. It had to assign addresses between the endpoints (IPCP) as well as ensure that Internet service was being given to a paying user by authenticating them (PAP or CHAP).

In those simpler times, one physical machine at the ISP’s end would connect to both the Internet and the telephone network.

Some people, though, needed so much internet that they wanted to dial in with multiple lines, and have packets routed down both of them. This was known as “multilink PPP” and it had a problem. What if the ISP doesn’t have just one machine handling dial-up connections?

It could be the case that a client would attempt to set up a multilink PPP session, but their calls were on different machines, and The Internet couldn’t really route to both of them. So, faced with different PPP servers, the client had little choice but to hang up, dial in again, and hope that this time, they hit the same servers.

Tunneling

PPTP was largely motivated by the desire to solve this problem, by splitting Big Machines into two parts: one (or more) to handle dial-in lines, maintaining connections to one (or more?) machines which were actually on the Internet.

Handling calls from the clients were the PPTP Access Concentrators, or PACs, which would then deliver traffic to the PPTP Network Server, or PNS. The “tunnel” between these two parts was a single channel/circuit (I presume) that carried the PPTP control traffic and tunneled packets between PAC and PNS for de/multiplexing.

In this scenario, the client-to-PAC connection is still regular PPP; a portion of the PPP exchange (setting up multilink) is forwarded to the PNS, who can then see all client links, even across PACs. Thus, it can route effectively. From the point of view of the Internet, it really is routing all the traffic to one machine, the PNS.

The PNS must also handle the IPCP portion of PPP setup, since it’s the server with IP connectivity.

Much of the terminology in the PPTP RFC is defined with reference to this situation, and says nothing about VPNs themselves, because the protocol itself was not originally defined as a VPN protocol.

VPN puts a network in your networks

However, if the tunnel is encrypted then we can convert it into a VPN technology by moving the tunnel off ISP equipment and onto the public Internet.

In this case, the VPN client is playing the role of PAC and the server plays PNS again; the connection between them is a TCP channel over which all network packets are forwarded. Like regular PPP, there are control messages, and also data packets riding on the single established channel.

This is where performance trouble turns out to be lurking: if a TCP connection is forwarded, it gets forwarded reliably. It’s never going to be lost on the journey over the PPTP VPN. That inner TCP connection can’t detect the path MTU by packet loss, and this does not play well with systems that self-control their network rates based on packet loss.

(Ironically, recent bufferbloat mitigations that try to measure RTT and slow down when RTT increases are much better suited to this scenario.)

Layer 2 Tunneling (L2TP)

Someone decided to fix that, so an L2TP connection is carried over UDP packets instead. They implement their own reliability protocol for control messages, and rely on the tunnelled connections to handle any packet loss. That fixes the performance issues facing PPTP.

L2TP is also simplified a bit from PPTP, in that it doesn’t offer any encryption of its own. That’s why it has to be run over IPSEC to form a VPN; but otherwise, it’s conceptually near identical to PPTP.

(Of course, all those “P” abbreviations also become “L,” but actually crossing them all out cluttered the image too much.)

With encryption provided by the IPSEC layer, any weakness of L2TP becomes irrelevant. But it seems like a horrible hack, because IPSEC isn’t concerned with hosts; it’s concerned with networks and routing.

Like PPTP, the L2TP RFC is not that worried about defining all this stuff in terms of how an L2TP VPN would be set up in practice.  Rather, it reads like a minimal number of changes were made to the PPTP RFC in order to make the required implementation changes obvious to people already writing PPTP VPN software.

IP Security (IPSEC)

IPSEC itself establishes an encrypted channel with integrity checking; it’s just anonymous.  L2TP provides a point to get a specific username/password pair after the IPSEC key exchange (IKE) is done.

If you’re using a pre-shared key (PSK), this oddly means that you need two passwords to connect up the VPN: one site-wide password to establish the IPSEC channel, and a separate per-user password to set up the L2TP tunnel inside.

Sediment

There you have it: VPNs today are generally complicated and awful because people have been adapting other tools to the job, and the layers have just been accumulating.

It should be noted that, as long as this post is, it does not cover other approaches to VPNs: it remains laser-focused on L2TP. OpenVPN and sshuttle are out of scope.

Quickie: Diffie-Hellman Groups

Relying on others' suggested magic numbers for crypto is probably a Bad Idea, so recently I studied Diffie-Hellman a while to understand what the "DH Group" parameter was in my IPSEC setup, and my PuTTY settings.

DH turns out to be a lot like RSA, so bit lengths are comparable between the two and neither is directly comparable to symmetric ciphers like AES.  A specific Diffie-Hellman exchange happens using some parameters: a generator for the base, and a prime to use as modulus.  (An exponent remains secret.)  DH Groups refer to specific, pre-chosen prime-and-generator pairs so that, for example, SSH can negotiate "group 14" instead of transferring the complete parameters themselves.

These groups have been standardized in RFC 2409, with additional groups defined in RFC 3526.  The latter RFC defines the bit lengths of the groups explicitly, stating that group 5 is 1536 bits, group 14 is 2048, and group 16 is 4096 bits.  As far as I can tell, groups 1 and 2 defined in the earlier RFC are only 768 and 1024 bits, respectively.

Note well: I believe this means DH groups 1 and 2 are dangerously short and should not be used to set up an IPSEC VPN today.  Likewise, PuTTY should really be configured out-of-the-box to warn about the use of anything less than DH group 14.  However, before I take my own advice, I need to do some experiments to determine whether the IPSEC client in iOS actually handles DH groups other than 2.  Edit from THE FUTURE: iOS 4.x does not accept other groups; iOS 5.x no longer accepts group 2, AFAICT.  I haven't gotten a working IPSEC VPN set up again, though, since it's not very important to me.  (Work provides a PPTP VPN.)