Porting ProL2TP to a new platform: Part 2 - Kernel Modules

Wed 07 August 2013
By Chris Elston

Tux jigsaw puzzle with a missing piece

In the first of this series we covered cross compiling ProL2TP for the Ubiquiti EdgeRouter LITE. In order to have a fully functioning L2TPv3 capable system we'll also need kernel driver support for the L2TPv3 drivers.

These drivers have been available in the mainline kernel since 2.6.35, but EdgeOS v1.0.2 is based on so we'll need to backport the drivers to this version. This article will cover the process of setting up a kernel build environment and compiling kernel modules for use with EdgeOS.

If you're not interested in the process of building the drivers, and you just want to get L2TPv3 driver support up and running for your EdgeRouter, you can skip ahead to Installing the Drivers.



To build ProL2TP we were able to use the Emdebian MIPS toolchain. The specific compiler version used was not critical because the kernel ABI sits between our userspace software and the kernel. This is not the case when you're building kernel modules, which will be loaded into the kernel itself. It's important that compiler and binary utils match the ones originally used to compile the kernel - if not you risk inducing critical kernel errors. So I headed on over to the Cavium Forums (requires registration) and downloaded the 2.0.0 version of their OCTEON SDK - which has the tools appropriate for use with EdgeOS v1.0.2.


The kernel sources used by Ubiquiti are published as part of their GPL release (the kernel source archive kernel_4539683-g7b3312f.tgz is inside the archive GPL.ER-e100.v1.1.0.4543695.tbz2). We'll need the kernel source in order to do an out-of-tree kernel module build.

The L2TPv3 drivers were first merged in kernel 2.6.35, and the EdgeRouter is running There have been several updates to the drivers since their original submission. Internally we maintain support for kernels as old as 2.6.32, and it's this version of the L2TP drivers that I'll be using.

Building the Drivers

First we unpack the sources:

$ tar zxf cnusers_sdk_2.0.0.tgz
$ tar zxf kernel_4539683-g7b3312f.tgz
$ tar zxf l2tp-drivers.tar.gz

Which will leave us with three directories at the top level: OCTEON-SDK, l2tp-drivers and kernel. We'll need the cross versions of the build tools in our PATH:

$ export PATH=`readlink -f tools/bin/`:$PATH
$ cd ..

Which we can confirm by checking gcc's version:

$ mips64-octeon-linux-gnu-gcc --version
mips64-octeon-linux-gnu-gcc (Cavium Networks Version: 2_0_0 build 95) 4.3.3
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO

Now we can configure the kernel and prepare it for building external modules:

$ cd kernel
$ make silentoldconfig ARCH=mips
$ make prepare scripts ARCH=mips

At this point we may then build the drivers:

$ export L2TP_KCONFIG="CONFIG_L2TP=m \
$ export L2TP_KCONFIG_DEFINES=`echo $L2TP_KCONFIG | sed 's/CONFIG_/-DCONFIG_/g'`
$ make M=../l2tp-drivers/src EXTRA_CFLAGS="-DL2TP_EXTERNAL -I../l2tp-drivers/src/include \

There's a fair bit going on here, so it warrants some explanation:

  • The source directory for the external kernel modules is passed to kbuild (the kernel build system) with the M= parameter.

  • The kernel configuration settings we need to enable building of the L2TP drivers aren't present in the Ubiquiti kernel config. So here we're passing them in as both make environment variables (stored temporarily in the variable L2TP_KCONFIG), and we're also passing them as defines via EXTRA_CFLAGS (stored in the variable L2TP_KCONFIG_DEFINES, and generated from the contents of L2TP_KCONFIG).

And finally, we install the drivers to a temporary location from which we make an archive to transfer to the EdgeRouter.

$ make M=../l2tp-drivers/src INSTALL_MOD_PATH=../ $KCONFIG modules_install
$ cd ..
$ find lib
$ tar czvpf lib/

Installing the Drivers

EdgeRouter users can download the compiled driver archive here (md5sum)

To install the drivers, copy the archive to the EdgeRouter then extract it and run depmod:

ubnt@ubnt:~$ cd /
ubnt@ubnt:/$ sudo tar xvpf /home/ubnt/
ubnt@ubnt:/$ sudo depmod

We can now load the drivers in the usual way:

ubnt@ubnt:/$ sudo modprobe l2tp_eth                
ubnt@ubnt:/$ sudo modprobe l2tp_ppp
ubnt@ubnt:/$ sudo modprobe l2tp_ip
ubnt@ubnt:/$ sudo lsmod | grep l2tp
l2tp_ip                 6528  0
l2tp_ppp               16112  0
l2tp_eth                4320  0
l2tp_netlink            8864  2 l2tp_ppp,l2tp_eth
l2tp_core              17792  4 l2tp_ip,l2tp_ppp,l2tp_eth,l2tp_netlink
pppox                   1696  1 l2tp_ppp
ppp_generic            20656  2 l2tp_ppp,pppox
ipv6                  328048  22 l2tp_core,cvm_ipsec_kame

Using the Drivers

There are two options for setting up L2TPv3 tunnels and sessions, you can either get a trial version of ProL2TP, or you can use the 'ip' utility from the iproute package to set up unmanaged tunnels.

The version of 'ip' from the iproute package in EdgeOS v1.0.2 lacks support L2TP. So I've produced a package iproute-l2tp (md5sum) which replaces the ip utility with one which has L2TP support. It saves the original version, which will be replaced when you uninstall iproute-l2tp should you need to revert it. Because it replaces the file of another package, you'll need to specify '--force-overwrite' on install:

ubnt@ubnt:/$ sudo dpkg -i --force-overwrite /home/ubnt/iproute-l2tp_20100519-3_mips.deb
Selecting previously deselected package iproute-l2tp.
(Reading database ... 24230 files and directories currently installed.)
Unpacking iproute-l2tp (from .../iproute-l2tp_20100519-3_mips.deb) ...
Setting up iproute-l2tp (20100519-3) ...
ubnt@ubnt:/$ ip l2tp
Usage: ip l2tp add tunnel
          remote ADDR local ADDR
          tunnel_id ID peer_tunnel_id ID
          [ encap { ip | udp } ]
          [ udp_sport PORT ] [ udp_dport PORT ]
Usage: ip l2tp add session
          tunnel_id ID
          session_id ID peer_session_id ID
          [ cookie HEXSTR ] [ peer_cookie HEXSTR ]
          [ offset OFFSET ] [ peer_offset OFFSET ]
       ip l2tp del tunnel tunnel_id ID
       ip l2tp del session tunnel_id ID session_id ID
       ip l2tp show tunnel [ tunnel_id ID ]
       ip l2tp show session [ tunnel_id ID ] [ session_id ID ]

Where: NAME   := string
       ADDR   := { IP_ADDRESS | any }
       PORT   := { 0..65535 }
       ID     := { 1..4294967295 }
       HEXSTR := { 8 or 16 hex digits (4 / 8 bytes) }

Next Steps

So now we have ProL2TP installed on our target platform, and we have a kernel which supports L2TPv3. All that remains is to put them through their paces to ensure that it's all operating as expected. Because ProL2TP is very flexible it has many configuration options, to test all of them manually for each new platform or release would take a long time. So in parallel with the development of ProL2TP, we also maintain a stress and regression testing harness.

In the next article of this series, I'll cover some of the interesting details of how the harness operates.