Porting ProL2TP to a new platform: Part 1 - Cross Compiling ProL2TP

Wed 31 July 2013
By Chris Elston

Ubiquiti EdgeRouter LITE

I've recently been working on a port of ProL2TP to the EdgeRouter LITE, which is a Linux based router from Ubiquiti Networks. Over the course of my next few posts, I'm going to take you through the process we use when bringing ProL2TP to a new platform, using the EdgeRouter as a case study.

Porting ProL2TP to a new platform usually requires three tasks:

  1. Building ProL2TP for the target system
  2. Building L2TPv3 kernel modules for the target system
  3. ProL2TP qualification using our test harness

In this first article of the series, I'm going to cover the steps taken to build ProL2TP packages for the Ubiquiti EdgeRouter LITE. I've outlined the high-level view first. Those wishing to compile their own third-party packages for the EdgeRouter can read on for the gory technical details.

Overview

The EdgeRouter runs EdgeOS, which is a Debian Squeeze (6.0.6) based platform produced by Ubiquiti. Its CPU is a Cavium OCTEON Plus, which is a 64 bit MIPS dual-core processor.

Since it is a Debian platform with apt-get based package management, it made sense for us to aim to produce MIPS Debian packages. This would make it easy for customers to add and remove the package. Our autotools build is already capable of producing Debian packages, and a bit of reading around convinced me that using Debian multiarch plus an Emdebian MIPS cross compiler would allow me to cross compile straight to a Debian package.

This presents us with our first issue. Multiarch support was added in Debian Wheezy (7.0), but we're targeting a Squeeze based system. Fortunately for us, apt-get allows us to install packages from earlier repositories, so with a little careful work we can actually build a Debian package on a Wheezy system which depends on the Squeeze version of it's dependencies. Neat, huh?

The final piece of the puzzle is that in order to work, dpkg-buildpackage requires that *-mips-cross versions of dependencies and tools are installed. Handily, the Emdebian folks have released the xapt tool which can produce cross versions of packages from the Debian repositories on-the-fly.

To embedded developers who aren't concerned with targetting a specific package format, all of this might seem like a complicated way to go about things. However it turns out that dpkg-buildpackage on a multiarch system nicely hides most of the complexity of producing cross-compiled binary packages. So that once you've got your build system set up, the process of building is very simple. I found that we only needed minimal changes to our build system to produce working MIPS Debian packages.

Building for The EdgeRouter

Setting up the Build Environment

If you've stuck with me this far, I'll assume that you're interested in producing an EdgeRouter cross-compilation environment for yourself. You'll need to base it on a Debian Wheezy system. Since we support ProL2TP on a wide variety of platforms, we use virtual machines to do the majority of our builds. I'm using a Wheezy VM created using vagrant, meaning I can quickly run through the build environment setup in the future using the Vagrant provisioning scripts.

The first step is to set up access to the Emdebian repositories. We're specifying the Squeeze Emdebian apt source, since we want the Squeeze versions of the Emdebian tools:

$ apt-get -y install emdebian-archive-keyring
$ echo "deb [arch=amd64,i386] http://www.emdebian.org/debian/ squeeze main" > \
/etc/apt/sources.list.d/emdebian.sources.list

Then we add the core Debian Squeeze repositories to our apt sources. My closest Debian mirror is in the UK, you should select your own nearest mirror:

$ echo -e "deb http://ftp.uk.debian.org/debian/ squeeze main\ndeb-src \
http://ftp.uk.debian.org/debian/ squeeze main" >> /etc/apt/sources.list
$ apt-get update

Then on to package installation. We install the Emdebian MIPS toolchain we need, specifying that it should come from the Squeeze Emdebian repository:

$ apt-get -y install -t squeeze linux-libc-dev-mips-cross libc6-mips-cross \
libc6-dev-mips-cross binutils-mips-linux-gnu gcc-4.4-mips-linux-gnu g++-4.4-mips-linux-gnu

Next up are the native build-time dependencies required by your build, these come from the default (Wheezy) repository. Note that build-essential:native, debhelper, and dpatch are probably common to all builds, whereas flex and bison are specific build-time dependencies of ProL2TP:

$ apt-get -y install build-essential:native debhelper dpatch flex bison

I found that xapt required a small modification so that I could point it at the Squeeze repositories to fetch the packages for converting to cross packages

$ apt-get -y install xapt
$ cat xapt-force-squeeze-repo.patch
Index: /usr/sbin/xapt
===================================================================
--- .orig/usr/sbin/xapt 2013-07-02 12:33:26.000000000 +0000
+++ /usr/sbin/xapt  2013-07-02 12:35:13.000000000 +0000
@@ -181,7 +181,7 @@
}
$host=`dpkg-architecture -qDEB_HOST_ARCH`;
chomp ($host);
-$config_str = '';
+$config_str = '-t squeeze';
$config_str .= " -o Apt::Get::Download-Only=true";
if (($arch ne $host) or (defined $ignore_status)) {
    $dpkgdir = "${dir}/${arch}/dpkg/";
$ patch -p0 < xapt-force-squeeze-repo.patch

This change will send xapt to the Squeeze repositories every time you use it. I don't mind doing this since my system is a single-use VM. If you're following this procedure on a machine which you need to use for other purposes you should consider implementing a less restrictive way to pass the target respository to xapt.

This next command tells dpkg that we want to be able to install packages for MIPS without having to specify --force-architecture:

$ dpkg --add-architecture mips

And finally we use xapt to install the cross versions of build dependencies. The build-essential package installs the toolchain and related utilities, while libnl2-dev and libssl-dev are ProL2TP dependencies:

$ xapt -a mips build-essential libnl2-dev libssl-dev

Building ProL2TP

Having set up a build environment, the next step was to try building with dpkg-buildpackage. This revealed a couple of problems:

  • Our build system only handled native Debian package builds.
  • When cross compiling, dpkg-buildpackage looks for native versions of build-time dependencies which aren't present.

These were pretty easy to solve. I hooked up the standard configure script '--host=' argument to the dpkg-buildpackage '-t' parameter, and passed '-d' to dpkg-buildpackage to tell it to skip dependency checks when cross-compiling. For those building their own Debian packages, the interesting bit is the dpkg-buildpackage commandline we end up calling:

$ dpkg-buildpackage -tmips-linux-gnu -b -d

Success!

This article has detailed the process I took to create a Debian build system to compile cross packages for the EdgeRouter. At this point, we have a working ProL2TP package which can be installed on the EdgeRouter using dpkg.

But we're not quite home free yet, because the ProL2TP userspace package is only half of the story. In order to get us up and tunnelling, we'll need the kernel drivers - which will be covered in the second post in this series.