Xenomai on Ubuntu 12.04 (LTS)

posted 2014.09.27

  • xenomai
  • realtime
  • tutorial

Introduction

This is a tutorial on building an x86 (32- or 64-bit) Linux kernel with the Xenomai patches on Ubuntu Linux. The end result is a debian package which, when installed, properly registers itself with the Ubuntu package management system so that you don’t clutter up your machine and can quickly iterate if you need to try different configuration options.

Note that there are several ways to get “real-time” support on Linux, and you should make sure that you’re using the approach that requires the least amount of effort to fulfill your needs.

The following is adapted from:

This tutorial was originally hosted on google code, but has moved here since that page might go away at some point.

There are additional details for other options as well numerous comments on a similar Xenomai build given in another tutorial by Sean Kelley here.

Contents

Build and Install a Xenomai-Patched Linux Kernel

Download and Install Ubuntu 12.04

Ubuntu 12.04 Precise Pangolin is an Ubuntu LTS (Long-Term Support) release, and an install iso can be found on Ubuntu’s servers. While Ubuntu 14.04 Trusty Taher is the most recent LTS release at the time of this writing, it has not been sufficiently tested by the robotics community.

Package Prerequisites

sudo apt-get install kernel-package
sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge
sudo apt-get build-dep linux
sudo apt-get install git-core libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev
sudo apt-get install qt3-dev-tools libqt3-mt-dev

Pick a Xenomai Version

Xenomai 2.6.3 is the latest stable release as of this writing. Xenomai supports the following kernels on x86 architectures and has been tested by us with the following Ubuntu distributions. Note that some combinations of Xenomai patches and kernels have had observed issues.

  Xenomai Version   Mainline Kernel   Ubuntu 12.04 LTS   Issues  
  2.6.1   3.2.21   √   No support for PEAK PCIe Card.  
  2.6.2   3.5.7   x   Debian packages not building.  
  2.6.2.1   3.5.7   √   None  
  2.6.3   3.5.7   √   None  
  2.6.3   3.8.13   x   Observed intermittent deadlocks and network floods on Intel E5 platform.  

For more information on the kernels used by various Ubuntu distributions, see the Ubuntu kernel version map.

Set up Convenience Variables and Workspace

First navigate to an empty directory to act as the workspace. Then set up the following environment variables based on which versions of Xenomai and Linux you plan on using:

export linux_version=3.5.7
export linux_tree=`pwd`/linux-$linux_version
export xenomai_version=2.6.3
export xenomai_root=`pwd`/xenomai-$xenomai_version
export build_root=`pwd`/build
mkdir $build_root

Download and Unpack Xenomai

wget http://download.gna.org/xenomai/stable/xenomai-$xenomai_version.tar.bz2
tar xf xenomai-$xenomai_version.tar.bz2

Download and Unpack the Linux Kernel Source

For more details on building kernels in debian, see the Ubuntu kernel compiling documentation.

First, download the linux kernel sources, and untar them:

wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-$linux_version.tar.bz2
tar xf linux-$linux_version.tar.bz2

Second, copy the kernel config that your machine is already using as the basis for the real-time kernel:

cp -vi /boot/config-`uname -r` $linux_tree/.config

Patch the Kernel

This will modify the vanilla kernel yource you just downloaded.

$xenomai_root/scripts/prepare-kernel.sh --arch=x86\
  --adeos=`find $xenomai_root | grep $linux_version-x86`\
  --linux=$linux_tree

Configure the Kernel

This is the most important step in creating a Xenomai kernel and involve platform-specific options. Below are several which you need to set regardless of your platform or application.

cd $linux_tree
make menuconfig

Required Options

The following required options are listed below as they appear in the kernel configuration:

  • Real-time sub-system
    • ENABLE Xenomai
    • ENABLE Nucleus
  • Power management and ACPI options
    • ACPI (Advanced Configuration and Power Interface) Support
      • DISABLE Processor
    • CPU Frequency scaling
      • DISABLE CPU Frequency scaling
  • Processor Type and Features
    • SET Processor family

NOTE: If your kernel does not boot, and gets stuck at Xenomai: debug mode enabled try being more agressive with disabling kernel options suggested on the Xenomai wiki.

RTSocketCAN Options

If you plan on using RTSocketCAN or other hardware interfaces, you need to enable these manually under Real-time sub-system. For example, if you plan on using a 2-port PEAK PCI CANbus card for robotics applications, you should enable the following:

  • Real-time sub-system
    • Nucleus
      • Drivers
        • CAN drivers
          • ENABLE RT-Socet-CAN, CAN raw socket interface
          • SET Maximum number of devices = 4
          • SET Size of receive ring buffers (must be 2^N) = 1024
          • SET Maximum number of receive buffers per device = 32
          • ENABLE Philips SJA1000 CAN controller
            • ENABLE PEAK PCI Card

Options for Many-Threaded Systems like Orocos

If you plan on running Orocos with many threads (like when using ROS integration), you should increase the number of Xenomai registry slots.

  • Real-time sub-system
    • Nucleus
      • SET Number of registry slots = 4096

Build the Kernel

Now you can build the kernel into a debian package and install it:

export CONCURRENCY_LEVEL=7
fakeroot make-kpkg --bzimage --initrd --append-to-version=-xenomai-$xenomai_version kernel-image kernel-headers modules
cd ..
sudo dpkg -i linux-image-*.deb
sudo dpkg -i linux-headers-*.deb

Configure Xenomai User Libraries

In order to write code that uses xenomai, you also need to build and install the xenomai userspace libs.

cd $build_root
$xenomai_root/configure\
  --enable-shared\
  --enable-smp\
  --enable-posix-auto-mlockall\
  --enable-dlopen-skins\
  --enable-x86-sep
make
sudo make install

Add xenomai Group For Non-root Usage

Normally only root can run Xenomai processes, unless you elevate a specific group’s privileges.

First add the “xenomai” group:

sudo addgroup xenomai

Then, add the users who will be allowed to use realtime tasks to the group. For the current user you can simply execute:

sudo adduser `whoami` xenomai

Then, add the xenomai GID to the xenomai configuration. First, get the group id:

cat /etc/group | sed -nr "s/xenomai:.:([0-9]+):.*/\1/p"

Then, modify the grub defaults (located at /etc/default/grub) so that GRUB_CMD_LINE_LINUX_DEFAULT defines the xeno_nucleus.xenomai_gid variable. For example, if the xenomai GID from above is 1001, this line should read:

GRUB_CMDLINE_LINUX_DEFAULT="xeno_nucleus.xenomai_gid=1001"

Then update grub:

sudo update-grub

Boot and Test the Xenomai Kernel

Now you can reboot. If you can’t boot up for some reason, review that you followed all of the steps with no errors and look at the troubleshooting notes at the bottom of this tutorial.

Estimate Clock Latency

For the best timing performance, once you’ve got your system running, you should estimate your clock latency with the xenomai latency program. You should run this program under heavy load, and then store the resulting value in /cat/proc/latency. For more details on this see this Xenomai list thread.

Troubleshooting and Other Configuration Options

IO-APIC

Depending on your hardware, you might need to reconfigure grub after trying to boot. If you get the error:

MP-BIOS bug: 8254 timer not connected to IO-APIC

Then you need to add the “noapic” option to your grub config defaults (located at /etc/default/grub). To do this in Grub2, add noapic to GRUB_CMD_LINE_LINUX_DEFAULT so that it looks like this (with the appropriate xenomai GID):

GRUB_CMDLINE_LINUX_DEFAULT="xeno_nucleus.xenomai_gid=1001 noapic"

Then update grub:

sudo update-grub

Nouveau

If your system gets through the boot sequence (shown as long as you don’t have “quiet” enabled in the grub config) and then the screen goes black, it might be a problem with the Nouveau graphics driver. In this case, modify the grub config defaults (located at /etc/default/grub) so that GRUB_CMD_LINE_LINUX_DEFAULT reads like so:

GRUB_CMDLINE_LINUX_DEFAULT="splash noapic rdblacklist=nouveau nouveau.modeset=0"

Then update grub:

sudo update-grub

Peak CANBus Driver (peak_pci)

If you’re using the PEAK CANBus driver, you may need to blacklist the non-realtime peak_pci kernel module by adding the following to /etc/modprobe.d/blacklist-pcan.conf:

blacklist peak_pci

Without this, the non-rt driver can take precedence and prevent you from accessing the device.

IRQ Problems

IRQs (interrupt requests) can sometimes collide between different hardware. If you’re having trouble using a certain kind of PCI hardware, this might be the source of the problem.

First, to determine if you’re having an IRQ problem, grep your kernel log for mentions of IRQ collisions, like so:

grep -i irq /var/log/kern.log
grep -i irq /var/log/kern.log.1

If you see something describing an IRQ “already being in use” then you’re having an IRQ collision problem.

There at least two ways to fix this. The first, is to simply move your PCI cards around. This is viable if the IRQs are from two different PCI devices. If the two devices trying to use the same IRQ are on the same PCI, device, however, this won’t make a difference. This is currently the case with PEAK PCIe dual-port CANBus cards.

In this case, you can enable IRQ sharing in your kernel config, and re-build your kernel. This option can be found here while running make menuconfig:

  • Real-time sub-system
    • Nucleus
      • ENABLE Shared interrupts

For more help debugging IRQ problems, see Ubunutu’s IRQ debugging help.

Nvidia Cards and Nvidia Drivers

You can’t use current Nvidia drivers with a Xenomai kernel.