Xenomai on Ubuntu 12.04 (LTS)

A Xenomai tutorial for roboticists.
Sat, Sep 27, 2014

Introduction

This is a tutorial on building an x86 (32- or 64-bit) Linux kernel with the Xenomai patches on Ubuntu Linux. The end result is a debian package which, when installed, properly registers itself with the Ubuntu package management system so that you don't clutter up your machine and can quickly iterate if you need to try different configuration options.

Note that there are several ways to get "real-time" support on Linux, and you should make sure that you're using the approach that requires the least amount of effort to fulfill your needs.

The following is adapted from:

This tutorial was originally hosted on google code, but has moved here since that page might go away at some point.

There are additional details for other options as well numerous comments on a similar Xenomai build given in another tutorial by Sean Kelley here.

Contents

    Build and Install a Xenomai-Patched Linux Kernel

    Download and Install Ubuntu 12.04

    Ubuntu 12.04 Precise Pangolin is an Ubuntu LTS (Long-Term Support) release, and an install iso can be found on Ubuntu's servers. While Ubuntu 14.04 Trusty Taher is the most recent LTS release at the time of this writing, it has not been sufficiently tested by the robotics community.

    Package Prerequisites

    sudo apt-get install kernel-package
    sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge
    sudo apt-get build-dep linux
    sudo apt-get install git-core libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev
    sudo apt-get install qt3-dev-tools libqt3-mt-dev 
    

    Pick a Xenomai Version

    Xenomai 2.6.3 is the latest stable release as of this writing. Xenomai supports the following kernels on x86 architectures and has been tested by us with the following Ubuntu distributions. Note that some combinations of Xenomai patches and kernels have had observed issues.

    Xenomai Version

    Mainline Kernel

    Ubuntu 12.04 LTS

    Issues

    2.6.1

    3.2.21

    No support for PEAK PCIe Card.

    2.6.2

    3.5.7

    x

    Debian packages not building.

    2.6.2.1

    3.5.7

    None

    2.6.3

    3.5.7

    None

    2.6.3

    3.8.13

    x

    Observed intermittent deadlocks and network floods on Intel E5 platform.

    For more information on the kernels used by various Ubuntu distributions, see the Ubuntu kernel version map.

    Set up Convenience Variables and Workspace

    First navigate to an empty directory to act as the workspace. Then set up the following environment variables based on which versions of Xenomai and Linux you plan on using:

    export linux_version=3.5.7
    export linux_tree=`pwd`/linux-$linux_version
    
    export xenomai_version=2.6.3
    export xenomai_root=`pwd`/xenomai-$xenomai_version
    
    export build_root=`pwd`/build
    mkdir $build_root
    

    Download and Unpack Xenomai

    wget http://download.gna.org/xenomai/stable/xenomai-$xenomai_version.tar.bz2
    tar xf xenomai-$xenomai_version.tar.bz2
    

    Download and Unpack the Linux Kernel Source

    For more details on building kernels in debian, see the Ubuntu kernel compiling documentation.

    First, download the linux kernel sources, and untar them:

    wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-$linux_version.tar.bz2
    tar xf linux-$linux_version.tar.bz2
    

    Second, copy the kernel config that your machine is already using as the basis for the real-time kernel:

    cp -vi /boot/config-`uname -r` $linux_tree/.config
    

    Patch the Kernel

    This will modify the vanilla kernel yource you just downloaded.

    $xenomai_root/scripts/prepare-kernel.sh --arch=x86\
      --adeos=`find $xenomai_root | grep $linux_version-x86`\
      --linux=$linux_tree
    

    Configure the Kernel

    This is the most important step in creating a Xenomai kernel and involve platform-specific options. Below are several which you need to set regardless of your platform or application.

    cd $linux_tree
    make menuconfig
    

    Required Options

    The following required options are listed below as they appear in the kernel configuration:

    • Real-time sub-system
      • ENABLE Xenomai
      • ENABLE Nucleus
    • Power management and ACPI options
      • ACPI (Advanced Configuration and Power Interface) Support
      • DISABLE Processor
      • CPU Frequency scaling
      • DISABLE CPU Frequency scaling
    • Processor Type and Features
      • SET Processor family

    NOTE: If your kernel does not boot, and gets stuck at Xenomai: debug mode enabled try being more agressive with disabling kernel options suggested on the Xenomai wiki.

    RTSocketCAN Options

    If you plan on using RTSocketCAN or other hardware interfaces, you need to enable these manually under Real-time sub-system. For example, if you plan on using a 2-port PEAK PCI CANbus card for robotics applications, you should enable the following:

    • Real-time sub-system
      • Nucleus
      • Drivers
        • CAN drivers
        • ENABLE RT-Socet-CAN, CAN raw socket interface
        • SET Maximum number of devices = 4
        • SET Size of receive ring buffers (must be 2^N) = 1024
        • SET Maximum number of receive buffers per device = 32
        • ENABLE Philips SJA1000 CAN controller
          • ENABLE PEAK PCI Card

    Options for Many-Threaded Systems like Orocos

    If you plan on running Orocos with many threads (like when using ROS integration), you should increase the number of Xenomai registry slots.

    • Real-time sub-system
      • Nucleus
      • SET Number of registry slots = 4096

    Build the Kernel

    Now you can build the kernel into a debian package and install it:

    export CONCURRENCY_LEVEL=7
    fakeroot make-kpkg --bzimage --initrd --append-to-version=-xenomai-$xenomai_version kernel-image kernel-headers modules
    cd ..
    sudo dpkg -i linux-image-*.deb
    sudo dpkg -i linux-headers-*.deb
    

    Configure Xenomai User Libraries

    In order to write code that uses xenomai, you also need to build and install the xenomai userspace libs.

    cd $build_root
    $xenomai_root/configure\
      --enable-shared\
      --enable-smp\
      --enable-posix-auto-mlockall\
      --enable-dlopen-skins\
      --enable-x86-sep
    make
    sudo make install
    

    Add xenomai Group For Non-root Usage

    Normally only root can run Xenomai processes, unless you elevate a specific group's privileges.

    First add the "xenomai" group:

    sudo addgroup xenomai
    

    Then, add the users who will be allowed to use realtime tasks to the group. For the current user you can simply execute:

    sudo adduser `whoami` xenomai
    

    Then, add the xenomai GID to the xenomai configuration. First, get the group id:

    cat /etc/group | sed -nr "s/xenomai:.:([0-9]+):.*/\1/p"
    

    Then, modify the grub defaults (located at /etc/default/grub) so that GRUB_CMD_LINE_LINUX_DEFAULT defines the xeno_nucleus.xenomai_gid variable. For example, if the xenomai GID from above is 1001, this line should read:

    GRUB_CMDLINE_LINUX_DEFAULT="xeno_nucleus.xenomai_gid=1001"
    

    Then update grub:

    sudo update-grub
    

    Boot and Test the Xenomai Kernel

    Now you can reboot. If you can't boot up for some reason, review that you followed all of the steps with no errors and look at the troubleshooting notes at the bottom of this tutorial.

    Estimate Clock Latency

    For the best timing performance, once you've got your system running, you should estimate your clock latency with the xenomai latency program. You should run this program under heavy load, and then store the resulting value in /cat/proc/latency. For more details on this see this Xenomai list thread.

    Troubleshooting and Other Configuration Options

    IO-APIC

    Depending on your hardware, you might need to reconfigure grub after trying to boot. If you get the error:

    MP-BIOS bug: 8254 timer not connected to IO-APIC
    

    Then you need to add the "noapic" option to your grub config defaults (located at /etc/default/grub). To do this in Grub2, add noapic to GRUB_CMD_LINE_LINUX_DEFAULT so that it looks like this (with the appropriate xenomai GID):

    GRUB_CMDLINE_LINUX_DEFAULT="xeno_nucleus.xenomai_gid=1001 noapic"
    

    Then update grub:

    sudo update-grub
    

    Nouveau

    If your system gets through the boot sequence (shown as long as you don't have "quiet" enabled in the grub config) and then the screen goes black, it might be a problem with the Nouveau graphics driver. In this case, modify the grub config defaults (located at /etc/default/grub) so that GRUB_CMD_LINE_LINUX_DEFAULT reads like so:

    GRUB_CMDLINE_LINUX_DEFAULT="splash noapic rdblacklist=nouveau nouveau.modeset=0"
    

    Then update grub:

    sudo update-grub
    

    Peak CANBus Driver (peak_pci)

    If you're using the PEAK CANBus driver, you may need to blacklist the non-realtime peak_pci kernel module by adding the following to /etc/modprobe.d/blacklist-pcan.conf:

    blacklist peak_pci
    

    Without this, the non-rt driver can take precedence and prevent you from accessing the device.

    IRQ Problems

    IRQs (interrupt requests) can sometimes collide between different hardware. If you're having trouble using a certain kind of PCI hardware, this might be the source of the problem.

    First, to determine if you're having an IRQ problem, grep your kernel log for mentions of IRQ collisions, like so:

    grep -i irq /var/log/kern.log
    grep -i irq /var/log/kern.log.1
    

    If you see something describing an IRQ "already being in use" then you're having an IRQ collision problem.

    There at least two ways to fix this. The first, is to simply move your PCI cards around. This is viable if the IRQs are from two different PCI devices. If the two devices trying to use the same IRQ are on the same PCI, device, however, this won't make a difference. This is currently the case with PEAK PCIe dual-port CANBus cards.

    In this case, you can enable IRQ sharing in your kernel config, and re-build your kernel. This option can be found here while running make menuconfig:

    • Real-time sub-system
      • Nucleus
      • ENABLE Shared interrupts

    For more help debugging IRQ problems, see Ubunutu's IRQ debugging help.

    Nvidia Cards and Nvidia Drivers

    You can't use current Nvidia drivers with a Xenomai kernel. Instead, you need to use the open-source nouveau driver.