Kinect on APF

Introduction

Page under construction... Informations on this page are not guaranteed !!

This article will show you how to get the Kinect driver, released by the project OpenKinect, to run on the APF51 board. The end goal of this page will be a system where the Kinect dumps RGB and depth images to a folder on the device. The images will be in ppm and pgm format and can be viewed on a host PC in GIMP.

I'm pretty new to the Armadeus project association, so if I'm repeating information listed elsewhere or not following the general layout, I apologize on beforehand.

Installing required libusb component to rootfs image

The driver requires the libusb component to be installed on the APF51, luckily this component doesn't need to be cross-compiled but can be installed together with the Linux file system on the board.

On your host-pc go to the armadeus toolchains buildroot

 cd ~/armadeus/buildroot 

To configure the device for libusb execute

 make menuconfig 

Select libusb under Package selection for target -> libraries -> hardware handling -libusb You don't need to select libusb-compat.

Exit the menuconfig and save the changes

Build the new image by executing

 make

Move the image to tftpboot

 cp apf51-rootfs.ubi /tftpboot/.

Flash the rootfs image to the board

Plug in power to the board, and stop normal booting during the 20s countdown. You will then see Command on APF

 # BIOS>

Transfer image to APF51

 # tftpboot ${loadaddr} apf51-rootfs.ubi

Flash the image

 # run update_rootfs

Reboot into linux

 # boot

Compiling the kinect driver

If you don't have cmake already installed, install it (debian) by

 sudo apt-get install cmake 

Download the Kinect driver to you host PC

 cd ~
 git clone https://github.com/OpenKinect/libfreenect.git

Make a cmake cross compilation toolchain file, mine looks like this, and is called apf51-crosscompile.cmake

# This one is important
SET(CMAKE_SYSTEM_NAME Linux)

# This is properly also important
SET(CMAKE_SYSTEM_PROCESSOR arm)

#this one not so much
SET(CMAKE_SYSTEM_VERSION 1)

# specify the cross compiler
SET(CMAKE_C_COMPILER   	/home/andersrohde/armadeus/buildroot/output/build/staging_dir/usr/bin/arm-linux-gcc)
SET(CMAKE_CXX_COMPILER 	/home/andersrohde/armadeus/buildroot/output/build/staging_dir/usr/bin/arm-linux-g++)

# where is the target environment 
SET(CMAKE_FIND_ROOT_PATH  /home/andersrohde/armadeus/buildroot/output/target)

# search for programs in the build host directories
#SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# for libraries and headers in the target directories
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
SET(LIBUSB_1_INCLUDE_DIR /home/andersrohde/armadeus/buildroot/output/build/staging_dir/usr/include/libusb-1.0)

Go into libfreenect folder and edit the CMakeList.txt file. We need to deselect examples because they require OpenGL, and therefore the driver can't compile.

 cd libfreenect
 gedit CMakeList.txt 

Configure the options to look like this

OPTION(BUILD_EXAMPLES "Build example programs" OFF)
OPTION(BUILD_FAKENECT "Build fakenect mock library" OFF)
OPTION(BUILD_C_SYNC "Build c synchronous library" ON)
OPTION(BUILD_CPP "Build C++ Library (currently header only)" ON)
OPTION(BUILD_CV "Build OpenCV wrapper" OFF)
OPTION(BUILD_AS3_SERVER "Build the Actionscript 3 Server Example" OFF)
OPTION(BUILD_PYTHON "Build Python extension" OFF)

Create a build directory in the libfreenect folder and go into it

 mkdir build
 cd build

Cross compile the libfreenect with cmake and you cmake file (remember the 2 dots in the end)

 cmake -DCMAKE_TOOLCHAIN_FILE=~/apf51-crosscompile.cmake ..

Compile the software

 make

Installing the Kinect driver on the apf51

The kinect driver is installed in the library file libfreenect.so.0.0.1 stored under libfreenect/lib/libfreenect.so.0.0.1. Tranfer this to /tftpboot folder. Make sure you don't just tranfer a link but transfer the real library file.

 cp /lib/libfreenect.so.0.0.1 /tftpboot/

Transfer this file to the apf51

 # tftp -g -r libfreenect.so.0.0.1 -l /usr/lib/libfreenect.so.0.0.1 192.38.66.102

Link the lib file on the apf51

 # cd /usr/lib
 # ln -s libfreenect.so.0.0.1 libfreenect.so.0.0

Now the Kinect driver is installed but we don't have any software to test it with, since we deselected the examples to make the software compile

Building a test application on the host PC

The record.c application that is one of the examples doesn't require OpenGL to run, and is a good example of a test program.

On you host PC copy the record application to a developement folder.

 cd ~
 mkdir kinectsoftware
 cd kinectsoftware
 cp ~/libfreenect/fakenect/record.c .  

Compile the record application using the library file and header file compiled

/armadeus/buildroot/output/build/staging_dir/usr/bin/arm-linux-gcc -o record -I ~/libfreenect/include/  -L ~/libfreenect/build/lib -l freenect record.c

Transfer the record application to the tftpboot folder

 cp record /tftpboot/.

Transfering it to the device and testing it

Transfer record to APF51

 # tftp -g -r record -l /usr/bin/record 192.38.66.102

Give record executable rights

 # cd /usr/bin
 # chmod a+x record

Create a folder to store the images in

 # cd /tmp
 # mkdir testkinect

Run record app and store the images in testkinect

 # record testkinect

After minimum 10-20s stop the application with Ctrl + C.

Now folder testkinect contains accelerometer dumps, rgb images (ppm files) and depth images (pgm files). The PPM files can be viewed in GIMP, the PGM files are 11 bit so GIMP is complaining about them I viewed them in matlab but will write a solution to gimp soon.