Back

Installing Debian GNU/Linux in the NAND flash of a DockStar

Debian GNU/Linux on (not next to) a DockStar

by Christian Franke <nobody-at-nowhere-dot-ws>

At the time they became available I was quite eager to obtain a Seagate DockStar as it provided a platform quite similar to that of a Sheeva Plug, but at a much lower price.
Obviously, this gadget is only fun if it runs a nice, flexible operating system. As I am into quite weird networking stuff, and as I know it better than most other OSs, the choice was Linux. I looked a bit though the web and it seems at that time, there were two common ways to set that up:
At the one hand, people installed OpenWRT into the NAND flash of the DockStar. OpenWRT is a great project and its developers did a great job porting it to so many platforms, including the DockStar. I am using OpenWRT on some boxes. However, I am not that comfortable with it, mainly because it often gets in my way. I usually end up with replacing big parts by rewriting them in a manner which fits me, which obviously is neither elegant nor convenient nor updateable.
On the other hand, people used full blown distros like Gentoo or Debian, and installed those to an external harddrive. What I dislike about this idea is the fact of the DockStar being a nice, small device in which I need to plug two cables to get it up and running. I would like to keep it that way instead of making more external components a necessity.
Combining these requirements, I made it my goal to install a general purpose distro into the flash of the DockStar. I ended up choosing Debian as it nicely fits into the flash and will be quite easy to update, an aspect which should not be underestimated when it comes to embedded devices.

Acknowledgment

The base for my experiments was the excellent article "How to brick your DockStar and void the warranty" by Alexander Holler [1].

Getting the Toolchain

As I had to build some software for the dockstar (originally I intended to cross-compile Gentoo for it, however I dropped that because of the likely update hassle) a toolchain with support for the DockStar was obviously neccessary.
Running Gentoo on my build box, obtaining one was as simple as issuing 'crossdev --target armv5tel-softfloat-linux-gnueabi' and drinking a few cups of coffee. There might be packages available for other distros, other options would be using OpenWRT to build a toolchain, downloading one from CodeSourcery [2], etc.

Hooking up a serial console

The usage of a serial console makes work with the bootloader much easier, so I hooked one up to the dockstar. You can see the pinout in the image.

Warning: The serial console operates at 3.3V. Using a simple standard USB-RS232 converter will very likely break your dockstar.


    ,__,   ,__,   ,__,   ,__,   ,__,
    |  |   |  |   |  |   |  |   |  |                        _____________
    +--+   +--+   +--+   +--+   +--+         LED 1         /   __  ,__,  \
                                           ,_________,    /   '__| |__,   \
    ,__,   ,__,   ,__,   ,__,   ,__,       |   / \   |    \   |__, ,__|   /
 J1 |  |   |  |   |  |   |  |   |  |       |   \_/   |     \_____________/
    +--+   +--+   +--+   +--+   +--+       +---------+
                   RX     TX     GND
                                       edge of pcb
    --------------------------------------------------------------------------

Building and Installing the bootloader

The DockStar comes with a bootloader that runs a quite cut-down configuration. As the NAND flash of the DockStar already had some bad eraseblocks when it arived - a friend of mine even had some of those in the original bootloader itself - I decided it would be best to simply leave the original bootloader alone and load a second bootloader which runs a more comfortable configuration, as does Alexander in his article.
So I used his patches against u-boot to build a suitable bootloader. In difference to his bootloader, I configured mine to store the environment in the NAND, as it is quite nice to have the ability to reconfigure the bootloader without reflashing it and it does not add much effort.
The final tree I used to build my u-boot is available in my git. [3] The process of building it is quite trivial: run the build script 'mkDockStar.sh'. The first time I did this, many error messages showed up which resulted from u-boot being configured for the wrong target triplet, for which there was no toolchain available. After setting 'CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi' in 'arch/arm/config.mk', it worked fine. As a result I got the 324.1 KiB file u-boot.bin [4].
Now, I attached a screen to the serial console (115200n8), and powered up the DockStar, interrupting the autoboot by sending a character when told to. For downloading the new uboot to the DockStar, I used my favorite tftpserver, atftpd. It runs at 172.22.92.3 in this example: 

    CE>> set ipaddr 172.22.92.14
    CE>> set serverip 172.22.92.3
    CE>> tftpboot 0x1000000 /u-boot.bin
    Using egiga0 device
    TFTP from server 172.22.92.3; our IP address is 172.22.92.14
    Filename '/u-boot.bin'.
    Load address: 0x1000000
    Loading: #################################################################
    done
    Bytes transferred = 331876 (51064 hex)
    CE>> nand erase 0x100000 0x400000

    NAND erase: device 0 offset 0x100000, size 0x400000
    Erasing at 0x4e0000 -- 100% complete.
    OK
    CE>> nand write.e 0x1000000 0x100000 0x60000

    NAND write: device 0 offset 0x100000, size 0x60000

    Writing data at 0x15f800 -- 100% complete.
     393216 bytes written: OK

To describe what I did here: I loaded the bootloader image to offset 0x1000000 in the RAM, erased the second flash partition (0x100000 to 0x500000, i.e. len = 0x400000), and wrote the bootloader to the start of the partition. (0x60000 is the size of the bootloader, padded to match a multiple of the erase sector size 128k, which is 384k = 393216 = 0x60000 in this case)
Now I checked whether the new bootloader was working: 

    CE>> nand read.e 0xc00000 0x100000 0x51064

    NAND read: device 0 offset 0x100000, size 0x51064

    Reading data from 0x150800 -- 100% complete.
     331580 bytes read: OK
    CE>> go 0xc00000
    ## Starting application at 0x00C00000 ...


    U-Boot 2010.06-00712-g50a9107 (Dec 11 2010 - 19:59:38)
    Seagate FreeAgent DockStar

    SoC:   Kirkwood 88F6281_A0
    DRAM:  128 MiB
    NAND:  256 MiB
    *** Warning - bad CRC or NAND, using default environment

    In:    serial
    Out:   serial
    Err:   serial
    Net:   egiga0
    88E1116 Initialized on egiga0
    DockStar>>

Bingo! Notice that warning, it says the environment has been reset, as there is nothing of any sense in the flash - yet. A first change to the environment after which a save might be nice is setting the right mac. It can be found at the bottom of the DockStar or in the first stage bootloader. 

    DockStar>> set ethaddr 00:10:75:C0:FF:EE   <- your mac here
    DockStar>> saveenv
    Saving Environment to NAND...
    Erasing Nand...
    Erasing at 0x160000 -- 100% complete.
    Writing to Nand... done

For the DockStar to boot correctly, in the final setup, the first bootloader has to load the second one automatically. 

    DockStar>> reset
    resetting ...
    [...]
    Hit any key to stop autoboot:  0
    CE>> set bootcmd 'nand read.e 0xc00000 0x100000 0x51064; go 0xc00000'
    CE>> saveenv
    Saving Environment to NAND...
    Erasing Nand...Writing to Nand... done
    CE>> reset

All went right, a 'DockStar>>' prompt appeared shortly after the reset.

Getting an installation environment

I wanted to install Debian using debootstrap, so I needed an installation environment. OpenWRTs ram disk image is quite suitable for that job, so I built one containing the neccesary tools as well as support for UBIFS which I intended to use. My configuration is available at [5], to enable UBIFS for this target, I had to use 'make kernel_menuconfig' and select it manually. (This means enabling most of it except the debugging stuff and setting CONFIG_MTD_UBI_WL_THRESHOLD to 256 as we are on NAND here. As result, I got a 3.6 MiB uImage [6], which I also loaded and booted using tftp: 

    DockStar>> set ipaddr 172.22.92.14
    DockStar>> set serverip 172.22.92.3
    DockStar>> tftpboot 0x1000000 openwrt-dockstar-ramdisk-uImage-20101211
    Using egiga0 device
    TFTP from server 172.22.92.3; our IP address is 172.22.92.14
    Filename 'openwrt-dockstar-ramdisk-uImage-20101211'.
    Load address: 0x1000000
    Loading: #################################################################
             #################################################################
             #################################################################
             ############################################################
    done
    Bytes transferred = 3734532 (38fc04 hex)
    DockStar>> bootm 0x1000000
    ## Booting kernel from Legacy Image at 01000000 ...
       Image Name:   Linux-2.6.35.9
       Image Type:   ARM Linux Kernel Image (uncompressed)
       Data Size:    3734468 Bytes = 3.6 MiB
       Load Address: 00008000
       Entry Point:  00008000
       Verifying Checksum ... OK
       Loading Kernel Image ... OK
    OK

    Starting kernel ...

    Uncompressing Linux... done, booting the kernel.

When booting had finished, pressing return gave a prompt.

Creating the UBI Volume and UBIFS

As this was the first time I used UBIFS, it was a bit tricky to figure everything out, but I fiddled my way through.
The first step was to attach the mtd device/partition for use with UBI. The 'rootfs' partition has been used for that purpose here. For this to succeed, it had to be formatted first. (this was the most intersting pitfall, imho) 

    root@OpenWrt:/# ubiformat /dev/mtd2 -y -s 512
    root@OpenWrt:/# ubiattach /dev/ubi_ctrl -m 2

After that, I had to create a UBI volume and an UBIFS image, and finally apply this image to the volume. 

    root@OpenWrt:/# ubimkvol /dev/ubi0 -N ubiroot -m
    root@OpenWrt:/# mkdir /tmp/empty
    root@OpenWrt:/# mkfs.ubifs -m 2048 -e 129024 -c 1983 -o /tmp/ubi.img \
                    -R 5MiB -x zlib -d /tmp/empty
    root@OpenWrt:/# ubiupdatevol /dev/ubi0_0 /tmp/ubi.img

After that, it could be mounted as installation target: 

    root@OpenWrt:/# mkdir /mnt/debian
    root@OpenWrt:/# mount -t ubifs ubi0:ubiroot /mnt/debian

Installing and configuring Debian GNU/Linux

I then used debootstrap to install debian:

    root@OpenWrt:/# debootstrap --arch armel squeeze /mnt/debian \
                    http://ftp.de.debian.org/debian

This took some time, enough for another coffee break. Once this was finished, some configuration steps had to be completed. This is done best in the new environment, so I chrooted into it: 

    root@OpenWrt:/# mount -t proc proc /mnt/debian/proc
    root@OpenWrt:/# mount -t sysfs sysfs /mnt/debian/sys
    root@OpenWrt:/# mount -o bind /dev /mnt/debian/dev
    root@OpenWrt:/# cp -L /etc/resolv.conf /mnt/debian/etc
    root@OpenWrt:/# env -i TERM=$TERM HOME=/root chroot /mnt/debian /bin/bash
    root@OpenWrt:/# source /etc/profile
    root@OpenWrt:/# export 'PS1=deb:\w\$ '
    deb:/#

The fstab had to be filled in. I created a symlink for /etc/mtab at this occasion:

    deb:/# cat > /etc/fstab << EOF
    > # file system        mount point   type    options   dump   pass
    > ubi0:ubiroot         /             ubifs   defaults  0      0
    >
    > proc                 /proc         proc    defaults  0      0
    >
    > tmpfs                /tmp          tmpfs   size=30m  0      0
    > tmpfs                /var/tmp      tmpfs   size=30m  0      0
    >
    > varlog               /var/log      tmpfs   size=20m  0      0
    > EOF
    deb:/# ln -s /proc/mounts /etc/mtab

I put these mounts there to avoid continous writes onto the flash. There were two directories left in which I intended to use a tmpfs. These were /var/lock and /var/run. As those are populated quite early, Debian provides a special configuration option to mount a tmpfs there early. 

    deb:/# sed -e '/RAM[A-Z]*=/s/no/yes/' -i /etc/default/rcS

Another thing which had to be configured was the timezone: 

    deb:/# dpkg-reconfigure tzdata

Concerning time there was an even bigger problem: the dockstar has no battery to buffer the RTC in the Kirkwood SoC, in fact, there is not even a crystal hooked up to the RTC, so it is quite useless. (it is possible to solder crystal and battery to the dockstar to get a functional RTC, however that shall be left for another article) The most important thing is to have a continous time. This was achieved by storing the time when the system is shut down. To do this, I wrote a small initscript [7], this is what it looks like: 

    #!/bin/sh -e
    ### BEGIN INIT INFO
    # Provides:          timerest
    # Required-Start:    udev
    # Required-Stop:
    # Default-Start:     S
    # Default-Stop:      0 6
    # X-Interactive:     true
    # X-Start-Before:    checkroot
    # Required-Stop:     $local_fs
    # Short-Description: Restore time from timestamp file
    ### END INIT INFO

    PATH="/sbin:/bin:/usr/bin"

    . /lib/lsb/init-functions

    case "$1" in
        start)
        log_daemon_msg "Setting time from timestamp" "timerest"
        date -s @$(stat -c %Y /etc/timerest)
        log_end_msg $?
        ;;
        stop)
        log_daemon_msg "Storing time" "timerest"
        touch /etc/timerest && sync
        log_end_msg $?
        ;;

        restart)
        $0 stop
        $0 start
        ;;

        *)
        echo "Usage: /etc/init.d/timerest {start|stop|restart}"
        exit 1
        ;;
    esac

I stored this as /etc/init.d/timerest and made it executable. Then, I enabled it as a service:

    deb:/# insserv timerest

The next step was to configure networking. For the sake of simplicity, I used a standard config back then:

    deb:/# cat >> /etc/network/interfaces << EOF
    >
    > auto lo
    > iface lo inet loopback
    >
    > auto eth0
    > iface eth0 inet dhcp
    > EOF

The hostname had to be configured too: 

    deb:/# echo ikarus > /etc/hostname
    deb:/# sed -e '/127.0.0.1\s*localhost/s/localhost/ikarus &/' -i /etc/hosts

As it is probably a good idea to download security upgrades, I added the security repository to the repo list. 

    deb:/# cat >> /etc/apt/sources.list << EOF
    >
    > deb http://security.debian.org/ squeeze/updates main
    > EOF

As I do like to have a utf8 locale, I configured one: 

    deb:/# aptitude install locales
    deb:/# dpkg-reconfigure locales

Now, all that was left to do concerning userspace was setting a password and cleaning up.

    deb:/# passwd
    Enter new UNIX password:
    Retype new UNIX password:
    passwd: password updated successfully
    deb:/# for i in /tmp /var/tmp /var/log /var/lock /var/run; do
    > rm -Rf $i/*
    > done

Oh, and I almost forgot: There is of course no need for a terminal on tty1 through 6, however there is a need for one on ttyS0, so /etc/inittab had to be modified accordingly.

Configuring and installing the kernel

I chose to cross-compile the kernel as this is a quite painless process, especially compared to a native build on the DockStar.
With the right configuration, a recent vanilla kernel will run fine on the DockStar. However there is no real support for the DockStar yet. Actually, I made the box pose as a SheevaPlug, by using its arcNumber 2097. If you are interested in this, have a look at the bootloader configurations. The official arcNumber for the DockStar is 2998. Until there is support for the DockStar, the easiest thing to do is pretending to be a SheevaPlug. The only relevant difference for the linux kernel is the LED anyway.
So either a vanilla kernel could be used when the unusability of the LEDs is acceptable, or a simple patch [8] could be applied.
Having my kernel tree [9], I configured and built it. The configuration is available at [10]. 

    nihilus@phoenix ~/dockstar/kernel $ make nconfig ARCH="arm" \
                    CROSS_COMPILE="armv5tel-softfloat-linux-gnueabi-"
    nihilus@phoenix ~/dockstar/kernel $ make -j2 uImage ARCH="arm" \
                    CROSS_COMPILE="armv5tel-softfloat-linux-gnueabi-"
    nihilus@phoenix ~/dockstar/kernel $ make modules_install ARCH="arm" \
                    CROSS_COMPILE="armv5tel-softfloat-linux-gnueabi-" \
                    INSTALL_MOD_PATH="/home/nihilus/dockstar/modules"

The contents of '~/dockstar/modules' had to be copied to '/' on the dockstar. Also, 'arch/arm/boot/uImage' had to be copied to the DockStar, I chose to store it as '/boot/kernel-2.6.36.2-dockstar-00001-gbb1c092', which happens to be the kernel version), and to add a symlink '/boot/kernel' pointing to the kernel I'd like to boot: 


    deb:/boot# ln -s kernel-2.6.36.2-dockstar-00001-gbb1c092 kernel

Now that was done, all that was left to do was configuring the second bootloader. At this point I had to touch '/etc/timerest' so I would have a sane date and time when booting the new Debian installation. 

    deb:/# touch /etc/timerest

Then, I unmounted everything and rebooted.

Configuring the second stage bootloader

First, I did a test boot, to check out if everything worked fine.
The first step was to make the bootloader aware of the flash partitioning: 

    DockStar>> set mtdids 'nand0=nand0'
    DockStar>> set mtdparts \
              'mtdparts=nand0:0x100000(u-boot),0x400000(uImage),-(rootfs)'

Then I initialized UBI and mounted the UBIFS: 

    DockStar>> ubi part rootfs
    DockStar>> ubifsmount ubiroot

Now the kernel could be loaded. 

    DockStar>> ubifsload 0x1000000 /boot/kernel

The commandline had to be set, and I was good to boot. 

    DockStar>> set bootargs \
              'ubi.mtd=2 root=ubi0:ubiroot rootfstype=ubifs console=ttyS0,115200'
    DockStar>> bootm 0x1000000

Everything went well, the system booted to a login prompt. After logging in and rebooting once again, it was time to make the bootloader configuration persistent: 

    DockStar>> set mtdids 'nand0=nand0'
    DockStar>> set mtdparts \
                    'mtdparts=nand0:0x100000(u-boot),0x400000(uImage),-(rootfs)'
    DockStar>> set bootargs \
             'ubi.mtd=2 root=ubi0:ubiroot rootfstype=ubifs console=ttyS0,115200'
    DockStar>> set bootdelay 1
    DockStar>> set bootcmd \
                             'ubi part rootfs; ubifsmount ubiroot; \
                             ubifsload 0x1000000 /boot/kernel; \
                             bootm 0x1000000'
    DockStar>> saveenv

This was it. Another reset, and it booted all through to a login prompt.

Summary

Of course, there still are many things which leave room for improvement. For example, ntpdate or ntpd could be installed so a correct time was available. Installing ntpdate is sufficient, it is called somewhere in the boot process. Combining this with timerest, the results are sufficiently good for me. Speaking of good results: The goal is achived: Debian is installed to the NAND, with about 50% of the flash left, so it is not that crowded as I suspected at the beginning. The time from poweron to login prompt is at about 24sec here, a value which is also quite satisfying, in my humble opinion.

Update 2011-07-22:

Today, I installed the dockstar as a router for my home network. It will have to route between various internal zones, and also, to the Internet. For that purpose, it is attached to an 48 port gigabit ethernet switch, having a VLAN trunk configured for its port, so it has connectivity to all logical ethernets. Some simple testing without performance hunting showed a throughput of about 320MBit/s routed by the dockstar between two internal networks, without conntrack. After enabling contrack, throughput was at 300MBit/s.
If you are wondering about OpenVPN performance: as OpenVPN runs in userland, it needs a lot of context switches for each packet transferred. These are somewhat expensive on ARM. Therefore, OpenVPN throughput is much lower than the numbers above. Even without crypto, performance was at about 70MBit/s. With crypto it might get worse, though there is also crypto acceleration available. However I can't as of now give any details on that, as I decided that OpenVPN won't cut it for the problem which I needed to solve, so I did not research that further.

References