This post documents my recent experiments with Oleksandr Tymoshenko’s patches to get the FreeBSD 10-CURRENT kernel running in a Qemu-emulated Versatile Platform Board (VersatilePB) armv6 environment.
Background
I’ve been looking for a way to build packages for FreeBSD running on small ARM devices (e.g. the NS-K330 with 32MB RAM), and recently stumbled across Oleksandr Tymoshenko’s work here. He has developed patches enabling the FreeBSD 10-CURRENT kernel to be built for the Versatile Platform Board (VersatilePB) under Qemu. Oleksandr notes that his patch creates a kernel that is “…fully compatible with Raspberry Pi’s userland, or Pandaboard’s one. So you can use latest RPi SD card image.” Using the RPi SD card userland image provided here I can create an armv6 environment within which to build ARM packages for other devices that are too small to build Ports themselves.
Applying patches
On December 9th I updated my FreeBSD src tree to svn version r244046:
nsk330-builder# cd <mydevtree>/src nsk330-builder# svn up [...] U sys/socketvar.h Updated to revision 244046. nsk330-builder#
Now create a new directory for the VersatilePB files:
nsk330-builder# cd ./sys nsk330-builder# mkdir arm/versatile
Grabbed a copy of the patch from here, stored it as ~gja/versatilepb.diff and applied it:
nsk330-builder# patch < ~gja/versatilepb.diff
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|Index: boot/fdt/dts/versatilepb.dts
|===================================================================
|--- boot/fdt/dts/versatilepb.dts (revision 0)
|+++ boot/fdt/dts/versatilepb.dts (working copy)
--------------------------
(Creating file boot/fdt/dts/versatilepb.dts...)
Patching file boot/fdt/dts/versatilepb.dts using Plan A...
Hunk #1 succeeded at 1.
Hmm... The next patch looks like a unified diff to me...
[...etc...]
Patching file dev/smc/if_smc.c using Plan A...
Hunk #1 succeeded at 807.
done
nsk330-builder#
Building a bootable image
The kernel is statically configured to assume there will be 128MB of emulated RAM. After a brief email exchange with Oleksandr I ascertained that the following changes to sys/boot/fdt/dts/versatilepb.dts would create a kernel able to use 256MB:
memory {
device_type = "memory";
/* reg = <0 0x08000000>; /* 128MB */
reg = <0 0x10000000>; /* 256MB gja 11dec12 */
};
I wanted serial console on my emulated VersatilePB system, so first commented out the following lines in arm/conf/VERSATILEPB:
# NOTE: serial console will be disabled if syscons enabled # Uncomment following lines for framebuffer/syscons support #device sc #device kbdmux #options SC_DFLT_FONT # compile font in #makeoptions SC_DFLT_FONT=cp437
Downloaded the kernel build script from here, stored it as ~gja/build-versatile.sh , edited SRCROOT and MAKEOBJDIRPREFIX inside the script to point to <mydevtree>/src/ and /usr/obj/ respectively, then ran it:
nsk330-builder# ~gja/build-versatile.sh
--------------------------------------------------------------
>>> Rebuilding the temporary build tree
--------------------------------------------------------------
[....]
--------------------------------------------------------------
>>> Kernel build for VERSATILEPB completed on Tue Dec 11 18:47:38 EST 2012
--------------------------------------------------------------
4+0 records in
4+0 records out
4194304 bytes transferred in 0.003811 secs (1100612240 bytes/sec)
20+0 records in
20+0 records out
20 bytes transferred in 0.000339 secs (58992 bytes/sec)
48+1 records in
48+1 records out
3201528 bytes transferred in 0.002820 secs (1135287597 bytes/sec)
nsk330-builder#
The resulting image file is stored at <mydevtree>/src/sys/freebsd-versatilepb.flash
Running under Qemu 1.1.1
Although Oleksandr recommends Qemu 1.3.0, I got reasonable success using Qemu 1.1.1 (the latest current version in FreeBSD’s Ports collection under qemu-devel). There were random freezes during heavy load, but still workable enough to prove the FreeBSD kernel will run under Qemu/VersatilePB.
I retrieved and uncompressed this RPi SD card image to provide a World for my VersatilePB kernel (Note: The uncompressed freebsd-pi-r243778.img file is ~1GB.)
Started qemu’s ARM emulator (qemu-system-arm) with GUI to show the console:
% qemu-system-arm -M versatilepb -m 256M -kernel freebsd-versatilepb.flash -cpu arm1176 -hda freebsd-pi-r243778.img -net nic -net tap oss: Could not initialize DAC oss: Failed to open `/dev/dsp' oss: Reason: No such file or directory oss: Could not initialize DAC oss: Failed to open `/dev/dsp' oss: Reason: No such file or directory audio: Failed to create voice `lm4549.out' [...the qemu GUI came up and I was able to login...]
“dmesg -a” confirmed that the kernel recognises 256MB of available RAM. Yay!
With tap-based network support on the host, qemu is able to offer network access to the FreeBSD guest. Running “dhclient smc0” results in correct assignment of a DHCP address from my local LAN space, and I was then able to adduser a new local user and ssh into the VersatilePB FreeBSD VM from elsewhere on my home network.
Console log from a similar boot on December 14th shows how the VM appears to the new kernel:
root@raspberry-pi:/mnt # more /var/log/messages Dec 2 21:52:52 raspberry-pi newsyslog[315]: logfile first created Dec 14 09:34:39 raspberry-pi syslogd: kernel boot file is /kernel Dec 14 09:34:39 raspberry-pi kernel: KDB: debugger backends: ddb Dec 14 09:34:39 raspberry-pi kernel: KDB: current backend: ddb Dec 14 09:34:39 raspberry-pi kernel: Copyright (c) 1992-2012 The FreeBSD Project. Dec 14 09:34:39 raspberry-pi kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Dec 14 09:34:39 raspberry-pi kernel: The Regents of the University of California. All rights reserved. Dec 14 09:34:39 raspberry-pi kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. Dec 14 09:34:39 raspberry-pi kernel: FreeBSD 10.0-CURRENT #0 r244046M: Thu Dec 13 19:20:04 EST 2012 Dec 14 09:34:39 raspberry-pi kernel: root@nsk330-builder:/usr/obj/arm.armv6/usr/wr1403nd-dev/head/src/sys/VERSATILEPB-NFSCL arm Dec 14 09:34:39 raspberry-pi kernel: CPU: Sheeva 88SV581x rev 7 (Marvell core) Dec 14 09:34:39 raspberry-pi kernel: Supported features: ARM_ISA THUMB2 JAZELLE ARMv4 Security_Ext Dec 14 09:34:39 raspberry-pi kernel: WB enabled LABT branch prediction enabled Dec 14 09:34:39 raspberry-pi kernel: 4KB/32B 4-way instruction cache Dec 14 09:34:39 raspberry-pi kernel: 64KB/32B 4-way write-through data cache Dec 14 09:34:39 raspberry-pi kernel: real memory = 268435456 (256 MB) Dec 14 09:34:39 raspberry-pi kernel: avail memory = 256843776 (244 MB) Dec 14 09:34:39 raspberry-pi kernel: kbd0 at kbdmux0 Dec 14 09:34:39 raspberry-pi kernel: simplebus0: <Flattened device tree simple bus> on fdtbus0 Dec 14 09:34:39 raspberry-pi kernel: intc0: <ARM PL190 VIC> mem 0xf0140000-0xf0140fff on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: intc0: peripheral ID: 00041190 Dec 14 09:34:39 raspberry-pi kernel: intc0: PrimeCell ID: b105f00d Dec 14 09:34:39 raspberry-pi kernel: sic0: <ARM Versatile SIC> mem 0xcc044000-0xcc044027 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: uart0: <PrimeCell UART (PL011)> mem 0xf01f1000-0xf01f1fff irq 12 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: uart1: <PrimeCell UART (PL011)> mem 0xf01f2000-0xf01f2fff irq 13 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: uart2: <PrimeCell UART (PL011)> mem 0xf01f3000-0xf01f3fff irq 14 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: timer0: <SP804 System Timer> mem 0xf01e2000-0xf01e203f irq 4 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: Timecounter "SP804 Timecouter" frequency 1000000 Hz quality 1000 Dec 14 09:34:39 raspberry-pi kernel: Event timer "SP804 Event Timer 0" frequency 1000000 Hz quality 1000 Dec 14 09:34:39 raspberry-pi kernel: timer0: peripheral ID: 00141804 Dec 14 09:34:39 raspberry-pi kernel: timer0: PrimeCell ID: b105f00d Dec 14 09:34:39 raspberry-pi kernel: pcib0: <Versatile PCI controller> mem 0xcc045044-0xcc045047,0xcc046000-0xcc046fff,0xcc047000-0xcd046fff,0xcd047000-0xcf046fff on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: pcib0: PCI core at slot #11 Dec 14 09:34:39 raspberry-pi kernel: pci0: <PCI bus> on pcib0 Dec 14 09:34:39 raspberry-pi kernel: pci0: domain=0, physical bus=0 Dec 14 09:34:39 raspberry-pi kernel: found-> vendor=0x1000, dev=0x0012, revid=0x00 Dec 14 09:34:39 raspberry-pi kernel: domain=0, bus=0, slot=12, func=0 Dec 14 09:34:39 raspberry-pi kernel: class=01-00-00, hdrtype=0x00, mfdev=0 Dec 14 09:34:39 raspberry-pi kernel: cmdreg=0x0003, statreg=0x0000, cachelnsz=0 (dwords) Dec 14 09:34:39 raspberry-pi kernel: lattimer=0xff (7650 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) Dec 14 09:34:39 raspberry-pi kernel: intpin=a, irq=0 Dec 14 09:34:39 raspberry-pi kernel: map[10]: type I/O Port, range 32, base 0, size 8, enabled Dec 14 09:34:39 raspberry-pi kernel: map[14]: type Memory, range 32, base 0, size 10, enabled Dec 14 09:34:39 raspberry-pi kernel: map[18]: type Memory, range 32, base 0, size 13, enabled Dec 14 09:34:39 raspberry-pi kernel: sym0: <895a> irq 27 at device 12.0 on pci0 Dec 14 09:34:39 raspberry-pi kernel: Alloc resources 3, 00000000..ffffffff, 1024 Dec 14 09:34:39 raspberry-pi kernel: sym0: Lazy allocation of 0x400 bytes rid 0x14 type 3 at 0x50000000 Dec 14 09:34:39 raspberry-pi kernel: Alloc resources 1, 0000001b..0000001b, 1 Dec 14 09:34:39 raspberry-pi kernel: Alloc resources 3, 00000000..ffffffff, 8192 Dec 14 09:34:39 raspberry-pi kernel: sym0: Lazy allocation of 0x2000 bytes rid 0x18 type 3 at 0x50002000 Dec 14 09:34:39 raspberry-pi kernel: sym0: chip clock is 0KHz Dec 14 09:34:39 raspberry-pi kernel: sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking Dec 14 09:34:39 raspberry-pi kernel: sym0: open drain IRQ line driver, using on-chip SRAM Dec 14 09:34:39 raspberry-pi kernel: sym0: using LOAD/STORE-based firmware. Dec 14 09:34:39 raspberry-pi kernel: sym0: handling phase mismatch from SCRIPTS. Dec 14 09:34:39 raspberry-pi kernel: smc0: <SMSC LAN91C110FD or LAN91C111FD> mem 0xcf047000-0xcf056fff irq 25 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: smc0: revision 3057f Dec 14 09:34:39 raspberry-pi kernel: smc0: bpf attached Dec 14 09:34:39 raspberry-pi kernel: smc0: Ethernet address: 52:54:00:12:34:56 Dec 14 09:34:39 raspberry-pi kernel: clcdc0: <PL110 CLCD controller> mem 0xcf057050-0xcf057053,0xf0120000-0xf0120fff irq 16 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: clcdc0: QEMU VGA 640x480 Dec 14 09:34:39 raspberry-pi kernel: sc0: Unknown <16 virtual consoles, flags=0x300> Dec 14 09:34:39 raspberry-pi kernel: sc0: fb0, kbd0, terminal emulator: scteken (teken terminal) Dec 14 09:34:39 raspberry-pi kernel: kmi0: <PL050 Keyboard/Mouse Interface> mem 0xcf058000-0xcf058fff irq 31 on simplebus0 Dec 14 09:34:39 raspberry-pi kernel: kmi0: [GIANT-LOCKED] Dec 14 09:34:39 raspberry-pi kernel: kbd: new array size 4 Dec 14 09:34:39 raspberry-pi kernel: Implement me: kmi_set_leds Dec 14 09:34:39 raspberry-pi kernel: kbd1: kmi0, generic (0), config:0x0, flags:0x3d0000 Dec 14 09:34:39 raspberry-pi kernel: Timecounters tick every 10.000 msec Dec 14 09:34:39 raspberry-pi kernel: tcp_init: net.inet.tcp.tcbhashsize auto tuned to 4096 Dec 14 09:34:39 raspberry-pi kernel: lo0: bpf attached Dec 14 09:34:39 raspberry-pi kernel: (noperiph:sym0:0:-1:-1): SCSI BUS reset delivered. Dec 14 09:34:39 raspberry-pi kernel: sym0: unknown interrupt(s) ignored, ISTAT=0x1 DSTAT=0x80 SIST=0x0 Dec 14 09:34:39 raspberry-pi kernel: da0 at sym0 bus 0 scbus0 target 0 lun 0 Dec 14 09:34:39 raspberry-pi kernel: da0: <QEMU QEMU HARDDISK 1.1.> Fixed Direct Access SCSI-5 device Dec 14 09:34:39 raspberry-pi kernel: da0: 3.300MB/s transfers Dec 14 09:34:39 raspberry-pi kernel: da0: Command Queueing enabled Dec 14 09:34:39 raspberry-pi kernel: da0: 1024MB (2097152 512 byte sectors: 64H 32S/T 1024C) Dec 14 09:34:39 raspberry-pi kernel: GEOM: new disk da0 Dec 14 09:34:39 raspberry-pi kernel: pass0 at sym0 bus 0 scbus0 target 0 lun 0 Dec 14 09:34:39 raspberry-pi kernel: pass0: <QEMU QEMU HARDDISK 1.1.> Fixed Direct Access SCSI-5 device Dec 14 09:34:39 raspberry-pi kernel: pass0: 3.300MB/s transfers Dec 14 09:34:39 raspberry-pi kernel: pass0: Command Queueing enabled Dec 14 09:34:39 raspberry-pi kernel: pass1 at sym0 bus 0 scbus0 target 2 lun 0 Dec 14 09:34:39 raspberry-pi kernel: pass1: <QEMU QEMU CD-ROM 1.1.> Removable CD-ROM SCSI-5 device Dec 14 09:34:39 raspberry-pi kernel: pass1: 3.300MB/s transfers Dec 14 09:34:39 raspberry-pi kernel: pass1: Command Queueing enabled Dec 14 09:34:39 raspberry-pi kernel: Trying to mount root from ufs:da0s2a []... Dec 14 09:34:39 raspberry-pi kernel: warning: no time-of-day clock registered, system time will not be set accurately Dec 14 09:34:39 raspberry-pi kernel: start_init: trying /sbin/init root@raspberry-pi:/mnt # root@raspberry-pi:/mnt # dmesg -a [...] Setting hostuuid: 98050a67-3cca-11e2-b9b6-525400123456. Setting hostid: 0xcaaa7639. No suitable dump device was found. Entropy harvesting: interrupts ethernet point_to_pointsha256: /kernel: No such file or directory kickstart. Starting file system checks: /dev/da0s2a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s2a: clean, 176889 free (1345 frags, 21943 blocks, 0.5% fragmentation) Mounting local file systems:. Writing entropy file:. Setting hostname: raspberry-pi. Starting Network: lo0 smc0. lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet 127.0.0.1 netmask 0xff000000 smc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 52:54:00:12:34:56 Waiting 30s for the default route interface: ............................. Creating and/or trimming log files. realpath: /dev/dumpdev: No such file or directory /etc/rc: WARNING: Dump device does not exist. Savecore not run. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /etc/ld-elf.so.conf Recovering vi editor sessions:. Clearing /tmp (X related). Updating motd:. Configuring syscons: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting background file system checks in 60 seconds. Mon Dec 3 01:19:00 UTC 2012 Dec 3 03:15:18 su: gja to root on /dev/pts/0 root@raspberry-pi:/mnt #
Building Ports
I successfully built portmaster and tmux from Ports, although it took one or two tries before the builds succeeded without qemu 1.1.1 freezing up.
NFS-mounted world
I later experimented with adding the following options to the config file:
options NFSCL #New Network Filesystem Client
and rebuilding the kernel. This successfully enabled me to NFS-mount and use an entire World I’d previously built for the GUMSTIX platform (a different ARM-based device for which a FreeBSD config file also exists). The binaries built for the GUMSTIX world seemed to function correctly when executed by the VersatilePB kernel under Qemu 1.1.1. This suggests I could even completely NFS-boot a VersatilePB VM, and no longer require the RPi SD image.
Using Qemu 1.3.0
I briefly tried running this again under Qemu 1.3.0 — it was more reliable (no random freezes while spending 3+hrs building some Ports) but slowed by the fact that my Qemu 1.3.0 was running under WinXP, which was itself under VirtualBox on another FreeBSD host. When FreeBSD’s qemu-devel Port reaches 1.3.0 I’ll revisit this…
I am. Because. 