Qemu and networking under FreeBSD 8

For experimental purposes I have installed qemu-devel 1.0.10_1 on my main FreeBSD 8.3-PRERELEASE server (gjabkup2). This post documents for posterity the configuration steps I used to enable networking for qemu-devel instances when launched by non-root accounts and inside jails. The steps were inspired by this post and this post.

Basics of qemu networking

The key is to create a ‘tap’ network interface and bridge that tap interface across to one of your FreeBSD systems real interfaces. Qemu attaches to the internal tap interface, and is thus bridged to (and from) your local network.

On gjabkup2 I already have a bridge0, so my manual steps were to create tap0, ensure tap0 is automatically configured ‘up’ each time it is opened, create bridge1, and associate bridge1 with both tap0 and ale0 (my local LAN interface):

  gjabkup2# ifconfig tap0 create
  gjabkup2# sysctl net.link.tap.up_on_open=1
  net.link.tap.up_on_open: 0 -> 1
  gjabkup2#
  gjabkup2# ifconfig bridge1 create
  gjabkup2# ifconfig bridge1 addm tap0 addm ale0 up
  gjabkup2# ifconfig tap0 up

(The final step shouldn’t technically be required, given the net.link.tap.up_on_open=1, but no harm done for my first experiment. Also note that net.link.tap sysctl variables do not exist until the if_tap.ko kernel module is loaded. This occurs automatically if required during “ifconfig tap0 create“, but you can manually “kldload if_tap” as root earlier if desired.)

My system’s interfaces now appear as:

  gjabkup2# ifconfig
  ale0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	  options=c3098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
	  ether 00:26:18:72:08:11
	  inet 10.1.1.12 netmask 0xffffff00 broadcast 10.1.1.255
	  inet 10.1.1.30 netmask 0xffffffff broadcast 10.1.1.30
	  inet 10.1.1.10 netmask 0xffffffff broadcast 10.1.1.10
	  media: Ethernet autoselect (1000baseT <full-duplex>)
	  status: active
	  [..]
  tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	  options=80000<LINKSTATE>
	  ether 00:bd:88:1a:69:00
  bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	  ether 02:91:4b:96:b8:01
	  id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	  maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
	  root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	  member: ale0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
		  ifmaxaddr 0 port 5 priority 128 path cost 20000
	  member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
		  ifmaxaddr 0 port 16 priority 128 path cost 2000000
  gjabkup2#

A trial pxeboot

My home LAN has a pxeboot server running at all times, so this provided a good test of the networking functionality. I ran the following command:

  gjabkup2# qemu-system-i386 -boot order=n -vnc :0 -net nic,model=e1000 -net tap

(and used “ssvncviewer 10.1.1.12:5900” in another window to ‘view’ qemu’s VNC-based screen)

Yay, this worked! Qemu found and launched my home’s pxeboot service and was able to launch a couple of basic tools.

Enabling networking by non-root users

Two steps are required if you want qemu to run as a non-root user yet still have network connectivity: Set the net.link.tap.user_open sysctl (which allows non-root users to open a tap device) and add specific devfs rules to ensure the permissions are set appropriately on /dev/tap* devices.

Set net.link.tap.user_open to allow non-root users to open tapX devices:

  gjabkup2# sysctl net.link.tap.user_open=1
  net.link.tap.user_open: 0 -> 1
  gjabkup2#

Added a line in /etc/devfs.rules to ensure new tapX devices are initialised to mode rw and assigned to group wheel:

  gjabkup2# more /etc/devfs.rules
  [system=10]
         [...]
  add path 'tap*' mode 0660 group wheel
  gjabkup2#

Re-run devfs globally to pick up the changed ruleset:

  gjabkup2# /etc/rc.d/devfs restart

Then confirmed that I could now launch qemu-devel as a non-root user and successfully utilise networking via the tap0 interface (provided that user is a member of wheel).

Ensuring configuration after boot

To ensure the configuration survives reboots, I added the following lines to /etc/sysctl.conf

net.link.tap.user_open=1
net.link.tap.up_on_open=1

An added the following lines to /etc/rc.conf

cloned_interfaces="tap0 bridge1"
ifconfig_bridge1="addm tap0 addm ale0 up"

Qemu networking from inside jails
I run a number of jails on gjabkup2, managed via ezjail. Running qemu from inside a jail, yet having access to external networks, requires that /dev/tapX devices are visible inside the jail from which you wish to run qemu. In this example, I add the functionality to my jail “nsk330-builder” at 10.1.1.10.

Added the following to gjabkup2:/etc/devfs.rules so that ezjail would
have a custom rule set to apply to jails that need tapX access:

  # Custom rules to be used by ezjail to allow
  # access to tapX
  #
  [devfsrules_ezjail_tapX=20]
  add include $devfsrules_hide_all
  add include $devfsrules_unhide_basic
  add include $devfsrules_unhide_login
  add path 'tap*' unhide mode 0660 group wheel
  #

Re-run devfs globally to pick up the additional ruleset:

  gjabkup2# /etc/rc.d/devfs restart

Edit /usr/local/etc/ezjail/nsk330_builder to point to my new devfs rule:

  #export jail_nsk330_builder_devfs_ruleset="devfsrules_jail"
  export jail_nsk330_builder_devfs_ruleset="devfsrules_ezjail_tapX"

Restart the nsk330-builder jail

  gjabkup2# /usr/local/etc/rc.d/ezjail.sh stop nsk330-builder
  Stopping jails: nsk330-builder.
  gjabkup2# /usr/local/etc/rc.d/ezjail.sh start nsk330-builder
  Configuring jails:.
  Starting jails: nsk330-builder.
  gjabkup2#

The tapX interface now appears inside the jail’s /dev

  nsk330-builder# ll /dev/
	  [..]
  crw-rw----  1 root  wheel    0, 149 Apr  9 18:34 tap0
	  [..]
  nsk330-builder#

Success! The following line successfully started up qemu inside the nsk330-builder jail, which got network access.

  nsk330-builder# qemu-system-i386 -boot order=n -vnc :0 -net nic,model=rtl8139 -net tap

(Using “ssvncviewer 10.1.1.10:5900” to connect to qemu’s VNC-based screen.)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: