Full-on networking for virtual machines - bridged network configuration for QEMU-KVM

Today I shall discuss how to set up your workstation so virtualized computers running under QEMU-KVM may access your LAN just like the workstation can.

What's Virtualization?

First let's explain what QEMU-KVM is. QEMU-KVM is a way to simulate computer hardware so that you can run an operating system and all its applications in a container where it will not come to harm. So, for example, you can:

• Run Windows 7 on a virtual machine because your boss at work makes you run Outlook, or the boss at home makes you run Quicken. You are safely away from meddling with and damaging your hardware or setting up a whole separate computer for these purposes.
• Friends raving about the latest Ubuntu or Windows version? Take a test drive of the full installation of an operating system, without messing up currently working computer.
• Provide an online server/service. On its own virtual computer, it's easy to stop, start, examine, checkpoint, backup, and move to other hardware when your disk starts going bad. My virtual private server for my home domain runs this way.
• Run any standalone application or other Virtual Appliance like a LiveCD or USB thumb drive image on its own virtual computer.

It is often not as sluggish as you think. Thanks to a lot of work by clever people, QEMU-KVM runs most of the computer code that is able to run on the computer you have at nearly full speed and across multiple processor cores if you have them - we call this paravirtualization. It can also run code for completely different processors, although considerably slower - for example, take the Android mobile phone kernel and software, which is typically compiled for and runs on ARM processors).

Why is Full Networking Important?

Networking ends up being critical. These days, it's not enough to just run these nifty images in isolation from the rest of the world. Everything is connected these days to the Internet. Your mobile phone may have an app to order pizza from the closest Dominoes at the push of a button. Programs look for updates from the internet of themselves, the weather, sports scores, stock quotes, and something you might like to buy or vote for. Sneakier applications spam your your social media friends on the Internet (Facebook, Twitter), but well meaning ones may look for printers, routers, media devices or media services on your home or office, slag your game playing friend the next town over, and charge you for additional use of online features.

It's no different with applications running on virtualized hardware - they need networks too. These days, web-connected software usually needs to feel as if it is on a full, capable computer ready to make the network do their bidding -- or in the case of servers, to respond to requests of others to do their bidding. Half-baked, cheesy networking, implemented as 'usermode' networking or 'NAT' networking, aren't as valuable as you might think. Both tend to be the roach motels of the internet - packets come in but they can't get out. Our virtual machines demand better. In fact, they probably require it.

So now that we've established that virtualized computers need to be full network peers, you'll understand why today I want to talk about setting up Linux workstations to do just this. The non-techy non-Linux folks can probably skip the rest, but it came at enough cost of experimentation that I just had to share with those whose search engines really need to find it.

The Plan

In short, we'll run your qemu-kvm virtual machines on the same subnet their host machine. This is done by configuring bridged networking on your PC. I found a significant part of my answers at the Linux-KVM site.

Lets say your starting setup looks like this:

Ordinary Workstation Networking

We plan for an ending setup that looks like this:

Bridged Workstation Networking with Virtual Machine

The overall principle is to stop using the ethernet (eth0) on the workstation as the endpoint for IP traffic for the workstation. Instead, we will add a virtual bridge (br0) behind it to which we can attach multiple IP endpoints - multiple virtual network devices. The workstation configures its address onto one side of the bridge, and any number of attached tap devices connect endpoints for other things (like VM hosts) which can request and set their own addresses.

Step By Step

1. Disable ethernet networking:

    $ sudo /etc/init.d/network stop

2. Add bridge network configuration. Below is the Redhat/Fedora way. Debian and SuSE have their own ways (see the Linux-KVM site). First set up a bridge configuration based on the existing eth0 configuration:

    $ sudo cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-br0
    $ sudo vim /etc/sysconfig/network-scripts/ifcfg-br0

    DEVICE=br0
    TYPE=Bridge
    BOOTPROTO=none
    IPADDR=192.168.10.80
    PREFIX=24
    GATEWAY=192.168.10.1
    DNS1=192.168.10.1
    DNS2=216.155.210.2
    DNS3=205.238.26.97
    DOMAIN=mydomain.net
    DEFROUTE=yes
    IPV4_FAILURE_FATAL=yes
    IPV6INIT=no
    NAME="System br0"
    ONBOOT=yes

3. Now cut back the ethernet configuration to just the interface itself as a connection to the bridge:

    $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth0 

    DEVICE=eth0
    TYPE=Ethernet
    BRIDGE=br0
    NAME="System eth0"

4. start up networking again and check that it's all up. You should be no worse off than before you stopped the networking, but note that the eth0 device is no longer what's being used at the link layer, and you now have a nifty bridge device configured:

    $ sudo chkconfig network on
    $ sudo /etc/init.d/network start
    $ route -n
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 br0
    169.254.0.0     0.0.0.0         255.255.0.0     U     1025   0        0 br0
    0.0.0.0         192.168.10.1    0.0.0.0         UG    0      0        0 br0
    $ brctl show
    bridge name     bridge id               STP enabled     interfaces
    br0             8000.6c626d5b5a44       no              eth0

5. Now create a tap device for QEMU-VM, attach it to the bridge, and bring it up:

    $ sudo tunctl -u $USER
    Set 'tap0' persistent and owned by uid 1000
    $ sudo brctl addif br0 tap0
    $ brctl show
    bridge name     bridge id               STP enabled     interfaces
    br0             8000.6c626d5b5a44       no              eth0
    $ sudo ip link set tap0 up
    $ ip link show
    1: lo:  mtu 16436 qdisc noqueue state UNKNOWN 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    2: eth0:  mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
      link/ether 6c:62:6d:5b:5a:44 brd ff:ff:ff:ff:ff:ff
    22: sit0:  mtu 1480 qdisc noop state DOWN 
      link/sit 0.0.0.0 brd 0.0.0.0
    25: br0:  mtu 1500 qdisc noqueue state UNKNOWN 
      link/ether 6c:62:6d:5b:5a:44 brd ff:ff:ff:ff:ff:ff
    27: tap0:  mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
      link/ether a2:ca:11:8f:94:81 brd ff:ff:ff:ff:ff:ff
    

6. Ta-da! You are ready to start qemu-kvm:

    $ qemu-kvm suse-server.img -net nic,model=rtl8139,macaddr=DE:AD:BE:EF:25:20 -net tap,ifname=tap0,script=no

You can drop several of the arguments, but macaddr is useful for having your Router/Gateway hand out a predictable IP address to the instance with that address using DHCP, and if ifname is useful when you have several VM instances.

7. Turn off NetworkManager. It will try to initialize eth0 for you and get it an address assigned. You set up the ifcfg-eth0 and ifcfg-br0 scripts earlier and don't want this. One way to make it stop meddling with the ethernet device is to just turn it off:

    $ sudo chkconfig NetworkManager off