Xen3 and a Virtual Network

Şuraya atla: kullan, ara

Introduction

By default, Xen creates a bridge to which it attaches a physical network interface. To keep the host or priviledged system (dom0) working, it creates a virtual interface for it and attaches this to the bridge.

This works quite transparently on a workstation, but has issues on a server which also serves as a router and firewall. Standard answer, and best practice according to other Xen users, is to hide the physical network interfaces for dom0 and expose them in a domU and have it handle the routing and firewalling.

A Virtual Network

For my purpose, I would like to run a Xen guest incidentally on my SOHO server. This has some implications, especially when using a firewall. Packets coming from a standard bridge pass the firewall tables differently and will never be masqueraded. Unless you hack extra rules using ebtables, which is what iptables is for routed traffic.

My requiremenst were actually not to have a physical NIC renumbered, renamed and attached to a bridge, but to have a virtual network on my server which will be treated as a DMZ.

The Brouter Setup

To accomplish this, and to have the domUs on my DMZ masqueraded to the net, I tried a brouter setup. And it works wonderfully well.

A brouter is a combination of a bridge an a router. In this configuration, the interfaces attached to it do not have an IP address, but the bridge does. Sticking with the descriptions used in the Xen manual, it is a switch and network interface in one, attached to dom0 and to which the domUs are connected.

So the domUs are bridged to each other, but the connection to dom0 is routed. Hence the name 'brouter'.

It looks like this:

XenWithVirtualNetwork480.png

The startup procedure is in essence as follows:

  1. At the start of Xend
    • Start a bridge
    • Configure it with an ip address as if it was a normal network interface
    • Set the proper route to it and restart the firewall.
  2. At the start of the domU
    • Attach the vif to the bridge.
    • Start the domU

To accomplish this, I use a special script to be started when xend is started, but still use the regular (bridging) script for starting the domUs.

The Configuration

my /etc/xen/xend-config.sxp has contains the following settings for the network (disable any others):

## Use the following if the domU's are on their own virtual net
# and dom0 does all routing etc. This is actually a brouter
(network-script 'network-virtual bridgeip="10.98.98.1/24" brnet="10.98.98.0/24"')
(vif-script     vif-bridge)

and my domU configuration looks as follows:

disk    = [ 'file:/var/lib/xen/images/MyDomU/hda,hda1,w', 'file:/var/lib/xen/images/MyDomU/swap,hda2,w']
memory  = 128
kernel  = "/var/lib/xen/images/MyDomU/vmlinuz-xen"
ramdisk = "/var/lib/xen/images/MyDomU/initrd-xen"
name    = "MyDomU"
root    = "/dev/hda1"
vif     = ['mac=aa:cc:00:00:00:01']

You will notice the absence of any network settings in this configuration file. That is because I use the regular SUSE configuration system in my domU to set these.

The Script

Maybe it would be logical to show the script before the configuratione, but due to its size I thought it better to put it at the end of the page.

The script is called network-virtual and should be located in /etc/xen/scripts

#!/bin/sh
#============================================================================
# Default Xen network start/stop script.
# Xend calls a network script when it starts.
# The script name to use is defined in /etc/xen/xend-config.sxp
# in the network-script field.
#
# This script creates a bridge (default xenbr${vifnum}), gives it an IP address
# and the appropriate route. Then it starts the SuSEfirewall2 which should have
# the bridge device in the zone you want it.
#
# If all goes well, this should ensure that networking stays up.
# However, some configurations are upset by this, especially
# NFS roots. If the bridged setup does not meet your needs,
# configure a different script, for example using routing instead.
#
# Usage:
#
# vnet-brouter (start|stop|status) {VAR=VAL}*
#
# Vars:
#
# bridgeip   Holds the ip address the bridge should have in the
#            the form ip/mask (10.0.0.1/24).
# brnet	     Holds the network of the bridge (10.0.0.1/24).
# 
# vifnum     Virtual device number to use (default 0). Numbers >=8
#            require the netback driver to have nloopbacks set to a
#            higher value than its default of 8.
# bridge     The bridge to use (default xenbr${vifnum}).
#
# start:
# Creates the bridge
# Gives it the IP address and netmask
# Adds the routes to the routing table.
#
# stop:
# Removes all routes from the bridge
# Removes any devices on the bridge from it.
# Deletes bridge
#
# status:
# Print addresses, interfaces, routes
#
#============================================================================


dir=$(dirname "$0")
. "$dir/xen-script-common.sh"
. "$dir/xen-network-common.sh"

findCommand "$@"
evalVariables "$@"

vifnum=${vifnum:-0}
bridgeip=${bridgeip:-10.6.7.1/24}
brnet=${brnet:-10.6.7.0/24}
bridge=${bridge:-xenbr$vifnum}

##
# link_exists interface
#
# Returns 0 if the interface named exists (whether up or down), 1 otherwise.
#
link_exists()
{
    test -d "/sys/class/net/$1"
}


# Usage: create_bridge bridge
create_bridge () {
    local bridge="$1"

    # Don't create the bridge if it already exists.
    if [ ! -d "/sys/class/net/$bridge/bridge" ]; then
        brctl addbr "$bridge}"
        brctl stp "$bridge" off
        brctl setfd "$bridge" 0
    fi
    ip link set ${bridge} up
}

# Usage: add_to_bridge bridge dev
add_to_bridge () {
    local bridge="$1"
    local dev="$2"
    # Don't add $dev to $bridge if it is already on a bridge.
    if ! brctl show | grep -wq "$dev"; then
        brctl addif "$bridge" "$dev"
    fi
}

# Usage: show_status dev bridge
# Print address config and routes.
show_status () {
    local dev="$1"
    local bridge="$2"
    
    echo '============================================================'
    ip addr show "$dev"
    ip addr show "$bridge"
    echo ' '
    brctl show "$bridge"
    echo ' '
    ip route
    echo '============================================================'
}

op_start () {
    if [ "$bridge" = "null" ] ; then
        return
    fi

    create_bridge "$bridge"

    if link_exists "$bridge"; then
        ip addr add "$bridgeip" dev "$bridge"
        ip link set "$bridge" up
    fi

    if [ "$antispoof" = "yes" ] ; then
        antispoofing
    fi
    rcSuSEfirewall2 start
}

op_stop () {
    if [ "$bridge" = "null" ]; then
        return
    fi
    if ! link_exists "$bridge"; then
        return
    fi
    
    ip link set "$bridge" down
    ip addr del "$bridgeip" dev "$bridge"
    ##FIXME: disconnect the interfaces from the bridge 1st
    brctl delbr "$bridge"
    rcSuSEfirewall2 start
}

case "$command" in
    start)
        op_start
        ;;
    
    stop)
        op_stop
        ;;

    status)
        show_status "$netdev" "$bridge"
        ;;

    *)
        echo "Unknown command: $command" >&2
        echo 'Valid commands are: start, stop, status' >&2
        exit 1
esac