How to Build a Virtual Network

with Oracle Solaris Zones

by Yuli Vasiliev

Learn how to set up and maintain a virtual network with Oracle Solaris Zones to overcome the restrictions of physical network hardware.


Published December 2013


A virtual network allows you to use virtual network interface cards (VNICs) rather than physical devices directly. Since a physical device can have more than one VNIC configured on top of it, you can create a multinode network on top of just a few physical devices—possibly even on a single device, thus building a network within a single system. The ability to have a number of full-featured VNICs configured on top of a single physical device opens the door to building as many virtual servers (zones) as necessary and as many as the system can support, connected by a network and all within a single operating system instance.

Want to comment on this article? Post the link on Facebook's OTN Garage page.  Have a similar article to share? Bring it up on Facebook or Twitter and let's discuss.

This article explains how you might build a virtual network consisting of several zones whose VNICs are configured over a single physical NIC. It starts out by describing how to configure a VNIC, and then describes how to create a zone that will use that VNIC exclusively. You'll see new Oracle Solaris 11 commands—dladm and ipadm—in action when configuring a VNIC. Once a virtual network is built, you will look at how to organize flow control by setting bandwidth limits for the traffic following through the VNICs constituting the network.

Network Virtualization in Oracle Solaris

VNICs are a key component of network virtualization in Oracle Solaris. VNICs allow you to have a number of virtual network devices treated as actual physical interfaces but configured over a single physical NIC, thus providing an efficient way of utilizing hardware network resources.

By default, a newly created zone in Oracle Solaris 11 is an exclusive-IP zone. If you configure a zone using the anet resource of the zonecfg utility, the zone will use an automatically created VNIC each time the zone boots. (anet stands for automatic network interface.) However, if you use the net resource of zonecfg, the zone will use a preconfigured VNIC. Either case allows Oracle Solaris 11 network virtualization features to be applied to the zone. Figure 1 gives a simplified depiction of this design:

Figure 1

Figure 1. Zones using VNICs configured on top of a single physical network device.

An exclusive-IP zone must have exclusive access to at least one network interface, which can be a VNIC, a separate LAN, or a separate VLAN. If necessary, you may configure an exclusive-IP zone to have more than one dedicated network interface.

Preparing Your Working Environment

To go through the article examples, you'll need to have Oracle Solaris 11 installed on your system. You may have it installed virtually on Oracle VM VirtualBox cross-platform virtualization software, which is available for Microsoft Windows, Mac OS X, Oracle Solaris, and Linux. Once you have Oracle VM VirtualBox installed, you can download and import the Oracle Solaris 11 Admin VM appliance into it, which provides a convenient way to evaluate the Oracle Solaris 11.1 operating system.

Creating a Zone with an Exclusive-IP Network Stack

The procedures in this section explain how to create new zones to use VNICs. Since only exclusive-IP zones may use VNICs, let's look at an example of configuring such a zone. First, you create a VNIC that will be then used as the zone's physical interface:

  1. Let's start by checking out the data links presented in the system at the moment:

    $ dladm show-link
    

    Assuming you have a single physical NIC in your system, the output of the command above might look like this:

    LINK                CLASS     MTU    STATE    OVER
    net0                phys      1500   up       --
    
  2. Before you can configure a VNIC, you must become an administrator. The simplest way to do that is to become root:

    $ su -
    Password: <Type the root password>
    
  3. Now you can create a VNIC over the net0 physical NIC, using the dladm create-vnic command:

    # dladm create-vnic -l net0 vnic1
    
  4. Let's check the data links again. Now, the dladm show-link command should generate the following output:

    LINK                CLASS     MTU    STATE    OVER
    net0                phys      1500   up       --
    vnic1               vnic      1500   up       net0
    

The following steps will walk you through configuring and installing a new zone that will use the newly created VNIC:

  1. Using the zonecfg command, create a new zone, assigning the VNIC created in the preceding steps to be the zone's physical interface:

    # zonecfg -z zone1
    Use 'create' to begin configuring a new zone.
    zonecfg:zone1>create
    create: Using system default template 'SYSdefault'
    zonecfg:zone1>set zonepath=/export/home/zone1
    zonecfg:zone1>set autoboot=true
    zonecfg:zone1>set ip-type=exclusive 
    zonecfg:zone1>add net
    zonecfg:zone1:net>set physical=vnic1
    zonecfg:zone1:net>end 
    zonecfg:zone1>verify
    zonecfg:zone1>commit
    zonecfg:zone1>exit
    #
    
  2. Check the state of the zone you've just configured:

    # zoneadm list -cv
    
    ID NAME         STATUS     PATH                   BRAND    IP    
     0 global       running    /                      solaris  shared
     - zone1        configured /export/home/zone1     solaris  excl  
    
  3. Next, install the zone:

    # zoneadm -z zone1 install
    The following ZFS file system(s) have been created:
        rpool/export/home/zone1
    Progress being logged to /var/log/zones/zoneadm.20131011T015848Z.zone1.install
           Image: Preparing at /export/home/zone1/root.
    
    ...
    
  4. After the zone has been successfully installed, start it:

    # zoneadm -z zone1 boot
    
  5. Now, log in to the zone:

    # zlogin -C zone1 
    

    Upon the first login, the System Configuration Tool is launched. There, you'll be prompted to specify the configuration parameters for the zone, including the host name and IP address of the zone, the default route, name services, and so on.

  6. While logged in to the zone, issue the ipadm show-addr command to see the configured addresses within the zone:

    $ ipadm show-addr
    ADDROBJ           TYPE     STATE        ADDR
    lo0/v4            static   ok           127.0.0.1/8
    vnic1/v4          static   ok           10.0.2.18/24
    lo0/v6            static   ok           ::1/128
    vnic1/v6          addrconf ok           fe80::8:20ff:fe44:f3f0/10
    

    You should see that the VNIC has been assigned the IP address you specified with the System Configuration Tool.

  7. To make sure that the network is up and running correctly, issue some pings:

    $ ping 10.0.2.18
    10.0.2.18 is alive
    $ ping 10.0.2.2
    10.0.2.2 is alive
    $ ping technet.oracle.com
    technet.oracle.com is alive
    

Let's now install an application that will use the zone's VNIC to communicate with the outside world. The following steps assume that you're still logged in to the zone. Otherwise, you can use the zlogin command as you did in the preceding steps:

  1. First, become root:

    $ su -
    Password: <Type the zone's root password>
    
  2. Then, use the pkg install command to install an application, say, the Apache web server:

    # pkg install apache-22
               Packages to install:  7
           Create boot environment: No
    Create backup boot environment: No
                Services to change:  1
    
    DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
    Completed                                7/7       665/665      8.7/8.7  298k/s
    
    PHASE                                          ITEMS
    Installing new actions                       916/916
    Updating package state database                 Done 
    Updating image state                            Done 
    Creating fast lookup database                working /
    Creating fast lookup database                   Done 
    
  3. Next, enable the newly installed Apache web server:

    # svcadm enable apache22
    
  4. Then, change the htdocs/index.html file of the Apache web server so it displays a message specific to the zone.

    1. If this file is read-only, you'll first need to make it writable:

      # chmod a+w /var/apache2/2.2/htdocs/index.html
      
    2. Now you can open the file with vi:

      # vi /var/apache2/2.2/htdocs/index.html
      
    3. Then edit the file as follows:

      <html><body><h1>Hello from zone1!</h1></body></html>
      
  5. Launch a web browser and point it to the zone's VNIC address.

    As a result, your web browser should display the page shown in Figure 2:

    Figure 2

    Figure 2. The index page of the Apache web server installed in zone1.

So, you have now an exclusive-IP zone and a network application installed in it, which can be reached via the IP address assigned to the zone.

Creating More VNICs and Zones

Now that you have a zone configured with a VNIC to communicate with the outside world, you might want to create more such zones, connecting them into a network. To create a similar zone, you might use a technique known as zone cloning, through which the data from a source zonepath is copied to a target zonepath, thus eliminating the need to install packages from the Oracle Solaris 11 package repository. Moreover, with cloning, your custom installations will also be copied to the new zone.

First, though, you need to create a VNIC for the new zone, as discussed in the preceding section. Then you can configure a new zone, assigning the newly created VNIC as the zone's physical interface, as also discussed in the preceding section. Then you can install the new zone by cloning the already existing zone1 instead of installing the packages from the Oracle Solaris 11 package repository. So, the general steps to follow now are:

  1. Create a new VNIC over the physical NIC.
  2. Configure a new zone.
  3. Install the new zone by cloning an already existing one.

The first two steps are discussed in detail in the preceding section, with the only difference being the names of the objects being configured. This time, create a zone named vnic2 and then configure a zone called zone2 to use that VNIC.

The details for the third step are below:

  1. Make sure you're connected to the global zone as an administrator.
  2. Halt zone1 (the one to be cloned):

    # zoneadm -z zone1 halt
    
  3. Assuming you have already created a VNIC and a configuration for zone2, you can now install zone2 by cloning zone1:

    # zoneadm -z zone2 clone zone1
    
    The following ZFS file system(s) have been created:
        rpool/export/home/zone2
    Progress being logged to /var/log/zones/zoneadm.20131022T002220Z.zone2.clone
    ...
    
  4. After the zone has been successfully cloned, you can boot it.

    # zoneadm -z zone2 boot
    
  5. Then, log in to the zone:

    # zlogin -C zone2 
    
  6. As with zone1, upon the first login, the System Configuration Tool is launched. When prompted to enter an IP address, make sure you specify an address that is different from what you specified for zone1 but belongs to the same network.

While logged in to zone2, make sure that it can ping itself, ping zone1, ping the router, and ping an outside source:

$ ping 10.0.2.19
10.0.2.19 is alive
$ ping 10.0.2.18
10.0.2.18 is alive
$ ping 10.0.2.2
10.0.2.2 is alive
$ ping technet.oracle.com
technet.oracle.com is alive

Now let's test the Apache web server that must have been cloned from zone1, by pointing a web browser to the VNIC address of zone2: 10.0.2.19. As a result, you should see the same greeting message shown in Figure 2: "Hello from zone1!" To change this message to indicate that it belongs to zone2, edit the /var/apache2/2.2/htdocs/index.html file as follows:

<html><body><h1>Hello from zone2!</h1></body></html>

If you now point the browser to 10.0.2.19, you should see the "Hello from zone2!" message. In this same way, you can change the configuration of the Apache web server and its document set for zone2.

Controlling Network Traffic

Now that you have a virtual network with two zones that can communicate with one another and with the outside world by means of their VNICs, let's configure the network to manage quality of service, setting resource controls to process network traffic within the network.

With Oracle Solaris 11, there are several methods you can take advantage of to manage network resources on your system, obtaining a desired quality of service easily. For example, you might configure network lanes or organize packet traffic into flows to efficiently satisfy differing bandwidth requirements.

The idea behind flows is pretty simple: you organize network traffic into a flow according to an IP address, a transport protocol (for example, UDP), or an application port number (for example, port 80 for TCP). Also, when defining a flow, you assign it to a network device, thus specifying that only the packets flowing through this device will be subject to the flow rule. Figure 3 gives a conceptual depiction of such a design:

Figure 3

Figure 3. Using flows to limit maximum bandwidth for the traffic subsets going through a network device.

As you can see in Figure 3, a flow can be thought of as a filter applied to a subset of the traffic going through a specific network device—either a physical or a virtual device. Thus, you can define a flow to limit access to a certain application behind that network device, setting the bandwidth limit on the traffic going to and coming from this application. Also, you can define a flow between a network device and a remote host, setting a maximum bandwidth between the device and that host.

With the flowadm command, you can define flows on VNICs as well as on physical network devices and even on link aggregations. The command allows you to create and manage flows in both global and non-global zones. However, you'll be able to modify or remove a flow only from within the same zone in which it was created. For further details on flows, refer to the "Network Resource Management by Using Flows" section in the Using Virtual Networks in Oracle Solaris 11.1 guide.

The following example illustrates how you might define a flow on vnic1 created previously, limiting bandwidth between this VNIC and the remote host 10.0.2.17—in other words, limiting bandwidth between zone1 and the global zone:

  1. Connect to zone1:

    # zlogin -C zone1 
    
  2. After connecting to the zone's console, become an administrator of zone1:

    $ su -
    Password: <Type the root password>
    
  3. First, check whether you have any flows already defined in zone1:

    # flowadm show-flow 
    

    The output should show nothing, meaning you haven't defined any flows yet.

  4. Now, suppose you want to create a flow that will be in effect only for the traffic between the zone1's vnic1 and remote host 10.0.2.17 (in this particular example, this address is assigned to the global zone). Before defining such a flow, let's evaluate the network connection speed between vnic1 and 10.0.2.17, copying a big file from zone1 to 10.0.2.17 or vise versa.

    root@zone1:/tmp#  scp dummy.db jxyul@10.0.2.17:/home/jxyul
    Password: 
    dummy.db        100% |*****************************| 15081 KB    00:00    
    
  5. Now, let's create a flow on vnic1, setting the remote_ip attribute to 10.0.2.17:

    root@zone1:/tmp# flowadm add-flow -l vnic1 -a remote_ip=10.0.2.17 flow1
    
  6. Now set the flow's bandwidth property, limiting the flow's bandwidth to a maximum of, say, 1Mb per second:

    root@zone1:/tmp# flowadm set-flowprop -p maxbw=1M flow1
    
  7. If you now repeat the copying operation, you'll see that the time it takes increases significantly:

    root@zone1:/tmp#  scp dummy.db jxyul@10.0.2.17:/home/jxyul
    Password: 
    dummy.db        100% |*****************************| 15081 KB    02:05    
    
  8. It's important to note that the flow rule is applied to the traffic going in both directions. Thus, if you now try to perform a reverse operation and copy the same file from the global zone to zone1, it should take the same time for the entire operation:

    root@solaris:/etc/svc# scp dummy.db z1user1@10.0.2.18:/tmp
    Password: 
    dummy.db        100% |*****************************| 15081 KB    02:05
    
  9. Finally, you can always remove a flow using the flowadm remove-flow command:

    root@zone1:/tmp# flowadm remove-flow flow1
    

    The command above will remove the restrictions applied by the flow, returning the traffic characteristics to the initial level.

As you can see from the example above, flows provide a convenient way to specify bandwidth limits for network interfaces, allowing you to not only control the traffic to a zone but also to control traffic to a certain application installed in that zone.

Conclusion

Taking into account that a single Oracle Solaris instance can run a great number of zones, it is hard to imagine that each might be assigned to a dedicated physical NIC, and, therefore, using VNICs enables you to overcome the restrictions of physical network hardware. Once a VNIC has been created, you can monitor and manage it as if it were a physical NIC, using the same commands, such as ipadm and flowadm, with the same syntax.

See Also

Also see the following resources:

About the Author

Yuli Vasiliev is a software developer, freelance author, and consultant currently specializing in open source development, Java technologies, business intelligence (BI), databases, service-oriented architecture (SOA) and, more recently, virtualization. He is the author of a series of books on Oracle technology, with the most recent one being Oracle Business Intelligence: An Introduction to Business Analysis and Reporting (Packt, 2010).

Revision 1.0, 12/02/2013

Follow us:
Blog | Facebook | Twitter | YouTube