What You See Is What You Get Element

How I Decide Which Virtualization Technology to Use

by David Pipes

How to weigh security, architectural, and organizational requirements to determine which virtualization technology is most appropriate.


Published April 2012

When the time comes to update your hardware, some workloads that were comfortably ensconced in their own hardware need to be consolidated onto systems with virtual environments. But how do you choose between the various methods of virtualization?  For instance, Oracle Solaris lets you create a virtual network.  And virtual storage.  You can allocate memory and CPUs to workloads in interesting ways.  Oracle Solaris Zones (previously called Oracle Solaris Containers) lets you virtualize entire systems.  You also have different hypervisors to choose from.  And what about the hardware virtualization options in SPARC and x86 platforms -- how do they add to your options, and when should you use them?

OTN is all about helping you become familiar enough with Oracle technologies to make an informed decision. Articles, software downloads, documentation, and more. Join up and get the technical resources you need to do your job.

I prefer to focus on workloads rather than technology.  And the first question I ask myself is: do I need to consolidate workloads or separate them?

Workload separation simply means combining workloads on the same platform in the same OS instance. How is this virtualization? Put simply, it's not. It's consolidation. Why virtualize if you can simply consolidate? Oracle Solaris has many tools to let you consolidate workloads and keep their footprints separate in the system. Projects can keep groups of users out of each other's files and processes. The Fair Share Scheduler lets jobs coexist on the system without any one of them taking up too many CPU resources. If the workloads can live together on the system, and the users can keep to their own areas, then why not just consolidate them on the same system? Sometimes the answer to how to virtualize is "you don't actually need to."

But suppose there are security, architectural, or organizational requirements that mandate keeping workloads separate. In this case, Oracle Solaris has an excellent answer, one that is built in to the kernel and has extremely low overhead. Oracle Solaris Zones allow you to create a virtual system that looks like a real, independent hardware platform to its users. This technology creates security separation and resource separation, and it allows workloads to be stopped, started, and worked on without interfering with any other groups on the system. It's secure and scalable, and it has excellent performance. (Oracle runs some public benchmarks using Oracle Solaris Zones.) And it's available at no extra cost.

There are even tools to support the management of Oracle Solaris Zones in a system and to allow zone cloning, the creation of snapshots, and zone migration, among other tasks. And these tools can be accessed from Oracle Enterprise Manager modules as well as from the command line.

Oracle Solaris Zones can be created with as few as one hardware thread—one virtual CPU, as they report themselves to the system—but realistically, they should include enough CPU resources to ensure the job gets done. The more zones you have in a system, the more planning and consideration should be given to whether there are enough resources for all of them. RAM is typically the biggest limiting factor, and it should be the first consideration when consolidating workloads into Oracle Solaris Zones.

Oracle Solaris Zones do, however, require that each zone share a kernel. While this is not a downside, it can have consequences in situations where consolidation is important. All Oracle Solaris Zones on a system will share the OS version and patches installed in the global zone. Non-global zones will not be able to control hardware devices or the kernel itself. These limitations might not fit into a highly consolidated environment, even with the flexibility and extremely low system overhead of Oracle Solaris Zones. Also, Oracle Solaris Zones cannot be migrated live. If the situation requires that different versions of Oracle Solaris be run on the same system, or there is a need for occasional live migrations of environments, that moves consideration to the next method.

On SPARC T-Series systems, the next option is Oracle VM Server for SPARC. Oracle Solaris Zones don't actually use a hypervisor, such as VMware, or other virtualization software, which is why they have such a low performance impact. Oracle VM Server for SPARC does have a hypervisor, but a lot of work has been put into minimizing it—it's not a full-blown Type 1 hypervisor, such as VMware. Thus, it still has low overhead. It's also somewhat more secure than a heavyweight hypervisor (since it's pretty minimal to begin with), and it is relatively easy to maintain.

Since Oracle VM Server for SPARC is hypervisor-based, it allows each instance on a system to run a different version of Oracle Solaris 10 or 11, which is perfect when different groups require different patch levels of the OS as well. Oracle VM Server for SPARC even supports the use of resource management and Oracle Solaris Zones within it (although there might be licensing issues with some software in the latter case—definitely ask.) CPU, memory, and storage resources can be added over time as required to account for changing workload requirements.

Oracle VM Server for SPARC requires more planning than Oracle Solaris Zones, especially in the area of capacity growth, because there are choices to be made about whether to virtualize all I/O and create duplicate I/O domains for reliability, and other similar considerations. But we'll stick to the high level here.

If different versions of the OS are needed, Oracle VM Server for SPARC is the choice for SPARC T-Series systems. Granularity goes to the hardware thread level, but for various reasons it's best to start out considering units of one or more cores for Oracle VM Server. (Without delving deeply into a topic for another article, consider that each core gives not just 8 or 16 threads for use in the virtual system, but also allows the associated FPU and Crypto units to be easily available to the virtual domain. This means that two Oracle VM for SPARC environments don't have to trade off scheduling threads on the same physical resources.)

For SPARC Enterprise M-Series systems the technology that is available is called Dynamic Domains. This technology provides a traditional hardware partitioning of the system. CPU modules and RAM and I/O resources are allocated by a system controller (not by a hypervisor), and a separate instance of the OS is placed on the newly created domain. Domains are relatively inflexible, although resources can be shifted around if the domains are properly planned and configured to do that. But domains really keep workloads electrically separate, with minimal sharing of resources (sharing is generally only done if an I/O board is split.) That capability is not something any software hypervisor can provide. What's the downside? Mostly cost and the relative lack of flexibility compared to software-based solutions. However, Oracle Solaris Zones and resource controls can be run inside Dynamic Domains, so you have an additional level of flexibility available when using Dynamic Domains.

Let's recap here, bearing in mind that our goal in this exercise is primarily consolidation. If possible, the easiest and least expensive solution for virtualization of workloads is to simply combine them on the same platform. This approach has the advantages of being highly flexible and easy to implement, and it takes little additional effort to maintain. The downsides are that there is less security between groups of users, which might violate internal requirements, and there is always the worry that patching or updating one package might interfere with the functioning of others on the system. Further, the extra security and increased agility of virtualization are not available in a "consolidation by stacking" environment.

Oracle Solaris Zones are for when resource and security controls in the OS are not enough to meet requirements or when additional business agility is desirable. They have very low overhead and strong security separation (they are even run in Multi-Level Security [MLS] environments), and they can share scarce hardware resources, look to the users like actual systems, and be booted independent of other zones. In addition, Oracle Solaris Zones can be moved around easily and can communicate internally at memory speeds, and there is a good selection of command line and GUI tools available to manage them. The downside is the kernel is shared by all zones, so the OS and its patch levels are shared, too. If you need different OS versions on a system, Oracle Solaris Zones and resource separation will not be enough.

In those cases, you'll want to look at hardware domaining: Oracle VM Server for SPARC on SPARC T-Series systems and Dynamic Domains on SPARC Enterprise M-Series systems. These technologies provide all the traditional advantages of hardware separation within a chassis (although Oracle VM Server for SPARC does have the ability to share within hardware elements, such as single CPUs and RAM). However, they can incur additional planning and configuration costs on SPARC T-Series systems and extra hardware costs on SPARC Enterprise M-Series systems. Oracle VM Server for SPARC in some configurations can also incur some overhead in I/O, but not to the level of a full Type 1 hypervisor such as VMware. Also, it has the valuable ability to migrate active Oracle VM Server for SPARC domains to another system while they are in use.

Finally—we now have a spectrum of use cases to move through quickly and easily:

  • Need to consolidate? Try resource management.
  • That's not enough (typically for security or business agility reasons)? Oracle Solaris Zones fit the vast majority of workload virtualization needs and should always be considered as the go-to option for Oracle Solaris virtualization (free, secure, easy, agile, performant—what's not to like?).
  • Consider Oracle VM Server for SPARC and Dynamic Domains as the primary option only after Oracle Solaris Zones technology has been ruled out—not because Oracle VM Server for SPARC and Dynamic Domains are bad, but because Oracle VM Server for SPARC is more work to set up and maintain and Dynamic Domains technology is potentially more costly in hardware. They do, however, offer additional flexibility over just stacking apps on a server.

In fact, all these virtualization scenarios offer more security, flexibility, and scalability options than do non-virtualized environments. That's an interesting observation that merits its own discussion.

If you would like more information on this topic, please see the excellent book Oracle Solaris 10 System Virtualization Essentials by Jeff Victor, Jeff Savit, Gary Combs, Simon Hayler, and Bob Netherton. While I used it as a reference, the above analysis is based on my own experiences in the field and any flaws in it are my own, not theirs.

About the Author

David Pipes specializes in systems hardware and software pre-sales and has been a Systems Consultant at Sun Microsystems and Oracle since 1999 supporting the US Department of Health and Human Services. Before that, he worked for various government and private sector companies as a systems administrator and problem-solver. He received a B.A. from Hampshire College in 1985.

Revision 1.0, 04/12/2012

Follow us on Facebook, Twitter, or Oracle Blogs.