by Thierry Manfé, with contributions from Orgad Kimchi, Maria Frendberg, and Mike Gerdts
Published September 2013
A growing number of companies are looking at virtualization to consolidate many physical servers on a single platform. As a general-purpose engineered system, Oracle SuperCluster has the following characteristics, which are required to address consolidation needs:
This article provides guidance, best practices, and hands-on instructions for using Oracle Solaris Zones to consolidate existing servers onto SuperCluster. It focuses on the operating system and virtualization layers, on the P2V migration process, and on the associated tools for facilitating this migration.
This article is intended to help system administrators, architects, and project managers who have some understanding of SuperCluster and want to familiarize themselves with P2V migration and evaluate the possibility of conducting such a transition.
SuperCluster offers a lot of flexibility and numerous options for virtualization and consolidation. The following sections discuss some questions that should help you to quickly identify the best options for consolidating your physical servers on SuperCluster.
Oracle SuperCluster T5-8 runs Oracle Solaris releases 10 and 11 natively. SPARC-based servers running these releases can be consolidated on SuperCluster using native Oracle Solaris Zones. SPARC servers running Oracle Solaris releases 8 and 9 can also be consolidated using
solaris9 branded zones.
Note: Native zones run the same Oracle Solaris release as the global zone that is hosting them. This is the preferred configuration for performance: a native zone uses an up-to-date Oracle Solaris release while a non-native zone runs older Oracle Solaris binaries that might not take advantage of the latest hardware platforms and processors. Native zones use the
native brand on Oracle Solaris 10 and the
solaris brand on Oracle Solaris 11.
Here are the required updates for each of these Oracle Solaris releases:
solaris9branded zones. Use
isainfo -bto check the number of bits (32 or 64) in the address space of the server to be consolidated.
libthreadmight experience some problems. See Chapter 8 of the System Administration Guide: Oracle Solaris 8 Containers for more details on
solaris8branded zones. Use
isainfo -bto check the number of bits (32 or 64) in the address space of the server to be consolidated.
Also look at Oracle Solaris Support Life Cycle Policy—referenced on this page—to check whether your Oracle Solaris release is supported.
Non-SPARC servers (such as x86 servers) cannot be consolidated on SuperCluster using P2V migration. However, applications using non-native code—such as Java, Python, and others—can be migrated from non-SPARC servers to SuperCluster.
Note: For consolidating an application that was developed with a compiled language (such as C, C++, or FORTRAN) and that runs on a non-SPARC server, the application must first be recompiled on SPARC, which means you need access to the source code.
Are you planning to consolidate a server running a business-critical application that you want to update with future releases over upcoming years, or are you trying to get rid of an old server running a legacy application that will not be updated anymore?
In the first scenario—long-term migration of a critical application—native Oracle Solaris 10 (that is,
native brand) zones and native Oracle Solaris 11 (that is,
solaris brand) zones are preferred, because they provide native performance and are maintained by the SuperCluster quarterly full stack download patch (QFSDP) delivered by Oracle.
Non-native zones—such as an Oracle Solaris 10 zone running in an Oracle Solaris 11 global zone—should be limited to applications and operating systems that are not planned to be updated on a regular basis and for which performance is not critical (though, even with non-native zones, migrating from obsolete hardware to SuperCluster can provide a performance improvement). The QFSDP does not apply to non-native zones. Patching non-native zones is the responsibility of the operator. Within these constraints, non-native zones offer the required flexibility for consolidating legacy servers running Oracle Solaris releases 8 and 9.
Oracle SuperCluster T5-8 offers two options for the zone's storage location:
The two main criteria to be considered when choosing between these options are I/O performance and manageability.
The first thing to check on the source server to be consolidated is whether the data that generates the I/O load is located on the root file system. If not—which is typically the case with data located on SAN storage—the data will likely not be transferred to the zonepath as part of the P2V migration, and the location of the resulting zone won't have much impact on I/O performance. In such a case, the zone should be installed on the Sun ZFS Storage 7320 appliance. If the data is located on the root file system, it will be transferred to the zonepath and the zone's location will impact I/O performance. In this case, and if a large number of zones have their zonepaths on the Sun ZFS Storage 7320 appliance, local HDDs can be a good alternative to dedicate I/O bandwidth to a limited number of zones.
If you want to migrate zones between the SuperCluster domains, installing zones on the Sun ZFS Storage 7320 appliance is the best option. Creating a dedicated ZFS pool (zpool) from an iSCSI LU exported by the Sun ZFS Storage 7320 appliance for each zone greatly simplifies the migration, because it becomes a matter of transferring the zpool from the source to the target domain using
zpool export and
zpool import. No data migration or copy is required.
Note: A SuperCluster domain is simply an Oracle VM Server for SPARC virtual machine (aka a logical domain). On SuperCluster, zones are hosted in domains.
In addition, one zpool per zone matches the default zone provisioning schema from Oracle Enterprise Manager Ops Center, and if the zpool is located on the Sun ZFS Storage 7320 appliance, Oracle Enterprise Manager Ops Center can be used to perform the migration between domains.
Finally, zones installed on the Sun ZFS Storage 7320 appliance immediately benefit from the high availability designed into the SuperCluster: the storage and the network access to it are fully redundant.
Oracle Solaris Zones resulting from a P2V consolidation must be hosted in Application Domains. The Sun ZFS Storage 7320 appliance should be preferred for installing zone; however, if you plan to install zones on local HDDs, the Application Domain must have spare HDDs for creating an extra zpool dedicated to zones. For SuperCluster configurations with one or two domains per SPARC T-Series server from Oracle, the Application Domains have enough spare HDDs. For configurations with more than two domains per SPARC T-Series server, the
/u01 file system can be used instead of an extra zpool.
Oracle SuperCluster T5-8 comes with a set of logical domains (Oracle VM Server for SPARC virtual machines). These domains are configured on the SPARC T5-8 servers during the initial configuration of SuperCluster and they can be of two different types:
Oracle Solaris Zones resulting from a P2V consolidation must be hosted in Application Domains.
Note: P2V zones are not supported in Database Domains because all the computing resources in these domains are dedicated to Oracle Database 11g Release 2 to deliver the expected performance on the database side.
If the server to be consolidated runs Oracle Solaris release 8 or 9, you need an Application Domain that boots Oracle Solaris 10. If it runs Oracle Solaris 11, you need an Application Domain that boots Oracle Solaris 11. If it runs Oracle Solaris 10, an Application Domain that boots Oracle Solaris 10 is preferred; however, an Oracle Solaris 11 domain remains a valid option.
If you plan to install zones on the Sun ZFS Storage 7320 appliance, you can skip the rest of this section.
If you plan to install zones on local HDDs, the domain must have spare HDDs to create an extra ZFS pool dedicated to zones. This ZFS pool must be mirrored to provide redundancy. When a SPARC T-Series server hosts more than two domains, some of them do not have enough spare HDDs to create this extra ZFS pool. In this case, the
/u01 file system available in the domain can be used to install zones.
Application Domains running Oracle Solaris 11 offer the highest flexibility for zones' network configuration. Virtual network interfaces and InfiniBand (IB) interfaces can be created at will and dedicated to zones. Each zone can be connected to multiple VLANs, enabling a seamless integration with the data center network infrastructure. Each zone can also be connected to multiple IB partitions. In each zone, IP Network Multipathing (IPMP) can be configured on each VLAN and IB partition to provide network redundancy.
Oracle SuperCluster T5-8 comes with three networks:
This section focuses on connecting the zones to the client access network. If need be, and using the feature described for the client access network, zones can also be connected to the InfiniBand network. Typically, a zone that hosts an application that uses an Oracle Database 11g Release 2 instance is connected to InfiniBand.
By default, zones are connected to the management network. VLAN tagging can be used to improve network-based segregation between zones.
Each Application Domain is provisioned with a minimum of two network interfaces connected to the client access network that can be used for zones.
If network redundancy is required, IPMP can be used. With shared-IP zones, IPMP is set up in the domain's global zone and the different zones can benefit from it. With exclusive-IP zones, IPMP is set up in each zone as it would be in the global zone (as described for Oracle Solaris releases 10 and 11).
Note: Shared-IP zones share the network interface and the IP stack. The separation between zones is implemented in the IP stack. This type is useful when there is a shortage of network interfaces available for zones. With shared-IP zones, the network configuration is achieved in the global zone. Exclusive-IP zones have an exclusive access to the network interface and to a dedicated IP stack. This type is preferred when strong network segregation between zones is required. Some applications might also require the use of an exclusive-IP stack.
Configuring IPMP in an exclusive-IP zone hosted in an Application Domain running Oracle Solaris 10 requires two network interfaces (NICs) that are not already in use in the global zone or in another zone, but the number of NICs per domain can be limited to two. In this case, and if many zones must run in the same domain, VLAN tagging can be used to create more NIC instances in the global zone. These additional instances can be used in exclusive-IP zones. For more details on how to use VLAN tagging with exclusive-IP zones on Oracle Solaris 10, check the following blog: Solaris 10 Zones and Networking—Common Considerations.
Configuring IPMP in an exclusive-IP zone hosted in an Application Domain running Oracle Solaris 11 puts less constraint on network interfaces. If the NICs are already in use or must be shared, it is possible to create Oracle Solaris virtual network interfaces (VNICs) that can be used by exclusive-IP zones. VNICs are created on top of NICs, from the Application Domain, without stopping it. Access to the control domain is not required.
NICs can be listed using
dladm show-link from an Application Domain running Oracle Solaris 10:
# dladm show-link vnet0 type: non-vlan mtu: 1500 device: vnet0 ixgbe0 type: non-vlan mtu: 1500 device: ixgbe0 ixgbe1 type: non-vlan mtu: 1500 device: ixgbe1 ibd2 type: non-vlan mtu: 65520 device: ibd2 ibd0 type: non-vlan mtu: 65520 device: ibd0 ibd1 type: non-vlan mtu: 65520 device: ibd1
Here, one vnet (
vnet0), two NICs (
ixgbe1), and three InfiniBand interfaces are available in the domain. A NIC that is not used by the global zone does not appear in the
ifconfig -a output, which is shown in Listing 1:
# ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 ixgbe0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 10.129.184.66 netmask ffffff00 broadcast 10.129.184.255 ether 0:1b:21:c8:5e:b0 ixgbe1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 7 inet 0.0.0.0 netmask 0 ether 0:1b:21:c8:5e:b1 vnet0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 8 inet 10.129.183.119 netmask ffffff00 broadcast 10.129.183.255 ether 0:14:4f:f8:4e:3
VLAN setup in an Application Domain running Oracle Solaris 11 on the 10-GbE client access network is straightforward for both Oracle Solaris release 10 and 11 zones. Use an exclusive-IP zone, and create virtual network interfaces (VNICs) in the global zone on top of the required 10-GbE NIC using the
dladm create-vnic command and using the
-v option to set the
vlan-id attribute. From there, all the packets leaving the zone through the VNIC are tagged with
vlan-id. Also, as soon as a VNIC is created, a virtual switch is instantiated in the global zone, which ensures packet filtering: only the packets tagged with
vlan-id are forwarded to the zone through the VNIC. The virtual switch also guarantees that two zones running in the same Application Domain but connected to different VLANs cannot communicate with each other.
Similarly, for Application Domains running Oracle Solaris 10, VLAN-tagged interfaces are created in the global zone and assigned to an exclusive-IP zone. For Oracle Solaris 10, more details can be found on this blog.
Oracle recognizes capped Oracle Solaris Zones as licensable entities, known as hard partitions. This means that zones created as part of the P2V migration process can be capped to optimize the cost of your Oracle software license.
To create a zone that fits the licensing requirements set by Oracle, you first need to create a resource pool with the desired number of cores and bind the zone to this resource pool.
First create and edit a
licensePool.cmd file as follows:
create pset license-pset ( uint pset.min = 16; uint pset.max = 16 ) create pool license-pool associate pool license-pool ( pset license-pset )
The first line defines a processor set with sixteen virtual CPUs. On Oracle SuperCluster T5-8, since each core has eight virtual CPUs, this is equivalent to two cores. As a result, the actual number of cores considered for the licensing is two. It is worth noting that the number of virtual CPUs in the processor set should always be a multiple of eight.
When you are done editing the file, create the
# pooladm -s # poolcfg -f licensePool.cmd # pooladm -c
The new resource pool configuration can be checked using
pooladm, as shown in Listing 2:
# pooladm # ... # pool license-pool int pool.sys_id 2 boolean pool.active true boolean pool.default false int pool.importance 1 string pool.comment pset license-pset pset license-pset int pset.sys_id 1 boolean pset.default false uint pset.min 16 uint pset.max 16 string pset.units population uint pset.load 31 uint pset.size 16 string pset.comment ...
Modify the zone configuration using the
zonecfg command and add the following attribute to the zone:
pool=license-pool. The zone is now using the
Finally, reboot the zone. You can now connect to it and check the number of CPUs that are visible using the
Note that many zones can use the
license-pool cores without increasing the license cost, which remains based on two cores. All the zones associated to
license-pool share the two cores.
This section goes through the main steps of a P2V migration to SuperCluster. The example given is the consolidation of an Oracle Solaris 10 server into an Oracle Solaris 10 zone in an Application Domain running Oracle Solaris 10.
Note: General information about P2V migration—outside of the SuperCluster context—can be found in the Oracle Solaris Administration guide.
zonep2vchk is a tool that helps in assessing the source server before performing a P2V migration. It provides a good overview of the modifications between the source server and the target zone as a result of the P2V migration process. It is useful for identifying potential problems and performing remediation ahead of the migration.
zonep2vchk is shipped with Oracle Solaris 11 and—being a script—it can be directly copied and executed on an Oracle Solaris 10 system.
zonep2vchk does not run on Oracle Solaris release 8 or 9.
zonep2vchk should be first executed with the single
-T option that specifies the Oracle Solaris release in the domain that will host the zone (Oracle Solaris 10, in our example), as shown in Listing 3. In that mode, the tool lists services running on the server that are not available in a zone, identifies existing zones and additional zpools that won't be migrated, identifies any NFS-share file system that can't be shared in an Oracle Solaris 10 zone, and performs many other checks:
# ./zonep2vchk -T S10 - Source System: t5240-250-01 Solaris Version: Oracle Solaris 10 8/11 s10s_u10wos_17b SPARC Solaris Kernel: 5.10 Generic_147440-01 Platform: sun4v SUNW,T5240 - Target System: Solaris_Version: Solaris 10 Zone Brand: native (default) IP type: shared --Executing basic checks - The system is sharing file systems with NFS. This is not possible in the destination zone. The shares should be evaluated to determine if they are necessary. If so, this system may not be suitable for consolidation into a zone, or an alternate approach for supporting the shares will need to be used, such as sharing via the global zone or another host. Use "zonep2vchk -P" to get a list of the shared filesystems. - The following SMF services will not work in a zone: svc:/network/iscsi/initiator:default svc:/network/nfs/server:default svc:/system/iscsitgt:default - The following zones will be unusable. Each zone should be migrated separately to the target host using detach and attach. See zoneadm(1M), solaris(5) and solaris10(5): Zone State s10zone running - The following SMF services require ip-type "exclusive" to work in a zone. If they are needed to support communication after migrating to a shared-IP zone, configure them in the destination system's global zone instead: svc:/network/ipsec/ipsecalgs:default svc:/network/ipsec/policy:default svc:/network/routing-setup:default - The system is configured with the following non-root ZFS pools. Pools cannot be configured inside a zone, but a zone can be configured to use a pool that was set up in the global zone: cpool - When migrating to an exclusive-IP zone, the target system must have an available physical interface for each of the following source system interfaces: nxge0 - When migrating to an exclusive-IP zone, interface name changes may impact the following configuration files: /etc/hostname.nxge0 Basic checks compete, 11 issue(s) detected
Executed with the
zonep2vchk performs runtime checks and detects programs that use privileges not available in zones. In the example shown in Listing 4, the runtime check is performed for only five minutes. A longer period is recommended when the server is hosting a real application:
# ./zonep2vchk -r 5m -T S10 - Source System: t5240-250-01 Solaris Version: Oracle Solaris 10 8/11 s10s_u10wos_17b SPARC Solaris Kernel: 5.10 Generic_147440-01 Platform: sun4v SUNW,T5240 - Target System: Solaris_Version: Solaris 10 Zone Brand: native (default) IP type: shared --Executing run-time checks for 5m - The following programs were found using privileges that cannot be added to a zone. The use of these privileges may be related to the program command line options or configuration. Program Disallowed Privilege /usr/lib/fm/fmd/fmd sys_config Run-time checks complete, 1 issue(s) detected
Executed with the
zonep2vchk performs a static binary analysis of the file system or directory specified on the command line. It detects binaries that are statically linked, because they cannot execute in zones. In the example shown in Listing 5, the check is performed on the root directory; however, when the source server is hosting an application, it makes more sense to specify the application's home directory:
# ./zonep2vchk -s / - Source System: t5240-250-01 Solaris Version: Oracle Solaris 10 8/11 s10s_u10wos_17b SPARC Solaris Kernel: 5.10 Generic_147440-01 Platform: sun4v SUNW,T5240 - Target System: Solaris_Version: Solaris 10 Zone Brand: native (default) IP type: shared --Executing static binary checks Static binary checks compete, 0 issue(s) detected
Finally, executed with
zonep2vchk generates a template file based on the source server configuration, which can be used when configuring the zone on SuperCluster:
# ./zonep2vchk -c create -b set zonepath=/zones/t5240-250-01 add attr set name="zonep2vchk-info" set type=string set value="p2v of host t5240-250-01" end set ip-type=shared # Uncomment the following to retain original host hostid: # set hostid=853c66cc # Max lwps based on max_uproc/v_proc set max-lwps=40000 add attr set name=num-cpus set type=string set value="original system had 128 cpus" end # Only one of dedicated or capped cpu can be used. # Uncomment the following to use cpu caps: # add capped-cpu # set ncpus=128.0 # end # Uncomment the following to use dedicated cpu: # add dedicated-cpu # set ncpus=128 # end # Uncomment the following to use memory caps. # Values based on physical memory plus swap devices: # add capped-memory # set physical=65312M # set swap=69426M # end # Original nxge0 interface configuration: # Statically defined 10.140.250.120 (t5240-250-01) # Factory assigned MAC address 0:21:28:3c:66:cc add net set address=t5240-250-01 set physical=change-me end exit
This step consists of storing the data of the source server in a single Flash Archive (FLAR)file. This is accomplished using a single command:
# /usr/sbin/flarcreate -S -n s10P2V -L cpio /var/tmp/s10P2V.flar
flarcreate command creates an
s10server.flar image file. The
cpio options enforce the use of the CPIO format for the archive. This format should be used when the source system has a ZFS root file system. The
-S option skips a disk-space checking step for a faster process, and
-n specifies the internal name of the FLAR image file. You can use the
-x option to exclude a file, a directory, or a ZFS pool from the FLAR image file.
flarcreate is not available on Oracle Solaris 11. A ZFS stream from the global zone's rpool is used instead of a FLAR image file. More information can be found here.
If the source server has multiple boot environments (BEs), only the active one is included in the FLAR image file. With an Oracle Solaris 10 native zone, the different BEs can be listed but only one can be used: the active one. With an Oracle Solaris10 branded zone (running in an Oracle Solaris 11 global zone), the live-upgrade related commands are not available and the different BEs cannot be listed. As much as possible, BEs should be deleted from the source server before the P2V migration.
The FLAR image file is now ready to be copied to SuperCluster.
Before performing this step, you must decide whether you want to install the zone on local HDDs or on the Sun ZFS Storage 7320 appliance.
If you are installing the zone on local HDDs, check whether the SuperCluster configuration includes more than two domains per SPARC T-Series server. If there are more than two domains the
/u01 file-system remains an option for installing the zone on local HDDs.
The following example is based on a configuration with one Database Domain and one Application Domain per SPARC T-Series server.
format to identify the disks in the domain, as shown in Listing 7:
# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t5000C500429EFABFd0 <SUN600G cyl 64986 alt 2 hd 27 sec 668> solaris /scsi_vhci/disk@g5000c500429efabf 1. c1t5000C500429F6D87d0 <SUN600G cyl 64986 alt 2 hd 27 sec 668> solaris /scsi_vhci/disk@g5000c500429f6d87 2. c1t50015179596885CBd0 <ATA-INTELSSDSA2BZ30-0362 cyl 35769 alt 2 hd 128 sec 128> solaris /scsi_vhci/disk@g50015179596885cb 3. c1t50015179596667E2d0 <ATA-INTEL SSDSA2BZ30-0362-279.46GB> /scsi_vhci/disk@g50015179596667e2
In this case, we have two HDDs and two SSDs. Each HDD is divided in two slices:
s0 slices are already in use by the rpool, which can be checked with
zpool status, as shown in Listing 8:
# zpool status BIrpool-1 pool: BIrpool-1 state: ONLINE scan: resilvered 8.93G in 0h2m with 0 errors on Thu May 17 17:02:12 2012 config: NAME STATE READ WRITE CKSUM BIrpool-1 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t5000C500429F6D87d0s0 ONLINE 0 0 0 c1t5000C500429EFABFd0s0 ONLINE 0 0 0
s1 slices can be used to create an
s10p2vpool pool dedicated to the zone:
# zpool create s10p2vpool mirror c1t5000C500429F6D87d0s1 c1t5000C500429EFABFd0s1
Note: If SuperCluster is configured with more than two domains per server node, the
zpool create command can abort with a message saying that it cannot open one of the slices because the device is busy. It is likely that the slice is shared by the domain virtual disk server. You can check this by connecting to the control domain and running
ldm list-domain -l on the Application Domain. The busy device should appear in the VDS section of the output. The solution is to use other slices to create the ZFS pool. This example uses the SSDs.
To create the pool on the Sun ZFS Storage 7320 appliance, perform the following steps:
# iscsiadm add static-config iqn.1986-03.com.sun:02:847ad5ff-eff5-4bd7-8310-999756b3d568,192.168.30.5:3260
iqn.1986-03.com.sun:...is the target number associated to the LUN and collected in Step 1.
192.168.30.5is the IP address of the Sun ZFS Storage 7320 appliance on the InfiniBand network.
3260is the standard iSCSI port.
# iscsiadm modify discovery -s enable
# iscsiadm list target -S iqn.1986-03.com.sun:02:78af61c2-953 Target: iqn.1986-03.com.sun:02:847ad5ff-eff5-4bd7-8310-999756b3d568 Alias: - TPGT: 1 ISID: 4000002a0000 Connections: 1 LUN: 0 Vendor: SUN Product: COMSTAR OS Device Name: /dev/rdsk/c0t600144F000212834B1BE50A60A010001d0s2
# zpool create s10p2vpool c0t600144F000212834B1BE50A60A010001d0
Once the pool is created, create a ZFS file system on it for the zonepath. Before actually installing the zone, turn on ZFS compression. For optimal performance, also consider adjusting the ZFS
recordsize (see the "Performance Tuning" section):
# zfs create s10p2vpool/s10P2V # zfs set compression=on s10p2vpool/s10P2V # chmod 700 /s10p2vpool/s10P2V/
Configure the zone using
zonecfg with the configuration file created with
zonep2vchk -c. In the configuration file,
zonepath is set to
ip-type is set to exclusive, and
net physical is set to
# zonecfg -z s10P2V -f s10P2V.cfg
When the zone is configured, install it using the
zoneadm command and the FLAR image file. If you want to use the same OS configuration as the source server—including the same IP address—use the
-p option. Be aware that this can create an address conflict: the source server should be shut down or its IP address should be modified before booting the zone. If you want to use a different configuration, use the
-u option instead, which unconfigures the zone upon install:
# zoneadm -z s10P2V install -a /s10P2V.flar -u cannot create ZFS dataset nfspool/s10P2V: dataset already exists Log File: /var/tmp/s10P2V.install_log.AzaGfB Installing: This may take several minutes... Postprocessing: This may take a while... Postprocess: Updating the zone software to match the global zone... Postprocess: Zone software update complete Postprocess: Updating the image to run within a zone Result: Installation completed successfully. Log File: /nfspool/s10P2V/root/var/log/s10P2V.install13814.log
At this point, the zone is ready to be booted but its Oracle Solaris instance is not configured. If the zone is booted as such, connect to its console using
zlogin -C in order to provide the configuration parameters interactively. Alternatively, you can copy a
sysidcfg file to the zonepath to avoid interactive configuration:
# cp sysidcfg /nfspool/s10P2V/root/etc/sysidcfg # zoneadm -z s10P2V boot
Then you can connect to the zone:
# zlogin s10P2V
This section focuses on tuning the I/O performance of a zone resulting from a P2V migration. From a CPU and network point of view, a zone behaves like any other Oracle Solaris image, so there is no zone-specific tuning to be performed. However, on the I/O side, because a zone sits on a file system, performance can benefit from file system tuning.
In the case of a P2V migration to SuperCluster, the most important parameters for I/O performance are the ZFS
compression for the zonepath and—if the zone is located on the Sun ZFS Storage 7320 appliance—the NFS
If the source server hosts a database or an application that performs fixed-size access to files, the ZFS
recordsize should be tuned to match this size. For example, if it is hosting a database with a 4k record size, set the ZFS
recordsize to 4k.
Regardless of whether the zonepath is located on a local HDD or on the Sun ZFS Storage 7320 appliance, ZFS compression improves performance for synchronous I/O operations. It also improves asynchronous I/O operations when the zonepath is on the Sun ZFS Storage 7320 appliance. For these types of workloads, the recommendation is to set the zonepath's
on. The improvement is not that important for asynchronous I/O operations with the zonepath on local HDDs.
This section describes a P2V migration from an Oracle Solaris 8 server running Oracle Database 10.2.0.5 to an Oracle Solaris 8 zone hosted in an Application Domain running Oracle Solaris 10. The database data is located on attached storage connected through Fibre Channel on which an Oracle Automatic Storage Management file system has been created.
On the source system, as user
oracle and before creating the FLAR image file, stop the database listener and the Oracle Automatic Storage Management instances, as shown in Listing 9:
$ sqlplus "/as sysdba" SQL*Plus: Release 10.2.0.5.0 - Production on Sun Aug 26 13:19:48 2012 Copyright (c) 1982, 2010, Oracle. All Rights Reserved. Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL> shutdown immediate $ export ORACLE_SID=+ASM $ sqlplus "/as sysdba" SQL*Plus: Release 10.2.0.5.0 - Production on Sun Aug 26 13:21:38 2012 Copyright (c) 1982, 2010, Oracle. All Rights Reserved. Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL> shutdown ASM diskgroups dismounted ASM instance shutdown $ lsnrctl stop LSNRCTL for Solaris: Version 10.2.0.5.0 - Production on 26-AUG-2012 13:23:49 Copyright (c) 1991, 2010, Oracle. All rights reserved. Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521)) The command completed successfully
Once the database and Oracle Automatic Storage Management are stopped, create the FLAR image file as
root and copy it to the Application Domain. In the following example,
-S specifies that disk-space checking is skipped and that the archive size is not written to the archive, which significantly reduces the archive creation time. The
-n option specifies the image name, and
-L specifies the archive format.
# flarcreate -S -n s8-system -L cpio /var/tmp/s8-system.flar
At this point, move the SAN storage and connect it to the SPARC T-Series server. Then, from the control domain, make the LUN (
/dev/dsk/c5t40d0s6) available in Application Domain
# ldm add-vdsdev /dev/dsk/c5t40d0s6 oradata@primary-vds0 # ldm add-vdisk oradata oradata@primary-vds0 s10u10-EIS2-1
Note: LUNs often appear with different names on different servers.
In the Application Domain, the LUN is now visible:
# format Searching for disks...done AVAILABLE DISK SELECTIONS: ... 1. c0d2 <SUN-DiskSlice-408GB cyl 52483 alt 2 hd 64 sec 255> /virtual-devices@100/channel-devices@200/disk@2
Now you can create the zone.
Note: Prior to creating the zone, check that the Oracle Solaris Legacy Containers software is installed in the domain.
solaris8 branded zone (
zonecfg. Listing 10 shows the output of
zonecfg -z s8-10gr2 info after the configuration is complete:
zonename: s8-10gr2 zonepath: /cpool/s8-10gr2 brand: solaris8 autoboot: true bootargs: pool: limitpriv: default,proc_priocntl,proc_lock_memory scheduling-class: FSS ip-type: exclusive hostid: net: address not specified physical: ixgbe1 defrouter not specified device match: /dev/rdsk/c0d2s0 attr: name: machine type: string value: sun4u
Still in the Application Domain, install the zone and boot it using
-p, the configuration of the Oracle Solaris 8 image is preserved, and
-a specifies the archive location:
# zoneadm -z s8-10gr2 install -p -a /var/temp/s8-system.flar ... # zoneadm -z s8-10gr2 boot
Now that the zone is booted, it is possible to connect to it using
zlogin s8-10gr2. As
root, change the ownership of the raw device and as
oracle, start Oracle Automatic Storage Management and the database, as shown in Listing 11:
# chown oracle:dba /dev/rdsk/c0d2s0 # su - oracle $ lsnrctl start ... $ export ORACLE_SID=+ASM $ sqlplus / as sysdba SQL*Plus: Release 10.2.0.5.0 - Production on Sun Aug 26 14:36:44 2012 Copyright (c) 1982, 2010, Oracle. All Rights Reserved. Connected to an idle instance. SQL> startup ASM instance started Total System Global Area 130023424 bytes Fixed Size 2050360 bytes Variable Size 102807240 bytes ASM Cache 25165824 bytes ASM diskgroups mounted SQL> quit Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options $ export ORACLE_SID=ORA10 $ sqlplus / as sysdba SQL*Plus: Release 10.2.0.5.0 - Production on Sun Aug 26 14:37:13 2012 Copyright (c) 1982, 2010, Oracle. All Rights Reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1610612736 bytes Fixed Size 2052448 bytes Variable Size 385879712 bytes Database Buffers 1207959552 bytes Redo Buffers 14721024 bytes Database mounted. Database opened.
Oracle Solaris Zones are fully supported and integrated with Oracle SuperCluster T5-8. In addition, the P2V migration tools provided with Oracle Solaris greatly simplify the consolidation of physical servers to virtual machines on Oracle SuperCluster T5-8.
As an engineered system, SuperCluster offers a lot of flexibility in terms of configuration: Oracle Solaris Zones are the perfect receptacle for P2V migration. They provide a strong segregation—including network segregation—and can be used to optimize the licensing cost of the platform. Meanwhile, with Oracle Solaris Zones, the virtualization overhead is minimized.
Native Oracle Solaris Zones are patched and updated by the quarterly full stack downloadable patch for SuperCluster.
Oracle's Sun ZFS Storage 7320 appliance, which is included in SuperCluster, provides a large amount of redundant storage for Oracle Solaris Zones. Once installed on this shared storage, zones can be swiftly migrated between the different domains and computing nodes of SuperCluster. Oracle Solaris Zones can be connected to the 10-GbE client access network and to the InfiniBand I/O fabric. Network redundancy is available through IP Network Multipathing, and VLANs are available for a seamless integration in the existing data center network.
The highly scalable computing resources of Oracle SuperCluster T5-8 ensure that many Oracle Solaris Zones can run on this platform, while the powerful database can sustain the load of many applications running concurrently.
All these integrated features make Oracle SuperCluster T5-8 the platform of choice for server consolidation.
Thierry Manfé has been working at Oracle and Sun Microsystems for more than 15 years. He currently holds the position of principal engineer in the Oracle SuperCluster Engineering group.
|Revision 1.0, 09/11/2013|