The Logical Domains Manager on the source machine accepts the request to migrate a domain and establishes a secure network connection with the Logical Domains Manager that runs on the target machine. After this connection has been established, the migration occurs. The migration itself can be broken down into five different phases:
Figure 3. Live Migration Phase 1
The source machine begins to track which memory blocks of the active guest domain change during the migration operation. If the domain to be migrated is inactive or bound, the migration operation proceeds to Phase 5.
Figure 4. Live Migration Phase 5
The following examples use the
ldm migrate-domain command to migrate a guest domain from one system to another system.
You can optionally specify an alternate user for authentication on the target machine. See "Assign an Authorization to a User" in the Oracle VM Server for SPARC 2.1 Administration Guide of the documentation.
ldm migrate-domain -n command enables you to perform a dry-run migration, which performs migration checks but does not migrate the specified domain. Any requirement that is not satisfied is reported as an error, which enables you to correct any configuration errors before attempting an actual migration.
The following examples show various ways in which to perform domain migrations:
ldg1domain to a target machine called
# ldm migrate-domain -n ldg1 t4-1 Target Password: password
ldg1domain to a target machine called
# ldm migrate-domain ldg1 t4-1 Target Password: password
ldg1guest domain to a target machine called
-poption specifies the name of a file,
pfile, that contains the plain-text superuser password for the target machine, which enables you to access the
# ldm migrate-domain -p pfile ldg1 t4-1
Note: Ensure that you protect the file that contains the superuser password so that only the root owner, or a privileged user, can read it.
# chmod 400 pfile
ldg1domain to a target machine called
t4-1and specify an alternate user,
tracy, to be used for authentication on the
# ldm migrate-domain ldg1 tracy@t4-1
For information about assigning an authorization to a user, see "Managing User Profiles" in the Oracle VM Server for SPARC 2.1 Administration Guide.
# ldm list -o status ldg1
Note: The migration time might vary depending on the amount of memory assigned and the network utilization.
The SwingBench kit contains a load generator and benchmarks that are designed to stress test an Oracle Database. The code that ships with SwingBench includes the OrderEntry, SalesHistory, CallingCircle, and StressTest benchmarks.
For this paper, the SwingBench OrderEntry workload was used to generate workload on the guest domain being migrated. OrderEntry is based on the
oe schema that ships with Oracle Database 11g. The workload introduces heavy contention on a small number of tables, and it is designed to stress the system interconnects and memory.
The following describes the workload that is being generated on the guest domain:
sga_target): 18 GB
The system global area (SGA) is a group of shared memory structures that contain data and control information for one Oracle Database instance.
users) for workload: 50 users
Table 1 shows the results of live migration operations that ran between two SPARC T4-1 servers.Table 1. SPARC T4-1 Server Live Migration Results
|Number of CPUs on the Control Domain||Overall Migration Time||Suspension Time||Guest Domain CPU Usage|
|8 CPUs||8 min 12 sec||26 sec||70.00%|
|16 CPUs||4 min 2 sec||13 sec||80.00%|
|24 CPUs||2 min 3 sec||7 sec||85.00%|
The following describes the live migration operation behavior that is typically seen when migrating a domain from one machine to another machine. Note that you might observe slightly different results because the migration time depends on the number of CPUs and the amount of memory assigned to the control domain, as well as the workload type.
mpstatcommand to observe the effect of system memory tracking on the guest domain's CPU. See the
USRcolumn to view the usage for each CPU.
db_block_changesinformation in the Instance Activity Stats section of the generated AWR report. For more information about the AWR and AWR reports, see Section 5.2 ("Overview of the Automatic Workload Repository") in the Oracle Database Performance Tuning Guide.
This section contains information that is relevant to the Oracle VM for SPARC Live Migration feature, including configuring NTP and using CPU dynamic reconfiguration.
Ensure that the system clock on the domain to be migrated is synchronized by using NTP.
In this example, the control domain on the source machine is used as a time source, and it configured as an NTP server. Note that it is best to select an NTP server that can be a dedicated time synchronization source, so that other services are not negatively affected if the machine is brought down for planned maintenance.
The example in Listing 4 shows how to configure an NTP server.Listing 4. Configuring an NTP Server
# grep -v ^# /etc/inet/ntp.conf server 127.127.1.0 prefer broadcast 22.214.171.124 ttl 4 enable auth monitor driftfile /var/ntp/ntp.drift statsdir /var/ntp/ntpstats/ filegen peerstats file peerstats type day enable filegen loopstats file loopstats type day enable filegen clockstats file clockstats type day enable keys /etc/inet/ntp.keys trustedkey 0 requestkey 0 controlkey 0 # touch /var/ntp/ntp.drift # svcadm enable ntp
The following example shows how to configure an NTP client.
# grep -v ^# /etc/inet/ntp.conf server source prefer slewalways yes disable pll # svcadm enable ntp
Oracle VM Server for SPARC supports the dynamic reconfiguration of CPUs. CPUs can be dynamically added or removed from any active domain, including the control domain, by using the
ldm add-vcpu or
ldm rm-vcpu commands.
These commands can be issued from the control domain to increase or decrease the number of allocated CPUs. By increasing the number of CPUs on the control domain, migration time can be decreased. For systems that have discrete cryptographic units, such as the UltraSPARC T2, UltraSPARC T2 Plus, and SPARC T3 platforms, you should add cryptographic units to the control domain as you dynamically add CPU resources.
The following example shows how to add eight CPUs to the control domain:
primary# ldm add-vcpu 8 primary
After the migration operation is completed, you can remove the virtual CPUs from the control domain and allocate them to a different domain. The following example removes eight CPUs from the control domain and allocates them to the
ldg1 guest domain:
primary# ldm rm-vcpu 8 primary primary# ldm add-vcpu 8 ldg1
Note: When reducing the number of CPUs, ensure that a sufficient number of CPUs are still available to efficiently handle the domain's workload. For instance, an Oracle Database single-instance guest domain should always have at least eight CPUs.
An IT organization can reach tactical and strategic goals by choosing the right hardware and software to best manage application availability and to optimize data center resources.
This paper demonstrated the benefits of using the Oracle VM Server for SPARC 2.1 Live Migration feature to manage the production lifecycle of an Oracle Database 11g Release 2 single-instance database. This paper showed the complete configuration process, including the creation and configuration of the domains, storage configuration, and software that were used to demonstrate these benefits. In addition to showing the benefits of the Live Migration feature, this paper described how to achieve better application availability during planned downtime and how to reduce hardware and software costs by means of dynamic hardware resource management in a production environment.
For more information about Oracle's virtualization solutions, visit Oracle's virtualization Web site.
Orgad Kimchi joined Sun/Oracle in September 2007. He currently works in the Independent Software Vendors (ISV) Engineering organization to assist software vendors in adopting Oracle technology that improves the performance of Oracle hardware and software. See Orgad's blog.
Roman Ivanov joined Sun/Oracle in January 2006. He currently works in the ISV Engineering organization to assist software vendors in adopting Oracle technology that improves the performance of Oracle hardware. See Roman's blog.
Here are URLs for the resources referenced earlier in this document:
|Revision 1.0, 02/23/2012|