by Anderson Souza
Published July 2013
This article is Part 1 of a seven-part series that provides best practices and recommendations for configuring VMware vSphere 5.x with Oracle ZFS Storage Appliance to reach optimal I/O performance and throughput. The best practices and recommendations highlight configuration and tuning options for Fibre Channel, NFS, and iSCSI protocols.
The series also includes recommendations for the correct design of network infrastructure for VMware cluster and multi-pool configurations, as well as the recommended data layout for virtual machines. In addition, the series demonstrates the use of VMware linked clone technology with Oracle ZFS Storage Appliance.
All the articles in this series can be found here:
Note: For a white paper on this topic, see the Sun NAS Storage Documentation page.
The Oracle ZFS Storage Appliance product line combines industry-leading Oracle integration, management simplicity, and performance with an innovative storage architecture and unparalleled ease of deployment and use. For more information, see the Oracle ZFS Storage Appliance Website and the resources listed in the "See Also" section at the end of this article.
Note: References to Sun ZFS Storage Appliance, Sun ZFS Storage 7000, and ZFS Storage Appliance all refer to the same family of Oracle ZFS Storage Appliances.
Table 1, Table 2, and Table 3 describe the hardware configuration, operating systems, and software releases used in the reference architecture described in this series of articles.Table 1. Hardware Used in Reference Architecture
|Storage||1 cluster (2 controllers)||Oracle ZFS Storage 7420 cluster |
256 GB DRAM per controller
2 x 512GB read cache SSD per controller
2 x 20 2TB SAS-2 disk trays
2 x dual-port 10GbE NIC
2 x dual-port 8Gbps FC HBA
2 x 17GB log device
|Network||2||10GbE network switch|
|Server||2||Sun Server X3-2 from Oracle |
2 internal HDDs
1 x dual-port 10GbE NIC
1 x dual-port 8Gbps FC HBA
|Microsoft Windows 2008 R2 (x64)||1||Microsoft Exchange Server|
|Oracle Linux 6.2||1||ORION: Oracle I/O Numbers Calibration Tool|
|Oracle ZFS Storage Appliance's Appliance Kit (AK) software||2011.04.24.4.0,1-1.21|
|Microsoft Exchange Server Jetstress verification tool||2010 (x64)|
|ORION: Oracle I/O Numbers Calibration Tool||220.127.116.11.0|
|VMware vCenter Server||5.1.0 (Build 880146)|
Virtual desktop infrastructures produce high random I/O patterns and need high storage performance as well as availability, low latency, and fast response time. To meet these demands, use a mirrored data profile. This configuration duplicates copies as well as produces fast and reliable storage by dividing access and redundancy, usually between two sets of disks. In combination with write SSDs' log devices and the Sun ZFS Storage Appliance architecture, this profile can produce a large amount of input/output operations per second (IOPS) to attend to the demand of critical virtual desktop environments.
The recommended minimum disk storage configuration for VMware vSphere 5.x includes:
Note: The example shown in Figure 1 through Figure 3 demonstrates 44 x 3TB SAS-2 7200 RPM disks.
Figure 1. Oracle ZFS Storage Appliance—disk pools configuration
Figure 2. Oracle ZFS Storage Appliance—log profile configuration
Figure 3. Oracle ZFS Storage Appliance—cache profile configuration
Note: For high availability and proper load balancing for a virtual desktop infrastructure, use an Oracle ZFS Storage Appliance model that supports clustering. Configure the cluster in active/active mode and use Oracle ZFS Storage Appliance software release 2011.1.4.2.x or greater.
The Oracle ZFS Storage Appliance software releases can be downloaded from the following URL:
If you are working with DE2-24C/P drive enclosure models, ensure that the system is working with Oracle ZFS Storage Appliance software release 2011.1.5.0.x or greater. Please refer to the following link for additional information:
Also, additional information on an Oracle ZFS Storage Appliance cluster configuration can be found in the Sun ZFS Storage 7000 System Administration Guide at:
The following combination and sizing of CPU, L1 cache (ARC) and L2 cache (L2ARC) is critical to meet the demands of compression and deduplication operations as well as overall performance in large deployments of a virtual desktop infrastructure. The minimum recommended configuration is:
Do the following to ensure that the network configuration where the NFS and iSCSI traffic will run has been designed to archive high availability and no single point of failure:
Note: Some network switch vendors do not support the LACP protocol. In this situation, set the LACP mode to "Off." Please refer to your switch vendor documentation for more information.
Figure 4. Figure 4. LACP, jumbo frame, and MTU configurations on Oracle ZFS Storage Appliance
When working with an Oracle ZFS Storage Appliance that has more than one disk shelf, try to split the workload across different disk pools and use the "no single point of failure" (NSPF) feature. This design will provide you with more storage resources as well as better I/O load balancing, performance, and throughput for your virtualized environment.
The following example uses only one disk shelf, a mirrored storage pool, one project and six different NFS shares. Table 4 lists the pool's projects and filesystem shares.Table 4. Projects and Filesystem Shares Created for Performance Test
| || || |
Figure 5 shows the share configuration and Figure 6 shows the filesystem and mountpoint configurations on the Oracle ZFS Storage Appliance browser user interface (BUI) for the performance tests. Details for the configuration choices follow.
Figure 5. Share configuration shown in Oracle ZFS Storage Appliance BUI
Note: The LZJB algorithm is considered the fastest algorithm and it does not consume much CPU. The LZJB algorithm is recommended for virtualized environments.
|Pool Name||Projects||Filesystems||Database Record Size|
| || || || |
Refer to the following document for more information if you want to enable this feature at the appliance level:
Figure 6. File systems and mountpoint configuration shown in Oracle ZFS Storage Appliance BUI
Figure 7 shows the minimum recommended configuration of Oracle ZFS Storage Appliance for VMware vSphere 5.x working with NFS protocol. Note that "Oracle ZFS head" in the graphic refers to the Oracle ZFS Storage Appliance head.
Figure 7. Oracle ZFS Storage Appliance—minimum recommended configuration for VMware vSphere 5 working with NFS protocol
Refer to the following websites for further information on testing results for Oracle ZFS Storage Appliance:
Also see the following documentation and websites:
Anderson Souza is a virtualization senior software engineer in Oracle's Application Integration Engineering group. He joined Oracle in 2012, bringing more than 14 years of technology industry, systems engineering, and virtualization expertise. Anderson has a Bachelor of Science in Computer Networking, a master's degree in Telecommunication Systems/Network Engineering, and also an MBA with a concentration in project management.
|Revision 1.0, 07/01/2013|