by Andrew Ness, February 2012
This article covers the following topics:
Oracle Solaris 10 and Oracle Solaris 11 come with a Fibre Channel initiator stack you can configure to integrate Fibre Channel (FC) LUNs presented by the Sun ZFS Storage Appliance into the Oracle Solaris environment. This article describes how to configure the Oracle Solaris Fibre Channel stack and how to configure the Sun ZFS Storage Appliance to set up FC LUNs for access by Oracle Solaris servers. These configurations can be accomplished using the browser user interface (BUI).
The following assumptions have been made in this article:
rootpassword for the Sun ZFS Storage Appliance is known.
rootpassword for the Oracle Solaris server is known.
To identify the Sun ZFS Storage Appliance and the Oracle Solaris server to each other, the FC World Wide Number (WWN) for each must be registered with the other. You must determine the WWNs for some forms of FC zoning implemented on the FC switch.
The FC WWN of the host is used to identify the host to the Sun ZFS Storage Appliance and is needed to complete the configuration procedure in this article.
The WWNs are taken from the FC Host Bus Adapters (HBAs) installed in the Oracle Solaris host and the Sun ZFS Storage Appliance.
In order to configure the Oracle Solaris FC stack, you will need to know the WWNs of the Sun ZFS Storage Appliance. In a traditional dual-fabric Storage Area Network (SAN), the Sun ZFS Storage Appliance will have at least one FC port attached to each fabric. Thus, you must determine at least two FC WWNs.
First, you need to establish a management session with the Sun ZFS Storage Appliance.
https://<ip-address or host name>:215
The login dialog box is displayed.
Figure 1. Sun ZFS Storage Appliance login
Once you have successfully logged in to the BUI, you can identify the WWNs through the Configuration tab.
Figure 2. Fibre Channel port configuration for SAN
The FC ports installed in the Sun ZFS Storage Appliance are shown. Since there is only one discovered port per HBA channel, this must be the HBA channel itself.
In the preceding case, port 1 has the WWN
21:00:00:e0:8b:92:a1:cf and port 2 has the WWN
The FC channel ports should be set to Target in the list box to the right of each FC port box. If this is not the case, the FC ports may be in use for another purpose. Do not change the setting until you have investigated the reason. (One possible reason may be for NDMP backups.)
If the Oracle Solaris host is already attached to the FC switch by the appropriate cables, use the following command to identify the WWNs.
root@solaris:~# cfgadm -al -o show_FCP_dev Ap_Id Type Receptacle Occupant Condition [...] c8 fc-fabric connected unconfigured unknown c9 fc-fabric connected unconfigured unknown root@solaris:~#
From this output, the controller numbers you need are
c9. You can also see that both ports are attached to an FC switch, as the port type is
fc-fabric. Next, interrogate these controllers to determine what WWNs are discovered.
root@solaris:~# prtconf -vp | grep port-wwn port-wwn: 210000e0.8b89bf8e port-wwn: 210100e0.8ba9bf8e root@solaris:~#
Alternatively, if FC devices are being accessed, the following command will show the FC HBA WWNs.
root@solaris:~# luxadm -e dump_map /dev/cfg/c8 Pos Port_ID Hard_Ad Port WWN Node WWN Type 0 2000 0 xxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxx 0x0 (Disk device) [...] 3 25600 0 2100000e08b89bf8e 200000e08b89bf8e 0x1f (Unknown type, Host Bus Adapter) root@solaris:~#
The last entry shown as type
Unknown type, Host Bus Adapter) gives the appropriate WWN under the port WWN entry. Repeat the command substituting
/dev/cfg/c8 for the other controllers identified in step 1.
c8has the WWN
The Sun ZFS Storage Appliance HBA and the Oracle Solaris host HBA WWNs can then be used to configure any FC switch zoning.
Once this has been completed, you can verify the proper zoning by running the following command:
root@solaris:~# cfgadm -al -o show_FCP_dev c8 c9 Ap_Id Type Receptacle Occupant Condition [...] c8 fc-fabric connected configured unknown c8::210100e08bb2a1cf unknown connected unconfigured unknown c9 fc-fabric connected configured unknown c9::210000e08b92a1cf unknown connected unconfigured unknown root@solaris:~#
You can now see the WWNs presented by the Sun ZFS Storage Appliance that are accessible by the Oracle Solaris host.
As a unified storage platform, the Sun ZFS Storage Appliance supports access to block-protocol LUNs using iSCSI and Fibre Channel protocols. This section describes how to use the Sun ZFS Storage Appliance BUI to configure the Sun ZFS Storage Appliance to recognize the Oracle Solaris host and present FC LUNs to it.
A target group is created on the Sun ZFS Storage Appliance to define the ports and the protocol by which the LUN to be presented to the Oracle Solaris server is accessed. For this example, an FC target group is created.
To define an FC target group on the Sun ZFS Storage Appliance, complete these steps:
Figure 3. Selecting the Fibre Channel port
Figure 4. Dragging the Fibre Channel port entry
nis an integer. See the example
targets-0in Figure 5.
Figure 5. Newly created Fibre Channel target group
Figure 6. Selecting the FC target group
targets-0, click the Edit icon () to display the dialog box shown in Figure 7.
Figure 7. Renaming the FC Target Group
targets-0. In this window, you can also add the second FC target port by clicking in the box to the left of the WWN you are choosing. In the example in Figure 7, this second port is identified as
PCIe 1: Port 2.
Figure 8. New FC target group with changes applied
An FC initiator is defined to allow access from one or more servers to particular volumes. Permissions to volumes should be configured so that a minimum number of FC initiators are allowed to access a particular volume. If more than one host can write to a given volume concurrently and a non-shared file system is used, inconsistencies may occur in file system caches on the hosts that can ultimately lead to corruption of the on-disk image. Typically a single initiator is given access to a volume, unless a specialized cluster file system is being used.
The FC initiator serves to define the "host" from the point of view of the Sun ZFS Storage Appliance. In a traditional dual-fabric SAN, the host will be defined by at least two FC initiators. The FC initiator definition contains the host WWNs. To identify the Oracle Solaris server to the Sun ZFS Storage Appliance, the Oracle Solaris FC initiator WWNs must then be registered with the appliance by completing the following steps.
Figure 9. Selecting the SAN configuration
If the zoning has been configured on the FC switches, the WWNs of the Oracle Solaris host should be displayed (assuming they are not assigned to an alias already).
Figure 10. New Fibre Channel initiator definition
Related FC initiators are combined into logical groups to allow single commands to execute on multiple FC initiators, for example, assigning LUN access to all FC initiators in a group with one command. For the following example, the FC initiator group will contain two initiators. Note that in a cluster, where multiple servers are treated as a single logical entity, the initiator group may contain many more initiators.
To create an FC initiator group, complete these steps:
Figure 11. Displaying the Move icon for the new FC initiator
Figure 12. Creating the FC initiator group
nis an integer, as shown in Figure 13.
Figure 13. Selecting the Fibre Channel initiator group
In this dialog box, you can add the other FC initiator(s) to the group by clicking the checkbox to the left of the WWN.
Figure 14. Renaming and completing the FC initiator group
Figure 15. Completed FC initiator configuration
To group related volumes, you can define a project in the Sun ZFS Storage Appliance. A project allows inheritance of properties for file systems and LUNs presented from the project. A project also allows you to apply quotas and reservations.
To create a project, complete the steps below:
Figure 16. Viewing a project
Figure 17. Create Project dialog box
Figure 18. Displaying the new project
Next, you will create a LUN from an existing pool of storage resources that the Oracle Solaris server will access. In the following example, a thin-provisioned 64-GB LUN called
DocArchive1 is created.
The target group will be the FC target group,
FC-PortGroup, which was created in the section Defining an FC Target Group, to ensure that this LUN can be accessed using the FC protocol. The initiator group defined in the section Defining the FC Initiator Group,
sol-server, will be used to ensure that only the server(s) defined in the
sol-server group can access this LUN. (In this example, this group contains only one server.)
To create a target group, complete the following steps:
Figure 19. Selecting Shares > Project > LUNs
DocArchive1, Volume size to
64 G, and select the Thin provisioned checkbox. Set Target Group to the FC target group
FC-PortGroupand set Initiator Group to
sol-server. Set Volume block size to
32kbecause the volume will hold an Oracle Solaris ZFS file system.
Figure 20. New LUN dialog box
Now that the LUN is prepared and available to the FC initiator group, the LUN must be configured for use by the Oracle Solaris server by completing the following steps:
root@solaris:~# cfgadm -al c8 c9 root@solaris:~# cfgadm -c configure c8::210100e08bb2a1cf root@solaris:~# cfgadm -c configure c9::210000e08b92a1cf root@solaris:~# cfgadm -al -o show_FCP_dev c8 c9 Ap_Id Type Receptacle Occupant Condition c8 fc-fabric connected configured unknown c8::210100e08bb2a1cf,0 disk connected configured unknown c9 fc-fabric connected configured unknown c9::210000e08b92a1cf,0 disk connected configured unknown root@solaris:~#
root@solaris:~# devfsadm -c ssd root@solaris:~# tail /var/adm/messages [...] Nov 30 06:30:00 solaris last message repeated 2 times Nov 30 06:32:22 solaris scsi: [ID 799468 kern.info] ssd42 at scsi_vhci0: name g600144f0f05e906c00004ed6096d0001, bus address g600144f0f05e906c00004ed6096d0001 Nov 30 06:32:22 solaris genunix: [ID 936769 kern.info] ssd42 is /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 Nov 30 06:32:22 solaris scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42): Nov 30 06:32:22 solaris Corrupt label; wrong magic number Nov 30 06:32:22 solaris genunix: [ID 408114 kern.info] /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42) online Nov 30 06:32:22 solaris genunix: [ID 834635 kern.info] /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42) multipath status: degraded, path /pci@1e,600000/SUNW,qlc@2/fp@0,0 (fp1) to target address: w210100e08bb2a1cf,0 is online Load balancing: round-robin Nov 30 06:32:22 solaris scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42): Nov 30 06:32:22 solaris Corrupt label; wrong magic number Nov 30 06:32:49 solaris genunix: [ID 834635 kern.info] /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42) multipath status: optimal, path /pci@1e,600000/SUNW,qlc@2,1/fp@0,0 (fp3) to target address: w210000e08b92a1cf,0 is online Load balancing: round-robin [...]
In this example, the multipath status is shown as
degraded initially because only one path had been identified at that point. Further on, the multipath status changes to
optimal due to the presence of multiple paths to the volume.
The disk device is now similarly available to an internal server disk.
root@solaris:~# format Searching for disks...done c1t600144F0F05E906C00004ED6096D0001d0: configured with capacity of 63.93GB AVAILABLE DISK SELECTIONS: [...] 4. c1t600144F0F05E906C00004ED6096D0001d0 <SUN-SunStorage7210-1.0 cyl 2078 alt 2 hd 254 sec 254> /scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 [...] Specify disk (enter its number): 4 selecting c1t600144F0F05E906C00004ED6096D0001d0 [disk formatted] Disk not labeled. Label it now? y FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels save - save new disk/partition definitions inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quit format> q
root@solaris:~# zfs createzpool create docarchive1 \ c1t600144F0F05E906C00004ED6096D0001d0 root@solaris:~# zfs list NAME USED AVAIL REFER MOUNTPOINT docarchive1 72K 62.5G 21K /docarchive1 rpool 25.4G 41.5G 97K /rpool rpool/ROOT 22.3G 41.5G 21K legacy [...] root@solaris:~# zfs create docarchive1/index root@solaris:~# zfs create docarchive1/data root@solaris:~# zfs create docarchive1/logs root@solaris:~# zfs list NAME USED AVAIL REFER MOUNTPOINT docarchive1 159K 62.5G 24K /docarchive1 docarchive1/data 21K 62.5G 21K /docarchive1/data docarchive1/index 21K 62.5G 21K /docarchive1/index docarchive1/logs 21K 62.5G 21K /docarchive1/logs rpool 25.4G 41.5G 97K /rpool rpool/ROOT 22.3G 41.5G 21K legacy [...]
The final two lines of the output from the
df(1) command show that approximately 64 GB of new space is now available.
This article has described how to configure the FC initiator stack supplied with Oracle Solaris 10 and Oracle Solaris 11 to enable an Oracle Solaris server to access Fibre Channel LUNs presented by the Sun ZFS Storage Appliance.
Here are resources for Oracle's Sun ZFS Storage Appliance and Oracle Solaris ZFS:
Here are some additional resources for Oracle Solaris:
|Revision 1.0, 02/24/2012|