by Dominic Kay
Published October 2012
With the advent of Oracle Solaris 11, all data is stored on ZFS file systems by default. ZFS also underpins much of the rest of Oracle Solaris 11, such as booting, root file systems, upgrading, and updating.
From a data management perspective, a strong case can be made for migrating data to ZFS, particularly in the areas of the scaleability and ease of data management required in the cloud as well as in the large number of no-cost data services that are available, such as encryption, compression, deduplication, snapshots, and remote replication. These services, along with the high availability and performance of Oracle Solaris 11 at scale, are strong incentives to consolidate on that platform.
While the UFS file system is still available in Oracle Solaris 11 (except as a root file system), ZFS is the preferred storage mechanism and provides far greater reliability and functionality than UFS.
Shadow migration allows you to migrate data from a legacy file system to a ZFS file system while simultaneously allowing access and modification to the data during the process. You can use shadow migration to migrate a local or remote file system to a local file system. To be more specific, you can migrate data from any file system that can either be mounted directly on the Oracle Solaris 11 host (UFS, ZFS, VxFS, and so on) or any remote file system that can be set to "read only" and mounted on the Oracle Solaris 11 host using NFS. This extends to filers such as NetApp and other operating systems such as AIX, HP-UX, and Linux.
The use cases for shadow migration extend beyond moving data to Oracle Solaris 11. When you want to enable ZFS services such as compression, deduplication, and encryption—where data needs to go through a transformation—shadow migration within the server between an existing and a new file system (to subsequently be renamed) might be the best method of achieving the data transformation.
The following were the principal design requirements for shadow migration:
Here is a quick demonstration of shadow migration that you can try out for yourself. This example migrates data from a local file system. However, the principles apply to remote file systems mounted over NFS. The syntax for remote file systems is in the administration manual (see the "See Also" section).
First create the file system you want to migrate
root@solaris:~# zpool create oldstuff c3t4d0 root@solaris:~# zfs create oldstuff/forgotten
Then populate it with some files:
root@solaris:~# cd /var/adm root@solaris:/var/adm# find . -print | cpio -pdv /oldstuff/forgotten
You need to install the
shadow-migration package, as shown in Listing 1:
root@solaris:~# pkg install shadow-migration Packages to install: 1 Create boot environment: No Create backup boot environment: No Services to change: 1 DOWNLOAD PKGS FILES XFER (MB) Completed 1/1 14/14 0.2/0.2 PHASE ACTIONS Install Phase 39/39 PHASE ITEMS Package State Update Phase 1/1 Image State Update Phase 2/2
Listing 1. Installing the
You then enable the
root@solaris:~# svcadm enable shadowd root@solaris:~# svcs shadowd STATE STIME FMRI online 7:16:09 svc:/system/filesystem/shadowd:default
Set the file system to be migrated to read-only:
root@solaris:~# zfs set readonly=on oldstuff/forgotten
Create a new ZFS file system with the
shadow property set to the file system to be migrated:
root@solaris:~# zfs create -o shadow=file:///oldstuff/forgotten mypool/remembered
shadowstat(1M) command to see the progress of the migration, as shown in Listing 2:
root@solaris:~# shadowstat EST BYTES BYTES ELAPSED DATASET XFRD LEFT ERRORS TIME mypool/remembered 92.5M - - 00:00:59 mypool/remembered 99.1M 302M - 00:01:09 mypool/remembered 109M 260M - 00:01:19 mypool/remembered 133M 304M - 00:01:29 mypool/remembered 149M 339M - 00:01:39 mypool/remembered 156M 86.4M - 00:01:49 mypool/remembered 156M 8E - (completed)
Listing 2. Checking Migration Progress
As noted above, if you had created
/mypool/remembered as encrypted, this would be the preferred method of encrypting existing data. The same is true for compressing or deduplicating existing data.
The migration of production data requires backup and thorough testing before the live migration. In particular, be aware of the following:
The performance of shadow migration feature depends on the following:
A design goal of shadow migration was to make the performance of the transfer unimportant to clients, the important factor being that clients continue to enjoy fast access to the data they require. However, if raw performance of the transfer between old and new media itself is important, you might not want to use shadow migration, because it is designed to be seamless and "invisible," but not fast itself.
Eric Schrock describes the motivation and design and also the engineering internals of shadow migration in his blogs. The formal treatment of shadow migration is in the Oracle Solaris Administration: ZFS File Systems manual.
Here are some additional resources:
Dominic Kay is a Principal Product Manager for Oracle Solaris and focuses on Oracle Solaris data management technology. Dominic joined Oracle in 2010 as part of the Sun Microsystems acquisition.
|Revision 1.0, 10/222012|