Oracle Cloud Infrastructure Storage Gateway helps you bridge your on-premises data center with Oracle Cloud Infrastructure Object Storage. Using Storage Gateway, any application that can store data on an NFS target can easily and seamlessly move data to Object Storage without having to first modify the application to uptake REST APIs. Storage Gateway exposes an NFS mount point that can be mounted to any host that supports an NFSv4 client. Storage Gateway performs the NFS to REST API translations for you. A local configurable cache enables asynchronous and optimized movement of the data to the cloud to work around bandwidth limitations or connectivity issues. Because frequently accessed data is cached locally, the cache also improves latency when reading data from the cloud. Each Storage Gateway NFS mount point maps to an Object Storage bucket. The NFS interface lets you use the traditional file system commands to interact with cloud storage.
Oracle Cloud Infrastructure Storage Gateway supports file to object transparency. This means that when the Storage Gateway uploads your data to the cloud, it preserves the original data format. The data in the cloud can be read or managed using supported Object Storage clients, such as APIs, SDKs, the CLI and Console, or other third party tools.
Using Storage Gateway, you can easily move data generated by traditional applications that support common file system protocols like NFS to Object Storage without having to modify the application to integrate with the REST APIs. Storage Gateway opens up virtually unlimited cloud storage to traditional on-premises applications, which previously could not take advantage of the seamless scalability, durability, and cost efficiency of Object Storage.
No, Storage Gateway must not be used as a general-purpose network-attached storage. Though Storage Gateway is in many ways a virtual shared file system, its primary function is data movement to the cloud. Storage Gateway is not built to handle the transactional workload that a full-featured shared file system can handle.
Storage Gateway is an efficient tool to move data from your on-premises data center to the cloud. You can use this service for use cases like backup/archive, disaster recovery, data migration, and hybrid (on-premises and cloud) workloads. Using Storage Gateway, you can easily script the movement of cold data from on-premises NAS servers to Oracle Cloud Infrastructure Object Storage.
General purpose network storage - Storage Gateway is not a general purpose storage filer and must not be used as a replacement for traditional network storage appliances.
File sync and share - Though Storage Gateway is an effective data mover, it's not a replacement for file sync and share services. Evaluate Oracle services like Oracle Document Cloud service for your file sync and share needs.
Content collaboration - Storage Gateway does not support multiple instances simultaneously reading from and writing to a single Object Storage bucket. Storage Gateway is not meant to be used as a tool that enables distributed teams to collaborate on creating and managing content.
Frequently modified files - When an existing file is modified, its content is read from Object Storage if not present in cache. The modified file is later uploaded as a new version/instance of the file. Therefore, files that are frequently modified are not ideal both in terms of consuming upload/download bandwidth and capacity utilization.
Storage Gateway is a free service for all Oracle Cloud Infrastructure customers.
Storage Gateway supports the NFSv4 protocol.
No, Storage Gateway does not currently support the Windows operating environment.
Yes, you can use Storage Gateway to replicate data from any storage, as long as the NFS client that is performing the data copy can either facilitate the replication or can use the Storage Gateway cloud sync feature.
Yes, we can facilitate this migration. Contact us about using the Storage Gateway cloud migration utility.
No, you do not need to use any Object Storage APIs to use Storage Gateway. Storage Gateway performs the NFS to REST API translation for you.
When you create a file system on the Storage Gateway, an Object Storage bucket with the same name is created in the Oracle Cloud Infrastructure region of your choice. Storage Gateway completely manages your data in the Object Storage bucket for you. You only need to interact with the local file system using normal file system commands.
No, Storage Gateway does not support compression or deduplication.
You can get support for Storage Gateway by contacting Oracle Support at https://support.oracle.com.
Yes, you can create multiple file systems on a single Storage Gateway instance. The hardware the host is running on dictates the maximum number of file systems that can be supported by a Storage Gateway instance. We recommend that you create no more than 5 active file systems per Storage Gateway instance. For best performance, host each file system on a dedicated Storage Gateway.
Storage Gateway can manage a maximum of 100 million files per file system. For data sets that contain more than 100 million objects, ensure that the objects are distributed across multiple file systems.
The maximum supported file size is 75% of cache space configured on Storage Gateway, but cannot exceed 10 TB. For example, if Storage Gateway is configured to have 1 TB of cache space, the maximum supported file size is 750 GB.
There is no upper limit on the number of Storage Gateway instances that can be installed for use within an Oracle Cloud Infrastructure tenancy.
As long as the local cache is sized accurately, you can upload an unlimited amount of data to Object Storage. Note, that Storage Gateway file systems should not exceed 100 million files.
The longest supported file name can be 255 characters. The total path length can be a maximum of 1024 characters, which is the maximum supported length for Object Storage object names. For example, /directory1/directory2/directory3/directory4/filename.txt is 57 characters in length. Note that the maximum supported length for Object Storage paths is shorter than the maximum of 4096 characters supported by Linux file systems. Trying to create a file or directory for which path is longer than 1024 characters generates an NFS error.
There is no limit to the number of directories, as long as the path length of each directory is 1024 characters or less.
There is no limit to the number of supported files in a single directory.
Yes, you can run the Storage Gateway in an Oracle Cloud Infrastructure compute instance using the same general installation instructions you follow for installing Storage Gateway on-premises.
Yes, Storage Gateway can be installed behind a proxy server. To set the proxy servers, use the following commands:
sudo ocisg configure proxy <http_proxy_server> <https_proxy server>
sudo ocisg down
sudo ocisg up
No, multiple Storage Gateway instances cannot run against the same Object Storage Bucket. Storage Gateway does not enforce object locking or consistency between different Storage Gateway instances.
Yes, data uploaded to the cloud using the Storage Gateway can be managed using any supported Oracle tools like APIs, SDKs, the CLI and Console, or third party tools like Cyberduck or CloudBerry.
Changes made to a Storage Gateway file system are eventually consistent with the Object Storage. Uploads are asynchronous and buffered for performance, so Storage Gateway file changes might not yet be reflected in Object Storage.
Changes made directly to the Object Storage bucket are not reflected back to the local Storage Gateway file system in real time. Click 'Refresh' in the Storage Gateway management console to ingest any data that was added or modified directly in Object Storage.
Yes, a Storage Gateway file system can be created against a non-empty bucket. When this happens, Storage Gateway creates metadata for all of the objects in the bucket with default permissions and attributes. Storage Gateway also creates a local inventory of the objects in the bucket. Depending on the number of objects in the target bucket, this process might take some time to complete. The Storage Gateway management console provides a progress bar showing the current state.
No, Storage Gateway does not support symbolic or hard links. Trying to create a link will result in an NFS error.
Yes, Storage Gateway supports the creation of sparse files. However, the corresponding Object Storage objects will be non-sparse and filled with 0x00s.
No, if a file system is deleted/removed, the data stored in the cloud is left intact. You can reclaim the deleted data by creating another file system backed by the same bucket.
Yes, deleting data from Storage Gateway permanently deletes the data from Object Storage.
Storage Gateway does not upload a new file or modified file until the file is closed. After a file is closed, Storage Gateway allows a 1 minute settling period before queuing the file for upload.
Depending on the number of files pending upload, it might be a while before the file is uploaded to the Object Storage bucket. You can view the upload status of files from the Storage Gateway management console.
Storage Gateway 1.2 provides an integrated cloud synchronization and replication feature that lets you backup and replicate NAS files to and from Oracle Cloud Infrastructure Object Storage buckets. This new feature replaces the independent, downloadable cloud sync utility that was available in previous Storage Gateway versions.
You can use the Storage Gateway management console or CLI to easily create, monitor, and manage sync jobs similar to other enterprise NAS backup/replication offerings. Cloud Sync runs as part of the Storage Gateway software inside the Docker instance on the host.
See Using Storage Gateway Cloud Sync documentation for details.
Customers with a lot of on-premises files to move into the cloud can do so using the Oracle Cloud Infrastructure Storage Gateway. Cloud Sync offers a way to copy files from the local storage into the Storage Gateway, where the files are asynchronously and efficiently moved to an Object Storage bucket. Customers who store database or data backups in local NAS and want to free up the space, or have a desire to move older backups into less expensive cloud storage, will find Cloud Sync an easy way to automate this kind of data movement.
Yes, when you create a file system, you specify which storage tier, Standard (the default) or Archive, to create the corresponding Object Storage bucket. The Standard Object Storage tier is used for storing data to which you need fast, immediate, and frequent access. The Archive Storage tier is used for storing data that is accessed infrequently and requires long retention periods. While Archive Storage is more cost effective than Object Storage for preserving cold data, you must first restore the objects before you can access them.
Each Cloud Sync copy operation provides comprehensive status to let you know the list of files selected, when they have been copied over to the Storage Gateway, and when they have been successfully uploaded to Object Storage. When a copy operation is in progress, Cloud Sync provides the start timestamp, elapsed time, estimated completion time, the total number/size of files to be synced, and the total number/size for files that have been already been synced.
After the file transfer is complete, the cloud sync utility compares the checksums of the files in the source and the target locations.
In performance benchmark tests with 10 Gb/s link speed, we observed the following:
Upload Speeds
Storage Gateway 1.2 provides a diagnostic command that you can use to test the bandwidth in your environment and ensure that you are getting the expected upload and download speeds.
See Testing Network Bandwidth documentation for more details.
Follow the documented best practices to get the maximum performance benefits from Storage Gateway.
Storage Gateway 1.2 is tuned for maximum upload and download performance by default. No additional tuning is needed.
In addition to having sufficient memory and file system cache space, we recommend using SSDs in a Storage Gateway environment. Storage Gateway needs to read from the cache to upload to Oracle Cloud Infrastructure. If the backend media is slow, NFS ingestion rate and thus the overall upload performance is negatively affected.
Using FastConnect in conjunction with Storage Gateway makes optimal use of full link speed. We have customers using FastConnect with 10Gb/s link speed and have seen each gateway achieve 400 - 450 MB/s uploads to Oracle Cloud Infrastructure.
See FastConnect documentation to learn more.