What is new in SecureFiles? Why is it a high performance solution for file data?
SecureFiles is a completely new infrastructure inside the Oracle Database for managing unstructured data. It offers the best of both the file system and database worlds. SecureFiles includes many design innovations that incorporate years of research in file systems and databases.
Write performance using SecureFiles is enhanced by a new cache that buffers writes before writing to disk. Advanced features such as compression and deduplication reduce disk I/Os further at the cost of a slight CPU overhead. Intelligent pre-fetching especially during streaming reads offer better performance during reads. SecureFiles uses an advanced space management routine that is optimized for large, contiguous I/Os. In addition, there are several innovations with the wire protocol, locking algorithms etc that make SecureFiles far more scalable than LOBs or file systems. For more information on SecureFiles, read the technical white paper here.
What versions of the Oracle Database support SecureFiles?
SecureFiles is available in Oracle Database 11g and 12c on all supported database platforms. It is not available in 10g or prior releases.It is now the default LOB storage in Oracle Database 12c.
I use LOBs. Why should I consider SecureFiles?
SecureFiles are 100% compatible with LOB interfaces. You can gain the performance advantages of SecureFiles with no change to your application. SecureFiles offers advanced file system features such as compression and deduplication for reducing the disk footprint and encryption for higher security. All these advanced features are application transparent and existing applications can take advantage of SecureFiles functionality with no changes.
My data is stored in LOBs. Do I need to migrate the LOBs to SecureFiles?
Yes, migration is needed to convert LOBs to SecureFiles. For more information refer to the Migration section.
Can I enable SecureFiles at the partition-level?
Yes, SecureFiles can be enabled only for new partitions while older partitions continue to use older LOBs or BasicFiles.
I don't get the performance from SecureFiles. What should I do?
In general, SecureFiles offers better performance compared to LOBs or BasicFiles keeping the same hardware (servers, storage etc) and network setup. The performance from SecureFiles depends on the network and storage setup. You should measure the network bandwidth between the client and the server. You should also measure the raw disk throughput to get the theoretical limit. You can also download the SecureFiles Test Kit to measure the throughput using SecureFiles and then compare it to that of BasicFiles.
My application currently stores data, typically small files in the file system. How will I get better performance using SecureFiles?
Your application will see increased performance with SecureFiles as it accesses both the file and meta data in one round-trip. Consider a DICOM application which accesses patient metadata in the database and images such as X-rays or scans. The image can be stored inside SecureFiles in the Oracle Database or as a file in a file system. The application in the case of file system needs to make two round-trips, one to retrieve the image data from the file system and another to retrieve the associated patient meta data from the database. SecureFiles avoids the overhead associated with two round-trips by retrieving both the patient metadata and image data in one roundtrip. SecureFiles would hence offer better overall application throughput.
What is 'secure' in SecureFiles?
Data stored in SecureFiles is more secure when compared to storing as LOBs or files in a file system. SecureFiles supports encryption using Oracle Transparent Data Encryption (TDE). Data is encrypted not only on disk but also in other images such as backups. File or unstructured data can be co-located with relational or structured data and more importantly, secured using Oracle technologies such as Virtual Private Databases (VPD), Fine Grained Auditing, Oracle Label Security etc. With SecureFiles, you have true transactional ability for all your data and full crash recoverability. Your backup and recovery procedures are common for all your data and you no longer have to deal with disparate procedures for file system data and relational data. You can also setup unified audit policies to track access to your data.
Oracle has a very active research organization (Oracle Labs) that is charged to 'Identify, explore, and transfer new technologies that have the potential to substantially improve Oracle's business'. One part of the organization is the External Research Office (ERO). The ERO is charged to ' ... invest in research collaborations that fit Oracle's long-term strategic goals. These collaborations are between university researchers and engineers/researchers throughout Oracle's various organizations'. The ERO webpage lists numerous current and past collaborations. Oracle provides funds and direct interactions with highly experienced developers.