The advanced features are available only for SecureFiles and do not apply to older LOBs or BasicFiles. The new features in SecureFiles - Deduplication, Compression and Encryption - can be setup independently or as a combination of one or more features. If all three features are turned on, Oracle will perform deduplication first and then compression followed by encryption. Deduplication
Oracle automatically detects multiple, identical SecureFiles data and stores only one copy, thereby saving storage space. In addition to storing only one copy, SecureFiles maintains 'references' for other duplicates. Deduplication is completely transparent to applications. Deduplication not only simplifies storage management but also results in significantly better performance, especially for copy operations. Duplicate detection happens within a LOB segment. The lob_storage_clause allows for specifying deduplication at a partition level. Duplicate detection does not span across partitions or subpartitions for partitioned SecureFile columns.
SecureFile data is compressed using industry standard compression algorithms. Compresion not only results in significant savings in storage but also improved performance by reducing IO, buffer cache requirements, redo generation and encryption overhead. If the compression does not yield any savings or if the data is already compressed, SecureFiles will automatically turn off compression for such columns. Compression is performed on the server-side and allows for random reads and writes to SecureFile data. SecureFile provides for varying degrees of compression: MEDIUM (default) and HIGH, which represent a tradeoff between storage savings and latency.
Compression tested on the Calgary data set consisting of doc, pdf and text documents. Actual compression might vary depending on the data loaded into SecureFiles. More details on Compression can be found in the SecureFiles and Large Object Developer's Guide
Oracle has extended the encryption capability to SecureFiles and uses the Transparent Data Encryption (TDE) syntax. The database supports automatic key management for all SecureFile columns within a table and transparently encrypts/decrypts data, backups and redo log files. Applications require no changes and can take advantage of SecureFiles using TDE semantics. SecureFiles supports the following encryption algorithms:
3DES168: Triple Data Encryption Standard with a 168-bit key size
AES128: Advanced Encryption Standard with a 128 bit key size
AES192: Advanced Encryption Standard with a 192-bit key size (default)
AES256: Advanced Encryption Standard with a 256-bit key size
Oracle database guarantees transactional consistency for unstructured data by logging both the metadata and data changes. File systems, on the other hand, usually log only the metadata and not the file data changes.
Oracle offers the ability to recover completely from media failures which is not available with generic file systems.
SecureFiles supports both the database logging and file system logging modes. A new logging level, FILESYSTEM_LIKE_LOGGING, which is similar to logging available with popular file systems, is supported. When SecureFiles logging is set to this level, Oracle writes only the metadata to the redo log. This setting is similar to the metadata journaling of file systems, which reduces mean time to recovery from failures and is sufficient for crash recovery or instance recovery. SecureFiles also supports database logging in which both the metadata and lob data are written to the redo log. This is especially useful when media recovery or standby database are required.
Oracle has a very active research organization (Oracle Labs) that is charged to 'Identify, explore, and transfer new technologies that have the potential to substantially improve Oracle's business'. One part of the organization is the External Research Office (ERO). The ERO is charged to ' ... invest in research collaborations that fit Oracle's long-term strategic goals. These collaborations are between university researchers and engineers/researchers throughout Oracle's various organizations'. The ERO webpage lists numerous current and past collaborations. Oracle provides funds and direct interactions with highly experienced developers.