Oracle Exadata Database Machine employs a modern scale-out architecture encompassing database compute, storage, network, and software. Uniquely designed to be the ideal platform to run Oracle Database, Oracle Exadata enables enterprises to add capacity and increase performance as workloads scale. Oracle Exadata utilizes state-of-the-art hardware and purpose-built software to drive the highest database performance. All database and storage servers are interconnected with each other, communicating over an intelligent, high-speed, low-latency network fabric.

Remote Direct Memory Access (RDMA) is an integral architectural component of Exadata. It is enhanced in every generation of hardware, Oracle Database, and Exadata System Software release. RDMA enables one server to interact with memory structures on another server without interacting with the OS and network stack on either machine. Each Database and Storage Server is equipped with RDMA capable network cards that directly read from and write to remote memory with no extra copying or buffering. RDMA is also used by Oracle Database when running on Exadata to optimize communication between Real Application Clusters (RAC) instances.

Fundamental to the architecture of Exadata is modular, shared, smart storage. Each Storage Server is capable of interpreting and processing data independently. The CPUs in the storage servers do not replace database CPUs. Instead, they accelerate database-intensive workloads similar to how graphics cards accelerate image-intensive workloads. Exadata System Software enables Exadata Storage Server's unparalleled performance without any of the bottlenecks of traditional storage. Automatic caching of data from disk into Exadata Smart Flash Cache and Exadata RDMA Memory (XRMEM) (a multi-level tiering of data into high-speed, low-latency storage media) enables staggering performance. Every storage server is accessible from every database server. As a result, flash and XRMEM can be shared effectively across multiple database workloads. By automatically caching data from disk into flash and XRMEM, the benefits of these technologies are available and realized by all workloads.

Oracle Exadata is comprised of Database Servers, Storage Servers, high-speed, low-latency network fabric, Oracle Database Software, and Exadata System Software. Together, these five components address the need for Oracle Databases shared-everything architecture of Real Application Clusters (RAC).

Starting with Exadata System Software release 24.1, Oracle Exadata Exascale transforms Exadata storage management by decoupling Oracle Database and GI clusters from the underlying Exadata storage servers. For new and existing deployments, you can deploy a configuration that uses Exascale, ASM, or both. This option enables new and existing customers to implement Exascale while keeping selected databases on the time-tested storage management infrastructure provided by ASM.

Oracle Database, running on the Exadata Database Servers, leverages a purpose-built protocol - iDB - and Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) to communicate with the Exadata Storage.

iDB is used to direct Smart I/O operations on the Storage Servers. Smart I/O includes Smart Scan (SQL Offload), Fast File Initialization, and RMAN Incremental Backup Offload. Exadata essentially can "function ship" or push processing from the database servers to the storage servers rather than move potentially hundreds to thousands of terabytes from disk to database server. When the storage server receives a request to perform a Smart I/O, Exadata System Software unpacks the iDB message and executes and processes I/O requests locally, including executing portions of Oracle Database code on the storage servers, before sending only the data that satisfies the request back to the database servers for final processing (such as aggregation).

Where iDB is used to offload I/O intensive processing, such as backups and Analytic queries, to the storage servers, Exadata also leverages the high-speed RoCE network fabric for OLTP workloads. High Capacity Exadata Storage Servers are equipped with Exadata RDMA Memory (XRMEM), flash drives, and traditional hard drives. The Exadata System Software automatically caches data into flash and XRMEM based on incoming workload. This automatically managed caching strategy ensures that the most appropriate storage tier is used to service I/O requests.

For example, Incremental Fast Backups are offloaded to the Exadata Storage Server. Since Exadata knows that blocks read during a backup are likely of little use for satisfying I/O beyond the backup itself, this data is not cached into either flash or XRMEM. At the same time, the Storage Server will cache blocks in both flash and XRMEM to satisfy read and write database requests for OLTP workloads.

Automatically caching data in this manner enables all workloads to run faster and resources to be used more efficiently for individual databases. It also enables high degrees of database consolidation.

Oracle Database and Exadata are engineered together. Oracle Database leverages the hardware and software Exadata provides. Exadata Storage Servers include XRMEM and are accessible over the RoCE network fabric. As OLTP workloads typically access and modify very small amounts of data - one or very few rows - caching this data on a fast device, such as XRMEM ensures the quickest read and write speeds for such data. As Exadata employs scale-out hardware and network architecture, placing the XRMEM in the storage servers ensures all database instances benefit from this cutting-edge technology.

Oracle Exadata System Software runs on all database and storage servers to enable Oracle Database to leverage every aspect of Exadata's hardware and network capabilities.

On each Database Server, in addition to the Oracle Linux Operating System, Oracle Grid Infrastructure, Automatic Storage Management (ASM), and Oracle Database software and instances, you will find the following processes and library:

• MS – Exadata System Software Management Server – the primary interface for managing the Exadata Database Server, including hardware and software monitoring, metric collection, alerting, and a command-line interface (DBMCLI).
• RS – Exadata System Software Restart Server – monitors and restarts the MS process if needed.
• libcell – Exadata-specific library linked into the Oracle Database and Grid Infrastructure that allows Oracle to communicate with the Exadata Storage Servers using iDB commands and RDMA.
• EGSB/EDSB – Exascale Global Services – per database instance services that maintain metadata about the Exascale cluster and vaults.
• EGS – Exascale Global Services – provides the core foundation for the Exascale system. EGS instances will run on database servers when the Exadata configuration has fewer than five storage servers.
• ESNP – Exascale Node Proxy – maintains information about the current state of the Exascale cluster, which it provides to local Oracle Grid Infrastructure and Oracle Database processes.
• EDV – Exascale Direct Volume – exposes Exascale block storage as raw block devices on Exadata compute nodes.
• BSW – Block Store Worker – primarily services requests from block store clients and performs the resulting storage server I/O.

Oracle Exadata System Software runs on all database and storage servers to enable Oracle Database to leverage every aspect of Exadata's hardware and network capabilities.

On each Extreme Flash and High Capacity Storage Server, in addition to the Oracle Linux Operating System, you will find the following processes and library:

• MS – Exadata System Software Management Server – the primary interface for managing the Exadata Storage Server, including hardware and software monitoring, metric collection, alerting, and a command-line interface (CELLCLI).
• RS – Exadata System Software Restart Server– monitors and restarts the MS process if needed.
• CELLSRV – Exadata Cell Server – The core Exadata System Software component responsibile for the majority of the services provided by the Storage Servers, including SQL Offload, I/O Resource Management (IORM), Exadata RDMA Memory (XRMEM) and Flash Cache tiering, and Storage Index creation and maintenance.
• IORM – I/O Resource Manager – A sub-process of the CELLSRV process. Responsible for ensuring multiple databases can share the same shared I/O resources, ensuring databases on the same infrastructure do not monopolize I/O bandwidth and degrade perfomance of other databases.
• CELLOFLSRVn – Cell Offload Server – A sub-process of the Cell Server processes that provides version-specific Storage Offload functions. Each version of Oracle Database uses an associated version-specific CELLOFLSRV process.
• EGS – Exascale Global Services – provides the core foundation for the Exascale system. Every Exascale cluster contains five EGS instances, each running on a storage server.
• ERS – Exascale RESTful Services – provides a management endpoint for Exascale management operations.
• EDS – Exascale Data Services – the collective name for the Exascale software services that manage file and vault metadata.
  • SYSEDS – System Vault Manager Service – serves and manages the metadata for Exascale vaults.
  • USREDS – User Vault Manager Service – serves and manages the metadata for files inside the Exascale vaults.
• BSM – Block Store Manager – serves and manages the metadata for Exascale block storage.
• BSW – Block Store Worker – primarily services requests from block store clients and performs the resulting storage server I/O.
• IFD – Instant Failure Detection – a dedicated lightweight service that quickly detects and responds to any storage server failure.