Overview

Oracle Real Application Clusters (RAC) Database require that all database instances are synchronized before a commit occurs, using the System Change Number (SCN) as the means to identify consistency across the cluster. In the traditional Broadcast-on-Commit protocol, the log writer (LGRW) process sends the SCN using messages to the Global Cache Service (LMS) on each remote instance. Concurrently, LGRW initiates redo log write activity on the local instance. The LMS process then updates its instances SCN and sends an acknowledgement message back to the originating instance. When log writer completes the log write IO, it checks for acknowledgement from each remote instance that the SCN messgae has been recieved and processed. If the SCN broadcast has been completed by all remote instances, the associated foreground completes the requested commit. If the SCN message has not been processed by one for more remote instances, the foreground process will wait until all instances have acknowledged the SCN update. In such events, high "log file sync" wait events can be seen in the AWR.

Oracle Database 21c on Oracle Exadata Database Machine uses Broadcast-on-Commit over RDMA, where the log writing process directly updates the SCN on each remote instance using remote atomic operations. By removing the message based mechanism, the commit protocol becomes faster as remote context switch latency, and CPU load does not affect the SCN synchronization process.