Oracle Database In-Memory, when used in a RAC environment, maintains its own discrete In-Memory Column Store (IMCS) on each instance. On Exadata each In-Memory Compression Unit (IMCU) in the Column Store can be replicated to one or more remote instances for both High-Availability, and even greater performance as both IMCU's can be used to satisfy queries.
The DUPLICATE clause must be added to the table definition to enable Oracle Database to automatically copy the In-Memory Compression Units between instances using the RDMA Network Fabric available on Exadata. For example:
ALTER TABLE LARGE_TABLE INMEMORY DUPLICATE;
will result in all IMCUs being distributed across every instance in the RAC cluster.
If an instance fails, only the IMCU's that now lacks a remote pair needs to be re-mirrored.
It is also possible to mirror each IMCU on every instance within the RAC Cluster by adding the ALL keyword to the above example:
ALTER TABLE LARGE_TABLE INMEMORY DUPLICATE ALL;
Rather than use rowid ranges, partiiton and sub-partition information to determine which IMCU is mirrored where, the ALL keyword tells Oracle to mirror every IMCU to each instance. This is particularly beneficial from a high-availability standpoint as no IMCU needs to be re-mirrored or moved if an instance fails or is restarted as the IMCU is already available, and likely being used, on the other instances in the cluster.
The In-Memory Column Store can contain objects that use either DUPLICATE or DUPLICATE ALL in their INMEMORY definition. Partitioned tables can also set the INMEMORY properties on the partition and sub-partition level. Such granular control allows administrators to ensure the most current and beneficial data is populated in the Database In-Memory Column Store.
Queries on star-schema typically join to a common set of dimension tables. To improve performance when joining such dimension tables to the fact tables, it can be beneficial to ensure a full copy of the dimension tables are available on every instance in the RAC cluster. This allows the join to occur locally. In this case, you may want to have a copy of the DIMENSION tables mirrored on every instance in the RAC cluster, while distributing the FACT table to improve parallelism.