You are here: How To Guides > Execution types

Execution Types

OEDQ is designed to support three main types of process execution:

Batch (to process a batch or batches of records)
Real time response (to provide interactive data quality services)
Real time monitoring (to act as a probe on new records for a limited period of time, without issuing responses)

However, in order to preserve the freedom to design processes independently of their intended execution mode and switch between types easily, the majority of execution options are configured at the process or job level.

Batch

Processes are normally initially designed as batch processes, as a set of data is normally required in order to test that a process produces the required results, even if it will eventually be deployed as a real time process.

Batch processes have the following characteristics:

They read data from staged data configurations (such as snapshots)
They may include any of the processors in the processor library, including those such as Duplicate Check that are not suitable for real time response processes.
They either write no results, or write their results to a staged data table and/or external database or file.
They are executed in Normal mode. That is, the process completes when the batch has been completely processed.

Real time response

Real time response processes are designed to be called as interactive services to protect data quality at the point of data entry. A real time response process may perform virtually any data quality processing, including checking, cleaning and matching data. Profiling would not normally be used in a real time response process.

Real time response processes have the following characteristics:

They read data from a real time provider (such as the inbound interface of a Web Service)

Note: A real time response process may also include Readers connected to staged data configurations (such as snaphots), for example when Real time reference matching - in this case the process must be executed in Prepare mode before processing requests.

They write data to a real time consumer (such as the outbound interface of a Web Service)

Note: A real time response process may include Writers connected to staged data configurations, for example, to write a full audit trail of all records processed and their responses. These writers will not write any results until the process stops, regardless of any interval settings.

They are typically executed in Interval mode, so that results can be written out while the process runs continuously.
They should not include processors that are unsuitable for real time response processing, such as Duplicate Check.

Note: If a real time process includes a processor that is unsuitable for real time response processing, it will raise an exception when the first record or message is received. The supported execution types of each processor are listed in the help page for the processor.

Note that real time response processes may use much, or all, of the same logic as a batch process.

Real time monitoring

Real time monitoring processes are designed to check data quality at the point of data entry, but not return a response to the calling application, so that there is no extra burden on the user modifying the data on the source system. As there is no need to return a response, there are fewer restrictions on what the OEDQ process can do - for example, it may include profiling processors that work on all the records for the period of time that the monitoring process runs.

Real time monitoring processes have the following characteristics:

They read data from a real time provider (such as the inbound interface of a Web Service)
They may include any of the processors in the processor library, including those such as Duplicate Check that are not suitable for real time response processes.

Note:If a real time monitoring process contains processors that are designed to process whole batches, and are therefore not suitable for real time response processes, it should be run in Normal mode, and not in Interval mode. The supported execution types of each processor are listed in the help page for the processor.

They either write no results, or write their results to a staged data table and/or external database or file.
The process completes when it is stopped or when a configured time or record threshold is reached.
They may be executed either in Normal mode (for a limited period of time, or processing a limited number of records), or in Interval mode.