You are here: Processor Library

Processor Library

The following is an overview of all the processors that are available in OEDQ.

Note that your purchased OEDQ configuration may not grant you access to the full set of processors. If a processor is in this list, but not available in the Tool Palette, it is because the OEDQ server does not have the appropriate functional packs enabled. Please contact your account representative if you need to purchase licenses for additional OEDQ functional packs.

Profilers

For general information about Profilers, see About profilers.

Icon	Processor Name	Description	Compatible attribute types
	Character Profiler	Analyzes a number of attributes and counts the instances of each character.	Strings only
	Contained Attributes Profiler	Analyzes records to find pairs of attributes where one attribute value commonly contains another.	Any
	Data Types Profiler	Analyzes attribute values for their data type - String, Number or Date - and assesses data type consistency.	Any
	Date Profiler	Analyzes a Date attribute for date distribution by day of week, day of month, day of year, month and year.	Dates only
	Equal Attributes Profiler	Analyzes records to find pairs of attributes that commonly have the same values.	Any
	Frequency Profiler	Analyzes value frequency across many attributes.	Any
	Length Profiler	Analyzes a number of attributes and measures the length of values by number of characters.	Any
	Max/Min Profiler	Finds minimum and maximum values - longest, shortest, lowest and highest.	Any
	Number Profiler	Analyzes a Number attribute for number distribution across user-defined bands.	Numbers only
	Patterns Profiler	Analyzes character patterns, and pattern frequency, across many attributes.	Any
	Quickstats Profiler	Analyzes high-level completeness, duplication, and value frequency across many attributes, and highlights possible issues.	Any
	Record Completeness Profiler	Analyzes records for their completeness across many attributes.	Any
	Record Duplication Profiler	Analyzes records for duplicates across many attributes.	Any
	RegEx Patterns Profiler	Analyzes a number of attributes for values that match a list of regular expressions.	Strings only

Audit processors

For general information about Audit processors, see About audit processors.

Icon	Processor Name	Description	Compatible attribute types
	Business Rules Check	Validates the input attributes using business rules defined externally to OEDQ	Any
	Cross-attribute Check	Checks one attribute's value against another, using a comparison	Any
	Data Type Check	Checks that a String attribute contains data of the expected data type	Strings only
	Duplicate Check	Checks for records that are duplicated across selected attributes	Any
	Email Check	Checks Email addresses are in a valid syntactic format	Strings only
	GBR Postcode Format Check	Checks GBR Postcodes are in a valid syntactic format	Strings only
	Invalid Character Check	Checks a String attribute for invalid characters.	Strings only
	Length Check	Checks values for an attribute for valid character length, and/or a valid number of words	Any
	List Check	Checks values for an attribute against lists of valid and invalid values	Any
	Logic Check	Checks values against a Logic expression	Any except Arrays
	Lookup Check	Checks for a valid number of related records in Reference Data	Any
	No Data Check	Checks whether or not values in an attribute contain any meaningful data, aside from whitespace characters	Any
	Pattern Check	Checks values for an attribute against lists of valid and invalid character patterns	Any
	RegEx Check	Checks values for a String attribute against lists of valid and invalid regular expressions	Strings only
	Suspect Data Check	Checks values for an attribute for common data 'cheats', such as repeating characters, or short values	Any
	Value Check	Checks values in an attribute are equal to, higher, or lower than a given value	Any

Transformation processors

For general information about Transformation processors, see About transformation processors.

Icon	Processor Name	Description	Compatible attribute types
	Add Current Date	Adds a new Date attribute with the current Date/Time as its value.	N/A
	Add Date Attribute	Adds a new Date attribute with a given value.	N/A
	Add Numeric Attribute	Adds a new Number attribute with a given value.	N/A
	Add String Attribute	Adds a new String attribute with a given value.	N/A
	Character Replace	Replaces characters with mapped characters.	Strings and String Arrays only
	Concatenate	Concatenates String values.	Strings and String Arrays only
	Convert Date to String	Converts one or more Date attributes into Strings.	Dates only
	Convert Number to String	Converts one or more Number attributes into Strings.	Numbers only
	Convert Number to Date	Converts one or more Number attributes into Dates.	Numbers only
	Convert String to Date	Converts one or more String attributes into Dates.	Strings only
	Convert String to Number	Converts one or more String attributes into Numbers.	Strings only
	Date Difference	Calculates the difference between two Dates.	Dates only
	Denoise	Removes noise characters from text attributes.	Strings and String Arrays only
	Enhance from Map	Adds a new attribute by matching an existing attribute against a map.	Any
	Extract Values	Extracts values that match a list from an attribute.	Any
	Generate Initials	Generates initials from text values.	Strings and String Arrays only
	Hash Generator	Generates hash keys from input values	Strings and String Arrays only
	Lookup and Return	Looks up and returns related records from Reference Data.	Any
	Lower Case	Converts String values to lower case.	Strings and String Arrays only
	Make Array from Inputs	Creates an array from a number of input String attributes	Strings only
	Make Array from String	Creates an array by splitting up a String attribute using specified delimiters.	Strings only
	Merge Attributes	Merges together data from a number of attributes to create new merged attributes, by selecting the first not null value.	Any, including arrays
	Metaphone	Generates a metaphone code for one or more String attributes.	Strings and String Arrays only
	Normalize No Data	Normalizes no data values to Nulls, or to a custom string value.	Strings and String Arrays only
	Normalize Whitespace	Removes leading and trailing whitespace, and normalizes inter-word whitespace to a single space.	Strings and String Arrays only
	Pattern Transform	Transforms values using a map of character patterns	Strings and String Arrays only
	Proper Case	Converts text values to Proper Case.	Strings and String Arrays only
	RegEx Match	Matches a String attribute against a regular expression.	Strings only
	RegEx Replace	Replaces a String value matching a regular expression with a given value, or part of the matching expression.	Strings only
	RegEx Split	Splits a String attribute using a regular expression as a splitter.	Strings only
	Replace	Replaces values in a single attribute using a map. Use for standardization.	Any, including Arrays
	Replace All	Replaces values in multiple attributes using a map. Use for standardization or to remove dummy values.	Strings and String Arrays only
	Return Array Size	Returns the size, in number of elements, of an Array attribute.	Arrays only
	Select Array Element	Selects a numbered element from an Array, and extracts it into a new attribute.	Arrays only
	Soundex	Generates a soundex code for one or more String attributes.	Strings and String Arrays only
	Split Records from Array	Normalizes data by outputting a record for each element in the largest input array.	Arrays only
	Strip Numbers	Removes all numbers from String attributes.	Strings and String Arrays only
	Strip Words	Removes all words that match a list from String attributes.	Strings and String Arrays only
	Transliterate	Trims String values down to a set number of characters, from left, right, or middle.	Strings and String Arrays only
	Trim Characters	Trims String values down to a set number of characters, from left, right, or middle.	Strings and String Arrays only
	Trim Whitespace	Trims whitespace from String values.	Strings and String Arrays only
	Upper Case	Converts String values to UPPER CASE.	Strings and String Arrays only

Matching Processors

For general information about Matching processors, see About matching processors.

Icon	Processor Name	Description	Compatible attribute types
	Advanced Match	Matches any number of working and reference data sets, with all options configurable	Any except Arrays
	Consolidate	Consolidates multiple data sets, identifying and merging duplicate records	Any except Arrays
	Deduplicate	Identifies and merges duplicate records in a single data set	Any except Arrays
	Enhance	Enhances a working data set by matching it against one or more reference data sets, and merging in the matching reference data	Any except Arrays
	Group and Merge	Groups records together by an attribute or attributes, and merges records	Any except Arrays
	Link	Links two data sets together	Any except Arrays

Maths Processors

For general information about Transformation processors, see About maths processors.

Icon	Processor Name	Description	Compatible attribute types
	Add	Adds together numeric attributes, or a constant to a numeric attribute or attributes.	Numbers only
	Divide	Divides a numeric attribute value by a constant.	Numbers only
	Multiply	Multiplies numeric attributes together.	Numbers only
	Round	Rounds a numeric attribute to a configurable number of decimal places.	Numbers only
	Subtract	Subtracts one numeric attribute from another.	Numbers only

Text Analysis Processors

For general information about Text Analysis processors, see About text analysis processors.

Icon	Processor Name	Description	Compatible attribute types
	Parse	Analyzes and classifies data in a number of attributes, and uses the classifications to transform the structure of the data.	Any
	Phrase Profiler	Analyzes text attributes for common words and phrases, or characters and character sequences.	Strings only

Third Party Processors

For general information about Third Party processors, see About third party processors.

Icon	Processor Name	Description	Compatible attribute types
	Capscan Matchcode	Uses the Capscan Matchcode API to verify and standardize address data.	Any
	Experian QAS	Uses the Experian QAS Batch API to verify and standardize address data.	Any
	Address Verification	Uses the Address Verification software and data to verify, match and enhance address data.	Any. One input attribute must be a country name or code.

Advanced Processors

For information about Advanced processors, see About advanced processors.

Icon	Processor Name	Description	Compatible attribute types
	Add Message ID	Adds in the message ID for each record.	N/A
	Add User Details	Adds in details of the authenticated (calling) user of a web service	N/A
	Expression	Uses OEDQ's expression language to define the processor logic.	Any, except Dates
	Expression Filter	Uses OEDQ's expression language to define a 'test' to filter records.	Any, except Dates
	Generate Warning	Generates a warning or triggers a process failure if a set number of records are processed.	N/A
	Script	Uses a script to define the processor logic.	Any
	Message Handling Script	Uses a script to define processor logic which can act across multiple records in a single message.	Any

Read and Write

For general information about Readers and Writers, see About readers and writers.

Icon	Processor Name	Description	Compatible attribute types
	Merge Data Streams	Merges a number of streams of data into one, preserving all records.	Any
	Reader	Reads data from a data source (Staged Data, View or Real time provider).	N/A
	Writer	Writes data from a number of input streams to a staged data table.	Any

Product Data

Note that the Product Data processors will only appear if OEDQ has been configured to connect to OEDQ-P (Oracle Enterprise Data Quality for Product Data).

Icon	Processor Name	Description	Compatible attribute types
	Process Product Data	Allows the functionality of OEDQ-P to be used in OEDQ processes.	Any

GNR

Note that the GNR processors will only appear if OEDQ has been integrated with IBM Global Name Recognition.

Icon	Processor Name	Description	Compatible attribute types
	GNR Get Best Culture	Identifies the most likely origin culture of an individual's Given Name and Surname using linguistic rules	Strings only
	GNR Get Cultures	Identifies all the possible origin cultures of an individual's Given Name and Surname using linguistic rules	Strings only
	GNR Parse	Parses names into a defined structure using linguistic rules	Strings only
	GNR Search	Searches names against a defined set of Reference Data using linguistic rules and returns possible matches with scores	Strings only