Release Notes – Oracle Enterprise Data Quality 9.0.8

This document lists all significant improvements and bug fixes in minor releases of major version 9.0 of Oracle Enterprise Data Quality. Each minor release has a three part version number. The latest generally available release is version 9.0.8. The document is intended to allow customers to assess the impact of upgrading to each minor release, as well as providing information on fixes to known issues in earlier versions. Oracle always recommend using the latest version of the product.

To jump to the contents for each specific version, please use the following links:

Installation

For installation details, please refer to either of the following documents:

These are available in the ZIP file that you received containing this software.

Version 9.0.8

Fixes v9.0.8:

The following issues have been fixed in v9.0.8:



16731651: Advanced installation process does not work

Issue

When deploying dndirector.war following the process described in the Advanced Installation Notes document, rather than installing the product with the Windows installer, the deployment process only works if a configdir.txt file is used to indicate the location of the configuration directory. The described process is to deploy the war file (or expanded directory from the war file) then use the web pages to point to the config directory and complete the installation. The bug meant users would see an HTTP 500 error when displaying the launchpad. See alert 1549361.1 on My Oracle Support.

Fix

The installation package has been modified to use a temporary config directory, as in previous versions, and configuration pages are now correctly displayed after deployment.

Deliverable

dndirector.war



16753602: Export locking is now too strict on Data Stores

Issue

EDQ 9.0.7 takes a WRITE lock on a whole data store when running a data export task, unless the task runs in Append mode. This means that jobs with several export tasks, each exporting data to different tables in the same target database, in the same phase , cannot run. Such jobs ran in EDQ 9.0.6.

Fix

All data exports now take READ locks only, making it possible to run several export tasks in the same phase to the same data store.

Deliverable

dndirector.war



16469508: ClassCastException running Lookup and Return with Dates

Issue

An exception could be encountered if using a DATE column as one of the lookup columns in a Lookup and Return processor.

Fix

Lookups have been modified to work correctly with DATE types.

Deliverable

dndirector.war



14031629: Password expiry warning message box counting the wrong units

Issue

If using the functionality to check password expiry dates in external LDAP systems, the countdown to the expiry date was calculated in the wrong units. Messages reporting the number of days until passwords expire contained high numbers even when only a few days were left.

Fix

The calculation of expiry days has been corrected to whole days.

Deliverable

dndirector.war

Version 9.0.7

New Features and Improvements v9.0.7:



DIR-13602: Improve memory usage when snapshotting Excel files

Issue

The previous Excel drivers used a large amount of JVM heap memory space – many times the size of the Excel file – when reading from (or writing to) large Excel files. In many cases this was over sensible limits for client and even server heap space, leading to errors when snapshotting large Excel files that are difficult to work around by upping heap limits.

Fix

A new 'streaming' driver has been provided for reading Excel files in the xlsx file format. Rather than reading all cells including metadata from the source file into memory before snapshotting, the new driver streams data from the file. This uses much less heap memory and means that large Excel files can be snapshotted on both clients and servers with sensible heap limits. Note that:

1. The new driver requires that any formulae in the spreadsheet are up to date before snapshotting.

2. The new driver is only used for files over 64KB in size, as smaller files will fit into memory comfortably.

3. The new drivers are only used for reading data from Excel currently (not writing data).

Deliverable

dndirector.war



DIR-13467: Updated PostgreSQL driver (version 9.1)

Issue

For customers using PostgreSQL as the repository database for EDQ, the previously shipped JDBC driver for PostgreSQL was only compatible with older versions of PostgreSQL (up to 8.2).

Fix

The 9.1 version of the PostgreSQL JDBC driver is now provided. This is compatible with versions of PostgreSQL up to and including 9.1, allowing the use of more up-to-date and better supported versions of PostgreSQL.

Deliverable

dndirector.war



DIR-13757: Relax locking when running a job that appends to a database table in an Export Task

Issue

Previously, if a job phase exported some data to a Data Store, that Data Store was always globally locked, even if the export was running in Append mode. That meant that if the same job, writing to the same Data Store, was run twice on the same server with 2 different run labels, the 2nd job would wait for the locks of the 1st job to be released. This should not be necessary where the export tasks are configured to append data.

Fix

Where an export is writing to a database table in Append mode, the Data Store is no longer globally locked. This allows multiple jobs with different run labels to be run on the same server, writing data to the same Data Store, simultaneously.

Deliverable

dndirector.war



DIR-13695 and DIR-13703: Address Verification processor needs to function at some level even if EDQ Address Verification libraries are not installed.

Issue

In order to enable automated installation of the EDQ Customer Data Services Pack and EDQ Address Verification, the dependency on the Address Verification libraries being installed to import configuration that uses the Address Verification processor needs to be removed.

Fix

Previously, if EDQ Address Verification was not installed, or was incorrectly linked to EDQ (via av.properties), the Address Verification processor was not available in the Tool Palette, and any processes using it could not be imported in Director.

Now, the Address Verification processor always appears in the Tool Palette and can always run. If the Address Verification libraries are not installed, errors will still appear in main0.log, and the processor will always produce empty output attributes, with the exception of the output accuracycode, which will have a value of -1.0.

Deliverable

dndirector.war


DIR-13514: Support for LDAP compare operation for password verification

Issue

For some installs, the LDAP compare operation may be the preferred method for password verification. EDQ should support this alongside bind and password comparison.

Fix

Support has been added for the LDAP compare method of password verification, using the syntax [realmname].auth.method = compare in config/security/login.properties.

Deliverable

dndirector.war


DIR-13713 and DIR-13714: Improve canvas drawing performance when adding links to large processes

Issue

Adding links between processors to large and complex processes could cause a significant delay in returning control to the user due to re-validation of the canvas, and data attributes, and especially in the detection of cyclic links.

The problem was significantly worse when dragging links earlier in the process rather than later.

Fix

Some types of validation have been made more efficient, and others have been removed from the canvas and into the process model, allowing much quicker return of control to the user.

Deliverable

dndirector.war


DIR-13568: API: Using transferfile.js to import a project from a package (dxi) file can fail silently

Issue

The transferfile.js script, a documented way of importing projects from dxi files from the command line, could fail, but given no indication of failure, if the project import failed due to attempting to overwrite a locked configuration object (for example, an open or running process).

Fix

An optional check on the return message was added to preserve backward compatibility but to allow the script to be used in a way that will exit if it encounters an issue in importing or overwriting configuration. The command line parameter to use this check is -exitonerrormsg.

Deliverable

transferfile.js


DIR-13758: Ability for jobs triggered by JMS to wait for locks

Issue

It is now possible to use a JMS queue to balance batch job requests between multiple instances of EDQ. When a job is invoked in this way, there must be an option to wait for locks on configuration objects rather than always running in nowaitmode.

Fix

When jobs are triggered by consuming a message from a JMS queue, each job will wait for locks on the configuration objects

Deliverable

dndirector.war


DIR-13617: Changes to the Address Verification processor

Issue

In order to use new functionality and options available in EDQ Address Verification, the Address Verification processor has been changed. The new processor allows the use of all inputs, options and outputs available in the underlying Address Verification (Loqate) API.

Fix

The Address Verification processor has been modified to include new inputs, options and outputs.

Note that for backwards compatibility, the previous version of the Address Verification processor has been kept in EDQ so that existing processes will still work after upgrade. The old processor cannot be newly added to a process, and instances of Address Verification should be switched to use the new processor (by replacing the existing processor instance with a new instance from the Tool Palette as the old processor will be removed in a later EDQ version.

Deliverable

dndirector.war


DIR-11338: Ability to specify a link to an online help page within the deployed location of EDQ for a custom or published processor

Issue

There was no mechanism available to specify a link to a help page as part of a custom or published processor, for example a help page provided as part of an EDQ extension such as the Customer Data Services Pack.

Fix

It is now possible to specify the location of the help file for a published processor; either as a URL (external or application relative) or as a path within a help 'blueprint' (i.e. a series of help pages packaged as an EDQ user application and linked from the Launchpad).

Deliverable

dndirector.war

Fixed Issues v. 9.0.7:

The following issues have been fixed in version 9.0.7:

16471621: Runtime cache takes up too much memory for small reference data sets

Issue

In stress tests (repeated processing using many small reference data sets) too much memory was being used for each set, potentially leading to Out of Memory errors on the application server.

Fix

The caching code has been modified to use a much smaller memory footprint for small reference data sets.

Deliverable

dndirector.war


16382310: Internet Explorer error displaying web pages running on a Weblogic server with SSL enabled

Issue

Due to a security update, Internet Explorer now blocks certificates with keys that are less than 1024 bits long. Unless specified otherwise, Weblogic uses a 512 bit certificate, leading to an error in displaying any EDQ web pages.

Fix

The Advanced Installation Notes for Weblogic have been modified to advise installers to change the certificate key size to 1024 or 2048 bits.

Deliverable

Oracle EDQ Advanced Installation Notes.pdf



16221838: Database migration script fails when attempting to upgrade EDQ to 9.0 on an Oracle 10 or 11 database which has been converted from Oracle 9.

Issue

Step 87 in the database migration script to upgrade a repository database for EDQ 9.0 on an Oracle database converted from Oracle 9 fails with the error ORA-00955: name is already used by an existing object.

Fix

The database migration script has been updated to drop the pk_dn_taskstatus index after migration, so avoiding the error.

Deliverable

migrate.scr



16184425: EDQ fails to start up on AIX with the IBM JRE and Weblogic

Issue

EDQ fails to start up with a Context initialization error on AIX with the IBM JRE and a Weblogic application server.

Fix

EDQ Web Services used classes that were only compatible with the Oracle JDK when running on Weblogic. These have been refined for compatibility with the IBM JDK also.

Deliverable

dndirector.war


15946566: Add Environment Details processor should use user cache

Issue

The Add Environment Details processor calls the authentication server to get user information when a new user ID is found. If the call takes a long time, for example because of LDAP server contention or lockup, the processor will hang with its internal map locked in a synchronized block.

Fix

The Add Environment Details processor has been modified to use the user cache.

Deliverable

dndirector.war


15891308: Real-time processing could crash with chained processes and multiple readers

Issue

A real time process with multiple readers and a match processor could fail as the real-time cache was being erroneously initialized using input data stream names. This can cause an error because data stream names do not have to be unique.

Fix

The initialization of the real-time cache has been modified to allow for non-unique input data stream names.

Deliverable

dndirector.war



14466643: On Data Export, an ORA-01400 error was not being caught in the GUI or Event Log

Issue

When running a scheduled export the export appeared to 'hang' due to trying to insert a NULL into a column in a target database where a NULL is not allowed. An error was written to the server logs but the Export task did not fail, and there was no error in the UI.

Fix

If an export task attempts to write a Null value to a Not Null column in a target database, the export task and job containing it now fail, with an appropriate error reported back to the UI.

Deliverable

dndirector.war



13344094: EDQ Online Help does not display correctly in Internet Explorer 7 on Weblogic

Issue

By default, Weblogic does not define a MIME type for XML files. Consequently, when using Internet Explorer 7 on Weblogic, the EDQ online help pages were not being displayed properly.

Fix

An explicit MIME mapping was added to the web.xml for the application to overcome this.

Deliverable

dndirector.war


Version 9.0.6

New Features and Improvements v9.0.6:

Improved Subversion Integration

Issue

Various improvements have been made to the integration between Director and Subversion. These only affect customers using this integration. Improvements include:

  • Custom processor images were previous ignored. Now they are handled in the same way as other server-level objects.

  • The Project Rename dialog was previously disabled. Now it is possible to use it to set the Description of a project.

  • The EDQ version compatibility check has been relaxed only to require that the first two digits of the EDQ release number (e.g. 9.0) must be the same when multiple users are working with EDQ and the same configuration files in Subversion.

  • An Update operation in Director now lists the file that have been changed.

  • It is no longer possible to modify (by reversion, update or any other Subversion action) any locked configuration objects, as this led to confusing behavior (for example, changes to processes that were not reflected in open canvases)

  • Added visual feedback for tasks that could take some time (for example a Commit on a large project)

  • Other minor UI improvements.

Fix

The Subversion integration has been improved in the above ways.

Deliverable

dndirector.war



DIR-13687: Case Management Extended Attribute data entry validation

Issue

The ability was requested to be able to validate data entered into Extended Attributes in Case Management.

Fix

It is now possible to specify regular expressions or lists of allowed values as validation conditions for extended attributes in flags.xml, and validate data entered into the attributes according to the stated rules.

Where a list of allowed values is specified, the Case Management UI shows a picklist to the user.

Deliverable

dndirector.war



DIR-13635: Manage custom images in the tree (for improved version control)

Issue

Performing version control for custom images would be easier if there was an Images node in the Project tree.

Fix

There is now an Images node at Server level in the Project tree.

Deliverable

dndirector.war



DIR-13629: Ability to overwrite (re-publish) a published processor

Issue

It is currently not possible to overwrite/re-publish an existing published processor without first deleting the existing one.

Fix

It is now possible to overwrite/re-publish an existing published processor as a single operation.

Deliverable

dndirector.war



DIR-13175: Appending to an empty export file in real-time should not cause job failure

Issue

During real-time operation Oracle Watchlist Screening includes an export to an audit file which appends a record for each request submitted.

When the real-time job is first initialized, an empty export audit text file is created (with no headers yet). When the first record is pushed through, or when the job is canceled, header information is added to the file.

If the real-time job fails ungracefully when no real-time requests have been submitted, the text file is left in a completely empty state. The next time the job runs, it will give an error because it can't reconcile the column information within the empty file.

Fix

Headers are now written immediately on file creation, so the error is prevented.

Deliverable

dndirector.war

Fixed Issues v. 9.0.6:

The following issues have been fixed in version 9.0.6:

14805224: Real-Time Aggregations can no longer be configured in Dashboard Administration

Issue

The Real-Time Aggregations node in Dashboard Administration was always greyed out and new Aggregations could not be added.

Fix

Real-Time Aggregations now work as expected.

Deliverable

dndirector.war



14785838: Null Pointer Exception after opening an issue

Issue

In certain conditions it was possible to see a Null Pointer Exception after opening an Issue in the Issue Manager UI.

Fix

The race condition that caused this error has been fixed by adding synchronization to the handleSignal method.

Deliverable

dndirector.war



14651154: Unsupported operation exception when launching Server Console

Issue

In certain circumstances, an Unsupported Operation exception could be seen when launching the Server Console client application.

Fix

The race condition that caused this error has been fixed.

Deliverable

dndirector.war



14762096: The JNI connector to the Experian QAS API was incorrectly shipped as 32 bit rather than 64 bit

Issue

The Experian QAS processor uses a JNI connector to communicate with the QAS libraries.

The 32 bit Linux library was wrongly shipped instead of the 64 bit library, meaning there was no 64 bit library available. This meant the Experian QAS processor could not be loaded.

Fix

Both the 32 bit and 64 bit libraries are now shipped.

Deliverable

dndirector.war


14645912: The Business Rules Check processor loses configuration when used in a compound processor

Issue

There is no check currently performed to stop Business Rules Check from being used in a constructed compound processor, but when this is done, the configuration of the Business Rules Check processor is not stored.

Fix

A check is now performed such that it is no longer possible to include Business Rules Check as a member processor of a constructed compound processor.

Deliverable

dndirector.war



14587828: Null Pointer Exception when changing password in some client applications if password due to expire soon

Issue

A Null Pointer Exception could be encountered when attempting to change password in Match Review, Case Management Administration or Web Service Tester when the password was due to expire shortly

Fix

The change password functionality has been changed in these applications to cope with password expiry.

Deliverable

dndirector.war



14482845: Real-time screening job can hang due to unsynchronized thread

Issue

In some cases, Oracle Watchlist Screening real-time screening jobs were hanging due to an unsynchronized thread.

Fix

The real-time processing framework has been modified to avoid the problem.

Deliverable

dndirector.war



15831635: Java Webstart bug in Java version 7 update levels 6-9 breaks Issue links, Case links and Login as... links in EDQ

Issue

Any JNLP links that use query parameters (the ?... syntax at the end of the URL links to download the JNLP files) do not work when running Java version 7, update levels 6, 7, 8 or 9, due to a bug with Java Webstart.

This affects direct links to issues and cases in EDQ, as well as the Login as... links on the Launchpad.

Fix

The query string is no longer used in any JNLP links, so avoiding the problem.

Deliverable

dndirector.war



15827516: Message queue stream termination option loses queue entries

Issue

When the stream termination option is enabled when reading in records from a JMS queue, records are lost on the receiving end if the queue contains multiple "sets" of terminated records. For example, with a stream such as...

Item 1
Item 2
<StreamTerminator#1>
Item 3
Item 4
<StreamTerminator#2>


...an EDQ process will read up to
<StreamTerminator#1> and finish, but any additional items in the queue beyond the terminator will have been read from the queue and lost.

Fix

The stream termination logic has been changed in order to ensure that when a termination message is reached, any subsequent messages in the stream are not lost from the queue.

Deliverable

dndirector.war



14616269: Files required for GNR integration not included in Third Party folder in distribution

Issue

The two files makefile and namehunter.o, required in order to enable the GNR integration with EDQ, are not currently included in the core distribution of EDQ.

Fix

The files (for AIX and Linux) have been added into Third Party Software/gnr in the main zip distribution.

Deliverable

makefile and namehunter.o



14849616: Always prevent Active Directory login when password is blank

Issue

In certain circumstances, and when using the 'simple' bindmethod in the login.properties configuration that controls the connection to Active Directory, it was possible to login with a blank password.

Fix

Blank passwords are now always rejected in the LDAP authenticator.

Deliverable

dndirector.war

Version 9.0.5

New Features and Improvements v. 9.0.5:

DIR-13142: French, Italian and Spanish Localization

Issue

Localization of user applications in the above languages is required.

Fix

These localizations are now incorporated in EDQ.

Note that documentation (including online help) is not currently localized.

Deliverable

dndirector.war



DIR-12149: Data Interfaces: Real-Time Process Chaining

Issue

While it is possible to connect Batch processes together with Data Interfaces, it was not possible to do this with Real-Time processes.

Fix

It is now possible to connect Real-Time processes together with Data Interfaces.

Deliverable

dndirector.war



DIR-12077: Support Nested Groups in LDAP Integration

Issue

Active Directory supports subgroups (i.e. nested groups), but EDQ does not.

Fix

EDQ now supports AD nested groups. This is controlled using settings in login.properties.

Deliverable

dndirector.war

Fixed Issues v. 9.0.5:

The following issues have been fixed in version 9.0.5:

14268382: Error when creating a pattern frequency sample in a 'dirty' Parse processor

Issue

If a Parse processor on the Process canvas is marked for rerun ('dirty'), no warning message is displayed when attempting to create a frequency sample in the Select sub-processor. Instead, the user is able to configure and run the process. Only then is an error message displayed.

Fix

In such a scenario the user is now offered an option to create a new frequency resource only if the parse processor has been run and is not marked for rerun. If not, then a warning message is displayed.

Deliverable

dndirector.war



14244397: Merge data streams in real time using first message record problem

Issue

When running a real-time message stream through a match and attempting to merge back in a single record with the Relationships result, the single record does not merge, despite passing the Select first record from message widget.

Fix

This scenario is now handled correctly.

Deliverable

dndirector.war



14089630: Unable to import CDS dxi due to corrupt reference data

Issue

Importing the EDQ-CDS.dxi results in an error.

Fix

This was caused by an issue in Java 1.7, update 4 when working with Astral plane Unicode characters (using the SAX parser) and does not apply to previous versions. A fix was included to handle this in EDQ so that the import will work regardless.

Deliverable

dndirector.war



14026481: Web Service creation and import can fail on Weblogic

Issue

EDQ ships with an implementation of the SAAJ APIs from the JAXWS reference implementation. Java 1.6 and later also contain an implementation, but with different package names. If other code in the application server makes use of the version in the JRE, the EDQ web service creation and import will fail with a java.lang.ExceptionInInitializer Error.

Fix

The SAAJ classes have been removed from the loaded distribution.

Deliverable

dndirector.war



14288373: Reference Data Loading is not ordered at Runtime

Issue

Some processors – such as Replace - may attempt to perform a single replacement on a string value using Reference Data with multiple possible matches, especially where uniqueness is not constrained in the lookup keys. The reference data row that is selected for an operation on a given string value may be different depending on the way that the Reference Data is loaded, leading to non-deterministic transformations.

Fix

The Reference Data is now ordered at runtime leading to deterministic behavior even where there are multiple possible lookup matches in the Reference Data.

Deliverable

dndirector.war



14146958: Lookups on Staged Data sets that are written multiple times do not work correctly when using Run Labels

Issue

When running with a Run Label, the Runtime resource manager cache is not correctly cleared if the same staged data set is written multiple times within the same job. This makes it possible for a staged data lookup to lookup to the wrong 'version' of the data (a cached version, rather than the version that has just been written).

This may affect users of the EDQ Customer Data Services Pack, which uses this technique.

Fix

The resource caching capability has been fixed to work with Run Labels in the correct manner.

Deliverable

dndirector.war



14127500: Oracle Data Store password doubling in size after each save

Issue

Where a configuration file (DXI) containing a database-type Data Store connection with a saved password is loaded into a new system, its password string is not decrypted and is being doubled in storage size.

Fix

Where a saved password cannot be decrypted it is set to blank.

Deliverable

dndirector.war

Version 9.0.4

New Features and Improvements v. 9.0.4:

DIR-13547: The Replace processor struggles to perform with large data sets

Issue

A Replace processor was configured to process data using a reference data set containing a million entries. The performance was poor.

Fix

The algorithm for selecting the data to replace has been improved for large reference data sets.

Deliverable

dndirector.war



DIR-13499: GNR integration needs updating to work with EDQ 9.0

Issue

The IBM Global Names Recognition integration needs to be updated to be released with EDQ 9.0.

Fix

The IBM Global Names Recognition integration has been further documented and repackaged to enable the end customer to deploy without external assistance.

Deliverable

dndirector.war



DIR-13143: Oracle EDQP integration

Issue

Oracle EDQ now integrates with Oracle EDQP at the DSA level.

Fix

There is a new processor that can send data to an EDQP DSA and retrieve the results of the EDQP processing to EDQ.

Deliverable

dndirector.war



DIR-11245: User names are not listed in Case Management history when a user is deleted

Issue

When a user is deleted from the internal user store or is made unavailable to EDQ in a external LDAP store, the user name disappears from Case Management history.

Fix

A record of user names is now kept in EDQ, so that it can be presented in appropriate history displays.

Deliverable

dndirector.war



DIR-10624: GNR integration needs updating

Issue

The IBM Global Name Recognition product has been updated.

Fix

The IBM GNR integration has been updated to work with IBM GNR 4.2.

Deliverable

dndirector.war

Fixed Issues v. 9.0.4:

The following issues have been fixed in version 9.0.4:

14074905: Memory consumption increase when using large parse tokenization maps

Issue

The parser is causing tokenization maps to be reloaded into the resource cache. Over repeated runs this can cause memory consumption growth.

Fix

The parser is now using the resource cache correctly.

Deliverable

dndirector.war



14064112: Unable to save externalized options for exports in a DXI file

Issue

When an export process is configured with externalized run time options, these options are not being saved to exported DXI files.

Fix

The configuration options are now saved to the DXI file.

Deliverable

dndirector.war



14021299: For certain types of matching, the full cluster group is loaded into memory even if it exceeds cluster group threshold

Issue

When matching a single working data stream (with no “compare against self”) against reference data, the Match process loads the entire cluster group into memory even if it exceeds the size threshold. For badly configured Match processes this can cause out-of-memory exceptions.

Fix

Match now enforces the threshold limit at the cluster group loading time.

Deliverable

dndirector.war



13986335: Running EDQ-CDS jobs can cause stack overflow errors

Issue

The Tomcat version shipped with the Windows installer had a small stack size compiled.

Fix

The version of Tomcat shipped in the Windows installer has been upgraded to 6.0.35. This version of Tomcat has a larger default stack size.

Deliverable

dnDirectorSetup.exe



13949392: Input Stream and Record ID column labels are reversed in a new Case Management source

Issue

When creating a new source, the Record ID column is labeled Input Stream and the Input Stream column is labeled Record ID.

Fix

The labeling has been corrected.

Deliverable

dndirector.war



13894306: Unable to create project-level data stores without permissions to create system-level stores

Issue

The permissions associated with the Create Data Store dialogs were incorrectly coded, so that a user needed both project- and system-level Data Store Creation permissions to be able to progress through the Create Data Stores dialogs.

Fix

This has been corrected.

Deliverable

dndirector.war



13865174: In WebLogic JMS message reception can stop before starting with WebLogic message queues

Issue

The WebLogic implementation of the JMS API will return a Null response if it receives a receive() call before a start call. Normally, other message queues block the receive() call until the start call is received.

Fix

The code has been reordered and synchronized to ensure the message queue is always started before the receive() call is made.

Deliverable

dndirector.war



13844784: Reference data with large code point ranges can cause memory problems

Issue

A number of the processors and transformers use an underlying character transformation engine. The engine uses reference data containing character ranges to perform transformations. The implementation expands the character ranges to allow fast transformation. For some international languages this can consume huge amounts of memory.

Fix

The character transformation engine no longer expands character ranges for fast transformation but instead holds the ranges in their original form and performs slower checks during which the reference data can be held in memory.

Deliverable

dndirector.war



13824558: Search text in case management needs to be entered in lower case

Issue

The quick search functionality in case management doesn't find entries if the search test is entered in upper case.

Fix

Internally the search terms are converted to lower case to align with the storage of the indexes.

Deliverable

dndirector.war



13737207: LDAP integration with Oracle Internet Directory causes a NPE at start up

Issue

The Oracle Internet Directory does not provide a namingContexts attribute in the RootDSE, which causes a Null Point Exception at start up.

Fix

The LDAP integration now handles the case where no namingContexts are provided in the RootDSE.

Deliverable

dndirector.war



13655643: Groovy script integration with input arrays is shifted by 1

Issue

In a script processor, the following script returns the second element of the input array and not the first:

#! language : groovy

output1 = input1[0][0];

Fix

Array handling for Groovy script integration has been corrected.

Deliverable

dndirector.war



13635825: Support for QAS on Linux

Issue

Historically we have not provided an integration of QAS on Linux.

Fix

The JNI components have now been ported to Red Hat Enterprise Linux 5.

Deliverable

dndirector.war



13623789: The Extended Attributes list does not have a vertical scroll bar

Issue

The Extended Attribute Selection list in the Filter Construction area does not have a vertical scroll bar. Therefore, when there is a large number of extended attributes some will be off-screen and therefore cannot be selected.

Fix

The Extended Attribute Selection in the Filter Construction area has been modified to use a dialog with Search and List functionality.

Deliverable

dndirector.war



13622928: Attempting to use the filter functionality in the Reference Data Editor opened from a Processor Configuration dialog causes an NPE

Issue

In a Processor Configuration dialog, open the Reference Data Editor using the > button. If the filter functionality is used in this dialog, and NPE is caused.

Fix

The filter functionality has been updated to reflect the alternative paths that it can be opened from.

Deliverable

dndirector.war



13613924: Attempting to run Real-Time Match with no working data input stream enabled causes an NPE

Issue

Attempting to run Real-Time Match with no working data stream enabled causes an NPE.

Fix

The process now provides a more helpful error message, indicating the Match processor needs as least one working data stream enabled.

Deliverable

dndirector.war



13608570: Unique constraint error when attempting to generate cases from a badly configured match processor

Issue

If two Match data streams are mapped to the same Case Management data stream, a unique constraint error can occur after migrating from 8.1 to 9.0.

Fix

Match processors now correctly indicate that such a configuration is invalid.

Deliverable

dndirector.war



13570687: The Configuration Analysis report does not report the Run Preferences content

Issue

The Configuration Analysis report does not generate content for the Run Preferences configuration of processes when saved to HTML.

Fix

The system now correctly generates HTML for the Run Prefer configuration.

Deliverable

dndirector.war



13566412: The Configuration Analysis report does not report export options.

Issue

The Configuration Analysis report does not generate content for the Export Options configuration when saved to HTML.

Fix

The system now correctly generates HTML for Export Options configuration.

Deliverable

dndirector.war



13566256: Configuration Analysis Comparison reports incorrect job names when comparing jobs

Issue

When two jobs in a project are selected and a Configuration Analysis Comparison report is run, both jobs will have their name reported as the first job name.

Fix

The Configuration Analysis Comparison report now correctly reports the job names.

Deliverable

dndirector.war



13566119: Configuration reports are generated in the server locale

Issue

Configuration reports are generated in the server locale, rather than the client locale.

Fix

Configuration reports are now generated in the client locale.

Deliverable

dndirector.war



13528962: Client application runs out of memory when running a Case Management report with one or two axes

Issue

When a large single axis or two axes Case Management report is run on a powerful server and the results are returned to a client, the rate at which the server generates data can overwhelm the client's ability to handle it. This results in the data being buffered for processing until the client runs out of memory.

Fix

The protocol handling between the client and server has been modified so that the client can pause the data flow.

Deliverable

dndirector.war



13514608: Importing and then deleting Match processes which contain decisions can cause table leaks.

Issue

If DXI containing a Match processor with a decision attached to it is imported, then the decision tables are imported but are not correctly linked to the Match processor. When the Match processor is next run it will connect to the tables; however, if it is deleted before it is run then the tables will have no reference, and therefore will leak.

Fix

When the decision tables are imported they are now correctly linked to the Match processor.

Deliverable

dndirector.war



13485976: Progress reporting for Record-Based Interval Real-Time processes is not working

Issue

Progress reporting for Record-Based Interval Real-Time processes is not working.

Fix

The record count is now being flushed, so it progresses at appropriate intervals.

Deliverable

dndirector.war



13432068: Configuration Analysis Comparison reports incorrect external tasks names when comparing external tasks

Issue

If a Configuration Analysis Comparison report is run on two external tasks in a project, then both external tasks will have their name reported as the first external task name.

Fix

The Configuration Analysis Comparison now correctly reports the names of external tasks.

Deliverable

dndirector.war



13370322: Running a job from the Server Console when all phases have been disabled by run profile produces an error in the log

Issue

Running a job from the Server Console when all phases of the job have been disabled by a run profile produces an error in the log.

Fix

The job now runs correctly, albeit with all phases disabled.

Deliverable

dndirector.war



13369120: NPE when accessing Event log

Issue

The Event log needs various aspects of the back-end configuration downloaded and cached in the UI before it becomes fully functional. If the controls are used before all the data is cached you get an NPE.

Fix

The controls are now disabled until all the caches have been successfully loaded.

Deliverable

dndirector.war



13261607: Explanation notes in run profiles example need expansion

Issue

The explanation notes in the run profiles sample file indicates that white spaces property names must be escaped, and how to do this. However, the notes do not make it clear that white spaces in the values of the properties do not need to be escaped.

Fix

The notes in the run profile template have been clarified.

Deliverable

config.zip



13117504: After deleting a job that had a schedule, the Scheduling dialog gives a NPE on opening

Issue

A job had been deleted without the schedule being cleared out, which caused an NPE in the Scheduler dialog. We were unable to reproduce this error.

Fix

The Scheduler dialog has been made more robust, so it will not fail it the schedule includes references to a job that no longer exists.

Deliverable

dndirector.war



13060407: A Parse processor downstream from a Match processor can cause Merge data stream progress to be incorrectly displayed

Issue

Under certain conditions the Run Time engine was not calling the finished() method, which indicated a data stream had completed.

Fix

The Run Time engine has been corrected.

Deliverable

dndirector.war



13049695: Canceling a process containing uninitialized staged data can cause the process to hang

Issue

Running a process which has uninitialized staged data and then immediately trying to cancel it can cause the process to hang.

Fix

Error handling improved when process about to fail and a cancel request is received.

Deliverable

dndirector.war

Version 9.0.3

New Features and Improvements v. 9.0.3:

DIR-13411: Put ‘Split Records from Array’ into the Audit license pack

Issue

Split Records from Array processor was only enabled when the Transformation functional pack was enabled.

Fix

This processor is now enabled when the Audit functional pack is enabled. This allows analysis of results from a Lookup and Return processor that returns several records (in array attributes).

Deliverable

dndirector.war



DIR-13405: Ability to restrict concurrent applications running on a per-user basis

Issue

Previously, it was not possible to restrict the number of concurrent logins a given user could make to any OEDQ application.

Fix

A limit can now be set in login.properties, either globally or per realm.

To set this globally (for all realms, including the ‘internal’ realm meaning the users set up and administered in OEDQ itself), use the following line:

sessionlimit = 1

To use different settings for different realms, specify the realm name before the parameter – for example:

internal.sessionlimit = 1

[external realm name].sessionlimit = 1

Deliverable

dndirector.war



DIR-13225: String displayed for a user in case assignments, history etc. should be configurable

Issue

In case assignment lists, the 'display' version of a user is shown - this is the user name or the username@realm. This information was not always useful, i.e. it was not always possible to quickly identify an actual user from their user name.

Fix

It is now possible to configure the string displayed with login.properties.

Deliverable

dndirector.war

Fixed Issues v. 9.0.3:

The following issues have been fixed in version 9.0.3:

13562979: Web Service authentication sharing fails with proxy server

Issue

A Proxy server situated between users and the EDQ application server prevents cookies being shared between applications accessing the EDQ server and applications accessing user-configured Web Services.

Fix

The logic determining the sharing of session cookies has now been improved.

Deliverable

dndirector.war



13557931: Process errors with NPE if it encounters a null match output from EDQ AV

Issue

The EDQ AV processor raises an error if no data is returned from the EDQ AV Server.

Fix

The EDQ AV processor has been configured to correctly handle null match outputs.

Deliverable

dndirector.war



13557822: Array index out-of-bounds exception when heavily loaded, multi-threaded matching web service is running

Issue

Multi-threaded match processing servicing real-time requests can (rarely) cause array index out-of-bound exceptions.

Fix

Synchronization between threads has been improved.

Deliverable

dndirector.war



13543326: Pasting data into a reference data grid in a German UI displays an English dialog box

Issue

The Confirmation Dialog displayed in the German interface when data is pasted into reference data contains the text “Add Data”. The comment and active status of the pasted data is also displayed in English.

Fix

The messages have been externalized for translation.

Deliverable

dndirector.war.



13540707: Non-ASCII user details are corrupted when saved

Issue

Non-ASCII characters in user detail fields (except for User Name and Full Name) can lead to the data being corrupted when it is saved.

Fix

All user details are now correctly encoded before being written to the database.

Deliverable

dndirector.war



13529285: Use TRUNCATE rather than DELETE when using JNDI for Oracle connection details

Issue

When using JNDI to specify database connection details, EDQ uses the DELETE statement rather than TRUNCATE to delete all the information in a table. This caused performance issues.

Fix

The code that analyzes the database type being connected can now identify JNDI details and therefore use the correct SQL optimizer.

Deliverable

dndirector.war



13524295: Importing certain reference data from a DXI can cause an NPE

Issue

Reference data created in Director 6.1 can cause a Null-Pointer Exception when imported into a later version of Director or EDQ.

Fix

The import migration has been corrected to cope with reference data created in Director 6.1.

Deliverable

dndirector.war



13506367: Emails are being sent on bulk updates

Issue

Email notifications are sent when cases are bulk updated.

Fix

Email generation is suppressed when bulk updates are applied to cases.

Deliverable

dndirector.war



13506253: Modified by and Date values are always changed on a bulk update even if the case isn't changed

Issue

The Modified By and Modified Date values of all cases targeted by a bulk update operation are changed, even if some of the cases within the operation have not changed.

Fix

Unchanged cases included within a bulk update are now successfully detected, and the Modified By and Modified Date values are preserved.

Deliverable

dndirector.war



13500511: Scheduling a job when using Subversion integration causes an error in the client

Issue

When using Subversion integration, attempting to schedule a job causes an error in the client.

Fix

The scheduling code has been modified to handle higher-level APIs that deal with accessing jobs, and can therefore operate with Subversion integration.

Deliverable

dndirector.war



13495797: Generate warning processor no longer displays warnings in notifications

Issue

Warning messages were not included in the text of job notifications.

Fix

Warning messages are now captured correctly.

Deliverable

dndirector.war



13491609: On migrating from 8.0 or 8.1 to 9.0, duplicate cases can be produced when running matching processes

Issue

Migrating cases from version 8.0 or 8.1 to 9.0 leads to duplicates being created. Specifically, when processing cases with alerts relating the records in different data sets, the migration code can create an alert key different to the key that would be generated during matching. This then leads to duplicate alerts being generated.

Fix

The migration code has been modified to agree with the matching alert key generation code.

Deliverable

dndirector.war



13491310: Job notifications are not being sent

Issue

Jobs running with the default job notification template do not generate email notifications because of an error in the default job notification mail template.

Fix

The template has been corrected.

Deliverable

dndirector.war.



13479899: Disabling a process in a job phase where many processes are fed from the same data source can cause the job to hang

Issue

In a job phase containing a staged data set that feeds many processes, disabling one of these processes can cause the job to hang.

Fix

The mission manager has been modified to cope with this scenario.

Deliverable

dndirector.war



13475850: Opening multiple workflows using the button in the Workflow Editor toolbar does not refresh the parameter table

Issue

If, when editing a workflow, a different workflow is opened using the button in the toolbar, the parameter table is not refreshed to the new workflow.

Closing the first workflow before opening the second for editing does refresh the parameter table correctly.

Fix

The GUI has been corrected to refresh the parameter display when a new workflow is opened.

Deliverable

dndirector.war



13470370: The parse results do not support sort filtering

Issue

Certain results views in the parser, for example Base Tokenization, cannot be sorted or filtered.

Fix

Sort filtering for parse results has been enabled.

Deliverable

dndirector.war



13464634: Index out-of-bounds exception when defining data associated with an alert

Issue

Deleting and immediately adding an attribute when editing data in Case Management can cause an index out-of-bounds exception.

Fix

The UI stored this data in a map, under the assumption that it was ordered. As the data was not ordered, this led to the index out-of-bounds exception. The in-memory attribute storage has now been corrected accordingly.

Deliverable

dndirector.war



13452712: The Case Source wizard has state names that do not comply with Oracle rules on translation

Issue

In the Case Source wizard the internal match states are referred to as AWAITING_REVIEW and POSSIBLE_MATCH. This type of string will not be translated by the Oracle translation group.

Fix

The match states are now displayed as localizable strings: “Awaiting Review”, “Possible Match” etc.

Deliverable

dndirector.war



13452291: When using the Subversion integration and editing a process's run preferences, the process is not marked as modified

Issue

When using EDQ integrated with Subversion, items with local modifications should be indicated as such in the project tree.

However, if the run preferences of an otherwise unmodified process are edited, the process is not marked as modified in the project browser, although the parent Processes node is marked as modified.

Fix

A process is now marked as modified if its run preferences are changed.

Deliverable

dndirector.war



13442369: NPE generated when committing changes to global reference data with Subversion integration

Issue

When using Subversion integration, committing changes to global reference data causes an NPE.

Fix

The Subversion integration has been taught to handle global reference data.

Deliverable

dndirector.war



13441180: The Windows installer database administration tool can no longer create meaningful useful databases

Issue

In accordance with good security practice, the Windows installer configuration utility creates new databases with randomly assigned passwords. However, the passwords are not communicated to the user.

Fix

The GUI database creator utility has been withdrawn and command line instructions provided.

Deliverable

dnConfig.jar



13440903: When pasting items that clash with an existing item, the Resolve Names dialog can disable the Rename checkbox

Issue

When pasting a number of items from one project to another, if there is a name clash the Resolve Names dialog is launched. The Rename check box is selected by default but marked as read only. If the box is deselected, it is impossible to reselect it.

Fix

The Rename checkbox is no longer marked as read only.

Deliverable

dndirector.war



13432107: Missing labels in configuration analysis of jobs including exports

Issue

When performing a configuration analysis of a job that includes export processes, the labels “Append”, “Replace” and “Delete” are missing from the report.

Fix

The labels have been added to the report.

Deliverable

dndirector.war



13431861: NPE when opening an alert generated from a consolidate processor

Issue

When a case alert generated from a consolidate processor is viewed in Case Management, an NPE is generated.

Fix

The rendering code in Case Management has been updated to support the data shape provided by the consolidate processor.

Deliverable

dndirector.war



13430847: NPE when canceling a matching process

Issue

Canceling match processors that are generating cases can cause an NPE.

Fix

The code has been made more robust to handle cancellation during case generation.

Deliverable

dndirector.war



13430596: Message displayed when canceling external tasks is malformed

Issue

The message displayed when canceling an external task includes the marker {2}, which should be substituted with the path to the executable.

Fix

The external executable path is now being passed as a parameter to the message.

Deliverable

dndirector.war



13428708: Show Characters functionality does not differentiate between carriage return, line feed and tab in comments

Issue

The Show Characters functionality in the Results view displays the correct values for carriage return, line feed and tab characters, but characterizes them as white space.

Fix

The Show Characters dialog no longer characterizes carriage returns, lines feeds and tabs as white space.

Deliverable

dndirector.war



13424225: Security permissions in default security groups are not retained correctly across migration

Issue

Default security groups do not retain Messaging System, View Dashboard, Dashboard Administration, Administer Issues or Edit Review Comments permissions across migration.

Fix

The permissions are now retained.

Deliverable

migration.jar



13419115: Web Service Tester Charting dialog is not displayed

Issue

The Web Service Tester Charting dialog was configured to use a deprecated library.

Fix

The dialog has been modified to use the new charting library.

Deliverable

dndirector.war



13417387: The Logged In As message in Case Management does not respect the local PC date format

Issue

The Case Management application displays a message indicating who a user is logged in as and when. The message does not respect the local date format.

Fix

The date format setting now respects the local machine date format.

Deliverable

dndirector.war.



13370362: NPE in real-time matching when using “Compare Against Self” option

Issue

If the Compare against self option is selected and the Match groups share working records option is deselected, running a real-time matching process with Case Management will generate an NPE.

Fix

Code has been corrected to handle these circumstances.

Deliverable

dndirector.war.



Version 9.0.2

Fixed Issues v. 9.0.2:

The following issues have been fixed in version 9.0.2:

13458206: Case source configuration files when exported using Oracle driver 11.2.0.1 cannot be imported

Issue

The internal representation of the INTEGER type changed in the Oracle 11.2.0.1 driver from NUMBER(38) to NUMBER(0,*). This cause problems when mapping the types back into EDQ.

Fix

The mapping of types into EDQ has been modified to cope with this change of representation.

Deliverable

dndirector.war.



13441667: Adding a new supplementary data attribute to a case source can cause cases generated with this new attribute to fail to open in Case Management

Issue

The attributes are held in a map. Indexes using Sequence IDs are held to refer back to the map. Adding new attributes can cause the IDs to change, depending on the hashes generated from the data.

Fix

Attributes are now held in an ordered list to ensure order is preserved.

Deliverable

dndirector.war.



13441456: Deleting Case Management supplementary data attributes and adding new ones with the same name can cause display information to be lost

Issue

Deleting Case Management supplementary data attributes and adding new ones with the same name can cause display information to be lost. If cases have been generated with the old information and for the same case with the new information, it will not be possible to view old information.

Fix

The ID for holding the attribute description is now also checked against display information to ensure it is not overwritten.

Deliverable

dndirector.war.



13416876: Incorrect message displayed when updating the configuration directory to an empty value

Issue

If the interface language is German, when the user attempts to set the configuration directory to an empty value via the Server Configuration link on the Launchpad, invalid characters are displayed in the returned error message.

Fix

The web pages now correctly set the encoding type to UTF-8.

Deliverable

dndirector.war.



13431856: After committing a new project at the project level, subsequent commits can fail

Issue

Creating a new project, adding items to it and then committing the project into source control using the menu option at the project level can cause subsequent commits at the project level to fail with a NegativeArraySizeException. A system restart will stop the issue occurring.

Fix

The initial commit was damaging an in-memory cache. This is now prevented.

Deliverable

dndirector.war.



13414799: Right clicking in the Column Headers when displaying the characters of an empty string causes the application to crash

Issue

In the Results Browser select “Show Characters” on an empty string. Once the Show Characters dialog is displayed, right click in the Column Headers. The GUI will now close with an IllegalArgument exception.

Fix

Check for null values returned from display grid.

Deliverable

dndirector.war.



13412498: Dashboard displays garbage rather than non-ascii characters

Issue

Dashboard values set to use a non-ascii character, for example è, are incorrectly displayed.

Fix

The web pages now correctly set the encoding type to UTF-8.

Deliverable

dndirector.war



13412092: Incorrect characters displayed on the German version of the Account Security page

Issue

When the Account Security page (found via the User Configuration page) is viewed in German, non-ascii characters are incorrectly displayed.

Fix

The web pages now correctly set the encoding type to UTF-8.

Deliverable

dndirector.war



13411663: The application names on the Application Groups page are set using the server locale rather than the client locale

Issue

When the EDQ is installed on a server with a German locale but with a browser set to use English, the application names on the Applications Groups page are incorrectly displayed in German.

Fix

The localization is now using the client locale rather than the server locale.

Deliverable

dndirector.war.



13411622: Some messages in the Results Book Configuration dialog have not been externalized for translation

Issue

In the Results Book Configuration dialog the string “Process” and “Latest” have not been externalized for translation.

Fix

These strings have now been externalized.

Deliverable

dndirector.war.



13411603: Tool tip on “>” button in Show Characters dialog is not externalized for translation

Issue

The tool tip on the > button in the Show Characters dialog is not externalized for translation.

Fix

The tool tip is now externalized.

Deliverable

dndirector.war.



13406777: Upgrading causes Group and Merge processor to be marked as invalid

Issue

When upgrading to 9.0.1, the Group and Merge processor was marked as invalid. Therefore it was not possible to run any processes containing these processors.

Fix

The migration of match processors to 9.0 was incorrectly marking the Group and Merge processor as changed. This has been corrected.

Deliverable

dndirector.war.



13405916: Setup Snapshot dialog title is not externalized for translation

Issue

When creating new snapshots the dialog title of Setup Snapshot is not externalized for translation.

Fix

The dialog title is now externalized for translation.

Deliverable

dndirector.war.



13405739: Configuration report for reference data has some messages which have not been externalized for translation

Issue

The Configuration report for reference data has not had the “Reference Data” and “Active” strings externalized for translation.

Fix

The messages are now externalized.

Deliverable

dndirector.war.



13405555: User session page has some messages not externalized for translation

Issue

On the web page displaying user sessions there are a number of messages that have not been externalized for translation.

Fix

The messages have now been externalized.

Deliverable

dndirector.war.



13402806: Confirmation dialog for deleting extensions has not been externalized for translation

Issue

When attempting to delete an extension, the confirmation message - “Are you sure you want to delete the selected files?” - is always displayed in English.

Fix

The confirmation message is now externalized for translation.

Deliverable

dndirector.war.



13402365: Attempting to log on to the Dashboard with invalid credentials gives a failure message in the server locale

Issue

When an invalid user name or password is entered when logging into the Dashboard page, the failure message is always displayed in the language set in server locale.

Fix

The Login Failure message now inherits the browser locale settings.

Deliverable

dndirector.war.



13400676: Results Filter dialog has messages not externalized for translation

Issue

The Results Filter dialog has a number of messages that have not been externalized for translation.

Fix

The messages have now been externalized for translation.

Deliverable

dndirector.war.



13386793: When running with a configuration repository in postgres and with version control enabled, reverting a reference data fails

Issue

When running with version control enabled and the configuration repository held in Postgres, reverting reference data fails. This is because when the reference data is reverted the reference data in the configuration repository is reloaded. The “DROP” SQL statement was correctly part of the reload translation but the “CREATE” SQL statement was not.

Fix

All of the necessary SQL is now correctly run within the correct transaction.

Deliverable

dndirector.war.



13383187: Derived state change not triggered when a new alert is added to a case

Issue

When a new alert is added to a case the derived state of the related case is not being updated.

Fix

When new alerts are added the derived state of the related case is now updated and associated trigger fired.

Deliverable

dndirector.war.



13370349: When multiple download tasks are run in the same job phase the task status is not being correctly recorded in the event log

Issue

The task_id for the download tasks was not being set. This is a key column in the dn_taskstatus table.

Fix

The task_id is now set for download tasks.

Deliverable

dndirector.war.



13364918: Running a process containing a Match processor where some data sources are not available can lead to the process becoming locked and uncancellable

Issue

Running a job that includes a process with a Match processor where not all sources of data are available can lead to some Match computation threads terminating without correctly notifying the rest of the Match computation threads. This leads to the process failing to close as the computation threads are waiting for threads that have already terminated to notify them that they have terminated.

Fix

Thread termination notification now occurs correctly in Match computations where some input streams are not available.

Deliverable

dndirector.war.



13363310: Exporting data from a Case Management filter exports all columns rather than just the columns that are being displayed

Issue

When exporting data from a Case Management filter the data for all the columns is being exported to Excel rather than just the columns that have been selected for display.

Fix

Case management export to Excel now respects the selected column.

Deliverable

dndirector.war.



13362562: The Configuration Report dialog has not externalized the “Complete” message

Issue

The Configuration Report dialog has not externalized the “Complete” message for translation.

Fix

The message has now been externalized.

Deliverable

dndirector.war.



13360286: The Group and Merge processor running in a real-time process produces cluster group size warnings

Issue

If the Group and Merge processor in a real-time process has a group size limit of 1 (set in the Advanced Options in the Group and Merge processor configuration screens) skipped group warnings are recorded in the logs.

Fix

The Group and Merge processor when connected to a real time data stream was erroneously initializing some elements of the match engine that weren't needed. This has been corrected, these elements of the match engine are no longer initialized.

Deliverable

dndirector.war.



13359845: Workflow Selection wizard in Case Source setup does not have all messages externalized for translation

Issue

The “Case” and “Alert” messages in the New Source Data dialog have not been externalized.

To view this dialog, within a Match processor select the Case Source Management tab in the Advanced Options dialog, and opt to create a new data source.

Fix

The message has now been externalized for translation.

Deliverable

dndirector.war.



13358483: When multiple jobs are running in the same project, the Current Tasks display in Server Console will cease displaying all jobs associated with the project when the first job finishes

Issue

When a job is completed, the project it is associated with is removed from the Server Console Current Tasks display.

Fix

The project is now only removed if no other jobs are running in the project.

Deliverable

dndirector.war.



13345013: The Job Editor cannot display a phase containing elements using a data interface if that data interface is deleted

Issue

The job editor is unable to display phases which contain elements that use a data interface that has been deleted.

Fix

The job editor can now cope with data interfaces that have been deleted.

Deliverable

dndirector.war.



13343511: Users without View Case permission can still view alerts if provided with the URL of the case

Issue

Case Manager provides a URL that will web start Case Manager and go directly to a case or alert. This works even for a user without permission to view cases and alerts.

Fix

The View Case permission is now checked correctly.

Deliverable

dndirector.war.



13340540: Pressing “Select Columns” in the Event Log window closes the window

Issue

Clicking the “Select Columns” button in the Event Log window closes the window.

Fix

The Column Selection dialog is now displayed when the button is clicked.

Deliverable

dndirector.war.



13340367: The Administrators group is missing two permissions on new installations

Issue

On a new installation, the Administrators group is missing the following permissions:

Director: Upload images

Director: Access User List Data Store

Fix

The Migration script has been updated to include these permissions.

Deliverable

migrate.jar



13340257: When snapshotting data from the “System information” CaseComments table in Director, a column ref “Source Name” error occurs

Issue

When capturing Case Comments data from the System information data store, a SQL error occurs. This is because changes to the data model made previously correct SQL ambiguous.

Fix

The SQL generating the Case Comment data has been corrected to be unambiguous.

Deliverable

dndirector.war.



13339854: Error appears in the log when attempting to view the results of a Group and Merge processor that hasn't been configured or run

Issue

When trying to view the results of a Group and Merge processor that hasn't been configured or run, an IndexOutOfBoundsException is logged in the Server logs.

Fix

Attempting to view the results of such a Group and Merge processor in these circumstances no longer returns an Exception error.

Deliverable

dndirector.war.



13339813: Exception logged when viewing the results of a Maths processor in a composite

Issue

An IndexOutOfBoundsException is logged when trying to view the results of a Maths process in a composite processor.

Fix

This has been corrected and results of the processor can be viewed.

Deliverable

dndirector.war.



13339348: Exporting through data interfaces fails

Issue

Attempting to export from a data interface fails with the error message: “Cannot run export from Data Interface”

Fix

System has been amended to allow exporting from a data interface.

Deliverable

dndirector.war.



13334772: Null Pointer Exception caused when comparing configuration of identical reference data and lookup

Issue

If there are two projects with identically named and structured lookups and reference data, comparing them will cause Null Pointer Exception.

Fix

Reporting fixed to handle this scenario.

Deliverable

dndirector.war.



13334685: Null Pointer Exception caused when comparing configuration of exports and Results Book exports of the same name

Issue

Comparing two projects where one has an export and the other than a Result Book export of the same name will cause a Null Pointer Exception.

Fix

Reporting fixed to handle this case.

Deliverable

dndirector.war.



13334534: The default folder for exporting Configuration reports is the current working directory

Issue

The default folder for exporting Configuration reports is the current working directory, which for web-started applications will default to the browser directory (e.g. c:\Program Files\Mozilla Firefox). If a relative path is specified the export normally fails.

Fix

The default folder is now the user's home directory.

Deliverable

dndirector.war.



13334268: Null Pointer Exception when comparing configuration of snapshots and staged data of the same name

Issue

Comparing two projects where one has a snapshot and the other has staged data with the same name causes an NPE.

Fix

Reporting fixed to handle this case.

Deliverable

dndirector.war.



13334207: Comparing configuration of export tasks displays property names rather than meaningful text

Issue

When comparing export tasks the report dialog displays the text “diff.title”, “diff.config” and others when meaningful text should be displayed instead.

Fix

Labels are now available to be substituted in report dialog.

Deliverable

dndirector.war.



13333786: Error reported when first setting configuration directory

Issue

When the configuration directory is first set through the launchpad pages an error is reported.

Fix

This was caused by the system attempting to make the connection secure over SSL. This has been corrected to use a redirect rather than a forward.

Deliverable

dndirector.war.



13330439: Removing published properties from a composite where the published property has been externalized can leave the externalized value but will remove the published property

Issue

This is an issue with externalizing properties of composite processes.

If a such a property is published, externalized so it can be viewed on the Externalized Properties dialog, but then is set to not be published, it will still be visible on the Externalized Properties dialog.

Fix

The Process Editor now correctly removes any externalized properties if they are removed from publication.

Deliverable

dndirector.war.



13324194: Closing the Externalization dialog without making changes marks the canvas as having unsaved changes

Issue

If you click OK on the Externalization dialog without having made any changes the canvas is marked as having unsaved changes even though no change has been made.

Fix

Clicking OK in the Externalization dialog now only marks the canvas as having unsaved changes if changes have actually been made.

Deliverable

dndirector.war.



13323606: A processor with an invalid family icon displays Datanomic logo rather than Oracle branding

Issue

Assigning a custom image to a processor and then deleting the custom image makes the processor display the Datanomic logo.

Fix

Default logo now defined with Oracle branding.

Deliverable

dndirector.war.



13323249: Project Browser not indicating when a process is modified when EDQ integrated with source code control

Issue

When EDQ is integrated with source code control, the Project Browser does not indicate that a process is modified when its run preferences are changed.

Fix

The Run Preferences change is now correctly broadcast.

Deliverable

dndirector.war.



13262191: Error message appears in Java Console when filtering in the Edit Reference dialog

Issue

When you select the Filter button on a Date column in the Edit Reference dialog, an error message appears in the Java Console indicating that “icon/calendar.png” could not be found.

Fix

The icon is now correctly located and is displayed correctly.

Deliverable

dndirector.war.



13262154: The Match Review dialog retains old orange icon rather than new blue icon

Issue

When reviewing results in the Match Review application the Review dialog contains the orange icon rather then the new blue icon.

Fix

The icon has been updated to the new blue icon.

Deliverable

dndirector.war.



13262102: The Server Console allows reference data to be edited; it should not

Issue

When reference data is being viewed in the Server Console using the “Show Characters” option it possible to edit single characters in the reference data.

Fix

This functionality has been removed from the Server Console.

Deliverable

dndirector.war.



13261868: Copying a large project can cause the canvas to appear to hang

Issue

The canvas handles events for all projects on the same server it is displaying a process from. When importing or copying projects this can lead to a significant load on the canvas.

Fix

The canvas now filters by project as well, so it only processes events from the same project as the process it is displaying.

Deliverable

dndirector.war.



13261593: Sample Run Profile Properties file is missing a necessary escape character

Issue

The sample Run Profile Properties file includes the sample property #phase.New Phase.enabled = yes. An escape character is required for the space character in the New Phase term.

Fix

Escape character added.

Deliverable

config.zip



13261282: When running processes with run profiles, overridden resources may not be locked

Issue

When running jobs with resources overridden using run profiles, the original resources are locked rather than the resources that they have been overridden to.

Fix

The locking is now correct and the resources actually being used are now locked.

Deliverable

dndirector.war.



13257266: When copying a filtered project all items are being displayed in progress even though only non-filtered projects are copied

Issue

When copying a filtered project only the visible items are actually copied. However, the task panel shows all the items in the progress.

Fix

Progress messages are now correctly broadcast to list items that are actually copied.

Deliverable

dndirector.war.



13257136: The File Download and External Task Externalization windows have not be externalized for translation

Issue

The messages of the File Download and External Task Externalization windows have not been externalized for translation.

Fix

The messages are now externalized.

Deliverable

dndirector.war.



13257049: Process Externalization dialog does not re-size well

Issue

On making the Process Externalization dialog larger, the contents are not made any bigger.

Fix

The contents now scale with the dialog.

Deliverable

dndirector.war.



13256798: Users without job execution privileges can create schedules

Issue

Users without job execution privileges can create schedules and configure the Scheduler to execute jobs.

Fix

Creating schedules now requires job execution privileges.

Deliverable

dndirector.war.



13256520: Using drag-and-drop to manage the display of supplementary data in Case Management does not behave as expected

Issue

When dragging and dropping attributes above separators in the Case Management Supplementary display, the attributes must be dragged above an attribute; otherwise the attributes and the separator become the selected items.

Fix

Drag-and-drop now behaves as expected.

Deliverable

dndirector.war.



13226255: A canceled job still fires post-phase triggers

Issue

When a running job is cancelled, the post-phase triggers associated with the job phase running at the time are still fired.

Fix

The triggers are no longer fired.

Deliverable

dndirector.war.



13253480: Some processors have attributes with poor default names when published to staged data

Issue

The default attributes names for the Business Rules check and Cross Attribute check include:

%businessrules.pass.column.percentage

%businessrules.fail.column.percentage

%crossattrcheck.nullass.column.check

Fix

The values are now correctly externalized and localized.

Deliverable

dndirector.war.



13253359: Some processors have duplicate attribute names when staging results

Issue

The Date Profiler, Invalid Character Check, Length Check, Enhance from Map, Extract Values, Pattern Transform and RegEx Match all have two attributes with the same name when staging results (generally multiple columns called percentage or “%”).

Fix

The column names have been made unique.

Deliverable

dndirector.war.



13252700: Application Configuration pages on launchpad have old orange branding

Issue

The Application Configuration pages on the launchpad have the old orange branding.

Fix

Updated to new blue branding.

Deliverable

dndirector.war.



13252692: Case Management administration still has some old orange branding

Issue

The subsections within Case Management administration still have orange branding, and open sub-dialogs are still displayed with the old orange icon.

Fix

Updated to new blue branding.

Deliverable

dndirector.war.



13252672: Launchpad has lost its favicon

Issue

The launchpad has lost its favicon.

Fix

Favicon was lost during re-branding, but has now been replaced.

Deliverable

dndirector.war.



13252345: Task window reports incorrect number of rows for export tasks

Issue

The progress reporting and status information in the Task window rounds the number of rows exported down to the nearest 100.

Fix

The correct number of rows exported is now displayed.

Deliverable

dndirector.war.



13251897: Snapshots that failed no longer stay in the task panel

Issue

Failed Snapshots used to stay in the Task panel in an error state. This is no longer the case.

Fix

The Task Panel has been updated to correctly remember these failed snapshots.

Deliverable

dndirector.war.



13249078: High CPU and unresponsive UI when clicking on processors in large processes

Issue

As it was possible to configure results publication before a process had been run, the process was being re-validated every time a processor was selected.

Fix

The ability to configure results has been reworked so that the re-validation only needs to occur if the user starts to configure results publication.

Deliverable

dndirector.war.



13243917: Jmxtools had a typo in logon failure message

Issue

When using jmxtools to start jobs remotely, there is a typo in the message that is displayed if the user name or password is incorrect.

Fix

Typo corrected.

Deliverable

jmxtools.jar.



13243862: Run profiles are not carried forward to triggered jobs when run via command line

Issue

When a job triggers another job to run, the Run Profile Configuration is not passed forward to the job being triggered.

Fix

Run time has been amended to pass forward Run Profiles through triggers.

Deliverable

dndirector.war.



13243738: Ability to get to processor help from the palette has been lost

Issue

It used to be possible to right click on processors in the palette and follow a link to the online help. This is no longer possible.

Fix

Functionality was lost with canvas re-factoring and has now been restored.

Deliverable

dndirector.war.



13147441: Abnormally long attribute names causes GUI layout issues in Case Management Supplementary Data display

Issue

Renaming an attribute for display in the Case Management Supplementary Data display and providing a new attribute name of 60,000 characters causes UI layout issues, requiring users to manually stop the application.

Fix

Renamed Case Management Supplementary Data attributes are now limited to 80 characters, although previously renamed attributes will be respected.

Deliverable

dndirector.war.



13095839: When logging into a second server with Server Console and the server supports SSO, a login dialog is presented

Issue

When logging into second server in Server Console where the second server supports SSO, a user is still presented with a Login dialog. Canceling the dialog logs the current user in with their SSO credentials.

Fix

Login code amended not to display Login dialog if SSO is attempted.

Deliverable

dndirector.war.



13091237: Windows installer does not enable upgrades if the repositories have been moved to alternative databases

Issue

When using the Windows installer if the system is reconfigured to use an externally managed repository the upgrade process fails. This is due to improved security defaults.

Fix

The installer has been configured to correctly handle external repositories with improved security defaults.

Deliverable

dndirector.war.



13050033: The default width of the Event Log dialog is too small

Issue

The default width of the event log dialog is too small.

Fix

The default width is now bigger.

Deliverable

dndirector.war.



13049645: Manipulating the Data Interface mappings requires saving the Data Interface between deleting and creating identically named mappings

Issue

In the data interface mapping dialog, deleting a mapping and then attempting to create a new one of the same name returns a error indicating that the mapping already exists.

Fix

The dialog now correctly validates against the update mapping model.

Deliverable

dndirector.war.



13049373: Review status values in Match review have not been externalized for translation

Issue

The review status values in Match review have not been externalized for translation.

Fix

The status values have now been externalized.

Deliverable

dndirector.war.



13049330: Case Management parameter population screen does not correctly validate selection function configuration

Issue

During configuration of the parameter mappings of case configuration, when changing the selection function being used the validation did not always validate the actually selected function. After changing the input stream value the selection function had to be changed to a different value and back again to get correct validation.

Fix

The validation is now correctly reactivated when selection function configuration changes.

Deliverable

dndirector.war.



13049309: Copy and pasting of multiple Match rules causes the rules to be pasted in reverse order

Issue

Copying and pasting multiple rules to another rule group that already contains rules causes the pasted rules to be inserted in reverse order.

Fix

The ordering of pasting rules has been corrected.

Deliverable

dndirector.war.



13049271: A warning should be written to the log files when long data is truncated in a snapshot

Issue

A warning should be written to the log files when long data is truncated in a snapshot. The warning is not currently written.

Fix

The warning is now written to the log files.

Deliverable

dndirector.war.



13049078: Icons can sometime disappear from the Project Browser

Issue

The Project Browser was frequently loading icon files. This could cause timing problems, which meant on some occasions the icons failed to load.

Fix

The icons are now cached.

Deliverable

dndirector.war.



DIR-11251: Improve free-text searching to prevent “TooManyClauses” error when using wild cards that produce a large number of terms

Issue

When running wild card searches in Case Management that can return many terms, the search libraries sometimes return a “TooManyClauses” error.

Fix

A less efficient but more robust querying methodology is now used instead.

Deliverable

dndirector.war.

Version 9.0.1

New Features and Improvements v9.0.1:

The following table summarizes the most important new features and improvements in the 9.0.1 release. For full information on these features, please consult the product documentation. If you require a detailed briefing and/or training on the release, please contact your account manager.



Change Reference

Title

Description

DIR-12320

Server Console UI

A new user application has been introduced, designed for the needs of enterprise Operations and Support users. The application can be connected to many OEDQ servers, and allows users to run and schedule jobs, monitor current activity, view the Event Log, and see selected staged results from completed jobs.

DIR-11257

Dynamic Configuration Interface

The Director application now allows users to externalize points of configuration so that they can be overridden when jobs are run, either using the Command Line Interface, or from the new Server Console user application. Externalizing configuration means that the same job can be run many times (concurrently, if required) with different data bindings (for example, different source and target files) and different override options (for example, different snapshot options, processor options, match rules etc.)

Run Labels are used to store results separately for multiple executions of the same job.

Run Profiles are used to save sets of regularly used configuration override settings.

DIR-12630

New Address Verification Processor

A new Address Verification processor has been added, providing global address verification for over 240 countries. The Address Verification software and data must be purchased and installed separately.

DIR-11920

Data Interfaces

Director now supports Data Interfaces, meaning processes can now be defined with no specific bindings to any physical source or target of data, and can therefore be reused in different jobs, with different mappings to source and target data.

Note that Data Interfaces replace the concept of 'Views' in previous versions of OEDQ.

DIR-6867

Stage Results Views

Director now supports the ability to write top-level processor results summary views to Staged Data. This allows such results views to be used as data in processes, and to be viewed by Operations and Support users in the new Server Console user application.

DIR-8183

Hash Key Generator Processor

A new standard processor has been added in the Advanced family to allow users to generate Hash Key values from one or many input attributes.

DIR-1078

Replace All Processor

A new standard processor has been added in the Transformation family to allow users to make replacements across many attributes in a single operation.

DIR-12237

Allow Export directly from Snapshot

It is now possible to export data directly from a snapshot in Director.

DIR-10818

Improved Event Logging

The Event Log has been improved to log events whenever jobs or tasks start and end, and to provide improved filtering capabilities.

DIR-11293

Improved Progress Reporting from Match Processors

Director and Server Console users can now see more detailed progress reporting from processes that include match processors, with each phase of match execution clearly visible, with detailed statistics on progress.

DIR-5042

Improved Permission Granularity

The system now provides highly granular functional access control, with separate permissions for all major functions in the various user applications, including separate Add/Modify/Delete permissions for all configuration objects managed in Director.

DIR-12470

Improved Active Directory Integration

It is now possible for OEDQ to interrogate Active Directory in order to determine if a user's password has expired or is shortly to expire, and pop up a configurable message.

DIR-12469

Customizable Launchpad

The Launchpad can now be customized to remove links to the administration pages.

DIR-12474

Rebranding

OEDQ 9.0 has been updated with a new color scheme for improved consistency, and splash-screens have been updated to include the Enterprise Data Quality family name.

DIR-11154

Import and Export of Case Management Filters

Filters in Case Management can now be imported from, and exported to, files.

DIR-10116

Links in Case Management

Case Management users can now add Comments with enabled hyperlinks.

DIR-11257

Improved filter selection in Case Management

Case Management filter controls are now more intelligent. If users select a Case Type and Case Source, the available workflow states are automatically filtered to display only the relevant states.

DIR-12287

Multiple links in Case Management

Case Management now supports multiple hyperlinks in a single attribute value, with all links enabled.

DIR-11979

Ability to delete all cases

It is now possible to delete all cases and alerts on a server, using the JMX API.

DIR-10326

Multi-select assignment in Case Management

Case Management now allows users to select multiple cases or alerts from a list and assign them in a single operation.

DIR-11530

Improved data display in Case Management

Case Management now supports the ability to define header attributes and spaces when displaying data from several sources. It is also possible to rename attributes directly in the display.

DIR-11149

Prohibit assignment out of permission group in Case Management

Case Management no longer allows users to assign a case or alert to a user that does not have permission to view it.

DIR-11152

Integrated Version Control

Director now supports integrated version control using a Subversion repository. Multiple OEDQ instances can be configured to use a single Subversion repository, with configuration mastered in the file system.



Upgrading

The Windows installer will upgrade previous versions of Oracle Enterprise Data Quality (formerly known as dn:Director). Depending on the version being upgraded, there will be the following effects:

All Versions

Version 8.1.5 and earlier

Version 8.1.3 and earlier

Note: This issue has been fixed in a number of maintenance releases. If you are upgrading from any of the following versions, the migration work has already been done:

  • Version 8.0.22 or later releases of version 8.0

  • Version 7.2.10 or later releases of version 7.2

Version 8.1.2 and earlier

Note: This issue has been fixed in a number of maintenance releases. If you are upgrading from any of the following versions, the migration work has already been done:

  • Version 8.0.22 or later releases of version 8.0

  • Version 7.2.10 or later releases of version 7.2

Version 8.0 and earlier

Security Note: It is strongly recommended that you enable the external task security enhancements after upgrading to version 8.1. Please refer to the appropriate installation notes document for further details.

Version 8.0.7 and earlier 8.0 releases where LDAP is in use,

ldap.referral = follow

Version 7.2 or earlier

Version 7.1 or earlier.

Version 6.2 or earlier.

Known Issues

There is a problem with Windows Server 2003 for which Microsoft have released hotfix 924054. If your operating system is affected by this issue and you are unable to install the hotfix please contact support for a workaround.

If you have any problems please contact Oracle support.



Release Notes Oracle Enterprise Data Quality v. 9.0.8

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.