COMMENT: Analyst's Corner
In Perfect HarmonyBy David Baum
Data integration tools improve movement of information among systems.
Oracle Magazine spoke with Bill Swanton, vice president of research at AMR Research, about the importance of data quality and the role of data integration technology in enterprise resource planning (ERP) and data warehouse initiatives.
Oracle Magazine: Why do companies use data integration tools?
Swanton: The main purpose of data integration tools is to improve data quality and gain control over the movement of data among systems, such as for data warehouses and business intelligence projects. In some cases, companies use data integration tools to manage multiple copies of applications among divisions, or to combine data from different types of ERP systems. These tools also help companies harmonize data, such as when an organization has different part numbers for the same parts across multiple plants.
If a company is running its entire business on a single ERP system such as Oracle E-Business Suite, theoretically it doesn’t have significant data integration requirements, since the suite runs on an integrated data set. However, most organizations have other applications too, and they need to make sure that their information is consistent.
Oracle Magazine: How do organizations address data quality in their processes?
Swanton: Organizations can’t accomplish their business goals without a high level of data quality. For example, if they store customer information in both a customer relationship management [CRM] system and an ERP system, they have to synchronize the data to deal with customers in a consistent way. Ad hoc methods don’t work very well, especially for large volumes of data. Integration tools help ensure consistency between these systems and also reliably propagate data from one place to another. Once you know the structure of the data and the validation requirements, it’s not difficult to duplicate the effort among other interrelated systems.
You also need to be able to do the data extracts from the data warehouse fast enough to deliver fresh data. High-performance ETL [extract, transform, and load] technology is very important. You want to be able to move data in a relatively narrow time window when people are not using the system, so the information is available first thing in the morning when people come in to do their jobs.
Oracle Magazine: What are some of the key data integration capabilities that help organizations quickly deliver data quality?
Swanton: A good profiling capability is important because it helps you figure out what data you have and how to transform it into a new system. You also need an automation process to flag records that do not meet your criteria and provide a way for the people running the integration tool to resolve issues and fix the data in the source system. Automating the process lets you manage these issues by exception, rather than trying to examine the data line by line.
Oracle Magazine: How do organizations look at the cost and value of data integration tools?
Swanton: We recommend that people use data integration tools for many kinds of integration and data migration projects, partly for their profiling and data quality capabilities. For small projects, you can’t always justify the cost. But if the tools are available, they will make the projects easier and improve data quality, which saves time and effort down the road. Many ERP projects allot only 5 percent of the budget for data integration or data migration, yet these tasks typically consume 15 percent of the budget. It’s money well spent. Lots of data warehouses and data marts never generate substantial value because nobody believes in the data.
Oracle Magazine: What is the role of data integration tools in master data management?
Swanton: Any master data management project will have two major parts: the initial migration/harmonization of the data, and then ongoing data stewardship. Many people think that they need only to clean up the data once and then that’s the end of it, but they need a permanent process for managing the quality of their data. Data integration tools do this very well. They can help you set up the initial structure and then continually scan your systems to make sure that there are no inconsistencies. People should not have to spend time fixing data errors that can be found automatically.
David Baum (email@example.com) is a freelance business writer based in Santa Barbara, California.
AMR Research provides subscription advisory services and peer networking opportunities to operations and IT executives in the consumer products, life sciences, manufacturing, and retail sectors.