Integrating InformationBy David A. Kelly
Companies turn to Oracle data integration for speed, accuracy, and consistency.
In an economy that trades on information, companies must be able to integrate vast amounts of data from disparate systems—and manage data movement, synchronization, and quality, as well as data management and services—while reducing costs and improving efficiencies.
One company that has succeeded in this effort is NYK Line, a 120-year-old shipping company. For this organization, which coordinates 700 ships and generates US$24 billion in revenue annually, handling tens of thousands of cargo containers from around the world and delivering them safely and on time is an everyday event.
But, like many companies, Tokyo, Japan-based NYK Line faced a business challenge: as part of implementing a new global shipping solution, it needed tighter integration between its key business software, its e-commerce systems, and its corporate management information system. For example, NYK Line needed to be able to easily address a U.S. Customs Service requirement for shipping companies to notify Customs 24 hours before a container was loaded on a ship in a foreign port. This meant a quantum shift in how the company’s data was managed and reported—and that meant NYK Line needed a new data integration strategy.
After considering several market-leading data integration solutions, including tools from Informatica, NYK Line’s IT group—NYK Business Systems (NBS)—turned to Oracle Data Integrator for several reasons.
“Oracle Data Integrator is very open in the sense that we have a lot of connectors to different databases and datasources including IBM MQ, flat files, XML drivers, and more,” says Durgaprasad Pulakkat, architect in the IT governance group within NBS. “That versatility is important to us.”
Oracle Data Integrator’s JDBC XML driver simplifies development and configuration, according to Mohan Loganathan, architect and manager, data services group at NBS. “You can connect to any target with a JDBC driver,” he says. “Oracle Data Integrator is easy to learn, and that’s one of its major advantages. We’ve trained about 20 developers in it so far. Everyone gets three weeks’ orientation, and at the end of the three weeks they’re expected to be delivering results using Oracle Data Integrator on projects.”
Delivering Integration Solutions
In the short term, NYK Line was able to meet the U.S. Customs Service requirements for notification. “In less than a month, we were able to implant new business processes using Oracle Data Integrator that report on container movements and provide that information to U.S. Customs,” says Loganathan. “That’s when our business managers realized how good the time to market is with Oracle Data Integrator-based solutions. Now when they need a solution, they ask that we do it in Oracle Data Integrator.”
The company began rolling out Oracle Data Integrator-based solutions in February 2006 and has rolled out new solutions using the technology every one or two months since.
“Right now we have about 60 to 70 Oracle Data Integrator-based solutions running in production, doing about 10,000 jobs on a daily basis and moving about 25 million transactions per day,” says Pulakkat.
NBS also uses Oracle Data Integrator for a wide variety of solutions, including data integration between e-commerce systems, partner integration with customers, legacy integration with regional systems, integration with enterprise resource planning (ERP) solutions such as SAP, and data integration with a corporate management information system. For example, NBS uses Oracle Data Integrator to provide the data integration services for its container-tracking application used by customers around the world, which needs to be accessible 24 hours a day.
“We need 24/7 availability primarily for our e-commerce data integration,” says Loganathan. “Just as FedEx, DHL, or UPS provides tracking numbers for shipments, NYK Line has a site, www.nykline.com, where customers can enter their container or booking or bill of lading number and see where the container is.”
Overall, the company supports about 5,000 users around the world for the global shipping solution, while the management information systems provide support for 500 users. “We do a lot of data integration,” says Pulakkat.
Committing to a data integration strategy—and having a reliable, high-performance data integration solution—is critical for companies that want to succeed, says Bill Swanton, vice president of research, AMR Research.
“People can’t accomplish what they want to do in business without a high level of data quality,” he says. “For example, if you’re trying to expose just one face to your customer, you need to have very tight control over the synchronization between your CRM [customer relationship management] and ERP systems. Doing integration ad hoc just doesn’t work well. Especially when you have a large volume of data, data integration helps ensure data consistency, it helps propagate data from one location to another, and it reduces the cost of maintaining the systems.”
In an ideal world, systems would be designed to talk to each other natively instead of requiring integration after the fact. But for most organizations, the real world is populated by a wide range of applications with diverse data models that need to be integrated or at least consolidated for management and reporting purposes.
“Data integration sits at the nexus of four important disciplines: database and data warehousing, business intelligence, SOA, and master data management,” says Miranda Nash, senior director of development at Oracle. “Because Oracle is a leader in each of these solution areas, we’re able to help our customers extract the full value of these disciplines with a comprehensive data integration solution.”
Delivering Better Services
For the City of Arnhem in the Netherlands, data integration was only one step toward delivering better services to its citizens. But it was an important step. The goal was to put online more than 4 million pages of documentation such as drawings, building applications, and permits from the city’s planning archive; to integrate and consolidate all this information into a single, accessible database for everyone from real estate agents to firefighters to homeowners; and to make the information easily accessible.
The job was huge. The planning archive contains documents dating back to 1917. Additional information related to some documents, such as permits, was recorded on paper as well as by computer systems. Previously, the documents were spread across three sites, making the task of locating and accessing documents time consuming and costly.
With assistance from Oracle partner Redora, the city of Arnhem implemented a flexible SOA-based infrastructure that would enable all its central office applications to access the consolidated information. This included using Oracle Data Integrator to import metadata and existing data records related to the documents from a wide range of diverse sources (from old DataFlex and Microsoft Access applications to existing Oracle applications) into the Redora Business and Data Warehouse. Next, the City of Arnhem digitized all the documents using Oracle Document Capture. Scanned files were stored in compressed PDF/A format to save space and speed downloads.
By the time the project is completed at the end of 2009, the solution will encompass about 25 applications from different suppliers sharing centralized information on properties, people, and buildings.
“To get all of the 25 applications in Arnhem to talk to each other is complex and requires a lot of interfaces and adapters. It’s also very expensive and time consuming to try and connect them manually,” says Hemmo de Groot, director, information management, City of Arnhem. “Oracle Data Integrator, combined with Oracle Enterprise Service Bus and Oracle BPEL Process Manager, gives us a universal connection interface that makes it possible to do it in an easy and standardized way. Now, using Oracle Data Integrator, we have one point to manage our data integrations. We can use one tool to do all those different kinds of data transportation across the organization.”
“The goals of a city are not commercial—they’re to provide good quality of services to your citizens, to be a good city to live in, to be safe, and to be a nice place,” says de Groot. “Oracle Data Integrator is helping us achieve those goals by enabling us to work more efficiently and standardizing our technology. It helps us provide better quality of data to our citizens, and we can manage it with fewer people because we have one technology platform.”
The key to successful data management, says AMR Research’s Swanton, is about the economy and how organizations can keep their employees productive. “If people spend their time fixing data errors that could have been found automatically, that’s wasting their effort, and you’re probably not doing what you need to do to bring revenue in,” he says. “It’s as simple as it gets: automating mundane data management tasks will free people up to do more-useful things.”
David A. Kelly (email@example.com) is a business, technology, and travel writer who lives in West Newton, Massachusetts.