Rock Around the Clock

Pythian’s 24/7 Service Helps Worldwide Technology Implement Innovative Oracle Real Application Cluster Environment

It’s a common paradox. Companies want to take advantage of cutting-edge technology that offers significant cost savings and performance gains, but risks can be huge if few enterprises have actually implemented the new technology. Moreover, if problems occur, support resources are often scarce. This was the dilemma faced by World Wide Technology (WWT) when it needed to migrate to a new platform to host its mission-critical Oracle E-Business Suite application. The company wanted to take advantage of an innovative Oracle Real Application Clusters (RAC) technology. But with tight project deadlines, finding a partner to provide technical assistance proved to be difficult. Fortunately, WWT connected with Pythian, an Oracle Platinum partner that offered an around-the-clock support services model to provide the expertise and experience WWT needed.

WWT, based in St. Louis, Missouri, has offices across the U.S. and the world, and is a leading systems integrator providing technology products, services, and supply chain solutions to customers. The company has more than 1,900 employees and $US4.2 billion in revenue. With business booming, WWT has experienced growth of more than 20 percent year over year between 2006 and 2011, a trend that is expected to continue. And while expansion is certainly positive, it has resulted in some challenges.

WWT automated its business processes using Oracle E-Business Suite to achieve competitive advantages in the supply chain, human resources, and back-office areas. Nearly every single business activity passes through the Oracle E-Business Suite environment. In fact, on a typical day, more than 2,200 transactions per second (up to 31,000 concurrent requests per hour) are processed through the system. In addition, all global warehouse operations interface with the system for critical functionality. For four years, the Oracle E-Business Suite system ran on a single enterprise-class server, but over time the system became costly to maintain and was no longer capable of meeting the processing requirements of the business. In addition, the system was not easily scalable or highly available, and did not offer effective disaster recovery functionality.

According to Todd Carlson, manager, DBA/ERP and EUC Teams at WWT, it was clear that WWT needed to make a change. One option was to simply replace the existing hardware configuration with similar hardware, a low-risk but prohibitively expensive choice still not meeting WWT’s new high-availability and data-recovery requirements. Further complicating matters, WWT had to upgrade its version of the Oracle database to retain support. The company needed to find a way to upgrade with a solution that would ultimately save money and provide better performance. After considering all alternatives, the ideal option was to build an Oracle RAC environment using Red Hat Linux on commodity Intel-based services using the latest version of the Oracle Database, 11g R2, while also making use of WWT’s existing Tier 1 storage from EMC.

“Very few had ever done what we were doing with a cross-platform transportable tablespaces migration for the Oracle E-Business Suite database that Oracle had certified only months before,” says Carlson. “It was technically difficult, but it would save us 20–50 hours of downtime, which is measured at $22,000 per hour,” Carlson says. “We had to do it as quickly as possible to get the return on investment.”

He adds, “My IT guys are fantastic; very, very knowledgeable, but they don’t have exposure to everything that’s going on in the marketplace, where external partnering does.” Initially, when WWT started the project internally, IT personnel ran into frustrating and expensive technical issues, which were further compounded because management could not find a partner with the requisite level of expertise to help with the implementation.”

With time ticking away, WWT faced considerable technical challenges with the project that were placing critical milestones in jeopardy when the tide changed. The key element to the turn-around? Finding Pythian by way of Pythian’s well-regarded and popular blog (pythian.com/news). One of Carlson’s team members follows the blog and saw that Pythian appeared to have the knowledge that matched the project demands at WWT.

Carlson says, “We got them on the phone and grilled them, asking pointed and detailed technical questions. We knew right then and there that these were the guys. They were better technically than any other firm we talked to.”

Pythian, which is based in Ottawa, Canada, and has offices worldwide, believes WWT is not an anomaly in using the blog as a bridge to hiring their firm. Keith Miller, vice president of sales at Pythian, explains, “The blog started in December 2004. It is a serious time investment for us and it provides our technical people an avenue to continuously write about solutions they find in the field.”

Since 1997, Pythian has specialized in remote database services and consulting for Oracle and MySQL. The company, which has been an Oracle Partner since 1999, has achieved Oracle PartnerNetwork Specialized status in GoldenGate, Oracle Exadata, and Oracle Data Warehousing.

“We’re uniquely positioned because we manage the 24/7 operations of the Oracle E-Business Suite stack for clients such as Harvard Business School Publishing and other large enterprises running in RAC configurations,” says Miller. “We’ve had years and years of dealing with EBS in the true integrated stack perspective, which is kind of rare.”

When Pythian came aboard, they quickly understood the scope of the issues faced by WWT. “WWT was moving a lot of parts at the same time. They were doing an upgrade, which is not trivial, and looking at new hardware and storage options,” says David Ashlock, service delivery manager at Pythian, “We looked at what they’d already planned and worked to troubleshoot issues in their existing environment, and then we suggested some additional enhancements.”

For example, Pythian suggested WWT consider revisiting the way storage functionality is used in the migration process, and in addition to the traditional storage-area network options leverage Oracle Automatic Storage Management (ASM) features. According to Ashlock, such tactical suggestions shaved 20–30 hours off the migration time required to complete the project.

Pythian approached the project as having five distinct points: a hardware upgrade, an OS change over to Linux, a database upgrade, converting to Oracle RAC, and converting storage to ASM. Ashlock says, “Any one of these five would normally be a project itself, but we had to combine them into a tight timeframe because of the constraints WWT had.”

Pythian was able to help WWT meet its six-month project deadline. After coming aboard at the three-month mark, the maintenance window was cut in half to 24 hours, down from 55 hours, and production migration was completed in just 32 hours.

A huge factor in making the project successful was Pythian’s innovative “follow the sun” project management model, which enabled remote personnel to work on the project around the clock by handing off the project to engineering teams based all around the world—from the U.S. to Australia, to India, to Europe. In one week, Pythian’s around-the-clock strategy enabled engineers to fix a majority of the bottleneck issues that had stymied WWT. “I’d never seen that approach before,” says Carlson. “We had a company to partner with that could work 24/7 on a project, and that was just amazing.”

Another aspect that impressed WWT was that Pythian was able to do all the work remotely. “They were never here,” says Carlson. “I didn’t want to agree to the remote strategy at first, but Pythian convinced me, and it was fantastic. Absolutely no problems.”

Indeed, a growing number of enterprises are seeing the appeal of outsourcing. According to analysis by Computer Economics, more IT organizations are outsourcing a greater portion of their IT functions in 2011 as compared to just a few years ago, and they are doing so to take advantage of productivity boosts provided by remote teams working around the clock.

However, some experts warn against potential problems with the follow-the-sun strategy because it requires a lot of communication that can break down while working in different time zones. Pythian was able to avoid such issues with WWT by stressing collaboration. Miller says, “We work best when we work together with the client to remove roadblocks and push toward a common objective. Todd and his team were wonderful to work with and were very engaging.”

Since going live in April 2011, WWT has achieved a multitude of benefits. The new system is at least twice as fast as the old system, is highly available, and there is 60 percent more capacity. The company achieved significant savings in hardware costs, as well as more than US$100,000 in annual maintenance costs.

And though WWT has experienced some minor issues postmigration, the problems have not been insurmountable thanks to WWT’s ongoing relationship with Pythian. Capitalizing on the consistency in teams and people, WWT maintains an ongoing relationship with Pythian. “In 2012, we’re starting our planning to upgrade to Oracle E-Business Suite R 12, and Pythian will be involved,” Carlson says.

He adds, “Whenever you’re implementing something really new, you need to get people who’ve done it before,” Carlson says. “It sounds like a no-brainer. We were smart enough to know that if we had tried to do our project on our own, it might have been ugly. So, finding the right partner was key.”