In anticipation of a big increase in site traffic, CNET Networks CIO Ted Cahall came up with a bold plan: use open-source software in conjunction with inexpensive two-way servers. However, he needed to take concrete steps to minimize the risks involved with open source implementations. The article covers: Cahall's careful consideration of risks and rewards associated with Linux His reliance on a secure foundation that includes Oracle Database and Oracle Discover. A brief guide to potential pitfalls — including high implementation costs and legal liabilities — and ways to avoid them.

As Published In February 2005

Brave New World
By Karen J. Bannan

CNET CIO Ted Cahall finds an "open source" of smart innovation.

Ted Cahall, CIO and senior vice president of engineering for CNET Networks, was on vacation three years ago when he got a call from the corporate development office, which wanted to talk to Cahall about how to handle an anticipated boost in the San Francisco-based media company's traffic. At the time, CNET's infrastructure could handle existing volume, but there were doubts that it would scale enough to handle significant increases in traffic. But additional eight-way servers like the ones that were already in use would cost US$60,000, used. New servers would cost US$250,000 each. Plus, because all of the company's business units shared one hub of 30 to 40 servers, a large increase in traffic might slow down existing Web sites, even with additional server space.

Cahall immediately started looking for a solution. He found his answer in an unconventional place: open source software in conjunction with simple—and cheaper—two-way servers. Cahall wasn't sure his proposal would be met with immediate approval, he says.

"This was radical. This was me walking in and saying, 'You need to double your infrastructure or more over the next couple of years and you want to do it for pennies on the dollar. This is the only strategy that will do that.' Everything else was going to be too expensive."

Practicing What They Preach

CNET Networks made its debut nearly a decade ago as a cable television network. Its online properties were a postscript, something that would support on-air activities. Although the television network never fully materialized, CNET Networks' online offerings have not only survived but they've also thrived. CNET Networks is a premier global online provider of content on personal technology, games and entertainment, and business technology. Approximately 89 million unique users visit the collection of sites—which include News.com, Download.com, photo-sharing site Webshots.com, and music site MP3.com—monthly, and that number has increased significantly over the past few years. For 2002 CNET Networks net revenues reached US$237 million. The company had an operating loss before depreciation, amortization, and asset impairment costs of US$30.4 million. As the company focused on returning to profitability, a plan that would improve revenues without taxing the bottom line was critical.

According to Cahall, CNET Networks needed to enter new markets and make acquisitions at a much lower cost. Many of those new markets and acquisitions would already be running open source software. And as a company that covers technology, it was important to stay on the leading edge. "Anybody who runs an IT organization that doesn't try to do things cheaply should be fired. So yes, we're trying to do leading-edge innovation, but we're also trying to do it smart," he says.

Open source, which was making inroads in the Web server market—many large sites were already running the Apache Web server—was gaining acceptance on the operating system side. Linux wasn't just free software anymore. Companies such as Raleigh, North Carolina-based Red Hat offered a version of Linux that would save the company more than US$1 million per year in licensing fees but still come with the support a large Web enterprise would require. Cahall says the company saves in both annual operating costs and capital expenditures. And best of all, its legacy and back-end software was already available for the fledgling operating system, so the company could move nearly everything over to the new platform.

Getting It Done

Making it happen was easier than Cahall thought it would be, although there were issues. Support was a concern in the beginning, because in 2002 the company didn't have its own in-house Linux expertise. Cahall needed Linux-savvy systems administrators to install and configure the Linux operating system and utilities. Additionally, he needed to port software and move Web content and applications from the old hub to the hundreds of new Linux systems. CNET Networks has over 1,000 servers and it was critical to make sure they were bulletproof, he says.

"We needed to be extra careful about security, because almost anyone can download the source code to Linux and learn about how to create a security exploit to compromise your system," says Cahall. "We took the time to do it right and put in extra security measures."

Cahall's team migrated all the applications and Web content from eight-CPU servers to US$6,000 dual 2.4GHz Xeon machines running Linux. The results were impressive, he says. "We were able to see that these systems were clearly outperforming our eight CPU RISC boxes on the same benchmark." Engineers ported software, including a special Apache client that handles CNET Networks' ad serving—a major source of revenue—and system management tools, search indexes, and the application servers. "The main issue was getting third-party libraries for our chosen version of Linux. We were able to move most of our environment over in about three months, and that was hundreds of machines," adds Cahall.

Information technology experts have heard war stories about moving from one server or operating system to another, but Cahall says that most employees never knew any of the changes had taken place. "I remember seeing [CNET executive] Dan Farber in the elevator and explaining that we were completely running on Linux now, and he was amazed. He had no idea we had changed everything out from under him," he says. "He saw no difference other than that things were faster."

Software, Soft Costs

Snapshot CNET Networks www.cnet.com Location: San Francisco, CA Industry: Media and Entertainment Employees: 1,700 Annual revenue: US$246.2 million in 2003 Oracle products and services: Oracle Database, Oracle Discoverer, and Oracle Financials

Adding open source technology did more than just save money and improve performance. Cahall says open source has also allowed employees to be more innovative. For example, pre-Linux, when all of the Web applications were on a single hub, three major business units, with different business and system requirements, coexisted on one platform. This inhibited innovation, as each business unit tried not to break the others' applications as they changed the infrastructure. For example, Download.com had to return quick, concise answers to user queries. News.com and ZDNet.com required immediate content publication and were not concerned with minor quality issues or the scale concerns of Download.com. Last, in contrast to the speed of innovation and scale required by the other business units, CNET.com and its shopping and product reviews properties required zero downtime and flawless presentation of merchants' products and prices. Slight mishaps due to moving too quickly on a new idea or because of the leanness of the scalability requirements were just not acceptable.

"Some business units were more concerned with innovation, some with scale, and some with extreme detail of quality, and all of these things had to coexist. You couldn't have all three characteristics, because they were all lumped onto the same hardware and set of servers," he says.

Moving to Linux and doing it with inexpensive hardware let each business unit have its own infrastructure. Although everything was architected and designed under the same underlying platform, no one was chained to a specific implementation. "At the same time that we moved all the business units to a common architecture, we literally split the business units' physical systems apart from each other. We let Download.com run on its own hardware and worried only about scaling to handle more pages a day. We moved CNET Shopper onto its own set of Linux servers and let it have a very high-quality control process, while making sure it never dropped one penny or upset one of its merchants," says Cahall. "We did this while letting the business technology group [ZDNet and News.com] have massive innovation and change things and break things occasionally. It was OK with little bumps in the night as it was trying out new ideas, which wouldn't have been OK for the other business units."

The Man Behind the Model

It isn't that surprising that CNET Networks' CIO would come up with such an innovative plan. Cahall is in the enviable position of being exposed to every bit of cutting-edge technology that's out there. He's also been working in the field since he was a teenager. "When I started, it was punch cards and mainframes—that kind of stuff—so I started programming at 14."

He augmented hands-on knowledge with formal education, receiving bachelor's and master's degrees in electrical engineering and computer science from the University of Illinois at Chicago. Management experience came from the trenches of Bank of America and AT&T Bell Laboratories and an executive MBA program at the University of North Carolina, Chapel Hill.

In the final analysis, Cahall is interested in continuing to use open source, but only to a point. "Oracle sells Oracle Financials, Oracle HR, its whole high-end enterprise application suite. Those applications and some other things that it's doing are a huge value-add to most corporations. Enterprise applications haven't really been touched in the open source space yet, and they need a rock-solid database such as the Oracle database server underneath them. In the current world of Sarbanes-Oxley compliance, you are probably not going to trust your financials to an open source database," says Cahall. "If there's a problem with your financials system and you're running Oracle Financials on top of an Oracle database, you can call Oracle up and they'll help get you through whatever the issue is. You can't do that with open source."

Let the Buyer Beware Open source is more than Linux—open source software includes business applications, games—even open source databases such as MySQL, PostGRES, Firebird, Ingres, and others. What are some of the risks of open source? Cost: One of the most misunderstood aspects of open source is cost. Although open source software is cheaper in the long run than typical software programs, that savings can be eaten up by implementation costs. And support fees often accompany open source software from the major suppliers. Compliance: Public companies, which are accountable to shareholders, are often cautious—and rightly so—about trusting mission-critical systems to software that doesn't come with a guarantee. "Anything auditors are going to ask about is still running on commercial implementations," says Gregory A. Jackson, a faculty member who is also vice president and chief information officer at the University of Chicago. "One of the considerations of doing one's own thing with open source is that if something goes wrong, you want someone there so you can sue." Lawsuits: The SCO Group has brought lawsuits against several major companies. The problem: The companies used Linux, and according to the SCO Group, it had developed parts of the open source operating system, so anyone using the program owes the SCO Group a licensing fee. This lawsuit and the Red Hat and IBM countersuits demonstrate one thing: If you're running open source, especially without a license, you run the risk of poaching someone else's intellectual property—or of someone's saying you are. The Solution? If you're looking for software to run your business and you're interested in exploring open source, look for software backed by major software and hardware vendors. Don't get scared off by implementation costs and support fees—you want the assurance of knowing you've got someplace to turn if you run into problems. And make sure you're paying attention to changes in the open source marketplace—you'll want to think strategically about your position in regard to open source as the market matures.